Bug 690225 - Predictable input processing when reading from a pipe
Summary: Predictable input processing when reading from a pipe
Status: RESOLVED WORKSFORME
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: PS Interpreter (show other bugs)
Version: 8.63
Hardware: Macintosh MacOS X
: P4 normal
Assignee: Ralph Giles
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-01-10 13:25 UTC by Markus Triska
Modified: 2009-02-17 15:55 UTC (History)
0 users

See Also:
Customer:
Word Size: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Markus Triska 2009-01-10 13:25:24 UTC
Consider the following shell command (using gs's X11 output device):

$ (echo "0 0 500 500 rectfill showpage"; sleep 5) | gs  -

ghostscript (8.63) won't show anything for five seconds, then briefly update the
X window, and finally exit. Previous gs versions seem to have read input earlier
(judging from this thread:
http://ghostscript.com/pipermail/gs-code-review/2001-September/001185.html),
however, I could not get them to compile to test it.

Currently, there seem to be two workarounds to enforce processing of all input
that was sent to gs: First, you can send enough comment characters to fill gs's
(apparent) internal buffer (if such a buffer really exists, then an option to
disable it or set its size would fix this problem!). Second, you can use
interactive mode instead of pipe mode (i.e., drop the "-" in the shell), and
send a newline at the end. However, both workarounds slow down gs, and the first
will stop working when the buffer size changes.

Other applications used with pipes show a different behaviour, for example,
using "cat" instead of "gs":

(echo "0 0 500 500 rectfill showpage"; sleep 5) | cat 

the piped input is processed instantly, and displayed on the terminal by "cat"
long before the pipe is closed. Only gs seems to delay it arbitrarily, which you
can see with:

(echo "0 0 500 500 rectfill showpage"; read) | gs  -

This delays drawing the black rectangle until you press ENTER.

My use case for predictable input processing are PostScript animations, as in:

www.logic.at/prolog/queens/queens.html

Here, a program sends PostScript commands to gs via a pipe, and at the end, the
pipe must stay open to view the finished picture. However, unless I fill the
buffer with sufficiently many comment characters, PostScript may not process
remaining (already sent) instructions that are necessary to finish the animation.

There was a very similar (currently unanswered) question on the mailing list
recently, which also requires predictable processing of all instructions sent
via a pipe:

http://ghostscript.com/pipermail/gs-devel/2008-September/007972.html

Thank you in advance!
Comment 1 Ralph Giles 2009-01-11 10:12:50 UTC
I experimented to see if sending an eof (echo -e "\004") would help the parser
complete. It didn't work, but I noticed the rect is being filled when the pipe
closes.

One may verify the input is being parsed by passing it invalid postscript:

$ (echo "badcmd"; sleep 5) | gs -
[5 second pause]
Error: /undefined in badcmd

or by running the original command with output to a file.

FWIW.
Comment 2 Markus Triska 2009-01-11 11:44:35 UTC
Thank you for looking into this! Regarding your comment about writing to a file:
The point is that I want to view the animation in real-time, i.e., each
PostScript instruction should be processed as soon as it is emitted. This is
because the main process that I want to visualise can take a very long time to
finish, or may not even finish at all (therefore, I cannot send all output to a
file), and while it runs, it continuously emits PostScript instructions that I
send to gs. To give immediate visual feedback about the ongoing process, I thus
want gs to work off the input as soon as it arrives in the pipe, which seems to
be what earlier gs versions and all other programs I tested do. As mentioned, it
works as expected when I drop the "-" to start gs in interactive mode. However,
in interactive mode, gs also displays a prompt (which I don't need and which I
have to actively send to /dev/null to prevent from showing up, which is not even
portable), and using a pipe also is much faster and generally seems to be the
most fitting option in this case.
Comment 3 Ralph Giles 2009-01-12 11:57:14 UTC
You can remove the prompts with 'gs -q -dNOPROMPT'. E.g. the following shows an
animation for me with nothing to stdout:

$ (echo "100 100 400 400 rectfill showpage"; sleep 1; echo "200 200 300 300
rectfill showpage"; sleep 1) | gs -q -dNOPROMPT
Comment 4 Markus Triska 2009-01-12 14:10:52 UTC
Thanks, -dNOPROMPT is indeed a better way to prevent the prompt. Still, as
interactive mode is significantly (about 40%) slower than a pipe in my use case
(even with -dNOPROMPT), I want to keep using piped mode if possible. Here is
another test case for this situation: The following shows nothing for 10
seconds, then briefly displays the rectangle, and finally exits:

(echo " 2 .setlanguagelevel 0 0 500 dup rectfill copypage "; sleep 10) | gs  -

In contrast, if you emit 1000 comment characters after the PostScript
instructions, then the rectangle is displayed instantly:

(echo " 2 .setlanguagelevel 0 0 500 dup rectfill copypage "; for i in {1..1000};
do echo "%"; done; sleep 10) | gs  -


Comment 5 Alex Cherepanov 2009-02-16 11:31:16 UTC
Ghostscript has to look ahead into the input stream to decide whether it's
PostScript, EPS, or PDF. That's why about 1K of characters are accumulated before
the processing starts. There's no buffering afterwards.

So the work around is easy. Fill the initial buffers with 1K of spaces and
enjoy unbuffered processing afterwards. Please confirm whether the proposed
work around is sufficient for you.
Comment 6 Markus Triska 2009-02-16 14:17:14 UTC
This seems not to solve the problem. Take for example show.ps from:

http://www.logic.at/prolog/show.ps

This file's size is several KB, yet the last few instructions are not processed
when you do:

$ (cat show.ps; read) | gs -g680x680 -r144 -dGraphicsAlphaBits=2 -

On the other hand, if you use gs interactively without the "pipe" flag, i.e.:

$ (cat show.ps; read) | gs -g680x680 -r144 -dGraphicsAlphaBits=2

the animation is shown completely. Even if I first emit 2000 blanks, the final
few instructions are not processed in pipe mode. (As an aside, 1000 characters
seem excessive to me at first sight to determine the file's type - in most cases
a few characters seem to suffice to me, but yes, it would yield a bearable
workaround if it were documented, would not grow bigger in future versions, and
worked as you say.)
Comment 7 Alex Cherepanov 2009-02-17 06:24:38 UTC
First, Adobe specifies that PDF file starts with 1024 bytes of junk. Actuality,
they say that the reader should search the first 1K of the file for the header,
presumably to process PDF file included in an email as text.

The size of the searched block may increase if Adobe chooses to do so.

Fortunately, PS file is recognized by the first 2 characters, '%!' before
GS tries to find PDF header. So all you need to do is to start your file with
'%!' . This is a standard practice, documented in every PS manual.

The graphic output is also buffered. You need to use operator 'flushpage'
to ensure that the graphic window is updated.

With '%!' header and 'flushpage' the animation works as expected.

Finally, 8 queens problem can be easily coded in PostScript. Long ago I coded it
on an early programmable calculator that had 14 memory cells, 98 program cells,
and 2 FLOPS speed.
Comment 8 Markus Triska 2009-02-17 12:39:20 UTC
After considering all issues, it seems best to simply stick to -dNOPROMPT and
pipe to gs the generated code without using the pipe flag, because only then the
display buffer seems to be predictably flushed without having to mention
(gs-specific) 'flushpage'. At the same time the prompt is disabled, and it works
with Level 3 PostScript. Only one issue remains: As we discussed on IRC, the
current documentation about -dNOPRPOMPT is wrong in that it states that this
switch disables the implicit flushpage, which is (luckily!) not the case. Thank
you in advance for fixing the documentation, and for looking into this!
Comment 9 Alex Cherepanov 2009-02-17 15:55:52 UTC
Documentation is fixed by the rev. 9484.

I don't understand what's wrong with Ghostscript extensions. Other
interpreters don't provide this functionality anyway. You can always
check whether extensions are available. For instance:

systemdict /flushpage known not {
    /flushpage {} def % define as no-op
} if

Since no Ghostscript code has been changed I'm closing this bug report as
WORKSFORME .