Bug 689577

Created attachment 3583 [details]
d01785-001

Bumping the priority since this bug effects a large number of users and has high
visibility.

Especially note that distros tend to the library version of Ghostscript, due to
my packaging work Ubuntu, Debian, and Mandriva use the library version already.
Some programs like GSView require the library. For printing the input files are
usually piped into Ghostscript (in most cases by foomatic-rip or pstoraster) and
acroread 8.1.1 ships also with most current distros. So the occurrence of the
problem is ievitable.

A fix by a small patch is appreciated, as it can more easily be applied as an
update for a released Linux distribution than a new release of Ghostscript.

The problem happens after the code:

%ADOBeginClientInjection: DocumentSetup Start "No Re-Distill"
%% Removing the following eleven lines is illegal, subject to the Digital
Copyright Act of 1998.
mark currentfile eexec
54dc5232e897cbaaa7584b7da7c23a6c59e7451851159cdbf40334cc2600
30036a856fabb196b3ddab71514d79106c969797b119ae4379c5ac9b7318
33471fc81a8e4b87bac59f7003cddaebea2a741c4e80818b4b136660994b
18a85d6b60e3c6b57cc0815fe834bc82704ac2caf0b6e228ce1b2218c8c7
67e87aef6db14cd38dda844c855b4e9c46d510cab8fdaa521d67cbb83ee1
af966cc79653b9aca2a5f91f908bbd3f06ecc0c940097ec77e210e6184dc
2f5777aacfc6907d43f1edb490a2a89c9af5b90ff126c0c3c5da9ae99f59
d47040be1c0336205bf3c6169b1b01cd78f922ec384cd0fcab955c0c20de
000000000000000000000000000000000000000000000000000000000000
cleartomark
%ADOEndClientInjection: DocumentSetup Start "No Re-Distill"

Apparently the 'cl' is getting swallowed by the eexec filter resulting in the
Error: /undefined in eartomark

Note that I was able to duplicate the problem on Windows with HEAD using:

cat bug_689577.ps | gswin32c -

Also on Windows, this works:   gswin32c bug_589577.ps

The encoded sequence converts to:

2f63757272656e7464697374696c6c6572706172616d73207768657265207b20706f70202f7064666d61726b2077686572650a7b706f7020285468697320506f737453637269
70742066696c652077617320637265617465642066726f6d20616e20656e63727970746564205044462066696c652e5c6e29207072696e740a28526564697374696c6c696e67
20656e6372797074656420504446206973206e6f74207065726d69747465642e5c6e29207072696e740a7573657264696374202f71756974206765742065786563207d69667d
2069660a63757272656e7466696c6520636c6f736566696c650a1604804bdadf7f4a8d878826f56dff7415478907310809f3cee4a38cae4f

which displays as:

/currentdistillerparams where { pop /pdfmark where
{pop (This PostScript file was created from an encrypted PDF file.\n) print
(Redistilling encrypted PDF is not permitted.\n) print
userdict /quit get exec }if} if
currentfile closefile
▬♦ΗK┌▀⌂Jμηκ&⌡m t§Gλ     ≤╬Σϊξ«O

Created attachment 3587 [details]
simplified.ps

A much simplified test case that fails in the same way.,

The problem appears to be the mem->gs_lib_ctx->stdin_is_interactive flag being
set incorrectly.  The following hack allows the file to be read, however this
likely breaks other things:

Index: ziodevsc.c
===================================================================
--- ziodevsc.c  (revision 8416)
+++ ziodevsc.c  (working copy)
@@ -99,7 +99,7 @@
     if (mem->gs_lib_ctx->stdin_fn)
        count = (*mem->gs_lib_ctx->stdin_fn)
            (mem->gs_lib_ctx->caller_handle, (char *)pw->ptr + 1,
-            mem->gs_lib_ctx->stdin_is_interactive ? 1 : wcount);
+            mem->gs_lib_ctx->stdin_is_interactive ? wcount : wcount);  /* mhw */
     else
        count = gp_stdin_read((char *)pw->ptr + 1, wcount,
                      mem->gs_lib_ctx->stdin_is_interactive,

More details:

The significant routine appears to be s_stdin_read_process() in ziodevsc.c:

/* Read from stdin into the buffer. */
/* If interactive, only read one character. */
static int
s_stdin_read_process(stream_state * st, stream_cursor_read * ignore_pr,
                     stream_cursor_write * pw, bool last)
{
    int wcount = (int)(pw->limit - pw->ptr);
    int count;
    gs_memory_t *mem = st->memory;

    if (wcount <= 0)
        return 0;

    /* do the callout */
    if (mem->gs_lib_ctx->stdin_fn)
        count = (*mem->gs_lib_ctx->stdin_fn)
            (mem->gs_lib_ctx->caller_handle, (char *)pw->ptr + 1,
             mem->gs_lib_ctx->stdin_is_interactive ? 1 : wcount);
    else
        count = gp_stdin_read((char *)pw->ptr + 1, wcount,
                      mem->gs_lib_ctx->stdin_is_interactive,
                      mem->gs_lib_ctx->fstdin);

    pw->ptr += (count < 0) ? 0 : count;
    return ((count < 0) ? ERRC : (count == 0) ? EOFC : count);
}


The difference between Ghostscript running as a shared library and as a
standalone problem when using "- <filename" is that when running as a program
the gp_stdin_read() routine is called to read stdin and it is passed
mem->gs_lib_ctx->stdin_is_interactive which it ignores:

/* Read bytes from stdin, unbuffered if possible. */
int gp_stdin_read(char *buf, int len, int interactive, FILE *f)
{
    return read(fileno(f), buf, len);
}

Whereas when Ghostscript is a library the call to read stdin is to gsdll_stdin()
(which is what *mem->gs_lib_ctx->stdin_fn is set to) and it doesn't take
mem->gs_lib_ctx->stdin_is_interactive as a parameter but instead is called with
wcount set to 1.

So my comment #8 is partially correct, mem->gs_lib_ctx->stdin_is_interactive
isn't set incorrectly, it's just ignored in the standalone case.

So the question is why does the shared library need to pay attention to the
stdin_is_interactive flag when the standalone program ignores it?  Does this
comment explain why (from imainarg.c):

        case 0:         /* read stdin as a file char-by-char */
            /* This is a ******HACK****** for Ghostview. */
            minst->heap->gs_lib_ctx->stdin_is_interactive = true;
            goto run_stdin;

Based on Ray's suggestion I tested -_ instead of - to enable buffered reading
and it works.  This is a workaround, so I'll suggest this to Till but leave the
bug open.

Here's a diff -c example of the hack from comment #10:

*** LaserJet-6MP-foomatic.ppd.old       Tue Dec  4 11:04:07 2007
--- LaserJet-6MP-foomatic.ppd   Tue Dec  4 11:13:52 2007
***************
*** 95,101 ****
 *FoomaticIDs: HP-LaserJet_6MP hpijs
 *FoomaticRIPCommandLine: "gs -q -dBATCH -dPARANOIDSAFER -dQUIET -dNOPA&&
 USE -sDEVICE=ijs -sIjsServer=hpijs%A%B%C -dIjsUseOutputFD%Z -sOutputFi&&
! le=- -"
 *End

 *FoomaticRIPOption Model: enum CmdLine A 100
--- 95,101 ----
 *FoomaticIDs: HP-LaserJet_6MP hpijs
 *FoomaticRIPCommandLine: "gs -q -dBATCH -dPARANOIDSAFER -dQUIET -dNOPA&&
 USE -sDEVICE=ijs -sIjsServer=hpijs%A%B%C -dIjsUseOutputFD%Z -sOutputFi&&
! le=- -_"
 *End

 *FoomaticRIPOption Model: enum CmdLine A 100

To not need to change all entries in the Foomatic database one by one, I have
simply modified the foomatic-gswrapper (the general Ghostscript problem
workaround layer of Foomatic) to replace any expression for stdin as the input
file for Ghostscript ny '-_'. Get the new version of the foomatic-gswrapper from

http://www.openprinting.org/foomatic-gswrapper

and replace your foomatic-gswrapper with it (usually in /usr/bin)

This makes all print queues with Foomatic-based PPDs (for example HP inkjets and
PCL lasers) printing correctly again.

If your printer uses a CUPS raster driver, the /usr/lib/cups/filter/pstoraster
needs to be patched. Replace the line

ifile="-"

by

ifile="-_"

(line 54 in current version). I have done this in the Ghostscript SVN repository
as rev 8423.

If you are using a driver of Rick Richardson's foo2zjs suite
(http://foo2zjs.rkkda.com/, HP LaserJet 10xx, P10xx, M1005 MFP, Color LaserJet
1500, 1600, 2600n, Konica Minolta magicolor DL/MF series, Samsung CLP-300/600,
Xerox Phaser 6110, Lexmark C500n) run the following command as root to patch all
the Ghostscript wrapper scripts of foo2zjs:

perl -p -i -e 's/(GAMMAFILE )- /\1-_ /' /usr/bin/foo2*wrapper

I have applied all this in Ubuntu Hardy now.

For foo2zjs the workaround was now applied upstream, too.

A fix of this bug is really needed, as workarounds cannot be applied everywhere.
Especially the closed-source drivers from Samsung call Ghostscript (with "-" as
input) out of a closed-source executable. Here one cannot do quick workarounds
as I have shown in my previous comments. See also the complaint of the user
"lunomad" in the Ubuntu bug report.

Can we get a nod whether a fix for this in ghostscript is coming or not?
Many applications like kghostview and evince are failing as well. I made
a small fix for kghostview (replacing - with -_ in one location) but if
ghostscript is going to be fixed we will have to take all these out again.
It would be nice to know. Thanks.

Till wrote in the previous comment:
"A fix of this bug is really needed, as workarounds cannot be applied everywhere.
Especially the closed-source drivers from Samsung call Ghostscript (with "-" as
input) out of a closed-source executable."

Since GPL doesn't allow closed source (isn't being able to fix it yourself part
of the point of GPL), this looks like a violation of GPL and thus a violation
of Artifex's license of Ghostscript under GPL. I'm not a lawyer, but I will
forward this to the appropriate people at Artifex Software.

----

As far as fixing the bug, it's still open and will get fixed.

A comment on needing to take out the '-_' ...

The '-' assumes that the input needs to be interactive and shouldn't be
buffered (needs to read 1 character at a time). The '-_' uses larger buffers from
stdin and so it generally is more efficient anyway.

I have a couple of suggestions as to how we can fix this without having to delve
into the atream code:

1.  Replace /usr/bin/gs with the standalone gs instead of the stub that calls
the library.

2.  Have the /usr/bin/gs stub scan the command line and replace '-' with '-_'
before calling the library Ghostscript.  

Both to these shouldn't break any existing code.  Since the standalone
Ghostscript has always had '-' mean buffered input anyone calling /usr/bin/gs
shouldn't be expecting anything else.

A patch that reduces the buffer in eexecDecode filter has been committed
as a rev. 8428. See:
http://ghostscript.com/pipermail/gs-cvs/2007-December/008011.html

Regression testing shows no differences.

The sample file is incorrect. It has to few 0's than specified in
PLRM. The affected fragment is inserted into PS files generated by
Adobe Acrobat from PDF files with usage restrictions. So the fragment
doesn't change between files and the proposed solution should be
robust enough.

The buffer size of 132 has been selected from a middle of small window that
fixes the bug but doesn't cause regression in comparefiles/fonttest.pdf.
Detection of EOF after seeing a run of 0's is worth to note as an alternative.

Should one keep the previous changes to cups etc. that replace - with -_ ?

An addition to my comment #14: A possible workaround is to replace Samsung's
closed-source driver ny the open-source driver SpliX, which supports most not
too old Samsung printers. From Samsung's driver use only the scanner driver if
you have an MF device. The SpliX driver (see http://www.openprinting.org/) is a
pure CUPS Raster driver which does not call Ghostscript by itself, my
workarounds from comment #12 are sufficient.

About comment #16: There is no GPL violation. Samsung's drivers are not
published under the GPL.

About comment #17: I will keep all my workarounds in place, not only because it
takes time until everyone will have a fixed Ghostscript but also as buffered
input is better for the performance.

About comment #18: The suggestion (2) can be implemented easily: Replace your
/usr/bin/foomatic-gswrapper by the newest one from

http://www.openprinting.org/foomatic-gswrapper

Rename /usr/bin/gs to /usr/bin/gs.orig and rename /usr/bin/foomatic-gswrapper to
/usr/bin/gs. Then edit /usr/bin/gs replacing 'my $gspath = "gs";' by 'my $gspath
= "gs.orig";'. Now every call of "gs", including calls by pstoraster and by
closed-source drivers, has all workarounds and fixes from foomatic-gswrapper
applied, especially the "-_" as input file.

About comment #19: As the fix is a really simple patch, I recommend to every
distro package maintainer to apply this patch without waiting for the next
official release of Ghostscript, and even apply this patch as an update for
already released versions of the distros.

Here is the patch:

----------
Modified: trunk/gs/src/seexec.c
===================================================================
--- trunk/gs/src/seexec.c 2007-12-08 13:57:53 UTC (rev 8427)
+++ trunk/gs/src/seexec.c 2007-12-09 06:33:00 UTC (rev 8428)
@@ -215,8 +215,10 @@
  * so that it will stay under the limit even after adding min_in_size
  * for a subsequent filter in a pipeline. Note that we have to specify
  * a size of at least 128 so that filter_read won't round it up.
+ * The value of 132 is small enough for the sample file of the bug 689577 but
+ * still sufficient for comparefiles/fonttest.pdf .
  */
 const stream_template s_exD_template = {
- &st_exD_state, s_exD_init, s_exD_process, 8, 200,
+ &st_exD_state, s_exD_init, s_exD_process, 8, 132,
     NULL, s_exD_set_defaults
 };
----------

Sammy, I recommend to keep the workarounds in place as buffered input improves
the performance.

Although the 200 -> 132 patch solves the problem for simplified.ps I have a PDF
for which the problem is not solved.

We get

"Error: /undefined in ec"

I can send the file privately.

This bug did not exist in ghostscript 8.15-22.

Created attachment 5002 [details]
pdfteste-hack.ps

Reduced PS file that has Adobe startup code only. The file displays blank page.


The bug can be reproduced by:
  gswin32c - <pdfteste-hack.ps

Seems that this problem applies to all PDFs with "security restrictions" (even
if printed is not restricted).

The bug is present on ghostscript 8.60-55 but not on (for example) 8.15-22 and
7.07-25.

Is this expected to be fixed inside ghostscript?

Reproduced pdfteste-hack.ps from #25 with r9948 on both Windows XP Pro SP3 + VS2008 Express and 
Windows 2000 Pro SP4 + VS6 SP6.  Did not reproduce on Mac OS X.  Looks like windows only problem.
Replacing '-' with '-_' removes error, this can be used as a workaround.

My mistake.  Please forget 'Did not reproduce on Mac OS X.  Looks like windows only problem.' part in 
above my comment.  I didn't try it using shared object.

In hex form of eexec stream, break out of reading loop and try to interpret
the accumulated data when a whitespace character is encountered. Skip whitespace
characters when the control returns to /eexecDecode filter.

Normally, eexec stream is followed by a large number of 0's to compensate
for read-ahead effects. The code fragment that Adobe Acrobat uses to prevent
re-distilling of the file has no trailing 0's at all. This patch tries to guess
the end of eexec stream after every whitespace character.

The following patch has been committed as a rev. 9989.
http://ghostscript.com/pipermail/gs-cvs/2009-August/009693.html
Regression testing shows no differences.