Bug 690704 - Regression: bad pdf files indeterministically generated by pdfwrite from pcl files
Summary: Regression: bad pdf files indeterministically generated by pdfwrite from pcl ...
Status: RESOLVED FIXED
Alias: None
Product: GhostPCL
Classification: Unclassified
Component: Regression PCL (show other bugs)
Version: master
Hardware: PC Linux
: P1 normal
Assignee: Marcos H. Woehrmann
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-08-12 22:21 UTC by Marcos H. Woehrmann
Modified: 2012-09-04 19:45 UTC (History)
1 user (show)

See Also:
Customer:
Word Size: ---


Attachments
c306-cut.bin (279 bytes, application/octet-stream)
2009-10-01 01:00 UTC, Ken Sharp
Details
c318-cut.bin (5.56 KB, application/octet-stream)
2009-10-01 01:00 UTC, Ken Sharp
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Marcos H. Woehrmann 2009-08-12 22:21:55 UTC
These PCL-XL FTS files seg fault with -sDEVICE=pdfwrite on my x86_64 box running
Ubuntu:

pxlfts/t306.bin
pxlfts/t307.bin
pxlfts/t421.bin
pxlfts/t422.bin

pxlfts2.0/t305.bin
pxlfts2.0/t306.bin
pxlfts2.0/t307.bin
pxlfts2.0/t419.bin
pxlfts2.0/t420.bin
pxlfts2.0/t421.bin
pxlfts2.0/t422.bin

pxlfts3.0/A001.BIN
pxlfts3.0/A003.BIN
pxlfts3.0/A005.BIN
pxlfts3.0/A006.BIN
pxlfts3.0/T306.BIN
pxlfts3.0/T307.BIN
pxlfts3.0/T335.BIN
pxlfts3.0/T337.BIN
pxlfts3.0/T421.BIN
pxlfts3.0/T422.BIN

The command line I'm using:

  main/obj/pcl6 -sDEVICE=pdfwrite -sOutputFile=test.pdf -dNOPAUSE ./t306.bin
Comment 1 Ken Sharp 2009-08-13 01:02:10 UTC
Marcos I believe I fixed this one yesterday, the problem being in pdfwrite, but
hadn't finished with PCL so I didn't commit it. I committed the fix today:

http://ghostscript.com/pipermail/gs-cvs/2009-August/009689.html

If you still see any crashes with PCL/PXL and pdfwrite please let me know. I
think that the XPS files should all run without crashing as well.

I still see that a number of PXL/PXL files produce invalid PDF files, one of the
issues is in the PXL error handling so I asked Henry to look at that one, I'm
looking into the others at the moment.

Comment 2 Marcos H. Woehrmann 2009-08-13 08:48:23 UTC
I can confirm that the current head (r9987) no longer seg faults, however, as
you noted, some files produce bad pdf.  I didn't do a complete check, but these
 include:

pxlfts/c318.bin
pxlfts/t306.bin
pxlfts/t307.bin
pxlfts/t421.bin
pxlfts/t422.bin
Comment 3 Ken Sharp 2009-08-13 09:00:48 UTC
My last commit might help some of the PCL files, maybe. It makes pdfwrite better
at noticing an object is actually used, not just defined. 

It helps a fair number of XPS files and 3 PostScript files, so it might well
help some of the PCL/PXL files.

I've punted one reference counting problem to Henry, I guess he'll fix it
shortly, which should improve a number of the C* and t* FTS files, which write
an error page, but I know there are more cases like the ones fixed by my last
revision. I'll get back to them after my vacation.....

Comment 4 Henry Stiles 2009-08-13 11:12:06 UTC
Adding analysis from Ken and note the "show_n" API (below) should be obsolete. 

Ken writes:

... This time its the cache device for bitmapped fonts. Now it looks like this
is stored in the text enumerator (dev_cache & dev_cache2) and these are
'retained'. I think they should be released when the enumerator is released.

The text in question actually comes from px_error_page_show, there's a loop
commented '/* Peel off the next line and display it. */' which uses the
enumerator 'pxs->error_page_show_enum.

Now, this enumerator is initialised once at startup, and properly released when
the interpreter has finished with it, which should address the reference
counting issues. Not only that but because the enumerator is retained we should
only set up the cache device for it once, when its first used.

What I see is that we frequently set up the cache device, incrementing reference
counts every time.

This appears to be because px_error_page_show calls show_n_begin and towards the
end of show_n_begin there is this code:

   /* Now we know pte points to a gs_show_enum. */
   *penum = *(gs_show_enum *)pte;


That sets the penum->dev_cache and penum->dev_cache2 members to 0 (from the pte
members) but does *not* count down the references to these devices. This may
have other unpleasant side effects as well of course...
Comment 5 Marcos H. Woehrmann 2009-08-13 12:36:13 UTC
I ran some additional files through the pipeline and these pcl files also produce bad pdf files:

tests/pcl/jason_ers.pcl
tests/pcl/jason_top.pcl
tests/pcl/thesis10.pcl
tests_private/pcl/pcl5cfts/fts.0510
tests_private/pcl/pcl5cfts/fts.0690
tests_private/pcl/pcl5cfts/fts.0730
tests_private/pcl/pcl5cfts/fts.0770
tests_private/pcl/pcl5cfts/fts.0790
tests_private/pcl/pcl5cfts/fts.0810
tests_private/pcl/pcl5efts/fts.2120
Comment 6 Henry Stiles 2009-08-13 22:30:53 UTC
> I've punted one reference counting problem to Henry, I guess he'll fix it
> shortly, which should improve a number of the C* and t* FTS files, which write
> an error page, but I know there are more cases like the ones fixed by my last
> revision. I'll get back to them after my vacation.....


Okay, the punt was fixed in revision 9993, haven't done much testing though. 
Assign it back to me if there is another one I should look at.
Comment 7 Ken Sharp 2009-09-25 02:20:22 UTC
revision 10087:

http://ghostscript.com/pipermail/gs-cvs/2009-September/009812.html

fixes another set of issues, looks like there are two classes of problem left at
this point.
Comment 8 Marcos H. Woehrmann 2009-09-30 21:31:02 UTC
Two of the files occasionally generate good pdf files, bad pdf files, or core dump:

  tests/pcl/jason_top.pcl
  tests/pcl/ieee0495.pcl

This indeterminism is annoying in the local cluster regression reports so I'm
going to disable these files; please re-enable when the problem is resolved.

Both produce many valgrind errors; let me know if you'd like it attached to the
bug report.
Comment 9 Ken Sharp 2009-10-01 00:14:57 UTC
I can't find the 'tests' sub-directory, can you give me a fuller URL please ?
Comment 10 Ken Sharp 2009-10-01 01:00:00 UTC
Created attachment 5425 [details]
c306-cut.bin

Thanks to Henry's recent fixes there are now very few files which fail, I still
see the following problems (apart from Marcos' indeterminisms):

cd60bw22.bin
Ends the job with a device reference count of 9. Even forcing this results in a
slightly damaged PDF file, I suspect some referenced object is not being
written (pdfwrite problem). The reduced version of this file *does* work, so
this is a different manifestation of the same problem (could be the same as
c318.bin below).

c306.bin, c307.bin (pcl6cet & pcl6cet3.0)
Seg fault, caused by the use of the clist to render large patterns. pdfwrite
has 2 methods for patterns, high level which is not supported by PCL and low
level. When using the low level method pdfwrite assumes the raster will be in
the pattern tile 'tbits' member, if we use the clist then this is a NULL
pointer.

c318.bin (pcl6cet & pcl6cet3.0)
device reference counting problem, the device has a reference count of 137 at
the end of the job. This is still the case even after Henry's latest fixes.

I will attach two test files, c306-cut.bin which has a single pattern causing
the seg fault, and c318-cut.bin which has two pages. I'm unable to reduce
c318.bin any further without the issue disappearing, I don't understand why....
Comment 11 Ken Sharp 2009-10-01 01:00:27 UTC
Created attachment 5426 [details]
c318-cut.bin
Comment 12 Marcos H. Woehrmann 2009-10-01 11:40:44 UTC
The files mentioned in comment #8 are in the svn repostitory: 
svn.ghostscript.com/svn/ghostscript/tests/pcl

They've renamed to .disabled by a recent commit.
Comment 13 Henry Stiles 2010-08-03 17:12:41 UTC
Should be fixed by one of the several reference counting fixes.  The disabled files run to completion and can be returned to the regression suite.  Assign it back to me if you want me to do it.
Comment 14 Marcos H. Woehrmann 2010-08-04 19:51:37 UTC
(In reply to comment #13)
> Should be fixed by one of the several reference counting fixes.  The disabled
> files run to completion and can be returned to the regression suite.  Assign it
> back to me if you want me to do it.

I've re-enabled:

  tests/pcl/jason_top.pcl
  tests/pcl/ieee0495.pcl