Bug 689705 - ExtraData Error with ReadImage JPEG Compression
Summary: ExtraData Error with ReadImage JPEG Compression
Status: RESOLVED LATER
Alias: None
Product: GhostPCL
Classification: Unclassified
Component: PCL interpreter (show other bugs)
Version: 1.51
Hardware: PC Windows XP
: P4 normal
Assignee: Henry Stiles
URL:
Keywords: bountiable
Depends on:
Blocks:
 
Reported: 2008-02-13 17:01 UTC by Jeffrey Vance
Modified: 2011-03-17 21:10 UTC (History)
4 users (show)

See Also:
Customer:
Word Size: ---


Attachments
PXL File which causes the ExtraData error. (1.27 MB, application/octet-stream)
2008-02-13 17:02 UTC, Jeffrey Vance
Details
Slimmed down testcase with same image (1.13 KB, application/octet-stream)
2008-02-13 17:02 UTC, Jeffrey Vance
Details
modified pxl with extra setColorSpace (1.27 MB, application/octet-stream)
2009-10-27 19:55 UTC, Hin-Tak Leung
Details
Patch for Bug689705 (1.15 KB, patch)
2011-01-24 22:43 UTC, Shailesh Mistry
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jeffrey Vance 2008-02-13 17:01:26 UTC
Hi all :)

I'm currently running the GhostPDL 1.51 release with a patch to fix a 
bitmasking problem before (essentially just the 1.51 release).  I'm also 
running on windows vista ultimate.

When running an output PXL file I'm getting an error when I hit a ReadImage:

PCL XL error
    Subsystem: KERNEL
    Error:     ExtraData
    Operator:  ReadImage
    Position:  4692

I've isolated out the image that appears to be causing the image to a test 
case that only contains that image, and even though the image data is 
identical, the slimmed down version of the test case throws no error and 
processes the file correctly.

The command line I've been using is:

pcl6 -SDEVICE=jpeg -r600 -sOutputFile="out%d.jpg" -dNOPAUSE pclDump.pxl

I also tried using the bitrgb and the pswrite devices and they threw identical 
errors with the same test file.

I've attached the pxl file that is causing an error first (pclDump_broke.pxl) 
as well as the slimmed down test case that causes no errors (pclDump.pxl).  
Let me know if you need anything else!

Thanks :)
Jeff
Comment 1 Jeffrey Vance 2008-02-13 17:02:25 UTC
Created attachment 3786 [details]
PXL File which causes the ExtraData error.
Comment 2 Jeffrey Vance 2008-02-13 17:02:54 UTC
Created attachment 3787 [details]
Slimmed down testcase with same image
Comment 3 Henry Stiles 2008-03-03 22:29:40 UTC
Yes, the pxl jpeg parser is sensitive to where the data is position within the
stream.  The stopping criteria for the parser is filling the final position in
the  destination output.  I would guess your jpeg data has 2 extra bytes that
have no effect but I didn't carefully dissassemble the jpeg.  We should try and
emulate HP better but we do have a lot of jpg pxl tests that seem to work okay,
so I don't know if it is a good idea to invest time in this.
Comment 4 Hin-Tak Leung 2009-10-18 18:33:09 UTC
attachment 3787 [details] no-longer causes error.
Comment 5 Hin-Tak Leung 2009-10-27 19:55:32 UTC
Created attachment 5560 [details]
modified pxl with extra setColorSpace

This appears to be a similar bug to bug 688320 . if you modify attachment 3786 [details]
and insert this pxl instruction:

ubyte eRGB ColorSpace
SetColorSpace

Just before the jpeg embedded data (as I did with this new attachment), pspcl6
would happily read the file to completion. So it looks like the generator of
the reporter's PXL stream had a different color space (eGray?) just before
encountering the color jpeg, thus causing the error. (I can't see where the
effective color space is set - the file was pushing and pop'ing graphic
contexts a lot).

Does a real HP printer accept attachment 3786 [details]? If it does, ghostpdl probably
should override the colorspace if it encounters a color'ed jpeg; if a real HP
printer doesn't accept attachment 3786 [details] either, the reporter probably should
report this issue to the generator program (it says "MTI XPS->PXL", " Monotype
Imaging, Inc." in the PJL header) to have the generator program fixed.
Comment 6 Hin-Tak Leung 2009-10-27 20:03:23 UTC
(Marketing-speak) Since I just fixed Bug 688320, and ghostXPS performs similiar
function as "MTI XPS->PXL", Artifex might consider selling ghostXPS incoporating
fix 688320 to the reporter and see if it does a better job than Monotype's XPS->PXL?
Comment 7 Hin-Tak Leung 2009-11-15 19:44:18 UTC
Had another thought on this - the submitted file only set color space once at
the beginning to RGB and the error location is a few operators after a PushGS,
so RGB should be in effect since it is the default. 
Comment 8 Shailesh Mistry 2011-01-24 22:43:18 UTC
Created attachment 7150 [details]
Patch for Bug689705

This bug results from the JPEG code failing to receive enough data to find the End of Image marker (FF D9).

The JPEG library code correctly decompresses the image and then reaches stage 4 where it looks for the EOI but at this exact point there is no more data in the buffer. The function jpeg_finish_decompress returns FALSE because it fails to find the EOI but the function read_jpeg_bitmap_data ignores the returned code and believes that everything is now completed. The internal GS code has correctly calculated that there are still 2 more bytes of data left and raises an error.

The attached patch accounts for the returned call asking for more data and then fills the buffer accordingly. This patch also solves the additional problem of the buffer finishing just after the FF of the EOI so that the buffer must be refilled to find the last D9.
Comment 9 Hin-Tak Leung 2011-01-25 00:40:17 UTC
(In reply to comment #8)
> Created an attachment (id=7150) [details]
> Patch for Bug689705

Is there any reason why this patch was marked private?
Comment 10 Henry Stiles 2011-01-27 23:49:12 UTC
(In reply to comment #8)
> Created an attachment (id=7150) [details]
> Patch for Bug689705
> 
> This bug results from the JPEG code failing to receive enough data to find the
> End of Image marker (FF D9).
> 
> The JPEG library code correctly decompresses the image and then reaches stage 4
> where it looks for the EOI but at this exact point there is no more data in the
> buffer. The function jpeg_finish_decompress returns FALSE because it fails to
> find the EOI but the function read_jpeg_bitmap_data ignores the returned code
> and believes that everything is now completed. The internal GS code has
> correctly calculated that there are still 2 more bytes of data left and raises
> an error.
> 
> The attached patch accounts for the returned call asking for more data and then
> fills the buffer accordingly. This patch also solves the additional problem of
> the buffer finishing just after the FF of the EOI so that the buffer must be
> refilled to find the last D9.

Thanks for working on this Shailesh.  I spoke to Chris and he said you were studying the problem further, perhaps new problems raised by your patch.  Let me know if I can help.  We'll make sure you get the "P2 or P1" bounty $1000, when this one is wrapped up.

Thanks

Henry
Comment 11 Henry Stiles 2011-02-02 18:19:40 UTC
The static analyzer discovered something I should have noticed, now pos_in_row is no longer used.  it looks like the code now depends on the jpeg stream being properly terminated which may not be the case, that said it is an improvement over the old code.
Comment 12 Henry Stiles 2011-02-24 16:50:39 UTC
The essential issue is fixed here, mark it later for adding the improvement comment #11.
Comment 13 Henry Stiles 2011-03-17 21:10:30 UTC
This bug was supposed to be marked "Later", see comment #12