Bug 689493 - Core dump with DEVICE=tiffsep with transparency
Summary: Core dump with DEVICE=tiffsep with transparency
Status: NOTIFIED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: Graphics Library (show other bugs)
Version: master
Hardware: PC Linux
: P2 critical
Assignee: Ray Johnston
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-10-03 10:30 UTC by Marcos H. Woehrmann
Modified: 2008-12-19 08:31 UTC (History)
0 users

See Also:
Customer: 190
Word Size: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcos H. Woehrmann 2007-10-03 10:30:45 UTC
The customer reports and I've verified that gs8.60 and gshead (r8268) both core dump with the attached 
file when using -sDEVICE=tiffsep.

The command line I used:

  bin/gs -sDEVICE=tiffsep -sOutputFile=test ./axa.pdf
Comment 1 Marcos H. Woehrmann 2007-10-03 10:31:05 UTC
Created attachment 3433 [details]
axa.pdf
Comment 2 Marcos H. Woehrmann 2007-10-03 10:32:02 UTC
The customer suggested a fix:

Update: in fact  the problem has nothing to do with spot colors: tiffsep
(and possibly other devices supporting spots) crashes immediately, with
all files having transparency.

The cause is this test  (in gdev_prn_allocate):

if (new_height < max_ulong/(ESTIMATED_PDF14_ROW_SPACE(new_width) >> 3))

which, at least on Linux, causes a floating point exception when
new_width=0, i.e. as soon as gdev_prn_open() is called.
This bugs does not affect other devices, because the test is only
executed when the "flag page_uses_transparency" is set, and for some
reasons it is set with tiffsep, but not with non-spot devices (at the
time gdev_prn_open() is called).

Now, I have tried 2 fixes:

 if (new_width>0 && new_height <
max_ulong/(ESTIMATED_PDF14_ROW_SPACE(new_width) >> 3))

Or

 if (new_width==0 || new_height <
max_ulong/(ESTIMATED_PDF14_ROW_SPACE(new_width) >> 3))

Both seems to work, but please tell me which one is better.


Comment 3 Ray Johnston 2007-10-12 13:31:59 UTC
fixed as of rev. 8289

The problem was that when the pdf14 device, several of the color procs were
substituted including get_color_comp_index. When the Separation name is unknown
we need to call the target (tiffsep) get_color_comp_index.