Bug 692219 - SEGV with transparent patterns in patterns
Summary: SEGV with transparent patterns in patterns
Status: RESOLVED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: Graphics Library (show other bugs)
Version: master
Hardware: PC All
: P4 normal
Assignee: Michael Vrhel
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-20 17:44 UTC by Robin Watts
Modified: 2011-06-01 16:10 UTC (History)
0 users

See Also:
Customer:
Word Size: ---


Attachments
xps6.pdf (1.98 KB, application/unknown)
2011-05-20 17:44 UTC, Robin Watts
Details
0001.patch (2.99 KB, application/octet-stream)
2011-05-20 20:05 UTC, Robin Watts
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Robin Watts 2011-05-20 17:44:09 UTC
Created attachment 7521 [details]
xps6.pdf

The following command gives a SEGV on both windows and linux:

  gs/debugbin/gswin32c.exe -o out.ppm -sDEVICE=ppmraw -r72 xps6.pdf

I've reduced the file (originally given from pdfwriting an xps file) as much as possible. It now has a page with a rectangle painted in pattern #1. Pattern #1 has no painting operations, but does set it's colorspace to Pattern #2. Pattern #2 has no painting operations at all, not even to set it's colorspace.

Pattern #2 DOES have a transparent colorspace (constant 50% alpha) declared in the ExtGState dictionary though, which leads me to suspect that we are deciding that Pattern #2 must be transparent.
Comment 1 Robin Watts 2011-05-20 20:01:24 UTC
I have a patch that allows this to run to completion with plausible rendering.

I think that there are 2 problems here.

Firstly when run with -Z@? to fill allocated memory with a known pattern, it dies with an out of memory error due to a calculation being based on uninitialised memory. It seems that a pattern tile caches transbuff structure is being allocated, but the number of channels is not being set.

For some reason (possibly because no colorspace is set/drawing operations are done) a memory allocation is done based upon 'n_chan' which is set to a1a1a1a1 - and hence the program fails to allocate and dies.

If we get past this, then later on, the pdf14 device is popped before anyone writes to the tile and a blend operation is done which SEGVs due to one of the data planes being an uninitialised one.

The patch attached sets n_chan to zero at allocation time. This solves the allocation problem. We add a check for n_chan being zero into the blending routine too.

This allows the file to complete, but I'm really not sure it's the correct solution - this could well be patching the symptom, not being a proper cure.

Left here for comments from MJV.
Comment 2 Robin Watts 2011-05-20 20:05:48 UTC
Created attachment 7523 [details]
0001.patch

This is a git format patch. To apply it, do:

 git am 0001.patch

That will make a new commit consisting of just that patch on the end of your current branch. This means you can test/poke/prod it and then get rid of it easily.

I assume tortoise git has the same kind of thing.
Comment 3 Robin Watts 2011-05-20 23:40:58 UTC
Assigning to Michael for his opinion.
Comment 4 Michael Vrhel 2011-06-01 16:10:07 UTC
Patch makes sense and has been committed.  Closing bug.