Bug 693422

Summary:	Pattern in SMask causes SEGV with '-r300' ('-r200' works)
Product:	Ghostscript	Reporter:	jsmeix
Component:	PDF Interpreter	Assignee:	Ray Johnston <ray.johnston>
Status:	RESOLVED FIXED
Severity:	normal
Priority:	P1
Version:	master
Hardware:	PC
OS:	Linux
Customer:		Word Size:	---
Attachments:	gts.pdf to reproduce the segfault Simplified sample file. Proposed patch.

Description jsmeix 2012-10-31 11:02:03 UTC

Created attachment 9044 [details]
gts.pdf to reproduce the segfault

We (i.e. openSUSE) got this bug report:
https://bugzilla.novell.com/show_bug.cgi?id=782737

In short:

$ gs -dBATCH -dNOPAUSE -sDEVICE=ppm -r300 -sOutputFile=gts.ppm gts.pdf

fails with Segmentation fault for the attached PDF but works with '-r200'.


To verify that this does not somehow depend on something in
the openSUSE environment (e.g. whatever system library)
I downloaded gs-906-linux_x86 from

http://downloads.ghostscript.com/public/binaries/ghostscript-9.06-linux-x86.tgz

and this also segfaults:

# ./gs-906-linux_x86 -dBATCH -dNOPAUSE -sDEVICE=ppm -r300 \
 -sOutputFile=gts.ppm gts.pdf
GPL Ghostscript 9.06 (2012-08-08)
Copyright (C) 2012 Artifex Software, Inc.  All rights reserved.
This software comes with NO WARRANTY: see the file PUBLIC for details.
Processing pages 1 through 2.
Page 1
Page 2
Segmentation fault

The segfault also occurs with other types of DEVICEs like pgm
when using -r300.


For now I selected "PDF Interpreter" as component for this issue
but I don't know if this is really the right one.

Comment 1 Alex Cherepanov 2013-01-13 21:04:33 UTC

Created attachment 9216 [details]
Simplified sample file.

Comment 2 Alex Cherepanov 2013-01-13 21:17:39 UTC

The problem is caused by the pattern loaded from clist. The pattern tile in
clist can be installed into the pattern cache but has no information to
re-load the pattern again. If the cache search fails, the code attempts to
use uninitialized ccolor and fails with SEGV.

If the pattern comes from the SMask, the following check in 
gx_pattern_cache_lookup() is not satisfied
   ctile->depth == dev->color_info.depth
which triggers an attempt to reload the pattern and the SEGV.

Comment 3 Alex Cherepanov 2013-01-13 21:37:50 UTC

Created attachment 9217 [details]
Proposed patch.

The patch is rather simple: just take out the check
   ctile->depth == dev->color_info.depth

However, rendering code is not my speciality, and this check might serve some
useful purpose. I'd appreciate review of this patch by somebody who is familiar
wit this part of gs.

Testing shows that the patch causes no regressions.

Comment 4 Ray Johnston 2013-01-18 21:54:48 UTC

A little more explanation added to Alex's...

The problem is that the transparency operations may be 'cropped' i.e.,
only written to some bands where needed, but the transparency ops may
change the color space. This file has a pattern that is used to paint
the SMask and the pattern is large, so it gets written to all bands,
but the colorspace is Gray for the pattern. When a band that didn't
have the transparency 'begin_mask' op, the colorspace for the device
was RGB, so the pattern logic didn't accept the pattern even though
it was in the cache because of the mismatch of 'depth'.

This is also the cause of several SEGV's during regression, so I will commit
this patch. Thanks to Alex for also finding it.

Note, that I am going to look at the painting code to see if I can find
a good place to put a test for the mismatch when it actually counts.

Comment 5 Ray Johnston 2013-01-20 20:56:58 UTC

Fix committed in:

2ae1a17 Fix Bug 693422. SEGV due to transparency and patterns

Michael Vrhel has suggested that we keep a 'debug build' option to print
something when this happens, since other cases may crop up due to the
difficulty of keeping the pdf14 colorspace synced up during clist playback,
but the above patch doesn't do this.