Bug 692368 - Segmentation fault when rasterizing a PDF file
Summary: Segmentation fault when rasterizing a PDF file
Status: RESOLVED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: PDF Interpreter (show other bugs)
Version: master
Hardware: PC Linux
: P4 normal
Assignee: Robin Watts
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-07-22 12:14 UTC by Till Kamppeter
Modified: 2014-02-17 04:42 UTC (History)
2 users (show)

See Also:
Customer:
Word Size: ---


Attachments
stars.pdf mangled by evince (124.77 KB, application/pdf)
2011-07-22 12:14 UTC, Till Kamppeter
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Till Kamppeter 2011-07-22 12:14:11 UTC
Created attachment 7700 [details]
stars.pdf mangled by evince

Rasterizing the attached file with this command line lets Ghostscript segfault:

gs -dNOPAUSE -dBATCH -sDEVICE=cups -sOutputFile=%stdout -r720x720 -dcupsBitsPerColor=8 -dcupsColorSpace=1 -dNOINTERPOLATE ~/ghostscript/testfiles/stars-evince.pdf > out.raster

Current GIT head, 64-bit on Ubuntu Oneiric.

Also using the PNG output device leads to a segfault:

gs -dNOPAUSE -dBATCH -sDEVICE=png16m -sOutputFile=%stdout -r720x720 -dNOINTERPOLATE ~/ghostscript/testfiles/stars-evince.pdf > out.png

So it is not a problem of the CUPS Raster output device.
Comment 1 Alex Cherepanov 2011-07-22 17:49:02 UTC
The problem is confirmed on a 64-bit GNU/Linux box.
The SEGV is caused by the incorrect value of ptile->ttrans->pat_trans_fill.
Old versions of gs don't crash but allocate gigabytes of memory.

#0  in st_free ()
#1  in tile_by_steps_trans at ./base/gxp1fill.c:560
#2  in gx_trans_pattern_fill_rect  at ./base/gxp1fill.c:856
#3  in gx_dc_pat_trans_fill_rectangle  at ./base/gxp1fill.c:802
#4  in gx_general_fill_path at ./base/gxfill.c:446
#5  in gx_default_fill_path at ./base/gxfill.c:665
#6  in clist_playback_band  at ./base/gxclrast.c:1804
#7  in clist_playback_file_bands  at ./base/gxclread.c:852
#8  in tile_pattern_clist at ./base/gxp1fill.c:285
#9  in tile_by_steps at ./base/gxp1fill.c:206
Comment 2 Ray Johnston 2011-07-22 21:39:48 UTC
Since Robin was the last person to touch this, assigning to him. It's either
him or Michael, so I am cc'ing him.
Comment 3 Robin Watts 2011-07-27 23:14:53 UTC
Fixed in:

commit b3dbb094608e2d5699c6d77d42f4919c0299a02a
Author: Robin Watts <Robin.Watts@artifex.com>
Date:   Wed Jul 27 16:25:19 2011 +0100

    Fix Bug 692368; SEGV with clist transparency pattern in clist rendering.

    The invocation/file given in the bug uses banded rendering (clist).
    While drawing a band, it uses a pattern with transparency, that is
    itself rendered by a clist (using the new pattern clist code).

    A fill rectangle is being done that tries to call the pattern clist
    tiling code without the fill_trans_buffer having been setup, and
    this causes a SEGV.

    The reason for this appears to be that the device used by the
    banded isn't a pdf14 device - instead it's a clipper device wrapping
    the pdf14 device. The clipper device does not call gx_forward_fill_path,
    but instead calls gx_default_fill_path, meaning that the special
    pdf14 handling done to setup the fill_trans_buffer isn't done.

    Making the commit call onwards to gx_forward_fill_path doesn't solve the
    problem either, as that results in no actual clipping being done.

    The fix, therefore appears to be to implement a clipping version of
    fill_path.

    No cluster differences expected.