Bug 690519 - Potential SEGV when dev_color substructures are moved.
Summary: Potential SEGV when dev_color substructures are moved.
Status: RESOLVED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: General (show other bugs)
Version: master
Hardware: All All
: P3 normal
Assignee: Ray Johnston
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-06-07 20:00 UTC by Marcos H. Woehrmann
Modified: 2011-12-09 14:53 UTC (History)
2 users (show)

See Also:
Customer:
Word Size: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcos H. Woehrmann 2009-06-07 20:00:24 UTC
Starting with r9772 the file PP0001G0.pdf generates a coredump with pkmraw output at 300 dpi.

The following command line demonstrates the problem:

  bin/gs -sDEVICE=pkmraw -r300 -o test.pkm ./PP0001G0.pdf

This problem occurs on my AMD64 box running Linux and my iMac running Mac OS X with both 
production and debug builds, so it appears to be independent of compiler and word size.
Comment 1 Marcos H. Woehrmann 2009-06-07 20:01:27 UTC
Created attachment 5076 [details]
PP0001G0.pdf
Comment 2 Marcos H. Woehrmann 2009-06-07 20:08:23 UTC
(gdb) run -sOutputFile=test.pkm -sDEVICE=pkmraw -r300 ./PP0001G0.pdf 
Starting program: /home/marcos/head/gs/debugobj/gs -sOutputFile=test.pkm -sDEVICE=pkmraw -
r300 ./PP0001G0.pdf
[Thread debugging using libthread_db enabled]
GPL Ghostscript SVN PRE-RELEASE 8.65 (2009-02-04)
Copyright (C) 2009 Artifex Software, Inc.  All rights reserved.
This software comes with NO WARRANTY: see the file PUBLIC for details.
Processing pages 1 through 1.
Page 1
Substituting font Times-Roman for TimesNewRomanPSMT.
Loading NimbusRomNo9L-Regu font from %rom%Resource/Font/NimbusRomNo9L-Regu... 3452456 
1827832 15293864 13985965 3 done.
[New Thread 0x7fec92caf750 (LWP 8759)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fec92caf750 (LWP 8759)]
0x000000000085931e in set_ht_colors_le_4 (pvp=0x7fff9acd2650, colors=0x7fff9acd1e50, 
sbits=0x7fff9acd2450, pdc=0x2c31800, dev=0x7fff9acd28c8, caches=0x7fff9acd2250, nplanes=4)
    at ./base/gxcht.c:860
860	        SET_PLANE_COLOR(1);
(gdb) where
#0  0x000000000085931e in set_ht_colors_le_4 (pvp=0x7fff9acd2650, colors=0x7fff9acd1e50, 
sbits=0x7fff9acd2450, pdc=0x2c31800, dev=0x7fff9acd28c8, caches=0x7fff9acd2250, nplanes=4)
    at ./base/gxcht.c:860
#1  0x00000000008584c0 in gx_dc_ht_colored_fill_rectangle (pdevc=0x2c31800, x=1102, y=1810, 
w=4, h=1, dev=0x7fff9acd28c8, lop=252, source=0x0) at ./base/gxcht.c:654
#2  0x00000000004ad97f in gx_dc_colored_masked_fill_rect (pdevc=0x2c31800, x=1102, y=1810, 
w=4, h=1, dev=0x29b4e48, lop=252, source=0x0) at ./base/gxp1fill.c:425
#3  0x00000000008ab89c in gx_fill_trapezoid_ns_nd (dev=0x29b4e48, left=0x7fff9acd7bf0, 
right=0x7fff9acd7be0, ybot=463340, ytop=463516, flags=0, pdevc=0x2c31800, fa=252)
    at ./base/gxdtfill.h:378
#4  0x00000000008aeba4 in gx_default_fill_trapezoid (dev=0x29b4e48, left=0x7fff9acd7bf0, 
right=0x7fff9acd7be0, ybot=463340, ytop=463516, swap_axes=0, pdevc=0x2c31800, lop=252)
    at ./base/gdevddrw.c:439
#5  0x0000000000877044 in loop_fill_trap_np (ll=0x7fff9acd7e20, le=0x7fff9acd7bf0, 
re=0x7fff9acd7be0, y=463340, y1=463516) at ./base/gxfill.c:1637
#6  0x0000000000877cf6 in slant_into_trapezoids__nd (ll=0x7fff9acd7e20, flp=0x7fff9acd80b0, 
alp=0x7fff9acd7fc8, y=463467, y1=463643) at ./base/gxfillts.h:81
#7  0x00000000008777fc in spot_into_trapezoids__aj_nd (ll=0x7fff9acd7e20, band_mask=-
2147483648) at ./base/gxfilltr.h:220
#8  0x0000000000879571 in spot_into_trapezoids (ll=0x7fff9acd7e20, band_mask=-2147483648) at 
./base/gxfill.c:2099
#9  0x000000000086aaf6 in gx_general_fill_path (pdev=0x29b4e48, pis=0x295ef38, 
ppath=0x297b970, params=0x7fff9acdab90, pdevc=0x2c31800, pcpath=0x2c31690) at 
./base/gxfill.c:521
#10 0x000000000086b7fb in gx_default_fill_path (pdev=0x29b4e48, pis=0x295ef38, 
ppath=0x297b970, params=0x7fff9acdab90, pdevc=0x2c31800, pcpath=0x2c31690) at 
./base/gxfill.c:687
#11 0x0000000000893768 in gx_fill_path (ppath=0x297b970, pdevc=0x2c31800, pgs=0x295ef38, 
rule=1, adjust_x=128, adjust_y=128) at ./base/gxpaint.c:49
#12 0x00000000008430b5 in fill_with_rule (pgs=0x295ef38, rule=1) at ./base/gspaint.c:310
#13 0x0000000000843160 in gs_eofill (pgs=0x295ef38) at ./base/gspaint.c:334
#14 0x000000000050d89e in zeofill (i_ctx_p=0x297aff0) at ./psi/zpaint.c:32
#15 0x00000000004bd264 in call_operator (op_proc=0x50d886 <zeofill>, i_ctx_p=0x297aff0) at 
./psi/interp.c:111
#16 0x00000000004c0264 in interp (pi_ctx_p=0x293c318, pref=0x7fff9acdb520, 
perror_object=0x7fff9acdb730) at ./psi/interp.c:1277
#17 0x00000000004bd917 in gs_call_interp (pi_ctx_p=0x293c318, pref=0x7fff9acdb660, 
user_errors=1, pexit_code=0x7fff9acdb74c, perror_object=0x7fff9acdb730) at ./psi/interp.c:496
#18 0x00000000004bd751 in gs_interpret (pi_ctx_p=0x293c318, pref=0x7fff9acdb660, 
user_errors=1, pexit_code=0x7fff9acdb74c, perror_object=0x7fff9acdb730) at ./psi/interp.c:454
#19 0x00000000004b1208 in gs_main_interpret (minst=0x293c280, pref=0x7fff9acdb660, 
user_errors=1, pexit_code=0x7fff9acdb74c, perror_object=0x7fff9acdb730) at ./psi/imain.c:214
#20 0x00000000004b1dc5 in gs_main_run_string_end (minst=0x293c280, user_errors=1, 
pexit_code=0x7fff9acdb74c, perror_object=0x7fff9acdb730) at ./psi/imain.c:526
#21 0x00000000004b1c82 in gs_main_run_string_with_length (minst=0x293c280, str=0x29b6d30 
"<2e2f50503030303147302e706466>.runfile", length=38, user_errors=1, 
pexit_code=0x7fff9acdb74c, 
    perror_object=0x7fff9acdb730) at ./psi/imain.c:484
#22 0x00000000004b1bef in gs_main_run_string (minst=0x293c280, str=0x29b6d30 
"<2e2f50503030303147302e706466>.runfile", user_errors=1, pexit_code=0x7fff9acdb74c, 
    perror_object=0x7fff9acdb730) at ./psi/imain.c:466
#23 0x00000000004b4bf0 in run_string (minst=0x293c280, str=0x29b6d30 
"<2e2f50503030303147302e706466>.runfile", options=3) at ./psi/imainarg.c:798
#24 0x00000000004b4b96 in runarg (minst=0x293c280, pre=0x8ebddb "", arg=0x29804d0 
"./PP0001G0.pdf", post=0x8ebe75 ".runfile", options=3) at ./psi/imainarg.c:788
#25 0x00000000004b483d in argproc (minst=0x293c280, arg=0x7fff9acdd8e6 "./PP0001G0.pdf") at 
./psi/imainarg.c:723
#26 0x00000000004b30c6 in gs_main_init_with_args (minst=0x293c280, argc=5, 
argv=0x7fff9acdc338) at ./psi/imainarg.c:207
#27 0x000000000040993d in main (argc=5, argv=0x7fff9acdc338) at ./psi/gs.c:77
(gdb) 
Comment 3 Marcos H. Woehrmann 2009-06-07 20:21:34 UTC
Here's the relevant valgrind output (earlier errors are also reported with r9771):

.
.
.
Substituting font Times-Roman for TimesNewRomanPSMT.
Loading NimbusRomNo9L-Regu font from %rom%Resource/Font/NimbusRomNo9L-Regu... 3513008 
1848276 15293864 13986051 3 done.
==27009== 
==27009== Use of uninitialised value of size 8
==27009==    at 0x867F8F: set_ht_colors_le_4 (gxcht.c:860)
==27009==    by 0x8671A2: gx_dc_ht_colored_fill_rectangle (gxcht.c:654)
==27009==    by 0x4AF75E: gx_dc_colored_masked_fill_rect (gxp1fill.c:425)
==27009==    by 0x8BA834: gx_fill_trapezoid_ns_nd (gxdtfill.h:378)
==27009==    by 0x8BDAAD: gx_default_fill_trapezoid (gdevddrw.c:439)
==27009==    by 0x885CB3: loop_fill_trap_np (gxfill.c:1637)
==27009==    by 0x8869CB: slant_into_trapezoids__nd (gxfillts.h:81)
==27009==    by 0x8864C8: spot_into_trapezoids__aj_nd (gxfilltr.h:220)
==27009==    by 0x88830E: spot_into_trapezoids (gxfill.c:2099)
==27009==    by 0x879617: gx_general_fill_path (gxfill.c:521)
==27009==    by 0x87A31C: gx_default_fill_path (gxfill.c:687)
==27009==    by 0x8A223F: gx_fill_path (gxpaint.c:49)
==27009== 
==27009== Invalid read of size 8
==27009==    at 0x867F8F: set_ht_colors_le_4 (gxcht.c:860)
==27009==    by 0x8671A2: gx_dc_ht_colored_fill_rectangle (gxcht.c:654)
==27009==    by 0x4AF75E: gx_dc_colored_masked_fill_rect (gxp1fill.c:425)
==27009==    by 0x8BA834: gx_fill_trapezoid_ns_nd (gxdtfill.h:378)
==27009==    by 0x8BDAAD: gx_default_fill_trapezoid (gdevddrw.c:439)
==27009==    by 0x885CB3: loop_fill_trap_np (gxfill.c:1637)
==27009==    by 0x8869CB: slant_into_trapezoids__nd (gxfillts.h:81)
==27009==    by 0x8864C8: spot_into_trapezoids__aj_nd (gxfilltr.h:220)
==27009==    by 0x88830E: spot_into_trapezoids (gxfill.c:2099)
==27009==    by 0x879617: gx_general_fill_path (gxfill.c:521)
==27009==    by 0x87A31C: gx_default_fill_path (gxfill.c:687)
==27009==    by 0x8A223F: gx_fill_path (gxpaint.c:49)
==27009==  Address 0xED8 is not stack'd, malloc'd or (recently) free'd
==27009== 
==27009== Process terminating with default action of signal 11 (SIGSEGV)
==27009==  Access not within mapped region at address 0xED8
==27009==    at 0x867F8F: set_ht_colors_le_4 (gxcht.c:860)
==27009==    by 0x8671A2: gx_dc_ht_colored_fill_rectangle (gxcht.c:654)
==27009==    by 0x4AF75E: gx_dc_colored_masked_fill_rect (gxp1fill.c:425)
==27009==    by 0x8BA834: gx_fill_trapezoid_ns_nd (gxdtfill.h:378)
==27009==    by 0x8BDAAD: gx_default_fill_trapezoid (gdevddrw.c:439)
==27009==    by 0x885CB3: loop_fill_trap_np (gxfill.c:1637)
==27009==    by 0x8869CB: slant_into_trapezoids__nd (gxfillts.h:81)
==27009==    by 0x8864C8: spot_into_trapezoids__aj_nd (gxfilltr.h:220)
==27009==    by 0x88830E: spot_into_trapezoids (gxfill.c:2099)
==27009==    by 0x879617: gx_general_fill_path (gxfill.c:521)
==27009==    by 0x87A31C: gx_default_fill_path (gxfill.c:687)
==27009==    by 0x8A223F: gx_fill_path (gxpaint.c:49)
==27009== 
==27009== ERROR SUMMARY: 1127 errors from 6 contexts (suppressed: 8 from 1)
==27009== malloc/free: in use at exit: 22,074,864 bytes in 580 blocks.
==27009== malloc/free: 1,749 allocs, 1,169 frees, 50,430,808 bytes allocated.
==27009== For counts of detected errors, rerun with: -v
==27009== searching for pointers to 580 not-freed blocks.
==27009== checked 27,936,760 bytes.
==27009== 
==27009== LEAK SUMMARY:
==27009==    definitely lost: 6 bytes in 2 blocks.
==27009==      possibly lost: 0 bytes in 0 blocks.
==27009==    still reachable: 22,074,858 bytes in 578 blocks.
==27009==         suppressed: 0 bytes in 0 blocks.
==27009== Rerun with --leak-check=full to see details of leaked memory.
Segmentation fault (core dumped)
Comment 4 Ray Johnston 2009-06-17 09:08:41 UTC
I'm able to reproduce this on peeves under the debugger. Working on it now.
Comment 5 Alex Cherepanov 2009-06-17 09:22:34 UTC
Bug 690505 and bug 690506 are other regressions that appeared in r9772.
Comment 6 Ray Johnston 2009-07-07 23:25:35 UTC
While rev 9772 caused the GC execution to change and resulted in this failure,
this was NOT the root cause. The problem was that the pgs->dev_color was stale
if a garbage collection ran during a pattern accumulation execution of the
(PostScript) PaintProc.

The fix that has been committed (rev 9846) performs gx_unset_dev_color so that
the pgs->dev_color pointer will be set correctly before use by filling/stroking.

It might be worthwhile to properly trace the dev_color pointer in the gs_state
structure, but this is adequate for now and limits the change to the PS interp
which is the only place that a "real" GC can move structures.

Once I get confirmation from nightly regressions, I will close this bug.
Comment 7 Ray Johnston 2009-07-12 19:15:55 UTC
Rev 9846 fixed (or at least works around) this rare case where a GC runs during
the collection of a pattern bitmap.

The underlying cause may not yet be resolved since further analysis shows that
the gstate dev_color structure is incompletely traced. The c_ht pointer in the
'colors' union was not being relocated, and probably the pointers in the binary
part of the union, e.g. b_ht, are not traced and relocated either.

Reducing the priority to P3 as a clean-up issue and assigning to Ralph. I am
also changing the description since it is now just a concept that MAY lead to
a seg fault.

I am available to help track this if desired.
Comment 8 Henry Stiles 2011-11-10 17:34:30 UTC
Back to reporter for re-testing.
Comment 9 Marcos H. Woehrmann 2011-12-09 14:53:07 UTC
I've confirmed that this issue does not occur with master and that no more valgrind issues are reported.