Bug 697762

I found another bug, but I think, the root cause is the same.

ASAN:

==15589==ERROR: AddressSanitizer: SEGV on unknown address 0x62a24e9cdbb8 (pc 0x00000192a0b0 bp 0x000000000100 sp 0x7ffdc55f5100 T0)
==15589==The signal is caused by a WRITE memory access.
    #0 0x192a0af in cursor_output_inrange_tr XYZ/ghostpdl/./base/gxscanc.c:2639:9
    #1 0x192a0af in cursor_always_inrange_step_right_tr XYZ/ghostpdl/./base/gxscanc.c:2766
    #2 0x192a0af in mark_line_tr_app XYZ/ghostpdl/./base/gxscanc.c:3131
    #3 0x19240dc in mark_curve_tr_app XYZ/ghostpdl/./base/gxscanc.c:3426:13
    #4 0x19240dc in mark_curve_tr_app XYZ/ghostpdl/./base/gxscanc.c:3426:13
    #5 0x19240dc in mark_curve_tr_app XYZ/ghostpdl/./base/gxscanc.c:3426:13
    #6 0x19240dc in mark_curve_tr_app XYZ/ghostpdl/./base/gxscanc.c:3426:13
    #7 0x1920fa6 in gx_scan_convert_tr_app XYZ/ghostpdl/./base/gxscanc.c:3514:21
    #8 0x17a468b in gx_general_fill_path XYZ/ghostpdl/./base/gxfill.c:499:24
    #9 0x17a468b in gx_default_fill_path XYZ/ghostpdl/./base/gxfill.c:702
    #10 0x1837e21 in gx_fill_path XYZ/ghostpdl/./base/gxpaint.c:52:12
    #11 0x1490499 in do_fill XYZ/ghostpdl/./base/gspaint.c:317:12
    #12 0x1490499 in fill_with_rule XYZ/ghostpdl/./base/gspaint.c:351
    #13 0x1a270fe in interp XYZ/ghostpdl/./psi/interp.c:1320:40
    #14 0x1a270fe in gs_call_interp XYZ/ghostpdl/./psi/interp.c:517
    #15 0x1a270fe in gs_interpret XYZ/ghostpdl/./psi/interp.c:474
    #16 0x19f9de2 in gs_main_interpret XYZ/ghostpdl/./psi/imain.c:247:12
    #17 0x19f9de2 in gs_main_run_string_end XYZ/ghostpdl/./psi/imain.c:665
    #18 0x19f9de2 in gs_main_run_string_with_length XYZ/ghostpdl/./psi/imain.c:623
    #19 0x1a05efe in run_string XYZ/ghostpdl/./psi/imainarg.c:978:16
    #20 0x1a05efe in runarg XYZ/ghostpdl/./psi/imainarg.c:968
    #21 0x1a05358 in argproc XYZ/ghostpdl/./psi/imainarg.c:901:16
    #22 0x19fddc3 in gs_main_init_with_args XYZ/ghostpdl/./psi/imainarg.c:238:24
    #23 0x547608 in main XYZ/ghostpdl/./psi/gs.c:96:16
    #24 0x7fd1c6cd582f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #25 0x47ba38 in _start (/usr/local/bin/gs+0x47ba38)

Created attachment 13586 [details]
Second POC to trigger OOBW

Passing to Robin to deal with scan converter problems, making P1 for a seg fault.

Exhibits on a Windows 32-bit debug build as well. Looks like its caused by the monstrous scale factor (2000,80 x 20,80 x 20,80).

The second file has 300000 8 scale, so I'm reasonably sure its the same problem.

I can confirm the bug and reproduce it on windows. Thanks for the report.

I should point out that it's

 -sOutputFile=

not

 -sOUTPUTFILE=

gs is case sensitive in the arguments it is given.

(In reply to Robin Watts from comment #4)
> I can confirm the bug and reproduce it on windows. Thanks for the report.
> 
> I should point out that it's
> 
>  -sOutputFile=
> 
> not
> 
>  -sOUTPUTFILE=
> 
> gs is case sensitive in the arguments it is given.

Apparently we have an exception hardcoded to cope with -sOUTPUTFILE. Ick.

Fixed in:

commit 8210a2864372723b49c526e2b102fdc00c9c4699
Author: Robin Watts <robin.watts@artifex.com>
Date:   Wed Apr 19 12:06:39 2017 +0100

    Bug 697762: Fix scan converter SEGVs.

    The SEGVs are caused by the scan converter allowing less room
    in the table than is actually required. This happens due to
    the mark_curve subdivision overflowing.

    For example, if we have a curve such as:

     0 0x80000000 moveto
     0 0x80000000 0 0x80000000 0 0x80000000 curveto

    Then the current subdivision logic does:

     fixed ay = (sy + c1y)>>1

    For sy = c1y = 0x80000000 this gives ay = 0, when clearly
    the average should be 0x80000000.

    An alternative formulation of the splitting code:

     fixed ay = (sy>>1) + (c1y>>1)

    gives the required result, at the expense of dropping accuracy at
    the low end.

    This commit therefore has 2 formulations of the splitting code, one
    for 'big' values (values that might overflow), and ones for normal
    values (values that won't).

    We select between the two pieces of code by comparing the top 2
    bits of each value; if they differ, then we use the big variant.

    Also, when calculating the size of the table required, we were
    incorrectly closing the curve more often than we needed to. This
    had no ill effect other than inflating the amount of memory we
    needed.

Many thanks for the report.