Bug 697453

Summary:	Divide by zero in intersect()
Product:	Ghostscript	Reporter:	Kamil Frankowicz <kamil.frankowicz>
Component:	Graphics Library	Assignee:	Robin Watts <robin.watts>
Status:	RESOLVED FIXED
Severity:	normal	CC:	omarandemad, robin.watts
Priority:	P1
Version:	master
Hardware:	PC
OS:	Linux
Customer:		Word Size:	---
Attachments:	POC to trigger FPE (gs)

Description Kamil Frankowicz 2016-12-23 02:00:43 UTC

Created attachment 13262 [details]
POC to trigger FPE (gs)

After some fuzz testing I found a crashing test case.

Git Head: 73060a27e554f8e64ae2aba4a1b03822207346c7

Command: s -dNOPAUSE -sDEVICE=bit -sOUTPUTFILE=/dev/null -dSAFER gs_fpe_intersect -c quit

ASAN + Output:

GPL Ghostscript GIT PRERELEASE 9.21 (2016-09-14)
Copyright (C) 2016 Artifex Software, Inc.  All rights reserved.
This software comes with NO WARRANTY: see the file PUBLIC for details.
Querying operating system for font files...
Can't find (or can't open) font file %rom%Resource/Font/.
Didn't find this font on the system!
Substituting font Courier for .
Loading NimbusMonoPS-Regular font from %rom%Resource/Font/NimbusMonoPS-Regular... 4265400 2800359 3055176 1770757 1 done.
ASAN:DEADLYSIGNAL
=================================================================
==6361==ERROR: AddressSanitizer: FPE on unknown address 0x0000017b02e4 (pc 0x0000017b02e4 bp 0x000004233630 sp 0x7ffdbf4dbc20 T0)
    #0 0x17b02e3 in intersect XYZ/ghostpdl/./base/gxfill.c:1754:22
    #1 0x17b02e3 in intersect_al XYZ/ghostpdl/./base/gxfill.c:1856
    #2 0x1795c41 in spot_into_trapezoids__aj_fd XYZ/ghostpdl/./base/gxfilltr.h:146:13
    #3 0x1795c41 in spot_into_trapezoids XYZ/ghostpdl/./base/gxfill.c:2084
    #4 0x1791139 in gx_general_fill_path XYZ/ghostpdl/./base/gxfill.c:557:16
    #5 0x1791139 in gx_default_fill_path XYZ/ghostpdl/./base/gxfill.c:700
    #6 0x1858d22 in gx_stroke_path_only_aux XYZ/ghostpdl/./base/gxstroke.c:1045:13
    #7 0x1858d22 in gx_stroke_path_only XYZ/ghostpdl/./base/gxstroke.c:1081
    #8 0x1853356 in gx_default_stroke_path XYZ/ghostpdl/./base/gxstroke.c:334:12
    #9 0x1823aea in gx_stroke_fill XYZ/ghostpdl/./base/gxpaint.c:69:12
    #10 0x148000c in do_stroke XYZ/ghostpdl/./base/gspaint.c:444:16
    #11 0x148000c in gs_stroke XYZ/ghostpdl/./base/gspaint.c:480
    #12 0x19fe37e in interp XYZ/ghostpdl/./psi/interp.c:1314:40
    #13 0x19fe37e in gs_call_interp XYZ/ghostpdl/./psi/interp.c:511
    #14 0x19fe37e in gs_interpret XYZ/ghostpdl/./psi/interp.c:468
    #15 0x19d1352 in gs_main_interpret XYZ/ghostpdl/./psi/imain.c:245:12
    #16 0x19d1352 in gs_main_run_string_end XYZ/ghostpdl/./psi/imain.c:663
    #17 0x19d1352 in gs_main_run_string_with_length XYZ/ghostpdl/./psi/imain.c:621
    #18 0x19dd2eb in run_string XYZ/ghostpdl/./psi/imainarg.c:977:16
    #19 0x19dd2eb in runarg XYZ/ghostpdl/./psi/imainarg.c:967
    #20 0x19dc748 in argproc XYZ/ghostpdl/./psi/imainarg.c:900:16
    #21 0x19d5283 in gs_main_init_with_args XYZ/ghostpdl/./psi/imainarg.c:238:24
    #22 0x5475d8 in main XYZ/ghostpdl/./psi/gs.c:96:16
    #23 0x7f4bf756982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #24 0x47b9d8 in _start (/usr/local/bin/gs+0x47b9d8)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: FPE XYZ/ghostpdl/./base/gxfill.c:1754:22 in intersect
==6361==ABORTING

Comment 1 Ken Sharp 2016-12-23 04:13:01 UTC

Its a scan-conversion problem, probably related to an overflow of the fixed point arithmetic (maybe). Unfortunately I can only reproduce this in a 64-bit Linux release build. Debug build on Linux and all Windows builds seem to work.

Assigning to Robin as a scan-conversion bug for more investigation.

Comment 2 Robin Watts 2016-12-29 08:05:58 UTC

I can reproduce this bug on 64bit linux in a debug build if I build using:

make debug XCFLAGS="-O2"

gdb --args debugbin/gs -dNOPAUSE -sDEVICE=bit -sOUTPUTFILE=/dev/null -dSAFER ~/Downloads/gs_fpe_intersect -c quit

Note, that it should be -sOutputFile, as these options are case sensitive. I haven't checked to see if this matters.

Comment 3 Robin Watts 2016-12-29 08:23:58 UTC

Fixed in:

commit 4bef1a1d32e29b68855616020dbff574b9cda08f
Author: Robin Watts <Robin.Watts@artifex.com>
Date:   Thu Dec 29 15:57:43 2016 +0000

    Bug 697453: Avoid divide by 0 in scan conversion code.

    Arithmetic overflow due to extreme values in the scan conversion
    code can cause a division by 0.

    Avoid this with a simple extra check.

      dx_old=cf814d81
      endp->x_next=b0e859b9
      alp->x_next=8069a73a

    leads to dx_den = 0