Created attachment 13262 [details] POC to trigger FPE (gs) After some fuzz testing I found a crashing test case. Git Head: 73060a27e554f8e64ae2aba4a1b03822207346c7 Command: s -dNOPAUSE -sDEVICE=bit -sOUTPUTFILE=/dev/null -dSAFER gs_fpe_intersect -c quit ASAN + Output: GPL Ghostscript GIT PRERELEASE 9.21 (2016-09-14) Copyright (C) 2016 Artifex Software, Inc. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. Querying operating system for font files... Can't find (or can't open) font file %rom%Resource/Font/. Didn't find this font on the system! Substituting font Courier for . Loading NimbusMonoPS-Regular font from %rom%Resource/Font/NimbusMonoPS-Regular... 4265400 2800359 3055176 1770757 1 done. ASAN:DEADLYSIGNAL ================================================================= ==6361==ERROR: AddressSanitizer: FPE on unknown address 0x0000017b02e4 (pc 0x0000017b02e4 bp 0x000004233630 sp 0x7ffdbf4dbc20 T0) #0 0x17b02e3 in intersect XYZ/ghostpdl/./base/gxfill.c:1754:22 #1 0x17b02e3 in intersect_al XYZ/ghostpdl/./base/gxfill.c:1856 #2 0x1795c41 in spot_into_trapezoids__aj_fd XYZ/ghostpdl/./base/gxfilltr.h:146:13 #3 0x1795c41 in spot_into_trapezoids XYZ/ghostpdl/./base/gxfill.c:2084 #4 0x1791139 in gx_general_fill_path XYZ/ghostpdl/./base/gxfill.c:557:16 #5 0x1791139 in gx_default_fill_path XYZ/ghostpdl/./base/gxfill.c:700 #6 0x1858d22 in gx_stroke_path_only_aux XYZ/ghostpdl/./base/gxstroke.c:1045:13 #7 0x1858d22 in gx_stroke_path_only XYZ/ghostpdl/./base/gxstroke.c:1081 #8 0x1853356 in gx_default_stroke_path XYZ/ghostpdl/./base/gxstroke.c:334:12 #9 0x1823aea in gx_stroke_fill XYZ/ghostpdl/./base/gxpaint.c:69:12 #10 0x148000c in do_stroke XYZ/ghostpdl/./base/gspaint.c:444:16 #11 0x148000c in gs_stroke XYZ/ghostpdl/./base/gspaint.c:480 #12 0x19fe37e in interp XYZ/ghostpdl/./psi/interp.c:1314:40 #13 0x19fe37e in gs_call_interp XYZ/ghostpdl/./psi/interp.c:511 #14 0x19fe37e in gs_interpret XYZ/ghostpdl/./psi/interp.c:468 #15 0x19d1352 in gs_main_interpret XYZ/ghostpdl/./psi/imain.c:245:12 #16 0x19d1352 in gs_main_run_string_end XYZ/ghostpdl/./psi/imain.c:663 #17 0x19d1352 in gs_main_run_string_with_length XYZ/ghostpdl/./psi/imain.c:621 #18 0x19dd2eb in run_string XYZ/ghostpdl/./psi/imainarg.c:977:16 #19 0x19dd2eb in runarg XYZ/ghostpdl/./psi/imainarg.c:967 #20 0x19dc748 in argproc XYZ/ghostpdl/./psi/imainarg.c:900:16 #21 0x19d5283 in gs_main_init_with_args XYZ/ghostpdl/./psi/imainarg.c:238:24 #22 0x5475d8 in main XYZ/ghostpdl/./psi/gs.c:96:16 #23 0x7f4bf756982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f) #24 0x47b9d8 in _start (/usr/local/bin/gs+0x47b9d8) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: FPE XYZ/ghostpdl/./base/gxfill.c:1754:22 in intersect ==6361==ABORTING
Its a scan-conversion problem, probably related to an overflow of the fixed point arithmetic (maybe). Unfortunately I can only reproduce this in a 64-bit Linux release build. Debug build on Linux and all Windows builds seem to work. Assigning to Robin as a scan-conversion bug for more investigation.
I can reproduce this bug on 64bit linux in a debug build if I build using: make debug XCFLAGS="-O2" gdb --args debugbin/gs -dNOPAUSE -sDEVICE=bit -sOUTPUTFILE=/dev/null -dSAFER ~/Downloads/gs_fpe_intersect -c quit Note, that it should be -sOutputFile, as these options are case sensitive. I haven't checked to see if this matters.
Fixed in: commit 4bef1a1d32e29b68855616020dbff574b9cda08f Author: Robin Watts <Robin.Watts@artifex.com> Date: Thu Dec 29 15:57:43 2016 +0000 Bug 697453: Avoid divide by 0 in scan conversion code. Arithmetic overflow due to extreme values in the scan conversion code can cause a division by 0. Avoid this with a simple extra check. dx_old=cf814d81 endp->x_next=b0e859b9 alp->x_next=8069a73a leads to dx_den = 0