Created attachment 8027 [details] file 1 that SEGVs The customer has supplied two files which, for me, cause a SEGV with the following command line: -sDEVICE=tiffsep -dNOPAUSE -dBATCH -dSAFER -r300 -dMaxBitmap=500000000 -dTextAlphaBits=4 -dGraphicsAlphaBits=4 -dFirstPage=1 -dLastPage=1 -sOutputFile=./out.tif In both cases the crash is in gc_reloc_ptr() called from pdf14_device_reloc_ptrs() during a vmreclaim. This is on a 32-bit debug version of Ghostscript, on Windows. I've put 'color' as the component though I'm not entirely sure this is correct. The Windows display device works correctly, even when handling separations.
P1 for crash and customer issue.
Assigning to Ray as he was able to see ilocate errors. I was not. I do see that we are unable to handle the number of colors in this file with our compressed color encoding though. I am not sure if these issues are related at all.
Created attachment 8033 [details] JPEG of error message Screenshot from the customer with the error they experience. Note that, like everyone else, they do not experience a Seg fault. The crash may be unique to my setup.
Created attachment 8035 [details] another example When run with the customer's command line on the customer system this results in a VM error in .discardtransparencygroup. I suspect this is simple VM exhaustion caused by the very large MaxBitmap. When run on my PC this also fails with a memory error (failed allocation) and crashes. When run using a lower MaxBitmap (on my PC), this crashes in the same way as the other files in this report, so I'm attaching it here
I have a fix for the seg fault. It doesn't address the non-encodable colors issue. The crash was due to the pdf14 device leaving dangling pointers to some of the devn and compressed color structures. I still see an issue where a debug build with -Z? gives two warnings in clist mode (without the -dMaxBitmap=500000000) for example: GPL Ghostscript GIT PRERELEASE 9.05: .\base\gsalloc.c(811): free_compressed_color_list: freeing 0x2b7e428, not owned by memory 0x1f38828! I have determined that this is happens with the pdf14 compositor device and a compressed color encoding device because the pdf14 compositor allocator is different to the base device, but when compressed color sub_levels are created they use the pdf14 allocator, NOT the same allocator as the target device used for the top level compressed_color_list. I'll chat with Michael about this, but probably will change the pdf14 device instead of adding a 'mem' to the compressed_color_list_t structure (either would work). The next issue I'll look into the why we are getting the non-encodable colors with the first file (so we can generate the correct separations).
OK, fix for the crashes and memory problems I observed with these files and the toffsep device are fixed in commit 7ebbcae comment: Fix bug 692618. Clear pointers to compressed color structured in pdf14 device. After the devn compressed color structures were freed, the pointers were not reset to NULL so subsequent GC would trace into freed or re-used memory. -Z? showed errors and, depending on memory contents and usage could result in a seg fault. Also add 'mem' element to the compressed_color_list structure to be used when freeing to avoid confusion about the correct allocator. Issue with non-encodable colors is _not_ fixed by this, only the segfault. Leaving the issue open while I investigate the non-encodable color issue.
Note that I did not see a VMerror with the 'another example' file run with: -r400 -dMaxBitmap=500000000 -sDEVICE=tiffsep -o y.tif -dTextAlphaBits=4 -dGraphicsAlphaBits=4 and a debug build with -Z: shows that clist is being used: [:]width=8375, band_width=8375, band_height=50, nbands=215 but with: -r400 -dMaxBitmap=1000000000 -sDEVICE=tiffsep -o y.tif -dTextAlphaBits=4 -dGraphicsAlphaBits=4 it uses page mode and completes (with the same debug build) in 93 sec. vs. 103 seconds in clist mode. With -dBufferSpace=500000000 the nbands=2 and the file completes in 91 sec. But no VMerror in any case. There is a possibility that my fixes for the crash also fixed a spurious VMerror, or that some previous patch fixed it (I was running rev. 7ebbcae
BTW, all my testing was with a 32-bit build.
(In reply to comment #7) > OK, fix for the crashes and memory problems I observed with these files and > the toffsep device are fixed in commit 7ebbcae comment: > > Fix bug 692618. Clear pointers to compressed color structured in pdf14 device. > > After the devn compressed color structures were freed, the pointers were not > reset to NULL so subsequent GC would trace into freed or re-used memory. -Z? > showed errors and, depending on memory contents and usage could result in a > seg fault. I'm afraid to say that I'm still seeing errors. The file RBR1103767.pdf still crashes for me with -dMaxBitmap=500000000 or =100000000. With 100000000 the GPF is in reloc_ptr and the call stack points back to gdevp14.c line 478: for (i = 0; i < pdev->devn_params.separations.num_separations; ++i) { RELOC_PTR(pdf14_device, devn_params.separations.names[i].data); When i=10. With -dMaxBitmap=500000000 the crash is in ialloc_validate_object called (eventually) from gs_vmreclaim. However the other example files listed here no longer seg fault for me with the commit above. They do still produce a rangecheck on showpage (as expected I think).
commit 97e6cd Fixes crash and Object not in any chunk error and VMerror. Actually two problems showed up after the previous commit related to this bug. A crash could occur (or cause "Object not in any chunk") because the pdf14 device did not use stable_memory for separation names. The second issue was a VMerror when the decision for banded (clist) mode was flawed, because the tiff_open the called gdev_prn_allocate_memory with new_width, hew_height = 0,0 and expected the 0,0 to be ignored. The new_width and new_height are only used from gdev_prn_reallocate_memory. This caused us to use page mode with at 400 dpi with the -dBufferSpace=500000000. The calculation for ESTIMATED_PDF14_ROW_SPACE (in gstrans.h) is not quite conservative enough when the device uses compressed color encoding because the potential num_components will cause more planes to be needed for the transparency (not just the 64-bit depth). Attempting to fix this showed LOTS of differences due to band height sensitivity, so I will have to look into those issues before improving the ESTIMATED_PDF14_ROW_SPACE calculation. The non-encodable pixel issue still remains (but only for file 1).
I don't quite understand why, but I don't get the non-encodable colors when using banded (clist) mode. I thought that it was because extra colors are generated by the pdf14 transparency blending so there are less colors in each band, and each band starts a new pdf14 compositor, so starts a new color list, but even when I run file 1 with '-dBufferSpace=500000000 -dMaxBitmap=10000' which uses clist mode, but only 1 band, there are no problems. This is something I want to investigate, but for the customer situation, there is a simple workaround.
I've sent the customer the applicable patches along with the workaround.
The 'workaround' doesn't actually produce the correct images for file_1 (the one that prints the 'non-encodeable color' message in page mode). In banding (clist) mode, there are still errors (white wedges around the 'BANG' on the left image). I don't understand where the white wedges come from and am investigating, but I have put a breakpoint where we would encode a compressed color as NON_ ENCODEABLE_COLOR and the breakpoint is not hit in clist mode as it is in page mode. More later.
The compressed_color_encoding has what I think is a bug -- when looking to pick a solid_not_100 color value, it can end up with value == 0 which is already taken into account by the 'colorants' bitmap. Investigating that fix to see what happens.
Change summary. The segfaults are all fixed, but the non-encodeable colors with file_1 still occur. We have an idea on how to get around this, but it is a fairly significant task, requiring a 'memsep' device that supports more than 8 planes of 8-bit data and a new color type to allow the gamut of colors to be passed to the memsep device. Design in process.
Note that the development of an alternative method to handle more colors/ components than can be packed into a 64-bit gx_color_index is a limitation so changing this to 'enhancement'
*** Bug 692895 has been marked as a duplicate of this bug. ***
Discussed this with Ray. The solution will roughly be to add high level encoding/decoding of DeviceN source colors into the clist along with the addition of a planar buffer for the separation devices so that we can maintain all the planes of data and not be limited with mem64 chunky buffers
Making good progress on this. I currently have the planar device getting installed for the tiff sep device. I am reading and writing DeviceN color values in and out of the clist. Next steps are to get the high level color filling working and fix the tiff sep device so that it properly assembles the data from the planar based buffers. There is likely going to be some work to do related to patterns as well as transparency in this fix.
The code is working now with some simple files. I am pushing on now to make sure all is OK with patterns, gradients and transparency.
This issue is currently fixed with the branch that I am working on. Commit is pending cluster testing review.
Fixed with http://git.ghostscript.com/?p=ghostpdl.git;a=commit;h=60640aeb33b18f9a9fcd76fc6f1083d7c7635f24