While investigating a memory issue for a customer I came across a strange memory usage pattern in Ghostscript. With gshead (r8445) converting the attached PDF file at 300 DPI to a ppmraw file uses about 950 megs of memory (an improvement over the ~4 gigs of memory that gs8.60 takes). However, this amount of memory is only used for a very short amount of time very early in the conversion process: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 15258 marcos 18 0 61332 11m 4832 R 30 0.3 0:00.15 gs 15258 marcos 18 0 61332 11m 4832 R 99 0.3 0:00.66 gs 15258 marcos 19 0 61132 11m 4932 R 101 0.3 0:01.20 gs 15258 marcos 20 0 947m 568m 4960 R 97 14.3 0:01.69 gs 15258 marcos 21 0 948m 900m 4980 D 67 22.7 0:02.04 gs 15258 marcos 21 0 948m 900m 4980 D 0 22.7 0:02.04 gs 15258 marcos 21 0 948m 900m 4980 D 0 22.7 0:02.04 gs 15258 marcos 18 0 948m 900m 4980 R 14 22.7 0:02.11 gs 15258 marcos 18 0 948m 900m 4980 R 101 22.7 0:02.64 gs 15258 marcos 19 0 69252 20m 4980 R 97 0.5 0:03.14 gs 15258 marcos 20 0 65788 16m 5204 R 99 0.4 0:03.65 gs 15258 marcos 21 0 67392 18m 5212 R 101 0.5 0:04.16 gs 15258 marcos 22 0 67392 18m 5212 R 101 0.5 0:04.67 gs 15258 marcos 25 0 67392 18m 5212 R 99 0.5 0:05.18 gs After this the memory usage rises slowly to 120 megs. Ghostscript 8.60 has the same memory usage behaviour, it allocate 948 megs, which it frees after a few seconds.
Created attachment 3637 [details] COZ_3200x2700.pdf
Created attachment 3638 [details] PDFDEBUG.log The allocation is due to allocating an SMask area. On my machine, I actually get a VMerror because the allocation fails. This happens when processing the object 401: %Resolving: [401 0] << /Intent /RelativeColorimetric /Subtype /Image /Length 6009714 /Filter /FlateDecode /Name /X /SMask 400 0 R /BitsPerComponent 8 /ColorSpace /DeviceCMYK /Width 2077 /DecodeParms << /Columns 2077 /Predictor 15 /BitsPerComponent 4 /Colors 4 >> /Height 1708 /Type /XObject >> The entire PDFDEBUG log up to that point is attached.
So what exactly is the bug here? Are we using more memory than the customer wants? Is I this just an enhancement to avoid the apparently unnecessarily huge allocation?
In general we want to avoid large allocations -- that's why Igor did the 'big pattern as clist' logic and why Dan switched the pdf14 to use the clist. Marcos didn't show the command line, so I assume that he was using the default x11 or display device. Neither of these devices uses the clist. I get large allocations even with -r72 to the display device on Windows. Testing with a device that DOES use the clist, however shows unexpected usage: gs -sDEVICE-pngmono -Z: -r400 -o x.png -dBufferSpace=10000000 COZ.pdf I get: [a+]gs_malloc(large object chunk)(1640566344) = 0x0: failed **** Warning: File encountered 'VMerror' error while processing an image. The behavior is quite strange, however, because at -r200 I see a brief allocation of > 200Mb, even though it is using banding (-Z: shows band_height=22, 889 bands). At -r300, the allocation spikes to >900Mb (band_height=15, 1977 bands). This clearly is not what we need for our customers that often print at resolutions of 600, 720 or higher.
A quick follow up comment to the previous. The large allocation happens BEFORE the -Z: printe out the clist_end_page message: [:]clist_end_page at cfile=56938445, bfile=760776 This shows that the allocation is happening as the clist is being written, so something is doing a large allocation and bypassing/ignoring the clist (probably a transparency related SMask or something).
Sorry for not including the command line (my initial comment did mention ppmraw and 300 DPI, but I should have been explicit): bin/gs -sDEVICE=ppmraw -sOutputFile=test.ppm -r300 ./COZ_3200x2700.pdf
Created attachment 3695 [details] ES_Outside_A3_PP.pdf Second customer supplied file that also exhibits the problem (this is a much smaller file, just over a meg).
As of r8554 this issue has not been fixed.
Reassigning to Leonardo who's been working on code in this area.
The huge block allocation described in Comment #4 is a dup of the bug 689080. Other issues mentioned here are pretty vague so I'll return the bug to the bug owner. Please provide command lines to reproduce your problems. If no problem besides Comment #4, please resolve as a dup of 689080.
I've confirmed this is fixed in 8.63. *** This bug has been marked as a duplicate of 689080 ***
Changing customer bugs that have been resolved more than a year ago to closed.