The following was reported with gs8.56, but the memory usage hasn't changed with gshead (r9355): The attached PDF file (rather large, sorry) consists of scanned image pages. The following trace from the first page is typical (skipping most of the %Resolving: lines): %Resolving: [1430 0] << /Subtype /Image /Width 2132 /Height 3334 /ColorSpace /DeviceRGB /BitsPerComponent 8 /SMask 1431 0 R /Length 7530 /Filter /JPXDecode >> stream %FilePosition: 6669 endobj %Resolving: [1428 0] << /Length 79 /Filter /FlateDecode >> stream %FilePosition: 1584 endobj q BT 0 600 Td ET Q q 3 Tr 0.17964 0 0 0.17964 0 0 cm Q q 383 0 0 600 0 0 cm /img0 Do %Resolving: [1429 0] << /Subtype /Image /Width 711 /Height 1112 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Interpolate true /Length 4681 /Filter /JPXDecode >> stream %FilePosition: 1826 endobj /Im001 Do %Resolving: [1430 0] %Resolving: [1431 0] << /Subtype /Image /Width 2132 /Height 3334 /ColorSpace /DeviceGray /BitsPerComponent 1 /Decode [ 1 0 ] /Length 1964 /Filter /JBIG2Decode >> stream %FilePosition: 14361 endobj Q %%EOF >>showpage, press <return> to continue<< The data for this page is roughly 3 x 2132 x 3334 + 3 x 711 x 1112 + 2132 x 3334 / 8 in size, i.e. under 25 Mb. However, when displaying this file, GPL Ghostscript 8.56 (2007-03-14) immediately acquires over 200M of address space. (evince, the PDF viewer distributed with Ubuntu Linux, does the same or even worse.) I could imagine a need for one or even two intermediate 32-bit-per-pixel buffers for handling the SMask, but that would only amount to about another 50M, not 150M. The time to display the page also seems rather long, around 10 seconds on my system (amd64, 2.something GHz, 512M RAM). Could you take a look at this?
Because of its large size the test image can be found on casper: /home/support/690230/fuguepro00prouuoft.pdf
The problem can be seen on just the first page (-dLastPage=1) in the execution of /Im0001 Do (-dPDFSTEP step #5535) The -ZA doesn't help because the large allocation is directly to 'malloc' from the jasper code. The call stack for the BIG allocation is: malloc(unsigned int nSize=28432352) Line 151 jas_malloc(unsigned int size=28432352) Line 171 jas_matrix_create(int numrows=3334, int numcols=2132) Line 126 jas_seq2d_create(int xstart=0, int ystart=0, int xend=2132, int yend=3334) Line 90 jpc_dec_tileinit(jpc_dec_t * dec=0x01a23bb0, jpc_dec_tile_t * tile=0x01948ac0) Line 714 jpc_dec_process_sod(jpc_dec_t * dec=0x01a23bb0, jpc_ms_t * ms=0x00000000) Line 566 jpc_dec_decode(jpc_dec_t * dec=0x01a23bb0) Line 396 jpc_decode(jas_stream_t * in=0x01a0fbd0, char * optstr=0x00000000) Line 253 jp2_decode(jas_stream_t * in=0x01a0fbd0, char * optstr=0x00000000) Line 249 jas_image_decode(jas_stream_t * in=0x01a0fbd0, int fmt=0, char * optstr=0x00000000) Line 381 s_jpxd_decode_image(stream_jpxd_state_s * state=0x019a5c28) Line 343 s_jpxd_process(stream_state_s * ss=0x019a5c28, stream_cursor_read_s * pr=0x019a5380, stream_cursor_write_s * pw=0x019a5d18, int last=1) Line 402 sreadbuf(stream_s * s=0x019a5cb8, stream_cursor_write_s * pbuf=0x019a5d18) Line 806 s_process_read_buf(stream_s * s=0x019a5cb8) Line 732 image_file_continue(gs_context_state_s * i_ctx_p=0x0192e728) Line 481 call_operator(int (gs_context_state_s *)* op_proc=0x00a05d60, gs_context_state_s * i_ctx_p=0x0192e728) Line 111 interp(gs_context_state_s * * pi_ctx_p=0x018e29bc, const ref_s * pref=0x0006f2e0, ref_s * perror_object=0x0006f3d0) Line 1162
Should see if libopenjpeg performs any better here.
To say that jasper is sloppy in its memory usage is an understatement. The 3 28 Mb buffers (one for each component) are bad enough (but why is it 4 bytes per pixel -- surely 2 would be enough), but then while processing the image, it allocates ANOTHER 91 Mb while reading parts of the file. Re-working the jasper code is definitely a big project. Perhaps the Luratech decoder would be better ?
Just to confirm Ray's analysis, I ran the whole file under valgrind's massif tool on linux x86_64. 95.92 percent of heap was allocated through jas_malloc(). About half was from jpc_dec_process_eoc() -> jpc_dec_tiledecode(), and about half from jpc_dec_process_sod() -> jpc_dec_tileinit(). The remaining 4% of memory is under gs_heap_alloc_bytes() so temporary buffers on the gs side aren't a significant contributor to the footprint. Jasper allocates 4 bytes per pixel component, so the 3334x2132 RBG image uses 81 MB internally; most of the rest is in temporary buffers also inside jasper. 32 bits is the maximum component resolution supported but the JPEG 2000 format; jasper makes no attempt to optimize footprint for the more common 8 bit case.
FWIW the luratech decoder uses around 80 MB on this file (200 MB virtual). It is also considerably faster.
Decoding the first 3334x2132 jpx image stand-alone, the openjpeg library also consumed about 108 MB, according to massif. So switching to this or the luratech decoder would produce the desired behaviour.
Closing. As Ray says in #4, fixing this is a major enhancement to 3rd party code, and we're not able to undertake it at this time. Potential workarounds are available in terms of other decoders. I've opened a bountiable enhancement (bug 690543) for libopenjpeg bindings to the gs stream library.
I know we closed this as an enhancement hoping that OpenJPEG would help, but I wanted to record the results I observed with two different gs command lines on my laptop that has a 3GHz Core i5 with 6Gb of 1366MHz DDR3. time gswin32c -sDEVICE=ppmraw -o /dev/null -r300 -dLastPage=1 bug_690230.pdf 4.9 seconds, max memory used 192M RAM time gswin32c -sDEVICE=bbox -r300 -dLastPage=1 -dBATCH -dNOPAUSE bug_690230.pdf 48.4 seconds, max memory used 1,358Mb RAM !!! I thought -r300 would make the bbox device operate at 300 dpi, but there is something going on here.
Changing customer bugs that have been resolved more than a year ago to closed.