Bug 689933

Summary: Memory allocation with high resolution
Product: Ghostscript Reporter: Leonid Lukomskij <leonidas>
Component: ImagesAssignee: Alex Cherepanov <alex>
Status: NOTIFIED FIXED    
Severity: normal CC: christinedelight.top85, henry.stiles
Priority: P3    
Version: 8.62   
Hardware: PC   
OS: Windows XP   
Customer: 582 Word Size: ---
Attachments: D35-100-A-28963303-MOR9.rar

Description Leonid Lukomskij 2008-07-02 14:01:31 UTC
Hello,

I have one more problem regarding to VM usage with high resolution output. I
tried to process pdf and had follows:

C:\gs\gs8.62\lib>..\bin\gswin32c.exe -dSAFER -dBATCH -dNOPAUSE -sDEVICE=tiff32nc
 -r2540 -sOutputFile=d35.tif D35-100-A-28963303-MOR9.pdf
GPL Ghostscript 8.62 (2008-02-29)
Copyright (C) 2008 Artifex Software, Inc.  All rights reserved.
This software comes with NO WARRANTY: see the file PUBLIC for details.
Processing pages 1 through 1.
Page 1

   **** Warning: File encountered 'VMerror' error while processing an image.

Ghostscript didn't generate output file because of 2gb output file size
limitation, but I saw output of my own device (which doesn't have this
limitation) and some images was lost.

I built debug version of Ghostscript and went inside with debugger. I found that
problem file contains type3 image. Ghostscript creates image8 memory device to
render this image. It tries to allocate about 1.6 Gb of memory for this device
and doesn't get this memory. It doesn't try to create command list as rendering
device (also base device for tiff32nc was command list).

Do you have any patches/suggestions to fix this problem?

Best regards

Leonid Lukomskij
Comment 1 Leonid Lukomskij 2008-07-02 14:07:49 UTC
Created attachment 4179 [details]
D35-100-A-28963303-MOR9.rar

Problem file (pdf inside)
Comment 2 Alex Cherepanov 2008-07-05 22:09:06 UTC
Excessive memory allocation happens in type 3 image processing.

gdev_mem_open_scan_lines() line 328
mem_open() line 308 + 19 bytes
make_midx_default() line 506 + 15 bytes
gx_begin_image3x_generic() line 230 + 107 byte
gx_begin_image3x() line 566 + 51 bytes

Fixing this may take a while. I cannot recommend anything better than using
lower resolution or installing more memory.
Comment 3 Ray Johnston 2008-07-06 08:24:16 UTC
This looks like a duplicate. Fixing this will probably require a clist based
accumulator similar to pattern clist accumulator, but other approaches are
possible. Such as writing a temp file, if disk is available. The only effective
advantage that clist acumulation will present is the zlib compression, and
this might be done simpler just using 'deflate' on blocks that are linked
together. With Type 3 images (AFAIK) access is sequential, so the random access
capability of the memory based clist isn't needed. 
Comment 4 Alex Cherepanov 2011-06-30 01:33:29 UTC
Memory allocation is has been reduced to about 50M but the file takes about 1 hour to render.

  %   cumulative   self              self     total           
 time   seconds   seconds    calls  Ks/call  Ks/call  name    
 27.76    448.19   448.19 53708306     0.00     0.00  pdf14_fill_rectangle
 24.10    837.31   389.12 6054822403     0.00     0.00  art_pdf_composite_pixel_alpha_8
  9.48    990.35   153.04    12286     0.00     0.00  pdf14_compose_group
  8.22   1123.04   132.69 627564679     0.00     0.00  gs_memcpy
  5.58   1213.17    90.13    96024     0.00     0.00  gx_build_blended_image_row
  5.03   1294.34    81.17   142264     0.00     0.00  image_render_color_icc
  5.02   1375.45    81.11  5562023     0.00     0.00  gs_memset
  3.68   1434.83    59.38 2913516529     0.00     0.00  art_pdf_recomposite_group_8
  2.76   1479.41    44.58 2913699880     0.00     0.00  art_blend_pixel_8
  1.34   1501.09    21.68                             art_pdf_union_mul_8
  1.18   1520.14    19.05  4383714     0.00     0.00  gx_fill_trapezoid_ns_lc
Comment 5 Ken Sharp 2011-08-25 09:21:33 UTC
(In reply to comment #4)
> Memory allocation is has been reduced to about 50M but the file takes about 1
> hour to render.

I think we should close the bug, as the memory usage has apparently been addressed. 

If we want to consider the performance then we should open a new bug for that, though I would point out that we already have several enhancement requests open against transparency rendering, and rendering a high resolution (2540 dpi) large page (40x40 inch) containing what appears to be mainly a huge image with some transparent objects laid over it (including a large soft mask), is probably never going to be 'fast'.
Comment 6 Henry Stiles 2011-08-25 14:25:44 UTC
Closing, if anyone knows the revision(s) responsible for the improvement please add them to the bug.