Bug 704236

Created attachment 21457 [details]
Bug704236.prof

This is the profile output (from gprof on linux) that shows that the dominant
time is spent processing images, in pdf_image_plane_data_alt, and much of that
is in s_image_colors_process doing s_image_colors_convert_to_device_color
calling gx_remap_ICC which is called 3,598,419,077 times.

I timed the 9.55 PRE-RELEASE rendering to raster with the command line:

bin/gswin64c -Z: -r200 -o x.pgm -sDEVICE=pgmraw -dMaxBitmap=3g Bug704236.pdf

and it took 211 seconds.

I used mutool for the same thing using:

mutool draw -r200 -o x.pgm -F pgm -s t Bug704236.pdf

and it took 112 seconds, so faster, but not the reported times or comparison
between gs and mutool.

The pdfwrite time, however is quite a bit slower: 1637 seconds.

Note that converting to gray colorspace still retains the transparency, but
the when the colors have been converted, transparency rendering may no longer
be the same as would result from rendering the original (color) image to gray.

If the goal is to get a gray color pdf, and if a specific resolution is OK,
then using the 'pdfimage8' device will result in a PDF that is a gray image at
a specified resolution.

For example, the command line:
bin/gs  -o z.pgm -sDEVICE=pdfimage8 -dBufferSpace=2g -r200 Bug704236.pdf

completes in 212 seconds, yielding a PDF that is 4.7Mb. This PDF can be
rendered to a gray image (such as TIFF) in 2 seconds (using gs).

The conversion using pdfimage8 is resolution dependent and -r400 takes 280 sec.
Going up to 600 dpi takes 901 sec.

I don't know if the pdfimage8 device would be a reasonable alternative output
device or not.

My mupdf test is with mudraw.exe version 1.7a, quite an old version because recent versions of mutool have significant regression in performance. The old version still renders very reliable and well.

Here is a copy paste of the command prompt on an i7-7700HQ CPU @ 2.80 Ghz

C:\temp>mudraw64.exe -s t -r 200 -c gray -F png -o "c:\temp\output.png" "c:\temp\Bug 704236.pdf"
page c:\temp\Bug 704236.pdf 1warning: not building glyph bbox table for font 'AppleSymbols' with 8794 glyphs
 13364ms
total 13364ms / 1 pages for an average of 13364ms
fastest page 1: 13364ms
slowest page 1: 13364ms

With under 14 seconds, this is about 10 times faster than the current version of "mutool draw" and probably subject to review what has changed regarding performance over the years.

To answer the second part of your test, Yes pdfimage8 is a valid alternative. Although vector output is always preferred because the print device is an HP PageWide with 1200dpi native resolution in this case. The output with pdfimage8 would need to be done at a reduced resolution to avoid excessive procession time, or accept some quality loss. In this case both 400 and 600dpi output produce acceptable prints. There are other cases where the output device has even higher max resolution (HP Indigo, Xerox IGEN, or an Offset Press)

thank you

I believe this is nothing to do with transparency, it is the vast number of patterns which are switched between frequently. Each switch involves rerunning the Pattern PaintProc and the file does this in excess of 10,000 times.

It may be possible to do something about caching the pattern, since we do cache the pattern tile, which would avoid rerunning the PaintProc many times. However I have no intention of trying this on the current PDF interpreter.

Assigning to myself to evaluate for the new interpreter.

I don't know enough of the inner workings and differences of the pdf interpreter of mudraw version 1.7a vs GS and if it is feasible to implement a similar strategy for such patterns or not, but I can say with confidence that the old version of mudraw is handling such and any file very efficiently.
thank you

As I suspected, the problem is nothing to do with transparency and everything to do with patterns. As noted in the commits below, the PostScript interpreter cannot reliably detect identical pattern reuse, and so the graphics library is built on the assumption that we cannot cache and reuse pattern bitmaps, except in the case of grestore'ing back to an earlier state where a Pattern was active.

The old PDF interpreter, being written in PostScript, cannot do anything about that. The new PDF interpreter, being written in C, can. At least up to a point; we are obviously still dependent on the capabilities of the graphics library.

This commit 626d5d3d7c7f7380af6fb3c7b6ee240ccdc1a213 followed by this commit 984b6bcba88a35a1a705480ad6b7cd14f60f9661 permit the new PDF interpreter to override the internal ID of tiles in the pattern tile cache. This allows us to uniquely identify such tiles, which means we can reuse them if the same pattern is used again while the tile is still in the cache.

There are a number of considerations in this, which I'm not going to document here as they are in the commit log. The summary is that with this code we can now render the 200 dpi comparison with an (old) version of MuPDF in 8 seconds, and can run the gray scale remapping of the PDF file in 16 seconds.

I consider this enhancement to be complete.