Bug 692290 - ps2write/pdfwrite have a limit of 4GB on temp files.
Summary: ps2write/pdfwrite have a limit of 4GB on temp files.
Status: NOTIFIED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: PS Writer (show other bugs)
Version: master
Hardware: PC Windows Vista
: P4 enhancement
Assignee: Ken Sharp
URL:
Keywords:
: 692127 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-06-20 10:46 UTC by Ken Sharp
Modified: 2013-01-03 20:45 UTC (History)
2 users (show)

See Also:
Customer: 661
Word Size: ---


Attachments
ps testfile for 4 Gbyte ps2write temp filesize limit fix (184.37 KB, application/postscript)
2011-06-29 15:22 UTC, steve166
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ken Sharp 2011-06-20 10:46:19 UTC
See bug 692172 for more details. 

The ps2write and pdfwrite temporary files have a maximum size of 4GB, though I'm not certain if this is because of the fact that (on Windows) we use a file system handle even though we create a 64-bit file, or because of our use of fseek.
Comment 1 Ken Sharp 2011-06-21 11:10:50 UTC
Committed changes in da11521 which should fix this:

http://ghostscript.com/pipermail/gs-cvs/2011-June/013027.html

Unfortunately I can't construct a test file which exercises this.
Comment 2 steve166 2011-06-29 15:22:58 UTC
Created attachment 7632 [details]
ps testfile for 4 Gbyte ps2write temp filesize limit fix

I have generated i testfile for this patch. With this patch the
temporary file grows beyond the 4 Gbyte limit as expected.
Unfortunately the generated ps output seems not to be valid, if
the temp file get larger than 4GByte. I have tested this with
Adobe Distiller 9.0 and the attached file using following commandline:

gswin32c -q -dBATCH -dNOPAUSE -sDEVICE=ps2write -dCompressPages=false
-dASCII85EncodePages=false -dCompressFonts=false -sOutputFile=out.ps ps2write4gbtemplimit_test.ps
(Testsystem Win XP Servicpack 2, 32 Bit, using Ghostscript Snapshot, compiled
with Mcrosoft VS 2005 using the supplied VS project file)

If you set the loop counter to 1200 "Pages" the temporary file stays slightly
smaller then 4 GByte and the distiller convertes the output without errors.
If you use 1250 as loop cntr the temp file size get larger than 4Gbyte
and the distiller crashes at pages 1247 with an ioerror (see attached log).
Comment 3 Ken Sharp 2011-06-30 08:22:05 UTC
(In reply to comment #2)

> If you set the loop counter to 1200 "Pages" the temporary file stays slightly
> smaller then 4 GByte and the distiller convertes the output without errors.
> If you use 1250 as loop cntr the temp file size get larger than 4Gbyte
> and the distiller crashes at pages 1247 with an ioerror (see attached log).

That was always the worry with this change. The calculation/manipulation of offsets needs to be 64-bit throughout as well, and its likely that I missed one (or more). I'll try and find the time to look into this but, obviously, its going to take a long time.
Comment 4 Alex Cherepanov 2011-07-30 16:44:45 UTC
pdfwrite and ps2write use 'long' extensively to represent various offsets.
For instance, see write_xref_section().

The problem is confirmed on 32-bit Windows, however 64-bit Linux build that
has 64-bit long type works just fine.
Comment 5 Ken Sharp 2012-04-17 08:46:35 UTC
*** Bug 692127 has been marked as a duplicate of this bug. ***
Comment 6 Marcos H. Woehrmann 2012-04-21 04:09:23 UTC
Moved customer number from Bug 692127.
Comment 7 Ken Sharp 2012-11-01 15:33:51 UTC
commit 9b10f8c659ffc643ab71c6180a70da4f39075578 resolves this for me. I have tested the 32 and 64 bit builds of Ghostscript running on Windows, using variations on the attached test file and command line.

The temporary file runs to 10GB, the produced PostScript file to 7GB (5000 pages) and the PostScript file to 6GB (3000 pages). The resulting PDF file can be opened by Ghostscript and Adobe Acrobat. The resulting PostScript file can be processed by Ghostscript and Adobe Distiller.

This relies upon commit c81c8f1b8740c484ec332080b5ce5f718357694d which *must* be applied before trying to use this code.