Bug 702345 - Large temp files with magick- prefix
Summary: Large temp files with magick- prefix
Status: RESOLVED WORKSFORME
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: PS Interpreter (show other bugs)
Version: unspecified
Hardware: Other All
: P4 critical
Assignee: Default assignee
URL: https://stockvectorsillustrations.com/
Keywords:
Depends on:
Blocks:
 
Reported: 2020-04-18 11:18 UTC by P Tamas
Modified: 2020-04-19 00:55 UTC (History)
0 users

See Also:
Customer:
Word Size: ---


Attachments
1 of the vectors (353.46 KB, application/postscript)
2020-04-18 12:02 UTC, P Tamas
Details
Jpeg (7.92 MB, image/jpeg)
2020-04-18 20:08 UTC, P Tamas
Details

Note You need to log in before you can comment on or make changes to this bug.
Description P Tamas 2020-04-18 11:18:51 UTC
Hello,

I use Ghostscript and ImageMagick with a software called Symbiostock on shared hosting environment.

Its a stock photo / stock illustration script. I use it for the latter with EPS (encapsulated postscript) files version 10, 2000x2000 pixels and large JPEG file combos.

On my shared hosting environment (it wouldn't afford a bigger plan),
large temp files are generated and doesn't gets deleted.

I upload 40 GB of EPS + JPEG combos...
- The JPEGs are 8000x8000 pixels at minimum, 12000x120000 at maximum
- The software --I think-- uses the EPS files to generate previews

...1.7 TB of temp files left in the TMP directory.

This is what my hosting sent, what brought me here:

"
What exactly are you converting?
I am actually curious how ghost script is coming into play as this command I feel is important to your issues:
gs -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 -sDEVICE=pamcmyk32 -dTextAlphaBits=4 -dGraphicsAlphaBits=4 -r1500x1500 -g54187x37500 -sOutputFile=/tmp/magick-25269EjNy90NAfnnR%d -f/tmp/magick-25269ENWpWU313ARs -f/tmp/magick-25269UQuoD7BLHow4
"

Here is some ongoing threads on ImageMagick forums:
https://imagemagick.org/discourse-server/viewtopic.php?f=24&t=37845&p=174167#p174167
https://imagemagick.org/discourse-server/viewtopic.php?f=3&t=37838&p=174121#p174121

Also, I'm now on a smaller plan, but when I was on a largest plan with 12 GB RAM, the process run by a cronjob, ate up all the resources:
- Ram
- IO processes

Any idea, or tip where should I find a solution?
Comment 1 P Tamas 2020-04-18 12:01:31 UTC
The temp files are 19 GB in file size. Aren't deleted automatically.
Comment 2 P Tamas 2020-04-18 12:02:40 UTC
Created attachment 19157 [details]
1 of the vectors
Comment 3 P Tamas 2020-04-18 20:08:02 UTC
Created attachment 19159 [details]
Jpeg
Comment 4 Ray Johnston 2020-04-19 00:55:31 UTC
The command line shown has TWO input files, not one, but running the EPS
file that was attached, I see that ghostscript creates only two temp files
(for the command clist which handles the banded processing).

The smaller "index" bfile was 302,608 bytes. The larger data file with the
commands to allow banded rendering cfile was 110,898,625 bytes.

Note that Ghostscript temp files are all created on linux using a method that
opens, then unlinks the file so that even if the process aborts, the files do
not exist.

Note that the OutputFile is NOT deleted by Ghostscript, even if the process
aborts. The full size output file in pamcmyk32 is 8,128,050,120 (7.5Gb).
With the "%d" in the -sOutputFile string, there will be one of these for each
page. Presumably, the second file creates a second page.

As far as can be determined, Ghostscript is running as expected, and if the
OutputFile 'pamcmyk32' format files created by Ghostscript are not being
deleted, there is nothing that Ghostscript can do about it.

Closing as WORKSFORME.

Note that a debug build on my machine took 200 seconds and used 22Mb of RAM.