Bug 688764 - Error: /stackoverflow in -file-
Summary: Error: /stackoverflow in -file-
Status: NOTIFIED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: PS Interpreter (show other bugs)
Version: 8.54
Hardware: PC Linux
: P4 normal
Assignee: Alex Cherepanov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-06-22 10:41 UTC by Hin-Tak Leung
Modified: 2008-12-19 08:31 UTC (History)
1 user (show)

See Also:
Customer:
Word Size: ---


Attachments
this is the file which causes stackoverflow (2.61 MB, application/postscript)
2006-06-22 10:45 UTC, Hin-Tak Leung
Details
this one also. (2.61 MB, application/postscript)
2006-06-22 10:48 UTC, Hin-Tak Leung
Details
pdf file from which the ps file was generated. (524.84 KB, application/pdf)
2006-06-23 04:50 UTC, Hin-Tak Leung
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hin-Tak Leung 2006-06-22 10:41:37 UTC
ps file output from xpdf causes this:

Error: /stackoverflow in -file-
Operand stack:
   --nostringval--
Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--  
--nostringval--   2   %stopped_push   --nostringval--   --nostringval--  
--nostringval--   false   1   %stopped_push   1   3   %oparray_pop   1   3  
%oparray_pop   1   3   %oparray_pop   1  3   %oparray_pop   .runexec2  
--nostringval--   --nostringval--   --nostringval--   2   %stopped_push
Dictionary stack:
   --dict:1118/1686(ro)(G)--   --dict:0/20(G)--   --dict:71/200(L)--  
--dict:63/75(L)--   --dict:18/24(L)--   --dict:0/15(L)--  --dict:0/15(L)--  
--dict:0/15(L)--   --dict:0/15(L)--   --dict:0/15(L)--   --dict:0/15(L)--  
--dict:0/15(L)--   --dict:0/15(L)--   --dict:0/15(L)--   --dict:0/15(L)--
Current allocation mode is local
Current file position is 1020116
AFPL Ghostscript 8.54: Unrecoverable error, exit code 1

files to follow.
Comment 1 Hin-Tak Leung 2006-06-22 10:45:15 UTC
Created attachment 2294 [details]
this is the file which causes stackoverflow

generated by xpdf from a 2x2 pdf file generated by pdfnup 2x2
Comment 2 Hin-Tak Leung 2006-06-22 10:48:57 UTC
Created attachment 2295 [details]
this one also.

this one also generates a stackoverflow (apparently the nup part is
irrelevant),
without the nup step.

Error: /stackoverflow in -file-
Operand stack:
   --nostringval--
Execution stack:
   %interp_exit   .runexec2   --nostringval--	--nostringval--  
--nostringval--   2   %stopped_push   --nostringval--	--nostringval--  
--nostringval--   false   1   %stopped_push   1   3   %oparray_pop   1	 3  
%oparray_pop   1   3   %oparray_pop   1  3   %oparray_pop   .runexec2  
--nostringval--   --nostringval--   --nostringval--   2   %stopped_push
Dictionary stack:
   --dict:1118/1686(ro)(G)--   --dict:0/20(G)--   --dict:71/200(L)--  
--dict:63/75(L)--   --dict:18/24(L)--	--dict:0/15(L)--  --dict:0/15(L)--  
--dict:0/15(L)--
Current allocation mode is local
Current file position is 1019374
AFPL Ghostscript 8.54: Unrecoverable error, exit code 1
Comment 3 Dan Coby 2006-06-22 14:41:59 UTC
This file is creating very large (over 61k elements) arrays by putting values 
on the operand stack.  This is overflowing the current maxomum operand stack 
size of 50,000 elements.

A work around is to increase the value of MaxOpStack in lib/gs_init.ps.
Comment 4 Alex Cherepanov 2006-06-22 20:40:03 UTC
The file creates large arrays using [ ... ] construct.
In one case an array of 61476 elements is creates this way.

The file can be fixed by inserting the following code before the large arrays

{ currentfile =string readline pop pop
  65535 array
  0
  currentfile 0 (]) /SubFileDecode filter
  { 3 copy token not { pop pop exit } if
    put
    exch 1 add exch
  } loop
  pop 0 exch getinterval
} exec
[
338 673 5 1
344 673 5 1
......
]

Unfortunately, there's no way to make an automatic patch using
idiom recognition.

PLRM specifies the maximum depth of the operand stack can be as low as 500
elements. The bug should be reported against xpdf/pdftops, which generated
highly unportable files.

Comment 5 Ray Johnston 2006-06-22 22:29:42 UTC
For Ghostscript, I don't think it is totally unreasonable to change the value
of MaxOpStack to 70000 or so. This would allow a maximum size array (65535)
plus lots of other junk on the op stack.

It's better than trying some 'IdiotRecongnition' (recognition of idiotic
PostScript producers). ;-)
Comment 6 Hin-Tak Leung 2006-06-23 04:50:58 UTC
Created attachment 2296 [details]
pdf file from which the ps file was generated.

The original pdf from which the two ps files were generated.

While I don't particularly object to classifying this as a pdf/pdftops 
problem, it is probably in the nature of documents from certain sources
(scientific?) - graphs with a lot of points?

I'll contact the xpdf people and see what they say about it anyway. Meanwhile,
ghostscript probably should try to cope, if there isn't too much penality in
doing so.
Comment 7 Hin-Tak Leung 2006-06-24 04:49:21 UTC
the ps file also "reliably" crashes and lock-up 
a postscript printer (Oki B6200) with an Adobe-licensed 
ps interpreter in the firmware.

I'll put the fault to xpdf and the pdf's generator,
but ghostscript probably ideally should cope with a warning...
Comment 8 Hin-Tak Leung 2006-06-24 04:54:51 UTC
I did manage to print that pdf with adobe acrobat reader 7.0.8 on linux;
contacted xpdf's author.
Comment 9 Alex Cherepanov 2006-06-25 06:55:46 UTC
The operand stack size cannot be made large than 65414.
Otherwise the program "{1}loop" fails with 
  Unexpected interpreter error -15.
  Error object: (b02)int 1
  Operand stack at 0x9f9814:
  0x9ede58: 0x0b int  -nF------ 0xcccc 0x00000001 = 1
  0x9ede60: 0x0b int  -nF------ 0xcccc 0x00000001 = 1
  0x9ede68: 0x0b int  -nF------ 0xcccc 0x00000001 = 1
  ....

We may look into the the cause of the unexpected error, but the maximum working
stack size is sufficient to process the sample files.
Comment 10 Ralph Giles 2006-06-28 09:37:22 UTC
just bump the limit to the current working maximum. It's worth spending an hour
or so trying to figure out why the 56414 limit and fixing it, but it's not worth
pursuing extensively.
Comment 11 Alex Cherepanov 2006-07-02 16:15:45 UTC
PostScript stack cannot exceed the size of a PostScript array (65535) because
the stack is copied to an array when the stack overflow.

The stack space is allocated in blocks. The stack overflow is checked when the
new block is allocated. If the MaxOpStack is set close enough to 65535 the
actual stack size exceeds 65535 at the moment of stack overflow.

This is detected when the stack is copied to an array during stackoverflow
processing, causing a low level stack dump. Cleaning up the stack handling code
is a substantial project. 

Increasing the stack to the maximum working size = 65414.



Comment 12 Hin-Tak Leung 2006-07-03 04:01:00 UTC
Got reply from Derek Noonburg (of xpdf) and he admitted it is a known problem,
but said that the acrobat reader's way also have problems. Here is an exerpt
of his comment:

...
The issue is masked images, for which Xpdf uses the rectclip PostScript
operator, with an array of rectangles.  Depending on the mask, the array
can get arbitrarily large.  This is a known problem in Xpdf, but I
haven't come up with a good solution for it yet.
...
I took a quick look at the PS file generated by Adobe Reader, and it
looks like they're defining a pattern using the image, then doing an
imagemask fill with the mask.  That runs into a different problem in
that defining the pattern uses up a lot of memory.
...
... Adobe's way requires storing the image in memory, and
Xpdf's way requires storing only the mask in memory.  In general, the
mask will probably be smaller.  And right now, both approaches are
killing ghostscript and at least some common printers.
...
Comment 13 Alex Cherepanov 2006-07-03 08:24:33 UTC
Regarding the reply from Derek Noonburg (of xpdf):

...
> Depending on the mask, the array can get arbitrarily large.
> This is a known problem in Xpdf, but I
> haven't come up with a good solution for it yet.
There may be no perfect solution but some solutions are better
than other.
- Constructing large arrays using [ ... ] is not portable. Allocating an
  array explicitely and using put operator is much better.
- The path can be constructed using multiple drawing operations with the
  clip operator applied at the end.
- Level 3 has masked images that can be used directly. On the level 2 devices
  the clipping path can be constructed from the sample data at run time.
- Large masked images can be split into stripes to reduce the size of the
  clip path.

> ... Adobe's way requires storing the image in memory, and
> Xpdf's way requires storing only the mask in memory.  In general, the
> mask will probably be smaller.
Most likely, every implementation buffers all the data in the clip list
before rendering.

> And right now, both approaches are killing ghostscript and at least
> some common printers.
Please submit an Adobe's file as a separate bug report.
Comment 14 Hin-Tak Leung 2006-07-03 11:16:17 UTC
(last part of comment 13) the problem with adobe-generated ps was filed as
bug 688774- "Error: /VMerror in --.imagemask1--", which has since been identified
as duplicated of bug 688396 - current status assigned.

I'll forward the rest of comment 13 to xpdf's author.
Comment 15 Alex Cherepanov 2006-07-04 14:34:59 UTC
The maximum size of the operand stack is increased to 65414 .
Tested, no differences.

The fix is committed as revision 6889.