Bug 691157 - Problem with image masking
Summary: Problem with image masking
Status: NOTIFIED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: Images (show other bugs)
Version: master
Hardware: All All
: P2 normal
Assignee: Alex Cherepanov
URL:
Keywords:
: 690351 690535 691348 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-03-02 10:50 UTC by Marcos H. Woehrmann
Modified: 2011-10-02 02:35 UTC (History)
4 users (show)

See Also:
Customer: 190
Word Size: ---


Attachments
proposed patch (5.30 KB, patch)
2010-11-24 19:04 UTC, Chris Liddell (chrisl)
Details | Diff
the correct patch..... (5.30 KB, patch)
2010-11-24 19:06 UTC, Chris Liddell (chrisl)
Details | Diff
Further simplification (319.47 KB, application/pdf)
2010-11-26 16:01 UTC, Alex Cherepanov
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Marcos H. Woehrmann 2010-03-02 10:50:36 UTC
The attache file has an issue when rendered with Ghostscript: head (r10837), 8.70, and 8.71 render it 
with one image missing and the other not correctly masked, earlier versions show both images but 
neither is masked correctly.  

The version that caused this change in behaviour is r9666:

r9666 | mvrhel | 2009-04-20 12:16:24 -0700 (Mon, 20 Apr 2009) | 128 lines

This is a merge of the smask_work branch into the trunk.  The merge fixes issues related to missing 
soft masks, improperly rendered soft masks and transparency group color spaces.  It addresses a 
number of bugs including 688728 689422 689512 689931 688601 689968 690958 690115 690157 
and 690170.    

The command line I'm using for testing:

  bin/gs -sDEVICE=tiff32nc -o test.tif ./Affiche2finaledefinale_c.pdf
Comment 1 Marcos H. Woehrmann 2010-03-02 10:51:07 UTC
Created attachment 6027 [details]
Affiche2finaledefinale_c.pdf
Comment 2 Michael Vrhel 2010-07-13 21:01:08 UTC
Made a bit of progress on this.  The soft mask consists of an image that has a soft mask of its own which has an inverted encoding.  The soft mask with the inverted encoding ends up having a bounding box region that is 0 and so it is not really used. However, with transparency the GrayBackground value needs to be considered outside the bounding box and I suspect that is not happening here since it is likely a value of 1 but we need to consider the inverted encoding of the image, which is not happening.  Just a theory at this point to explain what I am seeing.
Comment 3 Michael Vrhel 2010-07-14 20:22:34 UTC
Making more progress.  My initial assumption was incorrect.  There is a problem with the bounding box for the first softmask.  Digging into this now.
Comment 4 Michael Vrhel 2010-07-15 00:33:21 UTC
Verified that pdfwrite also fails and that the problems are caused by the following:

The contents for a simplified file (to be attached) is given by

stream
q 0 0 595 842 re W n q /GS0 gs /Perceptual ri 138.009903 16.561203 -12.4208984 103.5074005 271.7055054 234.8406982 cm 
/Im0 Do Q Q q 0 0 595 842 re W n q /Perceptual ri 139.9956055 16.7994843 12.1826019 -101.521698 258.4109039 337.2360992 cm /Im1 Do Q Q 
endstream

where the setting of /GS0 gs defines a softmask given by 

27 0 obj
<<
  /SMask 26 0 R
  /Type /ExtGState
>>
endobj

which is 

26 0 obj
<<
  /G 17 0 R
  /S /Luminosity
  /Type /Mask
>>
endobj

where the group is 

17 0 obj
<<
  /BBox [ 258.411 235.714 410.589 354.036 ]
  /Type /XObject
  /FormType 1
  /Group <<
    /CS /DeviceGray
    /I true
    /K false
    /S /Transparency
  >>
  /Length 178
  /Matrix [ 1 0 0 1 0 0 ]
  /Resources 25 0 R
  /Subtype /Form
>>
stream
q Q q 258.4109 337.2361 m 398.4065 354.0356 l 410.5891 252.5139 l 270.5935
235.7144 l h W n /Perceptual ri q 
139.9956 16.79948 12.1826 -101.5217 258.4109 337.2361 cm 
/Im28 Do Q Qendstream
endobj

The bounding box for the group is given as [ 258.411 235.714 410.589 354.036 ].  Note that there has not been a cm command at the time that the  /GS0 gs command occurred but that there is one ( 139.9956 16.79948 12.1826 -101.5217 258.4109 337.2361 cm ) within the stream for 17 0.  The problem is that the begintransparencymaskgroup does not occur until the /Im28 Do occurs within the stream of 17 0 (verified using -dPDFDEBUG).  At this point, the ctm for the imager state has been concatenated with 139.9956 16.79948 12.1826 -101.5217 258.4109 337.2361.  This matrix is applied to the bounding box when we reach pdf14_begin_transparency_mask using compute_group_device_int_rect(pdev, &rect, pbbox, pis);.  This is not correct, since the matrix in the stream should not affect the bounding box.  This puts us way outside the bounding box for the main buffer and the mask is not used.  

One approach to fixing this is to add a member variable to the image state that stores the ctm related to the current form xobject.  This ctm would then be used as needed to be applied to any bounding boxes for the xobject.  The application of the ctm has be be delayed since otherwise this would cause problems for high level devices.

Looking for some feedback on this issue from Ken, Ray and anyone else who wants to jump in.
Comment 6 Ken Sharp 2010-07-15 07:33:04 UTC
(In reply to comment #4)
 
> One approach to fixing this is to add a member variable to the image state that
> stores the ctm related to the current form xobject.  This ctm would then be
> used as needed to be applied to any bounding boxes for the xobject.  The
> application of the ctm has be be delayed since otherwise this would cause
> problems for high level devices.
> 
> Looking for some feedback on this issue from Ken, Ray and anyone else who wants
> to jump in.

I think I understand the problem. I haven't looked at the code in detail though, and in particular not the data types available at the time, so I may be talking rubbish.

Wouldn't it be possible to add the CTM to the object holding the Bounding Box though ? That is, when the form is defined take a snapshot of the CTM and carry it around along with the Bounding Box. I'm presuming the bounding box is not in the imager state, which may not be true.

Maybe that's what you are suggesting above, but I read it to be the general imager state.
Comment 7 Michael Vrhel 2010-10-04 23:10:12 UTC
OK.  So I have done a bit more work on this and things are a bit more complex than I originally realized.  

The problem is caused when we have PDF content like the following

/GS0 gs 138.009903 16.561203 -12.4208984 103.5074005 271.7055054 234.8406982 cm /Im0 Do

where /GS0 contains a soft mask.

The way the code currently works is that the graphic elements in /GS0 do not occur until /Im0 Do is encountered by the PDF interpreter.  Unfortunately, at that point the graphic states CTM has been altered by the cm command, and so the operations in the soft mask defined by /GS0 do not use the proper CTM.  Note that other graphic state elements could also have been corrupted (as far as the soft mask commands are concerned) in the above scenario if they were set between the /GS0 gs and /Im0 Do commands.

Talking with Ray and thinking a bit more about this, it seems that when a soft mask group is encountered by the PDF interpreter, the current graphic state should really be stored and then later used when the soft mask group is finally processed.  I am not sufficiently familiar with the PDF interpreter (or the greatest PS programmer) to know the best way to do this entirely within the interpreter.  If we choose a C-code approach, the only way I can think to do it is to add another z-command to the transparency code and a graphic state stack to the pdf14 context.  In this approach, when a transparency soft mask group is encountered by the interpreter, the new command is called, and the current graphic state is stored on the new stack.  Later, when the transparency soft mask group is finally processed, the graphic state is set to the one stored on the new stack and then when the soft mask group ends, the graphic state is restored to what is was at the time that the group processing started.  A stack is needed since you could have nested soft mask groups, each of which have this same issue (unlikely to occur but it could be constructed).

I would welcome suggestions or comments on this.
Comment 8 Michael Vrhel 2010-10-05 19:09:56 UTC
Passing off to Alex since this should be fixed in the interpreter.

Alex, You will need to store away the gstate and the extended gstate at the time that /gssmask occurs.  This is the gstate/extended gstate that you will use when the mask is handled. 

When /.execmaskgroup occurs you will need to store away the current gstate and extended gstate and set the gstate and extended gstate to what it was at the time that the gssmask occured.  

When the .endtransparencymask occurs (the one associated with the .begintransparencymaskgroup that occurs in .execmaskgroup) you will need to restore the gstate and extended gstate to what they were at the time that execmaskgroup was first handled.  

Be careful since .endtransparencymask is also called in doimagesmask and we should not have any gstate extended gstate operations occurring for that case.  You may just need to create a new operation to handle the .endtransparencymask associated with .execmaskgroup where you would do the restore and the .endtransparencymask  .
Comment 9 Ray Johnston 2010-10-05 19:23:40 UTC
As far as the requirement to "you will need to restore the gstate and extended
gstate to what they were at the time that execmaskgroup was first handled.",
it seems that this setting of the proper gstate+ExtGState can be done at the
start of the .execmaskgroup at:

/.execmaskgroup {	% <masknum> <paramdict> <formdict> .execmaskgroup -
    % Save our place in PDFfile. Do not use gsave-grestore when creating
    % a soft mask with .begintransparencygroup because high level devices
    % need to modify the graphic state by storing the soft mask ID.
    % Save the ExtGState (//nodict begin) BEFORE changing the colorspace
  mark currentcolor counttomark dup 4 add exch roll pop
  currentcolorspace 4 1 roll .getuseciecolor 4 1 roll //nodict begin

the '//nodict begin' would be replaced with a 'begin' to the saved dict
containing the ExtGState and a 'gsave setgstate' to the gstate that was saved
in gssmask.

Then the restoration of the states can be done at the end of .execmaskgroup:

  .setuseciecolor setcolorspace setcolor end	% restore colorspace, color and ExtGState (end)
} bdef

The 'end' suffices to restore the ExtGState and we just need to add (AFAICT)
a 'grestore'.
Comment 10 Chris Liddell (chrisl) 2010-11-16 20:51:17 UTC
*** Bug 690351 has been marked as a duplicate of this bug. ***
Comment 11 Chris Liddell (chrisl) 2010-11-17 11:41:17 UTC
*** Bug 691348 has been marked as a duplicate of this bug. ***
Comment 12 Chris Liddell (chrisl) 2010-11-24 18:36:59 UTC
*** Bug 690535 has been marked as a duplicate of this bug. ***
Comment 13 Chris Liddell (chrisl) 2010-11-24 19:04:45 UTC
Created attachment 6949 [details]
proposed patch

This patch sets the gstate and (most of) the ExtGState before executing a SoftMask Group content stream.

It doesn't quite set all the ExtGState parameters, but experimentation suggest it sets enough.

It shows cluster differences as follows:

Bug689968.pdf - renders differently after "conversion" thru pdfwrite, the differences are all but invisible.

Bug690022.pdf - renders considerably closer to Acrobat.
Bug690115.pdf - as above
Bug690208.pdf - as above

Bug690535.pdf - resolves the remaining issue with that job

fts_26_2601.pdf - again closer to Acrobat
fts_26_2603.pdf - as above

A few xps->pdf conversions thru pdfwrite render differently, but they were wrong before, and slightly differently wrong now. I believe the PDFs are incorrect.

Unfortunately, it doesn't completely fix this problem (the pictures don't "fade out" around the edges correctly), but it is a big improvement. It does fix the other bugs marked as duplicates of this one.
Comment 14 Chris Liddell (chrisl) 2010-11-24 19:06:19 UTC
Created attachment 6950 [details]
the correct patch.....
Comment 15 Alex Cherepanov 2010-11-26 16:01:59 UTC
Created attachment 6956 [details]
Further simplification

This is a further simplification of the sample file that shows that Ghostscript
cannot set a /SMask in the graphic state using an image with /SMask attribute.
instead, it uses the image and ignores /SMask attribute.
Comment 16 Alex Cherepanov 2010-11-26 19:20:07 UTC
I agree with the patch and recommend to commit it.
Comment 17 Chris Liddell (chrisl) 2010-11-29 14:51:23 UTC
r11922 partially fixes this bug, and fixes Bugs 690351, 691348 and 690535.

I'm opting to create a new bug to track the remaining issue with softmasks as exposed by the jobs for this bug.
Comment 18 Chris Liddell (chrisl) 2010-11-29 14:54:33 UTC
The remaining SMask problem is being tracked under: Bug 691803