Bug 690477 - Regression: Indeterminism since soft mask branch
Summary: Regression: Indeterminism since soft mask branch
Status: RESOLVED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: Color (show other bugs)
Version: master
Hardware: All All
: P2 normal
Assignee: Michael Vrhel
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-05-11 15:22 UTC by Marcos H. Woehrmann
Modified: 2009-10-26 21:40 UTC (History)
0 users

See Also:
Customer:
Word Size: ---


Attachments
Bug688631j.pdf (708.14 KB, application/pdf)
2009-05-11 15:22 UTC, Marcos H. Woehrmann
Details
valgrind.log (23.09 KB, text/plain)
2009-05-14 15:51 UTC, Marcos H. Woehrmann
Details
valgrind.log (21.70 KB, text/plain)
2009-05-14 22:43 UTC, Marcos H. Woehrmann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Marcos H. Woehrmann 2009-05-11 15:22:03 UTC
One of the nightly regression files has been changing md5sums the last few nights in response to 
changes in the code that shouldn't have any affect on this file.  The same portion of the output that is 
now behaving oddly also changed when the soft-mask branch merge happened, so I suspect it's 
related to that.

I've isolated the portion of the file that is changing behavior, the original file is the one attached to Bug 688631.

The command line that demonstrates the problem:

  bin/gs  -sDEVICE=pgmraw -o test.pgm -r72 Bug688631j.pdf

Here are some md5sum for recent revisions:

marcos@macbookpro:[200]% md5sum *pgm
MD5 (9721.pgm) = d75e0334a7d246262ab564b40ba0afd8
MD5 (9722.pgm) = 0ec5dc2c37d131e3b2fc85678e38b288
MD5 (9732.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9736.pgm) = d75e0334a7d246262ab564b40ba0afd8
Comment 1 Marcos H. Woehrmann 2009-05-11 15:22:27 UTC
Created attachment 5007 [details]
Bug688631j.pdf
Comment 2 Marcos H. Woehrmann 2009-05-11 17:12:44 UTC
This issue may be limited to Mac OS X and/or 32 bit machines.

On my x86_64 linux box the md5sum is more consistent:

7f3f03cc3e4fd2fc17bc3e9580e128ca  9721.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9722.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9723.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9724.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9725.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9726.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9727.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9728.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9729.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9731.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9732.pgm
7f3f03cc3e4fd2fc17bc3e9580e128ca  9733.pgm
759fb8be094cf68461b1ecea21f36b0a  9734.pgm
759fb8be094cf68461b1ecea21f36b0a  9735.pgm
759fb8be094cf68461b1ecea21f36b0a  9736.pgm
759fb8be094cf68461b1ecea21f36b0a  9737.pgm


my iMac Core 2 Duo:

MD5 (9721.pgm) = 0ec5dc2c37d131e3b2fc85678e38b288
MD5 (9722.pgm) = baa0caa4ee51adcfc1b608746796c3a4
MD5 (9723.pgm) = baa0caa4ee51adcfc1b608746796c3a4
MD5 (9724.pgm) = baa0caa4ee51adcfc1b608746796c3a4
MD5 (9725.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9726.pgm) = deadb7c55cb296afc382be0251920682
MD5 (9727.pgm) = deadb7c55cb296afc382be0251920682
MD5 (9728.pgm) = deadb7c55cb296afc382be0251920682
MD5 (9729.pgm) = deadb7c55cb296afc382be0251920682
MD5 (9730.pgm) = deadb7c55cb296afc382be0251920682
MD5 (9731.pgm) = deadb7c55cb296afc382be0251920682
MD5 (9732.pgm) = deadb7c55cb296afc382be0251920682
MD5 (9733.pgm) = e3ecdb645842b0442770aa2442a8c002
MD5 (9734.pgm) = 0ec5dc2c37d131e3b2fc85678e38b288
MD5 (9735.pgm) = 0ec5dc2c37d131e3b2fc85678e38b288
MD5 (9736.pgm) = 0ec5dc2c37d131e3b2fc85678e38b288
MD5 (9737.pgm) = 5b7d137d828a5bfe92188cecb2ebebab


and finally on my MacBook Pro Core 2 Duo:

MD5 (9721.pgm) = d75e0334a7d246262ab564b40ba0afd8
MD5 (9722.pgm) = 0ec5dc2c37d131e3b2fc85678e38b288
MD5 (9723.pgm) = 0ec5dc2c37d131e3b2fc85678e38b288
MD5 (9724.pgm) = 0ec5dc2c37d131e3b2fc85678e38b288
MD5 (9725.pgm) = baa0caa4ee51adcfc1b608746796c3a4
MD5 (9726.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9727.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9728.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9729.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9730.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9731.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9732.pgm) = 1f0e8f553cd970f9a7e809f59cbb006a
MD5 (9733.pgm) = b5b94bfb68ba6f2d46c3b1a7644736ba
MD5 (9734.pgm) = d75e0334a7d246262ab564b40ba0afd8
MD5 (9735.pgm) = d75e0334a7d246262ab564b40ba0afd8
MD5 (9736.pgm) = d75e0334a7d246262ab564b40ba0afd8
Comment 4 Marcos H. Woehrmann 2009-05-14 10:10:38 UTC
This issue appears to not fail with the debug build; I'm investigating.
Comment 5 Marcos H. Woehrmann 2009-05-14 15:51:30 UTC
Created attachment 5017 [details]
valgrind.log

The valgrind log file for the -O2 build is attached.
Comment 6 Marcos H. Woehrmann 2009-05-14 22:39:15 UTC
The problem does appear to be reading uninitialized memory.  I instrumented art_blend_luminosity_rgb_8():

void
art_blend_luminosity_rgb_8(int n_chan, byte *dst, const byte *backdrop,
                           const byte *src)
{
    int rb = backdrop[0], gb = backdrop[1], bb = backdrop[2];
    int rs = src[0], gs = src[1], bs = src[2];
    int delta_y;
    int r, g, b;

    /*
     * From section 7.4 of the PDF 1.5 specification, for RGB, the luminosity
     * is:  Y = 0.30 R + 0.59 G + 0.11 B)
     */
    delta_y = ((rs - rb) * 77 + (gs - gb) * 151 + (bs - bb) * 28 + 0x80) >> 8;
    r = rb + delta_y;
    g = gb + delta_y;
    b = bb + delta_y;
if_debug3('q', "[q]art_blend_luminosity_rgb_8 %3d %3d %3d\n",rb,gb,bb);
if_debug3('q', "[q]art_blend_luminosity_rgb_8 %3d %3d %3d\n",rs,gs,bs);
if_debug4('q', "[q]art_blend_luminosity_rgb_8 %3d %3d %3d %8d\n",r,g,b,delta_y);
    if ((r | g | b) & 0x100) {
        int y;
        int scale;
.
.
.

With the command line:

  debugobj/gs -sDEVICE=pgmraw -o test.pgm -r72 -Zq ./Bug688631j.pdf

the debugging output changes depending on if the code is compiled with -O0 or -O2.  

With -O0:

[q]art_blend_luminosity_rgb_8   0 255 191
[q]art_blend_luminosity_rgb_8 203 254 152
[q]art_blend_luminosity_rgb_8  56 311 247       56
[q]art_blend_luminosity_rgb_8   0 255 191
[q]art_blend_luminosity_rgb_8 200 254 152
[q]art_blend_luminosity_rgb_8  55 310 246       55
.
.
.

With -O2:

[q]art_blend_luminosity_rgb_8   0 255   0
[q]art_blend_luminosity_rgb_8 203 254   0
[q]art_blend_luminosity_rgb_8  60 315  60       60
[q]art_blend_luminosity_rgb_8   0 255   0
[q]art_blend_luminosity_rgb_8 200 254   0
[q]art_blend_luminosity_rgb_8  60 315  60       60
.
.
.

Replacing -sDEVICE=pgmraw with ppmraw eliminates the differences in the debugging output.
Comment 7 Marcos H. Woehrmann 2009-05-14 22:43:21 UTC
Created attachment 5018 [details]
valgrind.log

Valgrind log file from my linux system; reports the same problems as the
previous valgrind.log but includes filenames and lines numbers.
Comment 8 Henry Stiles 2009-05-30 16:34:53 UTC
I don't know if this related but -m64 breaks many files for me.  Ralph pointed
out I should use ./configure CFLAGS="-arch x86" without -m64 and that worked.
Comment 9 Ralph Giles 2009-05-30 17:11:38 UTC
That should be CFLAGS="-arch x86_64" for a 64-bit build, of course.
Comment 10 Marcos H. Woehrmann 2009-06-13 14:47:56 UTC
This is not a Mac OS X only problem, it also occurs on peeves with a 64 bit build
Comment 11 Marcos H. Woehrmann 2009-07-25 13:01:37 UTC
I've disabled the Bug688631.pdf file from the nightly regressions pending resolution of this issue.
Comment 12 Michael Vrhel 2009-10-26 18:14:14 UTC
I see the issue with this.  The gray_blending_procs are set to be the rgb 
blending type.  In fact there are no blending procs for gray due to the fact 
that gray transparency was always handled as RGB previously.  I will write 
some gray blending procs to fix this problem.   Thanks to Marcos for being so 
detailed in the problem description.
Comment 13 Michael Vrhel 2009-10-26 21:40:35 UTC
Should be fixed with rev 10228.