Bug 697585

Passing to ray to have a look.

Created attachment 13449 [details]
significantly simplified file

The simplified file contains (object 23 is the content stream) a rectangle drawn in RGB which is a shade of blue and an image (object 15 also in RGB) with a soft mask (object 14).

The mswinpr2 device (but not other RGB devices such as display or tiff24nc) renders the blue rectangle as brown.

Removing the SMask results in correct output, so this is definitely a transparency blending problem.

So the band height has an effect on this file.   

Running Ken's simplified file 

-sDEVICE=mswinpr2 -dMaxBitmap=1500000000  -Zv -dBandHeight=125 -dPrinted -dNoCancel -dBitsPerPixel=24 ./myinputs/mswinpr2_issue.pdf

and printing out to an Adobe PDF "printer"

I see the blue strip come out yellow.

Using a larger band size (e.g.)

-sDEVICE=mswinpr2 -dMaxBitmap=1500000000  -Zv -dBandHeight=250 -dPrinted -dNoCancel -dBitsPerPixel=24 ./myinputs/mswinpr2_issue.pdf

it comes out the proper blue  (0b 74 bf)

During interpretation, there is a pdf14 device push (as the page uses transparency), the blue rectangle fill occurs first and goes through gx_fill_path, pdf14_clist_fill_path, clist_fill_path.  In clist_fill_path, cdev->pdf14_needed is false as there have not been any group or softmask pushes at this point yet (nor any odd blend modes nor non unity alpha values).  So, the transparency bounding box maintained by the clist is not updated to include this fill.  This fill occurs at band 9 (the Adobe printer is 1200dpi).

There is then a transparency group push and sofmask for bands 2-9 which is the image at the top of the rect fill.

During clist playback, of band 9, the rectfill command occurs and can be caught by catching cmd_opv_closepath in clist_decode_segment or gx_fill_path_only in clist_playback_band.  This ends up going not the the pdf14_fill_path (since no pdf14 device was pushed for the band -- I am not sure why the pdf14 device is not getting pushed as the group and mask push were supposed to have occurred for band 9, they must be getting optimized away) but through the default and general fill path logic.   The device color at this point is the correct color and I watched the mem24 device get filled with the blue color.  I set a data break point to see if it changed to yellow at some point, but did not see anything.   Interestingly, if during interpretation, if I force cdev->pdf14_needed to true during the rectangle fill the proper blue color occurs. 

At this point, I am very puzzled and discussing with Robin to have a look.

Created attachment 13453 [details]
reduced3.pdf

Slightly tweaked simplified file.

Handing back to Ray... per IRC discusion

After much discussion between Michael, Ray and myself, we think we understand the problem.

The irc logs for #artifex on 20170303 from 18:30 onwards may be helpful, but I will try to summarise it here.

During clist writing, the colors are encoded into the clist using pdf14_encode_color, rather than the target devices encodings. If we subsequently decide that transparency is not required for a band, we then run without a compositor in place, and we treat those encoded values as if they had been encoded using the target devices encoding.

Now, it just so happens that a lot of devices use a 'standard' encoding that matches that of pdf14, so we get away with it.

The mswinpr2 device needs to get BGR out rather than RGB, hence the colors are reversed for 'skipped' bands.

The suggestion to solve this is to extend the separable_and_linear enumeration, currently:

typedef enum {
    GX_CINFO_UNKNOWN_SEP_LIN = -1,
    GX_CINFO_SEP_LIN_NONE = 0,
    GX_CINFO_SEP_LIN
} gx_color_enc_sep_lin_t;

to be something like:

typedef enum {
    GX_CINFO_UNKNOWN_SEP_LIN = -1,
    GX_CINFO_SEP_LIN_NONE = 0,
    GX_CINFO_SEP_LIN,
    GX_CINFO_SEP_LIN_NONSTANDARD,
    GX_CINFO_SEP_LIN_STANDARD
} gx_color_enc_sep_lin_t;

If we know that a device is not separable and linear, then it cannot be using a compatible encoding, therefore we cannot skip transparency for any bands.

If we know that a device is separable and linear, we can test all 768 values to see if it is compatible. If it is, then we safely skip transparency.

If a device is smart enough to set either GX_CINFO_SEP_LIN_NONSTANDARD or GX_CINFO_SEP_LIN_STANDARD to start with, this testing can be avoided too.

I'm going to have a crack at this.

Just a few minor comments -- there may be more than 768 tests if the pdf14
device has more than 3 components with 8-bits each. The pdf14 device only
handles 8-bit per component (currently) but it does support other target
device types including cmyk and cmykspot. The cmyk device has 4 components
so it would have 1024 evaluations (not bad). The spot color devices will use
the devn values, so the encoded color -> color_index value are probably not
relevant.

Good luck, Robin.

Fix committed as:

commit 64f5d62c80176085335d84c24e02cd49d9e032b9
Author: Robin Watts <robin.watts@artifex.com>
Date:   Fri Mar 3 19:42:29 2017 +0000

    Bug 697545: Extend "separable_and_linear" processing.

    First, avoid needless check_separable_and_linear tests.

    When we make a memory store for a target device, we use
    the encoding/decoding color functions from that target
    device. It seems reasonable to copy the "separable_and_linear"
    information from that device too, to avoid having to needlessly
    retest using the same encoding/decoding functions to get
    the same answer.

    Next, extend gx_color_enc_sep_lin_t to represent not just
    whether we know details about the device colors separability
    and linearity (and what those details are if known), but also
    to encode whether those are standard encodings or not. The
    standard encoding is defined to be "that which is compatible
    with the pdf14 compositor".

    Finally, we add a function, check_device_separable_encoding,
    that 'promotes' the record for a given space from being a
    known separable_and_linear one, to one that knows it's
    separable_and_linear, but also knows whether it uses the
    standard encoding.

    Call this in clist_close_writer_and_init_reader to ensure that
    the buffer device is created knowing whether it's a standard
    encoding or not. If called here, it only ever needs to run once.

    Call it again in pdf14_ok_to_optimize (just in case we ever slip
    through - this takes no time if it has been run already). This
    enables us to make a correct choice, and to avoid skipping
    transparency bands if we are not using the standard (pdf14
    compositor compatible) encoding.