Bug 691922 - Slightly wrong colors on RGBW model
Slightly wrong colors on RGBW model
Status: RESOLVED FIXED
Product: Ghostscript
Classification: Unclassified
Component: Color
master
PC Linux
: P4 normal
Assigned To: Michael Vrhel
Bug traffic
:
: 692831 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-02-01 20:12 PST by Oldřich Jedlička
Modified: 2014-02-17 04:42 PST (History)
12 users (show)

See Also:
Customer:
Word Size: ---


Attachments
Input file to generate the raster (414.59 KB, application/octet-stream)
2011-02-01 20:14 PST, Oldřich Jedlička
Details
pstoraster output (856.24 KB, application/gzip)
2011-02-01 20:14 PST, Oldřich Jedlička
Details
Raster file RGBW components saved as bitmap (1.06 MB, application/gzip)
2011-02-01 20:15 PST, Oldřich Jedlička
Details
Script to generate the raster and bitmap output (1.54 KB, application/x-sh)
2011-02-01 20:16 PST, Oldřich Jedlička
Details
Tool to extract bitmaps from RGBW raster file (quickly coded) (320.99 KB, application/gzip)
2011-02-01 20:18 PST, Oldřich Jedlička
Details
pstoraster output (GS 8.71) (871.17 KB, application/x-compressed-tar)
2011-02-02 20:37 PST, Oldřich Jedlička
Details
Raster file RGBW components saved as bitmap (GS 8.71) (737.39 KB, application/x-compressed-tar)
2011-02-02 20:38 PST, Oldřich Jedlička
Details
colorwheel (source, raster, bitmaps) as it goes through cups, GS 8.71 (3.87 MB, application/gzip)
2011-02-02 21:23 PST, Oldřich Jedlička
Details
grayramp (source, raster, bitmaps) as it goes through cups, GS 8.71 (445.95 KB, application/gzip)
2011-02-02 21:24 PST, Oldřich Jedlička
Details
colorwheel (source, raster, bitmaps) as it goes through cups, GS SVN trunk (4.76 MB, application/gzip)
2011-02-02 21:38 PST, Oldřich Jedlička
Details
grayramp (source, raster, bitmaps) as it goes through cups, GS SVN trunk (395.27 KB, application/gzip)
2011-02-02 21:38 PST, Oldřich Jedlička
Details
Print a test page of R G B C M Y and K (4.01 KB, application/postscript)
2011-02-03 21:33 PST, James Cloos
Details
Sample solid colors with RGB values as text (from inkscape) (15.63 KB, application/postscript)
2011-02-13 11:57 PST, Oldřich Jedlička
Details
Tool to extract bitmaps from RGBW raster file (v0.2) (321.18 KB, application/x-gzip)
2011-07-27 19:14 PDT, Oldřich Jedlička
Details
Skin colors are too pink when printing this file (2.95 MB, application/pdf)
2011-07-29 19:56 PDT, Till Kamppeter
Details
The PPD file for the command line of the previous comment (30.88 KB, text/plain)
2011-07-29 21:06 PDT, Till Kamppeter
Details
Script to generate the raster and bitmap output (1.45 KB, text/plain)
2011-08-15 05:49 PDT, Oldřich Jedlička
Details
Input file to generate the raster (414.60 KB, application/octet-stream)
2011-08-15 05:51 PDT, Oldřich Jedlička
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Oldřich Jedlička 2011-02-01 20:12:17 PST
When I print HP test page, the raster looks right at the first sight, but the colors are not completely right. When I translate the RGBW into RGB model with formulas that rasterview and hplip use (R'=R-255+W), there are arithmetic overflows. hplip is actually unusable to do any color printing without patching because of this.

I'm testing this on ghostscript from trunk. It looks like there is some color correction that causes this.
Comment 1 Oldřich Jedlička 2011-02-01 20:14:00 PST
Created attachment 7179 [details]
Input file to generate the raster
Comment 2 Oldřich Jedlička 2011-02-01 20:14:49 PST
Created attachment 7180 [details]
pstoraster output
Comment 3 Oldřich Jedlička 2011-02-01 20:15:36 PST
Created attachment 7181 [details]
Raster file RGBW components saved as bitmap
Comment 4 Oldřich Jedlička 2011-02-01 20:16:27 PST
Created attachment 7182 [details]
Script to generate the raster and bitmap output
Comment 5 Oldřich Jedlička 2011-02-01 20:18:20 PST
Created attachment 7183 [details]
Tool to extract bitmaps from RGBW raster file (quickly coded)
Comment 6 Oldřich Jedlička 2011-02-01 20:20:01 PST
When you look at the bitmaps, you see that the RGB part doesn't use full black and that the Gray Ramp isn't good in the W part.
Comment 7 Till Kamppeter 2011-02-01 21:45:48 PST
Can you supply simpler test input files, for example containing only one (or a few well separated) color square(s) with (a) unique color(s) with corresponding output files and also the information which colors you got and which colors you expected.

Did you write your Raster conversion from scratch or did you derive it from hpcups. If hpcups had a bug you could easily have it copied into your tool in the latter case.

Can you also try Ghostscript 8.71? Does the bug occur there, too?
Comment 8 Oldřich Jedlička 2011-02-02 07:57:42 PST
My tool just extracts the RGB and W part one-to-one, I've written it from scratch. There is no color correction, no calculations, so the bitmaps are exactly the same as in the raster file. This is contrary to the debug output of hplip, which writes the bitmap files after correction.

My personal expectation is that the W part takes the color "darkness" and the RGB part takes the rest. For example the Gray Ramp could be completely white in the RGB part, but should be perfectly visible in the W part.

Just have look at the Gray Ramp with Gimp or so. I would not expect that the darkest color is RGB 71,82,84 and W 30. This looks wrong, the W part should take the full "darkness"; the RGB values after translating from RGBW would be -154,-143,-141. I would expect that RGB is 255,255,255 and W 0.

The negative values cause problems in hplip (it uses unsigned byte values, so the value overflows) when it translates RGBW into RGB model and completely black pixels - I've reported this already. Anyway, fix in hplip would not fix the overflows in the source raster.

Anyway, I will try to generate some simpler test case (gray ramp and some color wheel) with Gimp and with Inkscape to have a clear starting point for image and vector-based drawing.

I will try to attach the raster from older Ghostscript too, but this used to work and there are reports in Gentoo bugs that it works with older version.
Comment 9 Till Kamppeter 2011-02-02 10:05:43 PST
I have looked into the SVN repository and there I have seen that the color conversion code of the CUPS Raster output device of Ghostscript was not touched at all after Nov 11, 2009 (rev 10314), assuming that global variable removal for multi-threading (rev 11135, April 27, 2010).

Michael, is it possible that there are any changes in color handling when the "cups" output device is used between GS 8.71 and 9.x? Are you sure that there are no color management bits breaking anything?

Oldřich, which version of Ghostscript is your "old" Ghostscript where color output was still correct?
Comment 10 Oldřich Jedlička 2011-02-02 12:50:36 PST
I will try the Gentoo's package ghostscript-gpl-8.71-r6 in the evening (something around 7 hours later) and see the result. The major change in 9.00 is the ICC profile support I think. The described color problem looks like some color profile correction.

There were some fixes in this area in January 2011 which improved color handling (some color profiles usage fixes), so it might be that some change to support ICC did this.
Comment 11 Michael Vrhel 2011-02-02 15:18:49 PST
Again,  Please create a very specific example with one or two color squares being filled and tell me what colors they are getting mapped to and what colors you expected to have.
Comment 12 Oldřich Jedlička 2011-02-02 20:37:08 PST
Created attachment 7189 [details]
pstoraster output (GS 8.71)
Comment 13 Oldřich Jedlička 2011-02-02 20:38:03 PST
Created attachment 7190 [details]
Raster file RGBW components saved as bitmap (GS 8.71)
Comment 14 Oldřich Jedlička 2011-02-02 20:41:55 PST
Please have a look at the last two attachments (or at least the bitmaps) - this is how it should look like. The RGB part is almost white, only color wheel and logo are there; the W part contains the rest image. I've used Ghostscript 8.71 from Gentoo package ghostscript-gpl-8.71-r6.
Comment 15 Oldřich Jedlička 2011-02-02 20:43:14 PST
I will create simpler examples too.
Comment 16 Till Kamppeter 2011-02-02 21:18:09 PST
Thanks for the output of GS 8.71. This clearly shows that the problem occurred on the transition from GS 8.71 to GS 9.x. So changes between these versions, probably the introduction of color management, are the culprit. The color conversion code in the "cups" output device did not change during that transition.

But anyway, please attach the simpler examples.
Comment 17 Oldřich Jedlička 2011-02-02 21:23:34 PST
Created attachment 7191 [details]
colorwheel (source, raster, bitmaps) as it goes through cups, GS 8.71
Comment 18 Oldřich Jedlička 2011-02-02 21:24:28 PST
Created attachment 7192 [details]
grayramp (source, raster, bitmaps) as it goes through cups, GS 8.71
Comment 19 Oldřich Jedlička 2011-02-02 21:38:15 PST
Created attachment 7193 [details]
colorwheel (source, raster, bitmaps) as it goes through cups, GS SVN trunk
Comment 20 Oldřich Jedlička 2011-02-02 21:38:48 PST
Created attachment 7194 [details]
grayramp (source, raster, bitmaps) as it goes through cups, GS SVN trunk
Comment 21 Oldřich Jedlička 2011-02-02 21:41:48 PST
I've printed simpler samples (as I don't know how to generate them in a better way) - grayramp and some colorwheel (both generated in Gimp). I've used borderless A4, so the pixels should start on the top-left.
Comment 22 Till Kamppeter 2011-02-02 22:05:36 PST
Can you please create one or more pages with one or more rectangles and each rectangle being of only one solid color? You can do this with GIMP, OpenOffice Draw, Inkscape, ... Then do your tests with these pages.
Comment 23 James Cloos 2011-02-03 18:22:07 PST
This one got me curious.

I had to change the raster-convert src to put height rather than -height into the bmp header.  I presume the posted src tries that to deal with the vertically flipped data, but it broke viewers, et al.  With that change I was able to pipe the bmps into bmptopnm to get something I could analyze.

First, didn't Michael post in another bug that the cupsraster device required device-specific icc profiles for the odd colourspaces (such as RGBW, KCMYcm, et al)?  It looks like trunk creates RGB_ when asked for RGBW (ie 8+8+8 bit RGB in a 32 bit field), leaving the W plane white.

Second, the cups spec says that the W plane should only have values 0 and ~0.
0 where the pixel represents black text and ~0 everywhere else.  It is not spec'ed to be 1-C,1-M,1-Y,1-K. but rather RGB plus a key plate for non-ditherred, non-halftoned black text.  (The spec serendipitously came up on the PWG lists; I happened to read it yesterday.)

Presumably, an RGBW profile would fix this bug.
Comment 24 James Cloos 2011-02-03 21:33:33 PST
Created attachment 7195 [details]
Print a test page of R G B C M Y and K

I used to have a nice bit s postscript for testing these kinds of issues, but could not find it.

I threw this one together, which uses recfill to show DefaultRGB and DefaultCMYK versions of red, green, blue, cyan, magenta, yellow and their half versions.

Please feel free to include it in gs/examples.

I’ll also keep it at:

 http://jhcloos.com/PostScript/rgbcmyk.ps
Comment 25 Till Kamppeter 2011-02-03 21:49:58 PST
The color output of these three commands 

gs -q -dBATCH -dSAFER -dQUIET -dNOPAUSE -sPAPERSIZE=a4 -r300x300 -sDEVICE=cups -dcupsColorSpace=1 -dcupsBitsPerColor=8 -sOutputFile=out-rgb.raster rgbcmyk.ps

gs -q -dBATCH -dSAFER -dQUIET -dNOPAUSE -sPAPERSIZE=a4 -r300x300 -sDEVICE=cups -dcupsColorSpace=17 -dcupsBitsPerColor=8 -sOutputFile=out-rgbw.raster rgbcmyk.ps

and 

gs rgbcmyk.ps

is all different. So I do not really know what the correct one is.
Comment 26 Oldřich Jedlička 2011-02-12 18:59:36 PST
(In reply to comment #23)
> Second, the cups spec says that the W plane should only have values 0 and ~0.
> 0 where the pixel represents black text and ~0 everywhere else.  It is not
> spec'ed to be 1-C,1-M,1-Y,1-K. but rather RGB plus a key plate for
> non-ditherred, non-halftoned black text.  (The spec serendipitously came up on
> the PWG lists; I happened to read it yesterday.)

You are right, I've found it in

  http://www.cups.org/documentation.php/doc-1.4/spec-raster.html

The RGBW is pure sRGB with W channel as 0 and ~0 called "dedicated black text channel". The specification says:

  The white channel is 0 for text (or "true") black, otherwise it must contain the maximum color value: 1 for 1-bit, 3 for 2-bit, 15 for 4-bit, 255 for 8-bit, or 65535 for 16-bit.

There is no word about the RGB values for text ("true" black), but I guess it can be R=0, G=0, B=0, W=0. For other values it should be RGB=real value, W=255.

So my expectations were wrong and the old ghostscript's cups driver didn't do the right job too. Anyway, there is some color correction in place that makes the RGBW colors wrong according to what was originally in ghostscript 8.71.

I will generate some solid color boxes on one page to have some sample data.
Comment 27 Oldřich Jedlička 2011-02-13 11:57:01 PST
Created attachment 7241 [details]
Sample solid colors with RGB values as text (from inkscape)
Comment 28 Oldřich Jedlička 2011-02-13 12:00:21 PST
I've attached sample colors from Inkscape, it was generated in the same way as previous samples (output of pstops from the cups filter).
Comment 29 Oldřich Jedlička 2011-02-13 12:02:49 PST
And the expectation (conforming to CUPS RGBW requirements) is that the RGB raster should match the source colors exactly, the W raster should be black only on the first box (which is black).

Do you need more information from me? Do you need raster file from my system too?
Comment 30 Oldřich Jedlička 2011-02-13 13:41:48 PST
(In reply to comment #23)
> I had to change the raster-convert src to put height rather than -height into
> the bmp header.  I presume the posted src tries that to deal with the
> vertically flipped data, but it broke viewers, et al.  With that change I was
> able to pipe the bmps into bmptopnm to get something I could analyze.

The first page of CUPS raster is stored left-to-right, top-to-bottom. The BMP has bottom-to-top pixel order for positive heights and top-to-bottom for negative heights (read from wikipedia http://en.wikipedia.org/wiki/BMP_file_format). Maybe bmptopnm cannot deal with negative heights or you tried to process second page (back-side for duplex print), which might have different pixel order according to CUPS spec.
Comment 31 Oldřich Jedlička 2011-07-27 19:14:59 PDT
Created attachment 7723 [details]
Tool to extract bitmaps from RGBW raster file (v0.2)

Updated RGBW tool extractor, version 0.2 handles BMP padding.
Comment 32 Till Kamppeter 2011-07-29 19:56:31 PDT
Created attachment 7725 [details]
Skin colors are too pink when printing this file

Run the attached file through this command line, using GS 9.04 RC1:

cat ~/ghostscript/testfiles/CityMap.pdf | PPD=/etc/cups/ppd/OJP8500.ppd gs -dQUIET -dPARANOIDSAFER -dNOPAUSE -dBATCH -dNOINTERPOLATE -sDEVICE=cups -sstdout=%stderr -sOutputFile=%stdout -sMediaType=Plain -sOutputType=0 -dDuplex -r600x600 -dMediaPosition=7 -dDEVICEWIDTHPOINTS=595 -dDEVICEHEIGHTPOINTS=832 -dcupsBitsPerColor=8 -dcupsColorOrder=0 -dcupsColorSpace=17 -dcupsInteger0=26 -scupsPageSizeName=A4 -I/usr/share/cups/fonts -c -f -_ out.raster

and the CUPS Raster file shows too pink skin colors, for example on the rightmost photo on the second page. Note that this is an RGBW color space.

The colors are correct with -dcupsColorSpace=1 (RGB).
Comment 33 Till Kamppeter 2011-07-29 21:04:48 PDT
Sorry, the command line must read

cat launch_leaflet.pdf | PPD=/etc/cups/ppd/OJP8500.ppd gs
-dQUIET -dPARANOIDSAFER -dNOPAUSE -dBATCH -dNOINTERPOLATE -sDEVICE=cups
-sstdout=%stderr -sOutputFile=%stdout -sMediaType=Plain -sOutputType=0 -dDuplex
-r600x600 -dMediaPosition=7 -dDEVICEWIDTHPOINTS=595 -dDEVICEHEIGHTPOINTS=832
-dcupsBitsPerColor=8 -dcupsColorOrder=0 -dcupsColorSpace=17 -dcupsInteger0=26
-scupsPageSizeName=A4 -I/usr/share/cups/fonts -c -f -_ out.raster
Comment 34 Till Kamppeter 2011-07-29 21:06:35 PDT
Created attachment 7726 [details]
The PPD file for the command line of the previous comment
Comment 35 Till Kamppeter 2011-07-29 23:28:34 PDT
Please try adding "-sOutputICCProfile=default_rgb.icc" to the Ghostscript command line. Does it solve the problems for you?
Comment 37 Oldřich Jedlička 2011-08-13 21:18:30 PDT
Finally I've got the latest git running (with modified 9.04 ebuild in Gentoo to work with Git) and the result is completely BW image. The RGB part is completely empty and only the W part contains a image.
Comment 38 Michael Vrhel 2011-08-14 00:43:30 PDT
Til,

Can you check what you get with this?
Comment 39 Till Kamppeter 2011-08-14 07:59:46 PDT
Oldřich, can you post the exact command line and attach the inpur file if it is not already attached? Thanks.
Comment 40 Oldřich Jedlička 2011-08-15 05:48:24 PDT
Confirmed, another reporter in the Gentoo bug has the same problem with 9.04 - BW picture. It is in https://bugs.gentoo.org/show_bug.cgi?id=353454
Comment 41 Oldřich Jedlička 2011-08-15 05:49:27 PDT
Created attachment 7779 [details]
Script to generate the raster and bitmap output
Comment 42 Oldřich Jedlička 2011-08-15 05:51:01 PDT
Created attachment 7780 [details]
Input file to generate the raster
Comment 43 Oldřich Jedlička 2011-08-15 05:55:38 PDT
(In reply to comment #39)
> Oldřich, can you post the exact command line and attach the input file if it is
> not already attached? Thanks.

I've uploaded the latest script and input file to generate the result. It uses gstoraster.bin, because gstoraster is a symlink to the script to catch the output of the filter. It also uses tool to extract bitmaps (v0.2 - raster-convert - attached earlier) to see the resulting RGB and W parts. If you want to run it, just change the gstoraster.bin to gstoraster and use your compiled raster-convert binary (or move it out). It has one argument - input file.
Comment 44 Oldřich Jedlička 2011-08-15 06:00:31 PDT
Just to correct the wording - it is not BW image, it is grayscale in the W part of the RGBW image.
Comment 45 Till Kamppeter 2011-08-15 07:21:22 PDT
I see the problem now. Color pictures come out with the correct color impression. The RGBW produced is simply the projector RGBW (Inverted CMYK, see my conversation (tkamppeter) with mvrhel2 and Robin_Watts on http://ghostscript.com/irclogs/20110729.html, from 17:36 on). HPLIP has no problem with it and does the right thing, but intended is output of CUPS RGBW which behaves like normal RGB in the RGB channnels and in the W channel it sets 0 for all black pixels (R=G=B=0) and the maximum (255 for 1 byte per color) on all non-black pixels (at least one of R, G, B is not 0).
Comment 46 Oldřich Jedlička 2011-08-15 09:51:31 PDT
Yes, this is exactly what the CUPS specification says for CUPS_CSPACE_RGBW as mentioned in comment #26 :-)
Comment 47 Till Kamppeter 2011-08-15 11:01:19 PDT
HPLIP supports both CUPS RGBW and projector RGBW and sets the true black flag for each pixel correctly in both cases. Also the color reproduction on both rasterview and the actual printer is correct.

It is more a problem that other printer manufacturers or driver developers could expect that Ghostscript sends CUPS RGBW and make there drivers supporting only CUPS RGBW getting wrong output from the projector RGBW which Ghostscript is sending.
Comment 48 Till Kamppeter 2011-08-15 12:21:23 PDT
Fixed in GIT rev. eb6b63157d90.

Now the CUPS Raster output device produces RGBW as defined in the CUPS documentation, comment #26,

I have tested also the color reproduction with rasterview and a real HP inkjet printer and the colors did not change.
Comment 49 Oldřich Jedlička 2011-08-15 13:12:44 PDT
Till, I still have it grayscale. The W part looks great, but the RGB part is without any color. I've used the same script/input file as before and ebuild for the Git version as mentioned in comment #37.
Comment 50 Till Kamppeter 2011-08-15 22:05:45 PDT
Fixed in GIT rev. 1920f21e245e.

What was missing was correctly setting the default output profile for all color spaces supported by the CUPS Raster output device. The patch does this now. User-supplied output profiles via "-sOutputICCProfile=..." are supported.
Comment 51 Till Kamppeter 2011-08-16 00:40:09 PDT
Please update to GIT rev d8da050fcfc. This improves on the recognition of black pixels on the CMYK -> RGBW conversion. Now all should work correctly.
Comment 52 Oldřich Jedlička 2011-08-16 05:50:40 PDT
I works for me. Many thanks :-)
Comment 53 Chris Liddell (chrisl) 2011-08-22 07:22:50 PDT
This was finally fixed properly in:
http://git.ghostscript.com/?p=ghostpdl.git;a=commit;h=781b73
Comment 54 Till Kamppeter 2012-02-05 19:35:50 PST
*** Bug 692831 has been marked as a duplicate of this bug. ***