I have a strange problem with gradations in certain cases. The attached PDF file contains a single radial gradation, going from a pure magenta to a pure white. When I rip this file at 60 dpi with my CMYK device (and with my custom version of GS), and with Color Management ON (meaning that CMYK colors are converted to an output ICC profile, in the procedure registered by get_color_mapping_procs), the result is wrong. I get a kind of triangulation effect, see the attached file "bad.tif". Now, when I disable Color management (i.e. when the CMYK mapping proc does not modify the colors), then the result is correct (see the attached file "noicc.tif") - and identical to what tiff32nc produces. This problems happens with both GS8.53 and GS8.60, but fortunately it's very rare. It depends of the file, of the resolution, and of the profiles used (e.g. if I rip the same file at 600 dpi instead of 60, then there's no problem). I've seen it 3 or 4 times in the last months. Of course, that could be a problem in my ICC color conversions, but for many reasons I think that's not the case. For example, when I apply the same ICC conversions in post-processing (i.e to the file "noicc.tif"), I obtain the correct result (attached file "ok.tif"). This is the one I should get directly from GS, normally. Now, I have tried to reproduce the problem with tiff32nc in the standard GS, by modifying it's mapping proc procedures, and making them return the same CMYK values than in my test case. But that doesn't work. I have no idea why, but with tiff32nc the CMYK mapping proc is not called for the colors of the gradation (not even for the pure magenta). I've even tried to modify the default function (cmyk_cs_to_cmyk_cm(), in gxcmap.c), but with no more success. So I hope you'll have an idea of what the problem could be by examining the attached files. Otherwise, I can retry with tiff32nc if you explain me why the color conversions are not called.
Created attachment 4203 [details] SimpleGradMag.pdf
Created attachment 4204 [details] noicc.tif
Created attachment 4205 [details] ok.tif
Created attachment 4206 [details] bad.tif
Additional information from the customer (including a command line, which should satisfy Henry): First, I've found the reason why I couldn't reproduce the problem with tiff32nc: it's because the gradation, in the file, is in a DeviceN colorspace, having only one component, "Magenta". And since tiff32nc uses the default get_color_comp_index procedure (gx_default_DevCMYK_get_color_comp_index), it identifies "Magenta" as one of it's colorant, so it uses gx_concretize_DeviceN(), and then finally cmap_devicen_direct(), to map all colors of the gradations. In my first attempt, I had tried to redefine the mapping proc CMYK->CMYK of tiff32nc, because I thought that the file was pure CMYK. God knows why, when you have a gradation in Illlusrator CS3 with only process CMYK colors, it always create the gradation in a DeviceN colorspace, when the output is PDF. Now, the reason our CMYK devices works differently than tiff32nc is that our get_comp_color_index() always return -1 for process inks. And the reason for that is, precisely, to force DeviceN colorspaces to be processed like DeviceCMYK, so that our mapping procs (which make an ICC conversion) are always used. Well, stricly speaking, the initial motivation was to disable overprint on process colors (that was a suggestion of Dan Coby a few years ago). I found out later that this has this rather happy side effect, too. Anyway, I finally could reproduce the initial problem with tiff32nc, by: a) redefining get_color_comp_index() to always return -1. b) redefining the cmyk mapping proc in such a way that it returns exactly the same values than my own device (with a look-up table). I attach the modified gdevtsep.c (my modifications are in #ifdef DO_MY_CMYK), for version 8.62. To produce the buggy TIFF, use the following command: gs -sDEVICE=tiff32nc -sOutputFile=a.tif -r60 -f SimpleGradMag.pdf (I attach again the problematic PDF file). As far as I can see, the problem is in gx_cspace_is_linear_in_triangle(). At first glance, it wrongly decides, at a certain point, that no more subdivisions are needed (besides, when I force gx_cspace_is_linear_default() to always return 0, I immediately get rid of the problem). However, I'm not so sure because if that was the case, I should normally end with a wrong gradation (wrong colors), but at least a smooth one. So maybe there is something else. Now, the real reason of the problem might be that our device declares itself as "separable and linear" whereas, in fact, it's not linear (ICC conversions, especially with digital printers, are almost never linear). But if that's the case then our device is buggy since the beginning. And I think that when tiff32nc processes an RGB gradation, it doesn't always make a strictly linear RGB->CMYK conversion. So maybe this bug is potentially present with the standard GS devices.
Created attachment 4207 [details] gdevtsep.c
I can replicate the problem with the tiff32nc device using the modified code that the customer provided. I am looking a bit more at this. My intuition is telling me that the customer's comment about there being an assumption of linearity when there it really is not, is going to be the issue, as the output shows every sign of this.
Created attachment 4265 [details] Bug689961_patch.diff I believe this should fix the issue. There was a problem with the testing to determine if a color space mapping was linear within a triangle and along a line. The old code assumed the number of components was the same in the client and device color space. In general this is not true, especially if we have a DeviceN color space with one color and a CMYK output device, which was the test case given.
The customer reports the patch in comment #8 does fix the problem. Please commit.