Various of the CET test files have changed rendering when unrelated bugs have been fixed. Either we are experiencing strange side-effects or the CET files are badly constructed (or both). Here is a partial list of the files and the revision responsible for the change on my amd64 linux box: 99-01.PS r9596 28-06.PS r9442 99-02.PS r9442 30-11.PS r9512 31-01.PS r9438 Here is the command line I'm using: bin/gs -sOutputFile=test.pgm -Ilib -I/home/marcos/artifex/fonts/ -sDEVICE=pgmraw -r300 -q -Z: -dNOPAUSE -dBATCH -K1000000 -dMaxBitmap=30000000 -dNOOUTERSAVE -dJOBSERVER -c false 0 startjob pop -f %rom%Resource/Init/gs_cet.ps - < ../99-01.PS
Created attachment 4876 [details] page 1 of 99-01.PS rendered with Ghostscript r9595.jpg
Created attachment 4877 [details] page 1 of 99-01.PS rendered with Ghostscript r9596.jpg
99-01.PS depends on the dictionary enumeration order. Perhaps, we need to modify the test and print the keys in the ASCIIbetical order.
At least in cet mode?
I found some more recent revisions that changed the same CET files, these also occur on my iMac so they are not system dependent: 99-02.PS r9596 28-06.PS r9593 31-01.PS r9596 This 30-11.PS file does not behave the same on the AMD64 linux box and the iMac. On the iMac r9599 and r9600 generate identical output, on the AMD64 linux box r9599 and r9600 differ.
Ralph, I suggest to modify the test file, not Ghostscript. Dependence on the enumeration order is a bug and, being good netizens, we need to fix it an send the patch upstream.
Created attachment 4878 [details] Patch for 99-01.PS And here's the patch for 99-01.PS 99-02.PS enumerates the dictionary returned by currentsystemparams and needs a similar patch.
28-06.PS also depends on the dictionary enumeration order. Sorting can be added to the test but it will obscure implementation details of the operator forall exercised by the test. 30-11-2 GS prints different numbers for every test in GLOBINT. The number printed is the VM usage for the test. I don't think anything can be done about it. 31-01-8 defines 2 or more idioms that match the target and expects the last definition to be used. PLRM (p. 121) says that the order is undefined. Ghostscript can be fixed to match the test if we really want it.
Alex, when I last looked at this (July 2008) I found the following: 09-56.PS - resourceforall 28-06.PS - forall 30-11.PS - forall 99-01.PS - forall 99-02.PS - forall All these were affected by the enumeration order of dictionaries. I agree completely that expecting the enumeration to be fixed is a bug in the test, but these are the Quality Logic (Genoa) test files, which customers may also run. If we modify them we want to remember that these are not the 'standard' files, maybe we should rename them (eg 99-01-modified.ps). Alternatively we can modify the forall operator behaviour in gs_cet.ps so that it sorts the dictionaries before reporting them. Unfortunately the QL tests' forall procedure argument leaves stuff on the stack, so this is kind of difficult. I also found these other problems: 08-04.PS - bar graph for memory usage, date and time from %Calendar% 23-32.PS - SubFileDecode error 31-01.PS - order idiom recognition finds idioms 31-07.PS - usertime used to draw a line 31-12.PS - usertime used to draw a line I think we already removed the 'usertime' files from the test suite and I never did get to the bottom of the SubFileDecode error (it may be gone by now). 08-04.ps uses vmstatus to get the memory and draw a graph, which is indeterminate, as well as using the %Calendar% device which obviously returns different values. Note that all these files are indeterminate for good reasons, unlike some of the other files which seem to be indeterminate for no readily apparent reason, usually tiny text differences.
On the argument that the issue is our regression testing, not that CET results are indeterminant--as this is generally known--I recommend fixing the files, and storing them under a -modified.ps filename as Ken suggested. That is, modify the tests to sort results before rendering them. We should report this issue to Quality Logic as well; there are other places in the tests where they do bother to sort so it may be an oversight.
This is a full list of differences in CET files reported in March. In many cases the cause is unknown and the differences are difficult to reproduce on a single box. Marcos, please help. Resource enumeration order - fixed by the attached patch 09-56.PS (pbmraw, pkmraw, ppmraw, 600dpi) Unknown cause 13-03.PS (pbmraw, pkmraw, ppmraw, 600dpi) 13-12.PS (pbmraw, pkmraw, ppmraw, 600dpi) 15-06.PS (pkmraw 600dpi) 16-06.PS (pbmraw, pkmraw, ppmraw, 600dpi) 23-13.PS (pkmraw, ppmraw 600dpi) 23-25.PS (pbmraw 600dpi) Arbitrary device selection 27-03.PS (pkmraw, ppmraw 600dpi) Unknown cause 27-05.PS (pbmraw, pkmraw, ppmraw 600dpi) 27-08.PS (pbmraw, pkmraw, ppmraw 600dpi) Directory enumeration order, fixed 28-06.PS (pbmraw, pkmraw, ppmraw 600dpi) Idiom selection order, I don't see a good fix for Ghostscript or test file. The affected parts of the test can be disabled. 31-01.PS (pbmraw, pkmraw, ppmraw 600dpi) Directory enumeration order, fixed 99-01.PS (pbmraw, pkmraw, ppmraw 600dpi) 99-02.PS (pbmraw, pkmraw, ppmraw 600dpi)
Created attachment 4900 [details] patch Patch for 99-01.PS, 99-02.PS, 09-56.PS
I've disabled those files that show an indeterminism and for which we don't have a fix pending (i.e. all except 31-01.PS, 99-01.PS, 99-02.PS, 09-56.PS). Please re-enable files as they are repaired.
I missed 30-11.PS but have now disabled it as well (it was listed in comment #9).
We have other indeterminisms, but these files are no longer an issue.