Created attachment 9724 [details] the "newer" pcl Some HP deskjet and Photosmart printers uses a raster compression scheme called Mode 10 for raster images. The attached is supposed to be just the usual tiger.
PageTech's PCL Reader can read the attached sample file correctly. Various sources indicate that one of the compression modes used by the CLJ3500/3550/3600 (bug 693661) is derived from/related to and a simplified version of mode 10 compression.
Here is the beginning of being ale to interprete that file correctly - pcl6 needs to add "ESC * g <nbytes> W" . This command sets compression mode, color mode and other info per component. More sophisticated ways of doing "ESC * r <byte> U" and friends. ------------------------------------------- diff --git a/pcl/pccid.c b/pcl/pccid.c index 01cc6a9..32bb41b 100644 --- a/pcl/pccid.c +++ b/pcl/pccid.c @@ -477,6 +477,36 @@ set_simple_color_mode(int type, pcl_state_t * pcs) } /* + * ESC * g <nbytes> W + * + * This command sets compression mode, color mode and other info per component. + * Repaces "ESC * r <byte> U" and friends. + */ +static int +pcl_configure_raster_data(pcl_args_t * pargs, pcl_state_t * pcs) +{ + int len = uint_arg(pargs); + byte *data = arg_data(pargs); + + dprintf1("Format:%d\n", data[0]); + dprintf1("ID:%d\n", data[1]); /* =31 for KRGB */ + dprintf1("#Components:%d\n", (data[2] << 8 | data[3])); + dprintf1("X Resolution:%d\n", (data[4] << 8 | data[5])); + dprintf1("Y Resolution:%d\n", (data[6] << 8 | data[7])); + dprintf1("Compression Mode:%d\n", data[8]); /* 9 */ + dprintf1("Orientation:%d\n", data[9]); + dprintf1("Bits:%d\n", data[10]); + dprintf1("Planes:%d\n", data[11]); + dprintf1("X Resolution:%d\n", (data[12] << 8 | data[13])); + dprintf1("Y Resolution:%d\n", (data[14] << 8 | data[15])); + dprintf1("Compression Mode:%d\n", data[16]); /* 10 */ + dprintf1("Orientation:%d\n", data[17]); + dprintf1("Bits:%d\n", data[18]); + dprintf1("Planes:%d\n", data[19]); + return 0; +} + +/* * ESC * v <nbytes> W * * This command creates only the basic element of the the palette: the cid_data @@ -597,6 +627,10 @@ pcl_cid_do_registration(pcl_parser_state_t * pcl_parser_state, gs_memory_t * pmem) { DEFINE_CLASS('*') { + 'g', 'W', + PCL_COMMAND("Configure Raster Data", pcl_configure_raster_data, + pca_bytes | pca_in_rtl | pca_raster_graphics) + }, { 'v', 'W', PCL_COMMAND("Configure Image Data", pcl_configure_image_data, pca_bytes | pca_in_rtl | pca_raster_graphics) ------------------------
The newer bug 694950 probably is "simplier", since it used the older set compression command.
Partial list of printers in this class: DeskJet 96x series, 98x series, 99x series, deskjet 6122, deskjet 6127, PHOTOSMART 1115, 1215 1218 1315, HP BUSINESS INKJET 2200, 2230, 2250, 2280 HP Color Inkjet CP1160 CP1700, deskjet 450 officejet d series offjetjet 7100 series
depend on Bug 694989
Marking this bountiable. I'm unclear if the code in comment #2 is a complete solution. I'm assuming it is not. If nobody is interested in working on this then we'll probably close it.
Hmm, it seems that I have a patch dated backed to Feb 2014 for a complete mode 10 compression itself, but I think it was missing some other pieces like https://bugs.ghostscript.com/show_bug.cgi?id=694989 to make it useful for any real world test files. I think somebody else submitted a patch for bug 694989 or related area in the last year or so.
(In reply to Hin-Tak Leung from comment #7) > Hmm, it seems that I have a patch dated backed to Feb 2014 for a complete > mode 10 compression itself, but I think it was missing some other pieces > like https://bugs.ghostscript.com/show_bug.cgi?id=694989 to make it useful > for any real world test files. Bug #694989 just wants to add 'ESC * g <nbytes> W', which I was under the impression was included in the code in comment #1. > I think somebody else submitted a patch for bug 694989 or related area in > the last year or so. The code here is the only patch I have seen in this area, and I went through all the outstanding PCL bugs. It is quite possible I missed something, not being entirely conversant with PCL but if a complete treatment exists I did not find it. If you can locate such a thing then I will try and track down the author. There is a supplied patch for KCMY colour space handling and I am in communication with the author of that patch. That's the only code I've seen being offered recently (bug 695047) and that's from March 2022 so slightly over a year but it's close enough in time, and I've seen no others.
Yes, I think bug 695047 is somewhat related. It talks about mode 9 compression. I think I have some work-in-progress diff which collides/co-incide with bug 695047 too - meant to look at that submitted patch for a while now.
(In reply to Hin-Tak Leung from comment #9) > Yes, I think bug 695047 is somewhat related. It talks about mode 9 > compression. I think I have some work-in-progress diff which > collides/co-incide with bug 695047 too - meant to look at that submitted > patch for a while now. I don't want to look at the code until we have a Contrubuter Licence Agreement, which I'm hoping to receive soon. All being well I will then apply it (fixing any problems with the code having moved on) and regression test it. If that passes then I'll push it into master.
I have looked at that patch but not actually build it. My first impression is that it probably will do what it claims to do for that test files, but break a lot of other/older files... i.e. it needs a bit more polish. Anyway, it has been 10 years, so I'll need to do a bit of digging as to why I stopped.
Hin-Tak, bug 695047 does not directly relate to compression. It addresses 4 plane CMYK simple color raster graphics that HP added to the DeskJet 550C and later PCL3 family of printers. HP later added Mode 9 compression (compressed replacement delta row encoding) which is documented in detail in "HP DeskJet 600, 700, 800, 900 & HP 2000 Series PCL Developer’s Guide". Mode 10 was added later, but I haven't been able to locate technical docs on it. All I know is that it is apparently the primary format that modern raster-only "PCL3GUI" DeskJets and DesignJets understand. None of the above was supported by PCL5c printers. Later HP DesignJets support it via RTL commands or PCL3GUI in the case of lower end units. For some reason the inkjets mostly stayed on PCL3 and were expanded in an incompatible way with PCL5c laser printers (and the handful of high end DeskJets that supported it).
Created attachment 24349 [details] compression mode 10 addition I think this code actually fixes "Bug 694950 - set_compression_method(): unsupported mode 10" .
okay, I think I have got to a clear story. I had a working mode 10 a long time ago. It works/worked with the test file in bug 694950, according to comment 694950#c6 . However, to get it to work with this particular test file, it needs "ESC * g <nbytes> W", which quoting myself in the other comment, ``` The other one (bug 694082) requires implementing a newer set compression method command; also, the incoming data is supposed to be of KRGB, with a mode-9 black plane and a mode-10 RGB plane. There doesn't seem to be existing code for coping with KRGB, or recombining two planes of different depth/color?` ``` The KRGB mode touches the same areas/code-path as the 4-plane CMYK code and quite related. The mode 10 code is correct I believe, and should work on the (private) test file in 694950 . Currently this test file segfaults in that code path: #0 0x00000000008ebb30 in set_dev_specific_default_palette () #1 0x00000000008ebc1e in set_default_entries () #2 0x00000000008ec54d in pcl_cs_indexed_set_num_entries () #3 0x00000000008ec95e in pcl_cs_indexed_build_cspace () #4 0x00000000008ee809 in pcl_palette_set_cid () #5 0x00000000008e8bda in install_cid_data.part () #6 0x00000000008e79eb in pcl_process () #7 0x0000000000901801 in pl_main_run_file_utf8 () #8 0x0000000000903c3e in pl_main_process_options () #9 0x0000000000904d4f in pl_main_init_with_args () #10 0x000000000045f229 in main () (gdb) I think I have some work-in-progress changes for "ESC * g <nbytes> W" which overlaps the 4-plane CMYK code paths, but it is not anywhere near complete. Anyway, since the compression code itself is functional (for bug 694950), I am posting it. The KRGB mode is possibly 2 planes 4 colours?.
(In reply to Hin-Tak Leung from comment #14) > The other one (bug 694082) requires implementing a newer set compression > method command; also, the incoming data is supposed to be of KRGB, with a > mode-9 black plane and a mode-10 RGB plane. There doesn't seem to be > existing code for coping with KRGB, or recombining two planes of different > depth/color?` I have no clue, there is certainly no graphics library support for that so it would have to be performed in the interpreter, I doubt there is any such support at all. > The mode 10 code is correct I believe, and should work on the (private) test > file in 694950 . Currently this test file segfaults in that code path: We can't have a seg fault. If that is the case then we should throw an error on this kind of data instead. > Anyway, since the compression code itself is functional (for bug 694950), I > am posting it. The KRGB mode is possibly 2 planes 4 colours?. I'll look at the seg fault before deciding what to do with this. OK so this is the same problem I noted in bug 695047, if you allow bits to be 3 then you get seg faults. If you limit bits to 2 then you don't. The file for bug 694950 works fine for me on Linux. The output file is **huge** but it seems correct (natural history dinosaur poster). The file for this report 'works', in as much as the initial page of garbage text no longer occurs and something recognisably like the right hand side of the tiger renders. So overall I believe this is fixed. Hin-Tak, I assume we have an Artifex Contributer Licence Agreement from you already ? (surely we must). I assume we have somehow remunerated you for work in the past as well. So regarding the bounty for this and bug #694990 can you drop me a note and tell me how to get these paid to you ? I'm leaving the two bugs open until we sort that out, just to make sure I don't forget.
*** Bug 694989 has been marked as a duplicate of this bug. ***
Yes, faxed a contributors agreement to Miles over a decade ago. We'll continue this part of exchange via private email. My misc-wip repo (which includes some wip stuff on the "ESC * g <nbytes> W" plus some color space related stuff overlapping Chris Mason's) does not segfault but returns -15 from the interpreter. If "bits >2..." doesn't segfault and give some recognizable rendering, probably just add a comment above the "bits >3.." and start a new bug to fix the bit >3 segfault? This test file needs a proper invocation of "ESC * g <nbytes> W".
Not planning to pursue this any further at this time.
Curses; closed the bug when I didn't mean to.....
Doesn't directly relate to fixing the bug, but I can confirm that "tiger-mode10.pjl" is PCL3GUI. The file prints on a modern HP Deskjet 2700 series, which surprisingly still supports printing to port 9100.
Fixed in commit 53c97a4b5fa208722537064291f94ce524390ee7
(In reply to Chris Mason from comment #20) > Doesn't directly relate to fixing the bug, but I can confirm that > "tiger-mode10.pjl" is PCL3GUI. The file prints on a modern HP Deskjet 2700 > series, which surprisingly still supports printing to port 9100. It is likely the file was even for that particular model, or something close to it. In 2014, I used hpijs (the official hp Linux driver) to generate a collection of spool files for different printer models at the time; then also looked at the window driver outputs particularly for mode 10. That was for an Artifex NRE for the CLJ3600 emulation which has a mode-10 like compression scheme. I have a bunch of other mode 10 files, but mostly with different colour models, so they basically need Bug #694989 the extended colour model command to work. The CLJ3600 work was completed but was reverted later - it does what it does, the HP driver plays with libjpeg internals to make the output, so it needs some deep hook inside libjpeg to work, and was a little lacking against system-wide libjpeg. I would argue that playing nice with system-wide libjpeg wasn't part of the initial requirement (and the official builds always have a private libjpeg), supplied a interim patch, and meant to revisit the playing-nice-with-system-wide-libjpeg part, still haven't got round to it. Anyway, I dug up the 2014 collection a few days ago and posted most of the rest which haven't been posted elsewhere; now some of the have model references (but don't work with ghostpcl in various ways).
I took a crack at getting "Configure Raster Data" working. No actual parsing work yet, but I forced the interpreter to switch to mode 10 compression with direct by pixel RGB raster data when a CRD statement is parsed. I then fed GPCL some PCL3GUI output in RGB only courtesy of HP's Windows driver (K+RGB like attachment 9724 [details] is going to need MUCH more work). The mode 10 compression routine works with a slight bug. It appears that leading white space on a raster line is rendered as black until there is actual pixel data. I don't know if this is due to my hacked up attempt at testing the decompression routine, but it is a good first start.
I may have gotten a bit further than you overall, I think (having a collection of test files helps). I think the pure mode 10 files works to some extent (wrong colour, stretched in either direction or both in different ways). The KRGB ones, the k-layer is mode 9, so I have got a b/w tiger outline for those, actually (ie just doing the mode 9 k layer and ignoring the RGB mode 10 part).
Colors and size appear correct for me, outside of the leading black issue. The RGB raster data appears to be organized the same as data for CID Mode 3, just compressed with Mode 10. The best I've gotten on K+RGB so far is the black-and-white layer as well. Best documentation I've found on CRD Mode 6 is the HPLIP source file "PrinterCommands.h". It documents the K-only, RGB-only, and K+RGB methods. CRD Modes 1-5 are documented here: https://developers.hp.com/system/files/attachments/PCL%20Implementors%20Guide-15-configure%20raster%20data.pdf There is what appears to be a very obvious error on page 15-3. It states that Format 1 sends raster planes in KCMY order, when it likely should say in CMYK order. I have found references to a CRD Mode 7, but have yet to find a driver that outputs it. I've found some HP DesignJets (ex: z3200 photo) that output CRD Mode 6 and use what appears to be Compression Mode "11", which maybe straight JPEG.