The attached file cannot be opened correctly with gshead (r8746), but instead gives the following warning: jbig2dec FATAL ERROR decoding image: symbol dictionary uses custom DW huffman table (NYI) (segment 0x03) The command line I'm using for testing: bin/gs -sDEVICE=tiff24nc -o test.tif ./scanned.pdf
Created attachment 4031 [details] scanned.pdf
The luratech build of Ghostscript opens this file correctly.
Does the patch in 689840 fix this? The example PDF is marked private, so I can't test it myself.
The file generates a C stack overflow for me, so I would say its not fixed. It is a different error though, maybe its partially fixed....
The patch from 689840 not resolve it for me. Will get back to this one after the next release.
Justin Greer (jgreer@nextpoint.com) reports: With the previous patches I've submitted, I was able to process this PDF without any errors. I took gshead (rev 9512) and applied the patches from bugs 689835, 689836, and 689840 (which required a bit of updating -- particularly against the make/build files), and that's it... $ bin/gs -sDEVICE=tiff24nc -o test.tif ./scanned.pdf GPL Ghostscript SVN PRE-RELEASE 8.65 (2009-02-04) Copyright (C) 2009 Artifex Software, Inc. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. Processing pages 1 through 1. Page 1 $ I'm including an updated diff, which includes these three patches against rev 9512. I've tested the updated patch (attached) with gshead (r9513) and it does allows the file associated with this bug to be read. I haven't tested the patch for unintended side effects, but recommend checking it in and seeing if any regressions are reported.
Created attachment 4819 [details] gs_jbig2.patch
I applied Marcos' patch to the latest gs & jbig2dec builds and this seems to address bugs 689836, 689840 and 689360. I have an explicit example which highlights bug 689836 quite well also - ask if you want to me to post it.
Just an update: Thanks, Marcos, for putting together the patch. I still need to verify some things about the patches, then it can go in. I'm on vacation the next couple of weeks; should happen in July.
Tried with r9974. Reproduced same symptom. I didn't try gs_jbig2.patch because it needs little hassle to apply to r9974, but it seems need little enhancement to go into gs source, e.g. jbig2_user_table.h doesn't have #ifdef guard, there are some C++ syntax (variable definition between sentences).
*** Bug 691080 has been marked as a duplicate of this bug. ***
I used gs_jbig2_patches.diff (gs_jbig2.patch) over r11067 and tried scanned.pdf, resulted "Bus error" on Mac OS X. Visual C++ 2008 throws compile errors on Windows. I am working on this.
I tried gs_jbig2_patches.diff (gs_jbig2.patch) over r9512, it worked on Mac OS X. But on Windows, same compile errors. I guess this patch has some conflicts with jbig2dec 0.11, and have not been tested on Windows.
Created attachment 6206 [details] b689853_c1.patch The same patch, but can be applied to r11093 without hustle. This patch worked on r11067 too. What I said in comment #12 was wrong, I probably had messed up my directory. Still same compile errors on Windows.
Created attachment 6233 [details] b689853_c2.patch Updated patch. Now compiles on Windows and processes the sample file (scanned.pdf). Needs more enhancements.
Created attachment 6245 [details] b689853_c9.patch Justin Greer's original patch included patches from bug 689835, bug 689836, and bug 689840. The patch for bug 689835 was committed in r6227, and I removed that part when I created b689853_c1.patch. This time, I removed the patch from bug 689836. I do not agree with this part and I will continue to discuss it on bug 689836. This patch is essentially the patch from bug 689840, with many modifications as well as enhancements. With this patch, gs renders the both sample file (scanned.pdf and BiasExA.pdf from bug 691080). Localcluster test reports no differences.
Created attachment 6317 [details] b689853_d1.patch Unfortunately, we couldn't get response from Justin Greer. I gave up his code and rewrote the function. This patch is functionally same with b689853_c9.patch.
*** Bug 689840 has been marked as a duplicate of this bug. ***
(In reply to comment #17) Amusingly, I just happened to check in on these old patches today. Having seen this message, I checked my spam folder and saw the messages you sent. Glancing at your patch in bug 689836 it seems like it would fix the issue correctly, in a different manner. It means that the standard huffman table definitions don't exactly match the representation in the specification, but the actual data is the same, so no worries there. It has been 2 years since I wrote all that code, too, so I'm pretty rusty on how all the little details worked. Sorry you ended up needing to rewrite the patches from me; I would have been happy to officially assign copyright (in fact, I thought I did, since I put the Artifex copyright headers in the new files). Anyway, it looks like your rewrite is significantly more concise. If I can find any of the files that I had problems with on this originally, I will test them and let you know if the new patches work correctly.
Dear Justin Greer, Glad to hear from you. I owe you a big to write that code. Big thanks to your original work. Please post a comment when you could confirm patches in this bug or bug 689836.
Created attachment 6319 [details] Patch for custom huffman tables, against r11305. I've had a chance to test your patch, and unfortunately it segfaults trying to decode a custom huffman table. Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0f977986 0x001b3435 in jbig2_image_compose () (gdb) bt #0 0x001b3435 in jbig2_image_compose () #1 0x001b7904 in jbig2_decode_text_region () #2 0x001b85ef in jbig2_text_region () #3 0x001ae02d in jbig2_data_in () #4 0x001ad945 in s_jbig2decode_process () #5 0x00091430 in sreadbuf () #6 0x000915bd in s_process_read_buf () #7 0x000b7bf8 in image_file_continue () #8 0x00085adb in gs_interpret () #9 0x0007ad36 in gs_main_interpret () #10 0x0007b325 in gs_main_run_string_end () #11 0x0007b40a in gs_main_run_string () #12 0x0007c2d5 in run_string () #13 0x0007c415 in runarg () #14 0x0007c606 in argproc () #15 0x0007e441 in gs_main_init_with_args () #16 0x00002347 in main () I've updated my patches so they work against revision 11305 (which is the version your patch is against). Using that instead, I can still correctly decode my test files.
Created attachment 6320 [details] Debug log of custom huffman tables in test PDF. Here is a log with basic info from my patched version of jbig2dec, decoding a PDF with custom huffman tables. This may be helpful in debugging the other change set, but I'm not sure. If you'd prefer to debug your patches instead of using the code I had written previously, I can help with testing on my file, and providing info/output. Unfortunately I still don't have any test files without confidential info in them.
Created attachment 6322 [details] b689853_d2.patch I was doing wrong when getting 'referred to' segments. It couldn't process a file attached in bug 691080 either. This is a revised patch. I believe this solves the problem Justin bumped into. If it didn't, please attach the output of command "debugbin/gs -Zw yourfile.pdf". (Now this patch has same debug output as Justin's)
Created attachment 6326 [details] Debug logs from each patch set. I tested the updated patch, and it still has a couple issues... By itself, it get an error "jbig2dec FATAL ERROR decoding image: Segment too short (segment 0x23)" on page 6 of this 29-page PDF. It also doesn't get the symbols into the correct places on the page. So I added your the patches from bug 689836 and tried again. It gets some symbols into the right places, but others seem to disappear completely. It still gets the segment-too-short error on page 6, but also seems to run into an infinite loop on page 15... I'm attaching a zip with 3 copies of the debug log. One with my patches in place ("jg"), and two with your patches in place ("mu"). The file numbered "2" is the log without the patches from bug 689836. There is also a backtrace in there, from when I broke out of the infinite loop (or whatever caused it to hang).
Justin, Thanks for the log upload. Looks like my code calls to build standard huffman table N, where your code calls table O in jbig2_text_region(). Probably some bits are missing in my code. Let me double check my code and get back to you. Thank you.
On second thought, interesting enough, the exchange of table N vs O begins far before than any custom table involved. The first wrong access to table N happens at segment 0x03 (line 1724 in gs_mu_patches_output.txt), the first custom table process begins at segment 0x07 (line 3610). This sounds like something impossible is happening to me. Justin, are you sure you are using right binary? Could you try rebuilding the binary (make debugclean debug) and see the same error still happens? (Oh! If only I could have your wonderful sample file in my hand!)
(In reply to comment #22) > Created an attachment (id=6320) [details] > Debug log of custom huffman tables in test PDF. > > Here is a log with basic info from my patched version of jbig2dec, decoding a > PDF with custom huffman tables. This may be helpful in debugging the other > change set, but I'm not sure. > > If you'd prefer to debug your patches instead of using the code I had written > previously, I can help with testing on my file, and providing info/output. > Unfortunately I still don't have any test files without confidential info in > them. I wasn't sure you were aware you can attach the files and mark them private, with that, only Artifex employees or contractors with non disclosure agreements can access the data.
Created attachment 6331 [details] b689853_e1.patch I leaned a little more about this problem today. First of all, my comment 25 and comment 26 was not correct. There shouldn't be any standard table exchange. I was misunderstanding the log message. I believe the Justin's file has tow issues. 1) it uses custom haffman tables, and 2) it is also affected by lower range table problem (bug 689836). But my patch for bug 689836 seems having another problems. So I would like to separate the issue if possible. Attached patch is my b689853_d2.patch + Justin's patch for bug 689836 (in comment 1 there), plus bunch of debug printing. If this can render Justin's file (and I believe so), we should be safe to assume that b689853_d2.patch solves this bug and Justin's patch for bug 689836 solves bug 689836. If so, I would like to close this bug by committing b689853_d2.patch and want to discuss about Justin's file in bug 689836. Justin, could you try this patch and let us know if it solves problems you have on your file? Also, can you tell me what application program produced the file you have? I would very much like to try that application. That would be second best thing to do, if you couldn't give your file to us. (You know we can make it NOT public, right?)
Created attachment 6514 [details] b689853_d3.patch This is a new patch modified to follow recent code style change in jbig2dec. I am going to commit this to include this in next release. I am sorry I haven't heard from Justin. My previous patch for bug 689836 had a serious problem and I believe that was the reason he saw problems with my patch. Bug 689836 was fixed in r11413 and I'm positive current code would work much better than ever. If you still see problems, please re-open this bug or open another entry for it.
b689853_d3.patch has been committed as r11526. Making this 'fixed'.
Changing customer bugs that have been resolved more than a year ago to closed.