I've generated an encrypted PDF document that GS opens but Adobe doesn't. It uses /Encrypt << /V 2 /R 3 /Length 64 >> (a 64 bits key). I guess our PDF interpreter isn't conforming.
Created attachment 453 [details] The encrypted PDF The document uses the owner password 'owner' and the user password 'user'. GS opens it fine, Adobe Reader 6 on Windows XP reports incorrect passwords.
Created attachment 454 [details] A better PDF Attached another PDF that Adobe can open. Only difference is the key length 128 rather than 64 in the old file.
XPDF also reports "incorrect password" with "The encrypted PDF". Probably we can analyze it's sources.
Compared with XPDF using a C debugger. When executing the sequence md5_init(&md5); md5_append(&md5, buf, 16); md5_finish(&md5, digest); when the 'buf' contains [0x0] 0xec 'ì' [0x1] 0x8b '‹' [0x2] 0xa0 ' ' [0x3] 0x4d 'M' [0x4] 0x1f '' [0x5] 0x15 '' [0x6] 0xbe '¾' [0x7] 0x5a 'Z' [0x8] 0x18 '' [0x9] 0x69 'i' [0xa] 0x98 '˜' [0xb] 0xcc 'Ì' [0xc] 0x20 ' ' [0xd] 0x72 'r' [0xe] 0xd8 'Ø' [0xf] 0x13 '' the result 'digest' appears different than XPDF : GS : [0x0] 0x0e '' [0x1] 0xcd 'Í' [0x2] 0xb3 '³' [0x3] 0xc7 'Ç' [0x4] 0xbd '½' [0x5] 0x5d ']' [0x6] 0x89 '‰' [0x7] 0x44 'D' [0x8] 0x86 '†' [0x9] 0xe1 'á' [0xa] 0x29 ')' [0xb] 0xfa 'ú' [0xc] 0x2f '/' [0xd] 0xc3 'Ã' [0xe] 0x83 'ƒ' [0xf] 0x15 '' XPDF: [0x0] 0x38 '8' [0x1] 0x52 'R' [0x2] 0xbe '¾' [0x3] 0xda 'Ú' [0x4] 0xe9 'é' [0x5] 0x58 'X' [0x6] 0x34 '4' [0x7] 0xce 'Î' [0x8] 0x54 'T' [0x9] 0x5e '^' [0xa] 0x41 'A' [0xb] 0x82 '‚' [0xc] 0x4d 'M' [0xd] 0xb8 '¸' [0xe] 0x84 '„' [0xf] 0xdb 'Û' I guess our md5 lies with short data.
OK, reading XPDF I found that in the mentioned code fragment the number '16' is wrong. It should be KeyLength/8. It explains why it worked with KeyLength=128. Thus it was my error. I'm sorry.
Oops, I've closed it prematurely. Rather PDF writer is now fixed, the PDF interpreter is not.
The patch http://www.ghostscript.com/pipermail/gs-cvs/2004-January/004016.html fixes << /V 2 /R 3 >>. With << /V 2 /R 2 >> the it is still incompatible to Adobe when Length is not 40, but it appears compatible to XPDF. Assigning to ralph for further analyzis.
Well, now I clarify the rest of the problem. Both GS and XPDF appears incompatible to Adobe in the case << /V 2 /R 2 /Length not40 >>. I am unable to understand for sure, what Adobe documentation means for this case (either PDF 1.4 or PDF 1.5). Maybe Adobe means that this case is prohibited. If so, we should set more constraints to the PDF interpreter and the PDF writer.
We're less worried about this since we've only been able to generate files with the problem parameters with ghostscript. We suspect a difference between the spec and adobe's implementation since xpdf works on the files. Setting to a lower priority.
So to summarise my understanding of this bug: 1) GS used to be able to generate files with a non-standard key length. These files could only be read by GS or XPDF (not Adobe). 2) The code has now been fixed so that we won't generate these files any more (we generate files that Acrobat can read). 3) The only "problem" remaining is that we continue to read these files where Acrobat refuses them. This doesn't seem like much of a bug to me. I am therefore closing this bug as WONTFIX. If anyone wants to challenge my reading of this bug or my choice of closing it, please just say.