Hello, When running the current 8.60 release of GhostScript on Windows Vista Ultimate (32 bit) I am running into an error when trying to print a special 'O' character through the interpreter. We are currently giving a TrueType Font whole-hog to the interpreter and wrapping it in a Type42 font (so not font conversion is taking place). When the interpreter tries to draw this particular character, however, we receive this message: ERROR: invalidfont OFFENDING COMMAND: .type42execchar OPERAND STACK: 0.0 0.0336914 123 123 123 --nostringval-- We've send the same file to an HP, Ricoh, and Adobe Distiller interpreter and all three have no problem printing out the character. You can download the problem file here: http://jeff.crazybaglok.com/junk/invalidfont.ps And a screenshot of the error here: http://jeff.crazybaglok.com/junk/invalidfont_error.gif And the expected output here: http://jeff.crazybaglok.com/junk/invalidfont_expectedoutput.gif Thank you again! Jeff Vance
I confirm that this problem exists in the current development version. TT interpreter detects an out-or-range index during TT program interpretation and bails out.
Created attachment 3412 [details] subsetted TrueType font in the sample document This issue is related to the fallback for TrueType composite glyph when the composite glyph refers non-existent component. The glyph index causing error is GID=123. According to MS-UCS2 cmap in the embedded TrueType font, GID=123 is a glyph for "latin small letter o with circumflex" (in TeX notation, \^{o}). The TrueType glyph instruction for GID=123 refers 2 other GIDs: GID=82 ("o") and GID=216. According to MS-UCS2 cmap, GID=216 is a glyph for U+02C6 "modifier letter for circumflex accent". However, the glyph instruction for U+02C6 is missing in the font (according to loca table, the length of glyf entry for GID=216 is zero). In fact, the expected result shows "o" with no accent. Therefore, the issue is: when TTF renderer (in ghostscript) finds a zero-sized component in composite glyph, TTF renderer (in ghostscript) should return an error immediately, or should ignore it and continue the rendering. Current TTF renderer in ghostscript is a modified version of FreeType2, so refering FreeType2 is consistent. Anyway, FreeType2 can render such composite glyph including. The result is of course without circumflex. I will investigate why vanilla FreeType2 can but FreeType2 in ghostscript cannot do.
Regarding comment 2 : The statement "Current TTF renderer in ghostscript is a modified version of FreeType2" is not fully correct. Actually it is a modified version of FreeType 1. So possibly FreeType 2 includes an improvement, which FreeType 1 and Ghostscript do not. Feel free to locate and port the improvement.
Oops, I slipped to change the status confirmed. And, sorry for my misunderstanding the code is based on FT1. It's very heavily modified and difficult to debug with side-by-side comparison with vanilla FT1. >Feel free to locate and port the improvement. Could you summarize the difference between FT1-based TTF renderer in gs and FT1/FT2 modules in FAPI?
Regarding comment #4: mpsuzuki wrote : "It's very heavily modified and difficult to debug with side- by-side comparison with vanilla FT1" I'm not sure what is "vanilla FT1", I've got the freetype-1.3.1 code from an official distribution site. Then it was locally modified, and I minimized the changes exactly to simplify the comparizon. Maybe mpsuzuki compares with another revision.
More regarding Commnet #4 : > Could you summarize the difference between FT1-based > TTF renderer in gs and FT1/FT2 modules in FAPI? The short answer is : "the FAPI/FteeType2 plugin does not work". This project was discontinued, because it made Artifex to pay for free type group for the free type development. I'm not sure what does "FT1/FT2 modules in FAPI" mean. gs/src includes few modules as an implementation of a bridge to FreeType2 : fapi_ft.c write_t1.c rite_t2.c . Note they are tuned for a modified revision of Free Type 2, and I don't know for sure whether that modification went to the official distribution of FreeType 2. Maybe it does, sinse a long time passed. In any case, FAPI/FreeType bridge does not include a TT interpreter, so the main difference is no interpreter.
This bug (rather, I want to call it as an intolerance against invalid TrueType bytecodes) exists in the revisions since SVN revision 4325, the changeset was by Igor. The older implementation is not interrupted by the bytecode. The behaviour seems to be caused by missing or instruction-less component in composite glyph. In current implmentation of ghostscript, the bytecodes of all components are collected before execution, by calling ttfOutliner__BuildGlyphOutlineAux() recursively, aslike: src/ttfmain.c 478 static FontError ttfOutliner__BuildGlyphOutlineAux(ttfOutliner *this, int glyphIndex, 479 FixMatrix *m_orig, ttfGlyphOutline* gOutline) 480 { ttfFont *pFont = this->pFont; ... 543 if (gOutline->contourCount == 0) 544 gOutline->pointCount = 0; 545 else if (gOutline->contourCount == -1) { ... 607 for (i = 0; i < nUsage; i++) { ... 624 code = ttfOutliner__BuildGlyphOutlineAux(this, e->index, m_orig, &out); ... 632 if (code == fPatented) 633 error = code; 634 else if (code != fNoError) { 635 error = code; 636 goto ex; 637 } If ttfOutliner__BuildGlyphOutlineAux() returns an error when a component is to be loaded, ttfOutliner__BuildGlyphOutlineAux() is interrupted as unrecoverable error. But, in this case, glyphID=216 is defined by loca/glyf, but zero-sized, so LoadGlyph() is successfully finished (glyph_size is set to zero), and ttfOutliner__BuildGlyphOutlineAux() does not return any error. src/ttfmain.c 508 if (!this->bOutline) 509 return fNoError; 510 if (!r->LoadGlyph(r, glyphIndex, &glyph, &glyph_size)) 511 return fGlyphNotFound; 512 if (r->Eof(r)) { 513 r->ReleaseGlyph(r, glyphIndex); 514 gOutline->xMinB = gOutline->yMinB = 0; 515 gOutline->xMaxB = gOutline->yMaxB = 0; 516 return fNoError; 517 } 518 if (r->Error(r)) 519 goto errex; ... As a result, the rasterization proceeds to the execution of TrueType bytecodes. Following is a comparison of executed TrueType bytecodes. freetype-1.3.1 ghostscript -------------- ----------- Ins_NPUSHB Ins_NPUSHB Ins_PUSHW Ins_PUSHW Ins_PUSHB Ins_PUSHB Ins_SRP0 Ins_MIRP Ins_MIRP Ins_DELTAP Ins_DELTAP Ins_MIRP Ins_MIRP Ins_MIRP Ins_MIRP Ins_DELTAP Ins_DELTAP Ins_DELTAP Ins_DELTAP Ins_DELTAP Ins_DELTAP Ins_MIRP Ins_MIRP Ins_SVTCA Ins_MIAP Ins_MIAP Ins_MIRP Ins_MIRP Ins_MIAP Ins_MIAP Ins_MIRP Ins_MIRP Ins_IUP Ins_IUP Ins_IUP Ins_IUP Ins_RS Ins_JROF Ins_SVTCA Ins_DELTAP Ins_DELTAP Ins_SVTCA Ins_DELTAP Ins_DELTAP Ins_DELTAP Ins_DELTAP Ins_NPUSHB Ins_NPUSHB Ins_SVTCA Ins_MIAP Ins_MIAP <error occurs interrupt> Ins_SHC Ins_CALL Ins_Goto_CodeRange Ins_GC Ins_GC Ins_ROLL Ins_ROLL Ins_GC Ins_GC Ins_ROLL Ins_ROLL Ins_PUSHW Ins_ROLL Ins_PUSHB Ins_ROLL Ins_PUSHB Ins_IF Ins_PUSHB Ins_PUSHW Ins_ELSE Ins_MSIRP Ins_ENDF Ins_Goto_CodeRange Ins_SHC <finishes successfully> Except of hinting-related operators, executed bytecodes are exactly same. Why freetype-1.3.1 can continue but ghostscript cannot? The original freetype-1.3.1 implementation is following. freetype-1.3.1/lib/ttinterp.c 4332 /**********************************************/ 4333 /* MIAP[a] : Move Indirect Absolute Point */ 4334 /* CodeRange : $3E-$3F */ 4335 /* Stack : uint32 uint32 --> */ 4336 4337 static void Ins_MIAP( INS_ARG ) 4338 { 4339 ULong cvtEntry; 4340 UShort point; 4341 TT_F26Dot6 distance, 4342 org_dist; 4343 4344 4345 cvtEntry = (ULong)args[1]; 4346 point = (UShort)args[0]; 4347 4348 if ( BOUNDS( point, CUR.zp0.n_points ) || 4349 BOUNDS( cvtEntry, CUR.cvtSize ) ) 4350 { 4351 if ( CUR.pedantic_hinting ) 4352 CUR.error = TT_Err_Invalid_Reference; 4353 return; 4354 } The modified version in ghostscript is following. src/ttinterp.c 3842 /**********************************************/ 3843 /* MIAP[a] : Move Indirect Absolute Point */ 3844 /* CodeRange : $3E-$3F */ 3845 3846 static void Ins_MIAP( INS_ARG ) 3847 { 3848 Int cvtEntry, point; 3849 TT_F26Dot6 distance, 3850 org_dist; 3851 3852 3853 cvtEntry = (Int)args[1]; 3854 point = (Int)args[0]; 3855 3856 if ( BOUNDS( args[0], CUR.zp0.n_points ) || 3857 BOUNDS( args[1], CUR.cvtSize ) ) 3858 { 3859 CUR.error = TT_Err_Invalid_Reference; 3860 return; 3861 } In the case of original freetype-1.3.1, the invalid reference error is returned if pedantic_hinting is enabled. But in the case of ghostscript, the invalid reference error is always returned. The boolean pedantic_hinting is used to make freetype-1.3.1 sensitive to invalid array access. freetype-1.3.1/lib/ttobjs.h 731 Bool pedantic_hinting; /* if true, read and write array */ 732 /* bounds faults halt the hinting */ This boolean is never introduced in ttobjs.h of ghostscript, and it seems that invalid array access is always taken too serious to continue. Anyway, if the part making unrecoverable error src/ttinterp.c 3859 CUR.error = TT_Err_Invalid_Reference; is commented out, the rasterization result is same with that expected. There's any variable to controle the severity of invalid array access, like pedantic_hinting? Or, also ghostscript should ignore the invalid array access that freetype-1.3.1 ignores by default?
In Comment #7 mpsuzuki wrote : "This bug ... exists in the revisions since SVN revision 4325". The revision number in the quote is wrong, because the patch 4325 is unrelated to the subject. Here is its log message : ------------------------------------------------------------------------ r4325 | igor | 2003-10-28 17:44:35 +0300 (Tue, Oct 28 2003) | 8 lines Fix (dropout prevention) : The expression for choosing a pixel to paint with narrow trapesoids was wrong. The old code sometimes unreasonably shifted stems in 1 pixel. EXPECTED DIFFERENCES : A massive difference in text rendering. ALMOST ALL comparefiles RENDER DIFFERENTLY.
>In Comment #7 mpsuzuki wrote : "This bug ... exists in the revisions >since SVN revision 4325". The revision number in the quote is wrong, Oh, sorry, the bug was since SVN revision 4324. The "new TrueType interpreter" was included in source tree since SVN revision 4234 but disabled by default. I enabled it, tested and found that the bug had already existed in SVN revision 4234. So, it is not important that how this bug was introduced - "new TrueType interpreter" had always included this bug. The problem is how ghostscript should fallback when invalid array access is found in TrueType bytecodes.
So the correct diagnozis is that the True Type interpreter isn't tolerant to some incorrect True Type instructions, which appear in the test case. Older gs revisions do not fail simply because they always ignore True Type instructions.
So my question to original author of current TrueType interpreter is: this intorelance should be relaxed? should be kept? should be controled? If should be controled, there's any existing switch to use such purpose?
The originasl author of the True Type interpreter is Free Type group. So the questions are wrongly placed to here. As to my opinion, see Comment #3.
Ah, I meant that "original author" is the person who simplified and removed the relaxation when he introduced FreeType derivatives. As I've written, original FreeType-1.3.1 has a relaxation control switch. 4348 if ( BOUNDS( point, CUR.zp0.n_points ) || 4349 BOUNDS( cvtEntry, CUR.cvtSize ) ) 4350 { 4351 if ( CUR.pedantic_hinting ) 4352 CUR.error = TT_Err_Invalid_Reference; 4353 return; 4354 } But ghostscript don't have. 3856 if ( BOUNDS( args[0], CUR.zp0.n_points ) || 3857 BOUNDS( args[1], CUR.cvtSize ) ) 3858 { 3859 CUR.error = TT_Err_Invalid_Reference; 3860 return; 3861 } this intorelance should be relaxed? should be kept? should be controled? If should be controled, there's any existing switch to use such purpose?
Regarding Comment #13 : 1. Comparing with the original ttinterp.c from freetype-1.3.1, I see all appearences of pedantic_hinting were replaced with 'true' and then simplified (please double check for sure). The goal probably was to remove the option. It didn't cause problems till now, so the problem looks pretty rare. 2. What to do with it ? To get a proper answer for this question, first I recommend to find out where the font comes from. Is it a popular font ? Are other popular font scalers able to handle this font ? For example, what happens if one installs this font to Windows ? Second, must know how UFST handles this glyph program. 2. I recommend to work more with svn history. ttinterp.c revision 6559 Modified Sat Feb 4 03:56:37 2006 UTC (19 months, 4 weeks ago) by alexcher reads : Following FreeType 2 implementation, ignore incorrect SHC command. Bug 688501 This may gives an additional tip. 3. In theory there are 3 possibilities : (3.1) return with error, (3.2) ignore the problem, (3.3) make it configurable. These 3 alternatives more or less correspond to what mpsuzuki assumes. We already have 3.1 . Please follow the point 2 to know whether we need other alternatives. If not (the bad font) close as invalid since it's not a customer request. 4. If we want to implement an options like pedantic_hinting, we need a new user parameter. The old parameter GridFitTT may be used as a prototype. However I'm not sure that it is the best way since I never heard that a popular font scaler (of some OS or a printer) provide such control. Regular users do not need such smart control, so it should be resolved somehow else. It would be a dumping of the headache from the developer's head to a healthy user's head - such kind of solutions never work for marketting. 5. I also recommend to check whether the glyph uses a patented instruction. Does it work fine with ignoring the bad instruction ? If for example it later fails with patented THROW_PATENTED, then it's another story, which to be discussed separately.
> 2. What to do with it ? To get a proper answer for this question, first I > recommend to find out where the font comes from. Is it a popular font ? Are > other popular font scalers able to handle this font ? For example, what > happens if one installs this font to Windows ? Second, must know how UFST > handles this glyph program. The font initially comes from an ATSIF testcase pdf file, which I assume subsetted the font in the creation of the PDF. Installing the font with windows isn't a possibility due to font's subsetted nature, it only includes the characters which are used directly in the test case [which may be the cause for the missing ^ component if the font was subset incorrectly] We haven't seen the same crash in any of the interpreters we have sent the test case to, including an HP printer, a Ricoh printer, and Adobe Distiller. Our internal rip which uses UFST also prints the font without error. I imagine each of these is ignoring the problem by default, similar to how the base version of FreeType2 works.
Regarding copmment #15 : 1. Adobe Distiller does not interpret font data. It only performs a subsetting, so testing with it doesn't help for this bug. 2. What True Type interpreter do the HP printer, the Ricoh printer implement ? Is it sertified by Apple ? If not, then they don't help for this bug, becase they may simply ignore TT instructions. 3. Reference to UFST does help for this bug, because it implies that UFST handles the font with no error. I suggested to check how UFST handles this glyph, so this way may give us the right solution.
Regarding Comment 16: To the best of our knowledge the HP printer uses a Zoran PostScript interpreter, and the Ricoh is using an Adobe PostScript interpreter. We're not certain exactly which TT interpreter each is using, however, and have no idea how to find out if each is Apple Certified.
It is known that Adobe CPSI does *not* include an Apple certified TT instruction interpreter. Therefore it incorrectly renders Dynalab fonts. Don't know aboout Zoran. Try to send a Dynalab font to there.
I searched around for a Dynalab font that I might have access to, and the only one I found was DFSongSd.ttf (and I'm not particularly certain that it belongs to Dynalab, though the wikipedia entry pointed to it). I printed a number of glyphs out to both printers and they both matched the expected output, is there a particular glyph range that makes the Adobe truetype interpreter show its problems? You can get the PS file I sent here: jeff.crazybaglok.com/junk/dynalab.ps
Created attachment 3441 [details] Mac OS X displays the invalid composite glyph by imcompleted form, by available components only. The patented hinting problem is another issue, I think. I extracted a TTF object from the PDF, see attachment 3412 [details], which includes invalid composite glyph. On Mac OS X 10.4.10 by Apple, its official TTF validator "Font Book" finds no problem of the font (oops), and I can install the font to the system. When I try to display the composite glyph including the invalid reference, by UCS2 code U+00F4 (the problematic glyph is "latin small letter o with circumflex", "small letter o" is included and refered, but circumflex is not included and cannot be refered), the glyph is displayed as "small letter o", as attached picture. For comparison, I displayed the same character U+00F4 by Times New Roman. I'm not sure if official TTF rasterizer in Mac OS X is certified by Apple, but the request to display the invalid glyph as http://jeff.crazybaglok.com/junk/invalidfont_expectedoutput.gif is reasonable request.
[redacted]
> is there a particular glyph range that makes > the Adobe truetype interpreter show its problems? Almost all composite CJK glyphs render wrongly when rendering Dynalab fonts. I recall those fonts are distributed with Windows NT (not sure).
Sorry guys we desided that we should not use UFST code for this bug due to possible legal problems. Please disregard my recommendation about that.
Created attachment 3449 [details] invalidfont.ps A local copy of the test file.
Regarding Comment #15, statement "Installing the font with windows isn't a possibility". I'm not sure what does this statement mean exactly. However the font perfectly installs to Windows XP. The font name is Z@R38A.tmp.
Created attachment 3450 [details] charmap.bmp.zip Here is how the font looks with Windows Character Map application.
Regarding Comment #7 "Ins_MIAP <error occurs interrupt>". ttinterp.c ln 3856 checks (index >= CUR.zp0.n_points), which gives 31 >= 27. It accounts 2 phantom points from the base subglyph. I'm not sure how many phantom points must appear when processing a composite glyph. The upper estimation is 2*2 + 2 == 6 (2 for each of 2 subglyphs and 2 for the composite glyph). It would give us 31 points. One more point can appear as the ending point of the empty subglyph (from its width from mtx), so we get 32 points, and the check would not fail. Well this is a hypothesis, which needs a proof. I'll be happy if mpsuzuki finds a proof. No I don't think FreeType 2 is a right proof - they know more than us, but still incomplete.
Regarding comment #20 "TTF validator "Font Book" finds no problem of the font (oops)" : Well, I don't think that the font "misses" a subglyph. I believe it *defines* an empty subglyph, so the font data is consistent. Particularly the empty glyph has a good metrix data. What the creator wanted to define and what does the glyph name (or char code) mean - it's another story, which is not known to the validator. So I think the font is good, and our interpreter is not.
Changing assignment due to no progress.
I've spent some time with several versions of the TrueType spec, the original FreeType 1 code and the current FreeType 2 code, and another TrueType interpreter looking into the problem. As far as I can see the speculation in comment #27 is not the case, there should only be one pair of phantom points per *outline*, an outline for a composite glyph is still only one outline, even though it consists of two glyph descriptions, and therefore only has two phantom points. The second glyph for the composite glyph is not 'empty' , its missing. The Microsoft Font Validator Tool is quite explicit in its error message: "E1115 Failure to load the component Glyph index 123, Component index 216" So the glyph is broken, I believe quite badly broken. Freetype 1 & 2, as noted above, ignore most problems with fonts, unless 'pedantic_hinting' is set. From previous work with TrueType fonts I believe this laissez faire attitude is common to TrueType interpreters, and is the reason this font does not cause a problem with most implementations. THe other TT interpreter I've checked also does not have a problem with this font, though for subtly different reasons. nevertheless it too allows this 'slack' error tolerance. So, Ghostscript's behaviour is technically correct, since the glyph and font are invalid. However, it would be appropriate for us to attempt to mimic the behaviour of other interpreters and allow this glyph to pass unchallenged. It would be nice if we could restore the 'pedantic_hinting' which was removed in the port of the FreeType code into GS, to allow customers flexibility in this kind of error handling, but that would now be a major undertaking. So I think the correct fix in this case is to abort the instruction if the point indexed is not valid, and continue with the next instruction, just as FreeType 1 & 2 do.
Ah, I see we already have precedent for the proposed change, in Ins_SHC, around line 3652 in ttinterp.c we have this: if ( BOUNDS( args[0], CUR.pts.n_contours ) ) { #if 0 /* A workaround for the Ghostscript bug 688501. * Ported from FreeType 2 */ CUR.error = TT_Err_Invalid_Reference; #endif return; } Which also effectively restores the FreeType no 'pedantic_hinting' behaviour.
Fixed in revision 9468, patch here: http://ghostscript.com/pipermail/gs-cvs/2009-February/009045.html