Bug 690448 - Issues with FreeType
Summary: Issues with FreeType
Status: RESOLVED FIXED
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: Font API (show other bugs)
Version: master
Hardware: PC Windows NT
: P4 normal
Assignee: Ken Sharp
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-04-24 07:17 UTC by Ken Sharp
Modified: 2010-04-08 18:39 UTC (History)
3 users (show)

See Also:
Customer:
Word Size: ---


Attachments
Suggested fix for regression in rev 10433+10434 (1.14 KB, patch)
2009-12-03 17:44 UTC, SaGS
Details | Diff
hinting.zip (111.65 KB, application/octet-stream)
2010-04-06 10:57 UTC, Ken Sharp
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ken Sharp 2009-04-24 07:17:51 UTC
Several of our standard test files fail when using FreeType as the font
rendering engine. Several of these appear to be due to the Quark 'q' font.

Files:
Bug687044.ps
Bug687489.ps
Bug687546.ps
Bug687845.ps
Bug688967.ps

There may be others, these fail using both pdfwrite and the Windows display.
Comment 1 Ken Sharp 2009-04-27 06:18:39 UTC
revision 9690, patch:

http://ghostscript.com/pipermail/gs-cvs/2009-April/009276.html

solves the first of these problems.
Comment 2 Ken Sharp 2009-04-27 09:28:10 UTC
Revision 9692, patch:

http://ghostscript.com/pipermail/gs-cvs/2009-April/009278.html

Should fix the problem with bug687546.ps caused by the FontBBox being
represented as a packed array.
Comment 3 Ken Sharp 2009-04-29 07:47:43 UTC
The file bug688967.ps has been fixed by one of the two previous patches. 

Bug687845.ps uses a Type 42 font where the sfnts data has no glyf or loca
tables, but instead uses a gdir table. The FAPI code does not currently support
this kind of Type 42 font.
Comment 4 Ken Sharp 2009-04-30 06:42:59 UTC
bug687845.ps is fixed by this patch:

http://ghostscript.com/pipermail/gs-cvs/2009-April/009300.html
Comment 5 Ken Sharp 2009-04-30 06:57:44 UTC
Additional files which fail outright with FreeType:

tpc2.ps       -  /typecheck on cshow
Bug690195.pdf -  /typecheck on .execform1
BCL_Easy.pdf  -  /typecheck on --run--
Bug689369.pdf -  /rangecheck on --run--
Bug687351.pdf -  /unknownerror on --run--
ca.pdf        -  /invalidfont on --run--
Bug689740     -  /invliadfont on definefont
0000728-simple-NG.ps - unknownerror on FAPIBuildGlyph
Bug690166.ps  -  unknownerror on FAPIBuildChar

These are only representatives for each error, other files fila in similar ways.
Comment 6 Ralph Giles 2009-05-03 22:15:18 UTC
Ken, BTW, I noticed the three latin glyphs look emboldened in Bug687845.ps with
the gs freetype branch, as compared to libpoppler and the gs native rendering.
Do you have that on your list as well?
Comment 7 Ken Sharp 2009-05-04 07:19:00 UTC
Ralph, I did notice it, but so far I'm concentrating on fatal problems rather
than quality issues. Checking against Adobe all the glyphs look like they should
be emboldened (but all the same), so this may be a 'better' result than the
regular GS output.

All the same, the glyphs should all have the same weight, the difference is
incorrect.

I'd like to keep this tracker for the errors rather than quality though, you
could open a new one if you want to track that ?
Comment 8 Ken Sharp 2009-05-07 05:48:23 UTC
revision 9725 fixes Bug689740.pdf, patch here:

http://ghostscript.com/pipermail/gs-cvs/2009-May/009311.html
Comment 9 Ken Sharp 2009-05-07 06:07:12 UTC
Revision 9726 fixes ca.pdf, patch here:

http://ghostscript.com/pipermail/gs-cvs/2009-May/009312.html
Comment 10 Ray Johnston 2009-05-07 07:50:14 UTC
If the FontDescriptor has a FontBBox, why can't our PDF interp (pdf_font.ps)
check the font, and if it is missing, insert the one from the FontDescriptor ?
Comment 11 Ken Sharp 2009-05-07 08:05:50 UTC
The PDF interpreter passes the CFF font data to the PostScript interpreter like
this:

  dup true resolvestream dup readfontfilter
		% Stack: pos resource streamdict stream filter
  3 index /FontDescriptor oget /FontName oget
  1 index 
  /FontSetInit /ProcSet findresource begin //true //false ReadData

The ReadData procedure builds a font dictionary from the CFF data, and then
calls definefont. The action of definefont eventually calls the FAPI code which
tries to modify the FontBBox in the font dictionary based on the actual values
returned from the font interpreter. This is where the problem arises.

The *only* data passed to ReadData is the CFF bytes and the font name, there is
no way to 'override' the FontBBox, and the CFF font has no FontBBox in the top
Font DICT.

Altering this looks like it would be horrendous, the internal GS implementation
of definefont must ignore the missing FontBBox, the FontBBox isn't actually very
useful anyway and modifying it ever so slightly is even less useful, so I
decided it was a lot simpler to just accept the missing information and carry on.
Comment 12 Ken Sharp 2009-05-11 08:29:47 UTC
revision 9737, patch here:

http://ghostscript.com/pipermail/gs-cvs/2009-May/009323.html

resolves the errors with the files Bug690166.ps and 0000728-simple-NG.ps.
However both of these files still render incorrectly with FreeType.
Comment 13 Ken Sharp 2009-11-23 08:00:41 UTC
The file tpc2.ps also now works correctly (fixed in an earlier patch I assume).

The remaining failing files:

BCL_Easy.pdf
Bug687351.pdf
Bug689369.pdf
Bug690195.pdf

Are all PDF files containing CIDFontType 2 fonts (CID Fonts with TrueType outlines).

These types of fonts are not documented as supported by FreeType which
apparently only supports CIDFonts with type 1 outlines (presumably also CFF).

Looks like we will have to add support for these ourselves, maintaining the type
0 structure in PostScript as we currently do, and having FreeType deal with the
descendant fonts as simple TrueType fonts.

Some of this is in place, we already deal with the descendant fonts as simple TT
fonts for FreeType. Currently I can see that the CIDToGID map is not being used,
which means that we are attempting to use the CID as the Glyph Index in the
TrueType font, which causes the 'unknown' errors, if the CID > the number of
glyphs in the TrueType font.

However this is not the whole story, hacking that to be correct still results in
other, different, errors. Looks like quite a lot to do on this.
Comment 14 Ken Sharp 2009-11-24 08:02:59 UTC
After a bit of head scratching....

It seems the problem is simply that the FAPI interface doesn't properly handle
the stack when we have a CIDFont with a CDevProc, it ends up leaving a
dictionary (the descendant font dictionary) on the top of stack before calling
the CDevProc, which naturally has no idea what to do with it, since its
expecting a CID or a name, and raises an error.

The problem looks like its probably more general than that, and *any* font with
a CDevProc will fail, but I haven't ascertained that for sure yet. CDevProcs are
unusual, but the GS PDF interpreter makes extensive use of them for handling PDF
files containing fonts with Widths arrays.

So it looks like if I can work out how to preserve the stack properly this
should solve the problem.
Comment 15 Ken Sharp 2009-11-26 02:28:46 UTC
revisions 10387 and 10388 address most of these issues. In particular:

BCL_Easy.pdf  -  /typecheck on --run--
Bug689369.pdf -  /rangecheck on --run--
Bug687351.pdf -  /unknownerror on --run--
Bug690195.pdf - /typecheck on .execform1

all now work correctly. The problem was with supplying a glyph name instead of
a CID to the CDevProc procedure, or supplying a CID instead of a GID to the
font interpreter when a CIDFont with TrueType outlines was processed.

These files still have problems:
0109848_cut.ps
Adobe Garamond CE.pdf
Bauhaus CE.pdf
Futura CE.pdf 
Bug687889.pdf 
Bug688154.ps 
Bug689516.pdf 
german.ps
pscharstrings5.ps
test-setweightvector.ps 
type1-ce1_setcurrentpoint.ps

In addition a number of files 'sometimes' cause crashes when reading TrueType
sfnt data (depends on memory setup whether a crash occurs) because the code
tries to read more data from a table than is available in the table.

Initially I'm going to protect against the crash by some length checking, need
to investigate these further to see why the sfnt reader is trying to read too
much data.
Comment 16 Ken Sharp 2009-11-26 08:55:14 UTC
Revision 10389 addresses the intermittent crashing problem which was caused by
confusion in the sfnts reading code over pointers and offsets.
Comment 17 Ken Sharp 2009-11-27 07:59:17 UTC
Revision 10393 fixes Adobe Garamond CE.pdf, Bauhaus CE.pdf and Futura CE.pdf.
Remaining files exhibiting problems:

0109848_cut.ps
Bug687889.pdf 
Bug688154.ps 
Bug689516.pdf 
german.ps
pscharstrings5.ps
test-setweightvector.ps 
type1-ce1_setcurrentpoint.ps

Comment 18 Ken Sharp 2009-11-28 03:08:51 UTC
revision 10398 fixes test-setweightvector.ps

Remaining files producing errors:
0109848_cut.ps
Bug687889.pdf 
Bug688154.ps 
Bug689516.pdf 
german.ps
pscharstrings5.ps
type1-ce1_setcurrentpoint.ps
Comment 19 Ken Sharp 2009-11-30 01:06:47 UTC
revision 10405 fixes 0109848_cut.ps, the problem was that the code did not 
cater for a 'mixed array' (internal GS type) FontBBox. 

Remaining files with errors:
Bug687889.pdf 
Bug688154.ps 
Bug689516.pdf 
german.ps
pscharstrings5.ps
type1-ce1_setcurrentpoint.ps
Comment 20 Ken Sharp 2009-12-02 06:21:11 UTC
Revision 10434 fixes Bug687889.pdf, remaining problem files:

Bug688154.ps 
Bug689516.pdf 
german.ps
pscharstrings5.ps
type1-ce1_setcurrentpoint.ps
Comment 21 SaGS 2009-12-03 17:44:35 UTC
Created attachment 5741 [details]
Suggested fix for regression in rev 10433+10434

The problem:
    Revision 10433 introduces a problem with Type42 glyphs that 
    have length == 0. The logic that attempts to deal with an 
    unsorted ‘loca’, which is used even if the ‘loca’ is sorted, 
    will make such a glyph identical to the next (in ‘glyf’ table 
    order) non-empty glyph, similar to the situation described in 
    Bug #688971 comment #7 ‘Huge performace problem (with large TT 
    font?)’ point (F).

To reproduce:
    gswin32c -dTextAlphaBits=4 "Attachment #2573 [details].ps"

    Without FreeType, it displays the word ‘Neckar’, with spaces 
    before and after. With FreeType and rev 10433, it displays two 
    exclamation points instead of the 2 spaces. The ‘!’ is the 
    non-empty glyph that follows the empty ‘space’ glyph.

    Note: I cannot say I understand why the ‘-dTextAlphaBits=4’ 
    makes a difference, but it does.

Suggested fix:
    See attached patch. gstype42.c::gs_type42_font_init() tries hard 
    to deal with unsorted ‘loca’s and compute the glyph lengths, 
    storing these lengths in gs_font_type42.data.len_glyphs[].

    As an added bonus, the new code is much faster.

    Note: not extensively tested.
Comment 22 SaGS 2009-12-03 17:45:14 UTC
Another file that does not work with the FreeType bridge is attachment #3313 [details] 
(from bug #689267 comment #6). It contains Type 1 (and Type 3) fonts with 
weird ‘/FontMatrix’ entries and with non-horizontal advance vectors for 
glyphs. Don’t know if this is a bug in the FreeType code itself or a problem 
in the GS glue code.
Comment 23 Ken Sharp 2009-12-04 07:31:37 UTC
SaGS thanks for the patch, I'll commit it shortly, I wasn't aware the glyph
lengths were already pre-calculated and had intended to add this later as an
optimisation. Nice to find out its already there :-) In fact I'll take the
opportunity to remove offset1 altogether as its only used for the glyph length
calculation, which we now don't need to do.

For now I'm concentrating on getting FT to work without producing errors, then I
have to check the quality (to make sure no glyphs are missing or badly
malformed) then think about performance if it seems required.

Regarding the other file, I'll have to look into it, I'm down to the weird and
wonderful bugs now :-( 
Comment 24 Ken Sharp 2009-12-04 07:41:09 UTC
revision 10449 fixes Bug688154.ps, remaining files with issues:
Bug689516.pdf 
german.ps
pscharstrings5.ps
type1-ce1_setcurrentpoint.ps
Bug689267.ps (my naming)

Comment 25 Ken Sharp 2009-12-04 07:53:50 UTC
revision 10450 should resolve the problem with 0 length glyphs introduced with
revision 10434, it should also show a small performance improvement since we now
use the pre-calculated glyph lengths instead of messing about with the LOCA
table again in order to calculate them.
Comment 26 Ken Sharp 2009-12-07 07:51:20 UTC
Bug689516.pdf contains multiple TrueType fonts with genuine problems. One of
them contains least three distinct fatal errors;
1) GLYF and LOCA tables contain more than NumGlyphs glyphs, and some of these
glyphs are used.
2) At least one glyph uses more points than the declared maxPoints value.
3) The FPGM is invalid.

It seems Acrobat ignores all these errors, and simply does not render the glyphs
with problems. Ghostscript's internal TT rendering engine has previously been
modified to issue warnings and do the same.

revision 10459 modifies the FreeType FAPI bridge to emit similar warnings to
regular GS and similarly ignores hte faulty glpyh(s).

Remaining files with problems:
german.ps
pscharstrings5.ps
type1-ce1_setcurrentpoint.ps
Bug689267.ps (my naming)
Comment 27 Ken Sharp 2009-12-08 03:00:02 UTC
revision 10467 resolves the problems with german.ps and
type1-ce1_setcurrentpoint.ps for the time being. These files contain fonts which
are technically invalid, but regular GS, Jaws and Acrobat Distiller all ignore
the error.

Remaining files with issues:
pscharstrings5.ps
Bug689267.ps (my naming)
Comment 28 Ken Sharp 2009-12-08 04:04:28 UTC
Hmmm. pscharstrings5.ps is a very peculiar (and silly) file. The job contains a
Type42 font, but the job adds a CharStrings dictionary to the font, the
dictionary includes some PostScript routines to draw what are, effectively, type
3 glyphs for two of the glyphs. One of the glyphs is explicitly added to the
Encoding, I'm not sure how the other one works yet. The other is already present
in the Encoding and apparently supersedes the type 42 sfnts.

It looks like we need to test the font to see if it has a CharStrings dictionary
before we pass it to FreeType, and if it does we need to extract and execute the
relevant procedure from the CharStrings dictionary.

I wonder if this is also a requirement for type 1 fonts....
Comment 29 Ken Sharp 2009-12-08 08:11:51 UTC
revision 10468 adds support for Type 42 fonts whose CharStrings dictionary
contains PostScript glyph descriptions. This fixes pscharstring5.ps, assuming a
regression test does not find any more problems I will then close this issue.

Sags, I'm aware this doesn't resolve your reported problem, but I want to tackle
quality problems separately, this bug is really for me to track errors and
crashes in the FreeType implementation. When I'm happy with this (now I hope) I
will open a new issue for quality problems, and transfer your fault to there.
Comment 30 Ralph Giles 2009-12-08 09:42:48 UTC
The setcurrentpoint issue (comment #27) is reported upstream as
https://savannah.nongnu.org/bugs/?28226
Comment 31 Ken Sharp 2010-01-13 08:33:56 UTC
All crashes and Ghostscript PostScript errors are resolved and the current
development snapshot of FreeType works 'as is'with Ghostscript.

QA testing of a good selection of our test suite reveals the following problems
still exist (at 72 dpi)

0000728-simpleNG.pdf 	- missing glyphs
030b-1.pdf 		- extra glyphs rendered ?
6800-0000000028.pdf 	- '-' glyphs missing
bug687615.ps 		- glyphs too large
Bug688728.pdf 		- TrueType issue - B, C, D, G and H malformed
bug689267.ps 		- many badly formed glyphs using custom type 1/3 fonts
bug690113.pdf 		- FT renderer error 'will do with native GS renderer'
bug690116.eps 		- glyph missing
bug690166.ps 		- spacing wrong
catx4279.pdf 		- spacing wrong
china2.pdf 		- misrendered TrueType
CIDDownloaded.PS 	- missing glyph on page 2
embed.eps 		- glyphs rendered huge 
EmbeddedFontCopies.pdf 	- spacing problem
german.ps 		- accents misplaced
gs_stroke_error.pdf 	- 'correct' checkmark rendered incorrectly
HeiSeiminstd.pdf 	- FT failure, uses GS renderer
ia3z0240.pdf 		- page 1 'A guide for...' text rendered very badly
japan-.ps 		- glyph rendered poorly
objlune-font2-mine.ps 	- glyph not rendered
shadowtext.eps 		- crash
T2cgarstring.eps 	- incorrect spacing
two_encodings.ps 	- text rendered too large (see embed.pd)
type11withvmtx.pdf 	- glyphs rendered incorrectly
wd0010l0.pdf 		- glyphs missing

Generally
============
Problem with TrueType .notdef glyphs, can't avoid rendering these at present.
Type 1 artificially italicised fonts are too slanted.
Comment 32 Ken Sharp 2010-01-15 10:52:18 UTC
More problems. The following files generate an error 'Can't render Font xxxx
with FAPI, will do with native GS renderer' and fall back to the Ghostscript
internal renderer:

Bug689267.ps
Bug690166.ps
chartab.pdf
HeiSeiMinStd.pdf
IAZ31869.pdf
missingchars.pdf
small.pdf
WEV_052_bad.pdf

These problems weren't picked up before as they don't raise PostScript errors,
crash, or produce bad output....
Comment 33 Ken Sharp 2010-01-18 00:42:37 UTC
Bug690113 is fixed by revision 10614:

http://ghostscript.com/pipermail/gs-cvs/2010-January/010359.html

A fault in the generation of a serialised CFF font for consumption by FreeType.
Comment 34 Ken Sharp 2010-01-18 07:59:35 UTC
Revision 10617 addresses many (but not all) of the problems with bug689267.ps.

The GS FAPI code was never using the vertical advance for glyphs, resulting in
the vertical advance always being 0, which led to many glyphs being misplaced.

In addition, FreeType was not maintaining the vertical advance from the type 1
font but was always setting it to a synthesized value based on the glyph
bounding box. This resulted in glyphs being misplaced even after the FAPI used
the vertical advance value.

The patch in revision 10617 resolves these issues and many of these tests now
work correctly, however there are still problems with FreeType and
rotated/skewed FontMatrix settings in type 1 fonts. This appears also to be the
problem noted above where artificially italicised fonts are rendered too
'slanted'. Note that in this case the horizontal and vertical advance values are
again slightly incorrect as well. I believe these problems are now in the
FreeType code.

These problems need to be addressed in further patches. Reference comment #22 by
SaGS which I had forgotten about (sorry about that).
Comment 35 Ken Sharp 2010-01-18 08:31:38 UTC
Revision 10617 supplied as patch attached to the FreeType bug report #28678:

https://savannah.nongnu.org/bugs/index.php?28678
Comment 36 Ken Sharp 2010-01-28 06:38:54 UTC
revision 10653 improves bug687615.ps and two_encodings.ps. These files (and my
minimalist test file) now render glyphs at the correct size. However
bug687615.ps has incorrect spacing and two_encodings.ps has many missing glyphs.
These are probably related to the other instances of these errors.
Comment 37 Chris Liddell (chrisl) 2010-02-04 07:48:35 UTC
With the full FT bytecode interpreter enabled, the glyphs in:

china2.pdf
japan-.ps

appear to be well formed, but clipped (probably due to a wider issue with FAPI
glyphs being unexpectedly clipped when caching).
Comment 38 Chris Liddell (chrisl) 2010-02-18 01:38:38 UTC
IAZ31869.pdf now doesn't cause a FAPI error, but does highlight an issue with
FT's incremental interface and Truetype hinting - reported as FT bug:
https://savannah.nongnu.org/bugs/?28905

Comment 39 Chris Liddell (chrisl) 2010-02-22 04:10:42 UTC
chartab.pdf
HeiSeiMinStd.pdf
IA3Z1869.pdf
missingchars.pdf
WEV_052_bad.pdf (& small.pdf)

All now rip without warnings using FAPI/Freetype (no longer fall back to GS
native) with revision 10782.

Comment 40 Chris Liddell (chrisl) 2010-02-22 04:15:49 UTC
"IAZ31869.pdf now doesn't cause a FAPI error, but does highlight an issue with
FT's incremental interface and Truetype hinting - reported as FT bug:
https://savannah.nongnu.org/bugs/?28905"

Patch for FT accepted.

Comment 41 Ken Sharp 2010-03-08 17:13:18 UTC
revision 10880 fixes problems caused by FreeType creating vertical metrics when none are actually present in the type 42 font. We now ignore these. This fixes a number of 'diagonal text' problems. Also this code fixes an uninitialised variable which was leading to mispositioned text when using a /Metrics override.

Known problems at 72 dpi:
0000468-In1Arial		Many missing glyphs, misplaced glyphs
0000728-simpleNG		Missing text '150'
05Aug01NewsFax250-1		Missing underlines on page 4
addesso7			page 2, underline missing
bug687111			some '-' missing
bug687845			missing glyphs
bug689516			text missing
bug690166			spacing incorrect
bug690395			'W' misformed in 'Win deze'
catx3717			missing underlines
catx4279			spacing wrong, multiple missing '-'
catx9214.pdf			missing underlines
CIDDownloaded			missing '1'
EmbeddedFontCopies		spacing incorrect
ia3z0815			/.notdef rendered as square
ia3z3476			page 17 underlines missing
ia3z35193			stroke and fill mismatch!
objlune-font2_mine		page 2, glyph missing 'A'
tpc2				page 2, several '.' missing
transflag			'_' underlines missing
type11withvmtx2			text rendered incorrectly (TT byte code ?)
userguide1			page 4 '-' missing
WD0010L01			glyphs missing

Additional known problems at 300 dpi
(1162)cmyk9.pdf			-malformed glyphs
0000468-In1Arial.pdf		-glyph shape/clipping
09-47A.ps			-page 1, 2, 3, 4, 5, 6, 7 malformed glyphs
12-05.ps			-glyph too small, right edge jagged (!), page 2&3 missing text
34-all.ps			-page 4 many missing glyphs (/.notdef ?)
bug687121.ps			-serif odd, and glyph too small
bug689918.pdf			-broken glyphs
bug690170.pdf			-broken glyphs
bug690395.pdf			-W broken and z broken
catx0383			-malformed glyphs
catx1525			-malformed glyphs
do826e00003			-malformed glyphs
font13				-malformed glyphs
HelveticaObliqueTest		-malformed glyphs
pattyp11			-malformed glyph ('S')
pattyp1cd			-malformed glyph ('S')
pdf-cid-tt-embed		-4 row 'C' misplaced
realcities.pdf			-malformed glyphs
scienceposter.pdf		-malformed glyphs ('N')
s_36-unc.pdf			-malformed glyphs
s_36.pdf			-malformed glyphs
Comment 42 Chris Liddell (chrisl) 2010-03-10 15:20:55 UTC
china2.pdf works correctly as of revision 10909
Comment 43 Chris Liddell (chrisl) 2010-03-11 17:33:47 UTC
Of the files thrown up by Marcos' test run, all but one are caused by Freetype bug #27442:
https://savannah.nongnu.org/bugs/?27442

It's been open for a while, so I submitted a patch, Werner has accepted it.

The one remaining file from Marcos' list is:

34_all.PS

which gives a *lot* of "TrueType parsing error" messages.
Comment 44 Ken Sharp 2010-03-22 15:09:36 UTC
The missing '-', underline and '.' glyphs all appear to be dropouts caused by rendering the text at low resolution (72 dpi). This is a FreeType issue, reported as issue #29298. This doesn't seem to be a critical problem, as the glyphs reappear when rendered at higher resolutions, but it would be nice if it were consistent (vertically flipped glyphs sometimes *do* render).
Comment 45 Chris Liddell (chrisl) 2010-03-22 17:15:29 UTC
(In reply to comment #43)
> The one remaining file from Marcos' list is:
> 
> 34_all.PS
> 
> which gives a *lot* of "TrueType parsing error" messages.

This was due to the CID handling in the FAPI code having a GDBytes value hardcoded as 2.

This is resolved with revision: 10953
Comment 46 Chris Liddell (chrisl) 2010-03-30 16:41:21 UTC
Two things, so they don't get lost.

When GS functions are called via the FT API callbacks (for example, when we resort to grabbing an outline), errors generated from the GS code tend to get lost en route back to through FT.

Second, FAPI should be updated to include a pointer to the GS context, and the use of FT should be changed to supply FT_MemoryRec so that FT uses GS managed memory (the GS context is needed for the GS memory manager).
Comment 47 Chris Liddell (chrisl) 2010-03-31 14:18:43 UTC
Testing at low res (100 dpi), with revisions 10997, 10998 and 10999:

0000468-In1Arial - now works
bug689516 - missing glyphs compared to trunk GS, but not compared to Acrobat.
bug690166 - now works
bug690395 - now works
EmbeddedFontCopies - now works
type11withvmtx2 - correct with the full byte code implementation
Comment 48 Chris Liddell (chrisl) 2010-03-31 14:44:47 UTC
Testing at low res (100 dpi), with revisions 10997, 10998 and 10999:


0000728-simpleNG.ps - now works.
Comment 49 Ken Sharp 2010-04-02 09:08:23 UTC
revision 11007 fixes the malformed glyphs at 300 dpi, The cause was a calculation in the FT scan conversion which calculates the slope of a line using a multiply and divide. The calculation did not account for the possibility of arithmetic overflow during the multiply.
Comment 50 Chris Liddell (chrisl) 2010-04-02 09:22:58 UTC
As of revision: 11008

0000728-simpleNG - now works.
Comment 51 Ken Sharp 2010-04-02 12:36:49 UTC
FreeType's hinting is worse than useless for type 1 fonts, its actually harmful.
It distorts the glyph shape because it hints an unscaled outline. While it would be best to fix FreeType's hinting this would be a large project.

Revision 11009 disables hinting with FreeType when the font is a type 1 font. This resolves a number of spacing issues as well as fixing the stroke/fill mismatch in ia3z3519.pdf and correctly scaling the face in womanface.pdf.

Remaining known issues at 72 dpi:
-----------------------------------
093-01.ps 		test 3 and test 10 look 'odd'
12-05.ps		'@' seems a little small

bug688154.ps		glyph shifted vertically

000728-simpleNG.PDF	missing glyphs '150'
bug689516.pdf		missing glyphs
CIDDownloaded		missing glyphs
objlune-font2_mine	missing glyphs
wd00l01.pdf		missing glyphs


093-01 may not be a concern, same for 12-05. The various missing glyphs may all be the same issue; wd00L01.pdf and 000728-simpleNG.pdf are believed to be the same.
Comment 52 Ken Sharp 2010-04-03 08:48:07 UTC
Couple more issues turned up when testing at 300 dpi

catx0383.pdf - this is wrong at both 72 and 300 dpi, but its hard to see at 72 dpi because the text is small. THe text labelling the roads is rendered incorrectly.

pdf-cid-tt-embed.pdf This is a test of PDF 'Tr' and one of the 'C' glyphs is mispositioned when rendered at 300 dpi 

Summary of outstanding problems:

093-01.ps             test 3 and test 10, missing/mispositioned glyphs

bug688154.ps          glyph shifted vertically

000728-simpleNG.PDF   missing glyphs '150'
CIDDownloaded         missing glyphs
wd00l01.pdf           missing glyphs

bug689516.pdf         missing glyphs (but now matches Acrobat, trunk does not)

objlune-font2_mine    missing glyphs
catx0383.pdf          misrendered glyphs
pdf-cid-tt-mbed.pdf   mispositioned glyph at 300 dpi
Comment 53 Ken Sharp 2010-04-03 09:12:37 UTC
In the case of pdf-cid-tt-embed.pdf the whole fourth line of text (misplaced 'C') should not be rendered. Truly strange.
Comment 54 Ken Sharp 2010-04-03 11:23:09 UTC
objlune-font2_mine.ps contains a glyph which is technically invalid, and is flagged as such by FreeType.

The 'A' glyph begins:

0 685 hsbw
245 730 rlineto


The hsbw operator sets the current point to 0 0 but specifically does not add that point to the current path. The type 1 specification states quite clearly that the first operation should be an rmoveto (obviously other moveto variants are acceptable) to create the first point in the path.

However, when distilled, the glyph is rendered by Adobe Acrobat, Jaws and the existing interpreter in Ghostscript (rather oddly in this case though, perhaps the initial point is incorrectly set). Ignoring the technical fault in FreeType, the glyph renders correctly.
Comment 55 Chris Liddell (chrisl) 2010-04-05 10:07:31 UTC
As of revision 11010 the following now work correctly:

000728-simpleNG.PDF   missing glyphs '150'
CIDDownloaded         missing glyphs
wd00l01.pdf           missing glyphs
Comment 56 Ken Sharp 2010-04-05 16:33:18 UTC
revision 11012 & 11014 resolve the issue with objlune_font2_mine.ps. The font has an invalid glyph (line before move immediately after hsbw) but CPSI, Acrobat current GS and Jaws allignore this error. The revision makes FT ignore the error as well.

revision 11013 resolves the problem with bug688154.ps. The font contains embedded bitmaps for use at certain resolutions. If we use these then FT creates vertical metrics, including advance, and that messes up our own advance calculations. Telling FT not to use embedded bitmaps solves the problem and produces a more consistent output.

pdf_cid_tt_embed.pdf is correct using FT, but incorrect using trunk at 300 dpi (though its fine at 72 dpi). Since the FT build is correct, we won't look further at this.

catx0383.pdf was fixed already by one of our submissions to FT, but we hadn't resynchronised our copy. Fixed now.

bug689516.pdf matches the Acrobat output, though not trunk. We will accept this as a decent definition of correctness, even though its different to trunk. (Trunk is probably incorrect).

This leaves only the QL test file 093-01.ps which has some peculiar (not necessarily incorrect) results which I intend to investigate a little further.
Comment 57 James Cloos 2010-04-05 19:05:04 UTC
(In reply to comment #51)
> FreeType's hinting is worse than useless for type 1 fonts, its actually
> harmful.  It distorts the glyph shape because it hints an unscaled outline.

> While it would be best to fix FreeType's hinting this would be a large project.

If anyone is interested, the plan is to replace the current t1 hinting module with one which uses the autofit module, providing the hints from the font instead of letting autofit guess them itself.

It hasn’t gotten done because autofit does a good job on its own (especially in its vertical-only mode, named light autohinting in the API) and most t1/cff fonts these days seem to be autohinted anyway.

FT_LOAD_TARGET_LIGHT|FT_LOAD_FORCE_AUTOHINT may even be a better choice for gs than FT_LOAD_NO_HINTING, for type1 and cff.

Also, with regard to 11009, does a_fapi_font->is_type1 match cff fonts, too?  I doubt ft’s cff hinting is any different than its type1 hinting....
Comment 58 Ken Sharp 2010-04-06 07:22:23 UTC
Hi James, I'm certainly interested to hear that someone is working on the type 1 hinting in FreeType. Turning off type 1 hinting isn't an approach we are happy with, but it was the only way to resolve the problems in a number of files.

I'm not sure what you mean by most fonts being auto-hinted, unless you mean the font generating application adds the hints itself automatically, rather than being added manually by a typographer.

I had intended to look into the autohinter anyway, after reading the paper Werner mentioned on the ft-devel list, especially with reference to anti-aliasing which is anoother area we still have to look at.

I'll give this a try with the worst offending job and see what it looks like. In this file we have a stroke which is supposed to lie around the outside of a piece of text. The stroke isn't generated using charpath though, its a specified path, and the existing T1 hinting made the filled text very different from the stroked text.

is_type1 does match CFF fonts as well, and no the hinting isn't any different for CFF outlines as far as I can tell.
Comment 59 Ken Sharp 2010-04-06 10:37:27 UTC
A few ad-hoc tests indicate to me that using FreeType unhinted with type 1 fonts produces output which is closer to the output of Adobe than using the auto hinter, testing at 72 dpi.

There are some files which from a subjective point of view are improved slightly using the auto hinter, but equally there are a number of files which are degraded.

For now we'll stick with disabling type 1 hinting, we should return to this question and do more testing. However it seems likely that anti-aliasing will yield better results for on-screen viewing anyway.
Comment 60 Ken Sharp 2010-04-06 10:57:36 UTC
Created attachment 6151 [details]
hinting.zip

The attached zip file demonstrates some selected files using type 1 fonts, and the results of various kinds of hinting.

TIFF files are output from Acrobat at 72 dpi, files labelled 'head' use the current Ghostscript font machinery. The other files use FreeType and are unhinted, FT type 1 hinted, FT auto-hinted or FT auto-hinted with light hinting.

womanface shows the main problem using FreeType's type 1 hinting, which compresses the glyph outline. (Note also the mess Acrobat makes of this). 

Bug690166 shows the more regular use of type 1 text.
Comment 61 Ken Sharp 2010-04-06 14:25:18 UTC
revision 11023 fixes test 3 in 093-01.ps. The FAPI and FreeType interface
were not correctly handling /Metrics overrides.

Test 10 in 093-01.ps uses a font with a broken CDevProc. The CDevProc reads:

    { 11 {pop} repeat 200 200 200 1200 1000 1000 200 -1000 500 1000 }

The return values from CDevPorc are fed into setcachedevice 2. From the PLRM, the numbers are respectively:

W0x W0y LLx LLy URx URy W1x W1y Vx Vy

LLx, LLy, URx and URy are the distances from origin 0 to the lower-left and upper-right corners of the glyph bounding box. As you can see URy is below LLy. These values are in glyph co-ordinate space so the CTM is not required to be accounted for.

The PLRM refers us to setcachedevice for more details, which says "The declared bounding box must be correct—in other words, sufficiently large to enclose the entire glyph. If any marks fall outside this bounding box, the result is unpredictable."

Since URy is less than LLy the entire glyph technically falls outside the bounding box, and even if we ignore the inversion there are marks well outside the defined height. So our result is as acceptable as any other. We have tried this file on several different interpreters and each one has drawn a different result.

This is the final known issue with FAPI and the FreeType integration. There is further work to be done, which will be tracked in separate (hopefully shorter!) entries in the bug tracker.
Comment 62 James Cloos 2010-04-08 18:39:55 UTC
(In reply to comment #58)
> I'm not sure what you mean by most fonts being auto-hinted, unless you mean the
> font generating application adds the hints itself automatically, rather than
> being added manually by a typographer.

Yes, exactly.  Adobe’s fdk and fontforge both have quality autohinters; it seems unlikely that anyone does it manually anymore.  tweaking, perhaps, but not from scratch.

> I had intended to look into the autohinter anyway, after reading the paper
> Werner mentioned on the ft-devel list, especially with reference to
> anti-aliasing which is anoother area we still have to look at.

Which reminds me that the autofit modules does do a better job when fitting for aa than for mono output.  That may explain some of the issues you saw, such as Bug6901661-light-autohint.ppm.