Bug 698305 - ghostscript cannot handle ps (lilypond output) with noto cjk font
Summary: ghostscript cannot handle ps (lilypond output) with noto cjk font
Status: RESOLVED INVALID
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: PS Interpreter (show other bugs)
Version: 9.21
Hardware: PC Linux
: P4 normal
Assignee: Ken Sharp
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-28 11:18 UTC by Tom Yan
Modified: 2017-07-30 03:39 UTC (History)
0 users

See Also:
Customer:
Word Size: ---


Attachments
lilypond file that generates the problematic ps (212 bytes, text/x-lilypond)
2017-07-28 11:18 UTC, Tom Yan
Details
Problematic PS file with Noto CJK font (XZ compressed) (12.21 MB, application/x-xz)
2017-07-29 11:53 UTC, Tom Yan
Details
Working PS file with WenQuanYi Zen Hei font (XZ compressed) (6.16 MB, application/x-xz)
2017-07-29 11:54 UTC, Tom Yan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tom Yan 2017-07-28 11:18:49 UTC
Created attachment 14027 [details]
lilypond file that generates the problematic ps

I am experiencing the following problem when opening a PostScript generated by lilypond 2.18.2 (http://lilypond.org/):

[tom@localhost ~]$ lilypond --ps gs.ly
GNU LilyPond 2.18.2
Processing `gs.ly'
Parsing...
Finding the ideal number of pages...
Fitting music on 1 page...
Drawing systems...
Layout output to `gs.ps'...
Success: compilation successfully completed
[tom@localhost ~]$ gs gs.ps
GPL Ghostscript 9.21 (2017-03-16)
Copyright (C) 2017 Artifex Software, Inc.  All rights reserved.
This software comes with NO WARRANTY: see the file PUBLIC for details.
Error: /undefinedresource in findresource
Operand stack:
   NotoSansCJKjp-Regular   CIDFont   NotoSansCJKjp-Regular
Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostringval--   --nostringval--   false   1   %stopped_push   1999   1   3   %oparray_pop   1998   1   3   %oparray_pop   1982   1   3   %oparray_pop   1868   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   1944   2   3   %oparray_pop   findresource   %errorexec_pop   --nostringval--   --nostringval--   --nostringval--
Dictionary stack:
   --dict:1208/1684(ro)(G)--   --dict:0/20(G)--   --dict:109/200(L)--   --dict:20/26(ro)(G)--
Current allocation mode is local
Last OS error: No such file or directory
Current file position is 36234603
GPL Ghostscript 9.21: Unrecoverable error, exit code 1
[tom@localhost ~]$ fc-match NotoSansCJKjp-Regular
NotoSansCJK-Regular.ttc: "Noto Sans CJK JP" "Regular"

If for example WenQuanYi Zen Hei (another Chinese font) is used instead, gs can handle it properly.

Attaching the lilypond file. Because the generated PS is pretty large (35M), so I am not sure if I should attach it. Let me know if I should.
Comment 1 Ken Sharp 2017-07-28 12:47:32 UTC
(In reply to Tom Yan from comment #0)

> Attaching the lilypond file. Because the generated PS is pretty large (35M),
> so I am not sure if I should attach it. Let me know if I should.

I don't have lilypond, and in any event probably wouldn't be able to run it in the same way, so you'll have to send the PostScript file.

Have you made the font available to Ghostscript ? The error suggests you have not and it is not present in the file. The mere presence of the font on the system is insufficient to make it available for Ghostscript to use, and you can't use TrueType fonts (even less so TrueType collections) from PostScript anyway as it doesn't support TrueType.
Comment 2 Tom Yan 2017-07-29 03:05:32 UTC
Okay I would see if I can upload it here as an attachment later when I get access to my computer.

I am not exactly sure how to differentiate the type fonts, but as far as I can tell, WenQuanYi Zen Hei is a TrueType Collection with TrueType (used in my test case) and bitmap glyph (not used). And for the non-working noto cjk fonts, the variant I got in Arch is apparently OpenType Collections
Comment 3 Tom Yan 2017-07-29 03:08:58 UTC
Okay I would see if I can upload it here as an attachment later when I get access to my computer.

I am not exactly sure how to differentiate the type fonts, but as far as I can tell, WenQuanYi Zen Hei is a TrueType Collection with TrueType (used in my test case) and bitmap glyph (not used). And for the non-working noto cjk fonts, the variant I got in Arch is apparently OpenType Collections with OpenType/CFF glyphs (though they have the extension name ttc for some reason).

How do I make them available for Ghostscript if they are supported? As far as I can tell the fonts are embedded in the PS for both cases, coz with noto cjk it's 10M larger or so.
Comment 4 Ken Sharp 2017-07-29 03:22:30 UTC
(In reply to Tom Yan from comment #3)
> Okay I would see if I can upload it here as an attachment later when I get
> access to my computer.

You can always use a compression tool (gzip, zip, etc) to reduce the file size, PostScript normally compresses well.

 
> I am not exactly sure how to differentiate the type fonts, but as far as I
> can tell, WenQuanYi Zen Hei is a TrueType Collection with TrueType (used in
> my test case) and bitmap glyph (not used). And for the non-working noto cjk
> fonts, the variant I got in Arch is apparently OpenType Collections with
> OpenType/CFF glyphs (though they have the extension name ttc for some
> reason).

OpenType is a successor to TrueType, OpenType fonts may contain either TrueType or CFF (Compact Font Format) outlines. Collections simply means the 'font' actually consists of a number of subsidiary fonts.

 
> How do I make them available for Ghostscript if they are supported? 

To make the fonts available you need to edit one or both of /ghostpdl/Reswource/Init/fontmap.GS for regular fonts. For CIDFonts (which are not the same thing) you need to edit cidfmap. If you are using a Ghostscript executable built to use the ROM file system you will then either need to use the -I switch to tell Ghostscript where to find the modified files, or rebuild Ghostscript with the modified files built in.


> As far
> as I can tell the fonts are embedded in the PS for both cases, coz with noto
> cjk it's 10M larger or so.

I need to see the file to have any chance of figuring this out. The fact that the PostScript includes the font doesn't mean it does so correctly, or in a portable fashion.

Sounds like Lilypond is including the entire font, which is at the very least hideously inefficient. Failing to incrementally download a subset as required suggests the author didn't understand fonts very well, so its not unlikely there's a problem with the PostScript program. But as I say, can't tell without seeing it.
Comment 5 Tom Yan 2017-07-29 11:53:00 UTC
Created attachment 14031 [details]
Problematic PS file with Noto CJK font (XZ compressed)
Comment 6 Tom Yan 2017-07-29 11:54:05 UTC
Created attachment 14032 [details]
Working PS file with WenQuanYi Zen Hei font (XZ compressed)
Comment 7 Tom Yan 2017-07-29 12:03:16 UTC
Some outputs that might be relevant:

[tom@archlinux ~]$ fc-match NotoSansCJKjp-Regular
NotoSansCJK-Regular.ttc: "Noto Sans CJK JP" "Regular"

[tom@archlinux ~]$ fc-query -f "%{postscriptname}\t%{fontformat}\n" /usr/share/fonts/noto/NotoSansCJK-Regular.ttc
NotoSansCJKjp-Regular	CFF
NotoSansCJKkr-Regular	CFF
NotoSansCJKsc-Regular	CFF
NotoSansCJKtc-Regular	CFF
NotoSansMonoCJKjp-Regular	CFF
NotoSansMonoCJKkr-Regular	CFF
NotoSansMonoCJKsc-Regular	CFF
NotoSansMonoCJKtc-Regular	CFF

[tom@archlinux ~]$ fc-match WenQuanYiZenHei
wqy-zenhei.ttc: "WenQuanYi Zen Hei" "Regular"

[tom@archlinux ~]$ fc-query -f "%{postscriptname}\t%{fontformat}\n" /usr/share/fonts/wenquanyi/wqy-zenhei/wqy-zenhei.ttc 
WenQuanYiZenHei	TrueType
WenQuanYiZenHeiMono	TrueType
WenQuanYiZenHeiSharp	TrueType
Comment 8 Ray Johnston 2017-07-29 16:57:57 UTC
I've looked into the supplied PostScript file -- the only thing that is
relevant to Ghostscript.

It goes to some length (543,590 lines of PS code) to create a Type42 font
with:
/FontName /NotoSansCJKjp-Regular def
/Encoding StandardEncoding def
/PaintType 0 def
/FontMatrix [1 0 0 1 0 0] def
/FontBBox [-1.002000 -1.048000 2.928000 1.808000] def
/FontType 42 def
/FontInfo 8 dict dup begin
/version (1.004) def
/isFixedPitch false def
/UnderlinePosition -0.125000 def
/UnderlineThickness 0.050000 def
end readonly def
/sfnts [
 ...
] def
/CharStrings 65535 dict dup begin
 ...
end readonly def
FontName currentdict end definefont pop

Then it tries to use this Type42 Font using:
 /NotoSansCJKjp-Regular /CIDFont findresource

But "definefont" defines this as a /Font resource (as appropriate for a Type
42 font) so the /CIDFont findresoresource gets an error.

Changing the /CIDFont to /Font before the findresource works, but then the
attempt to invoke "glypshow" results in an error since the PLRM says:
   glypshow shows the glyph for a single character from the current font or
   CIDFont; the character is identified by name if a base font or by CID if
   a CIDFont.
but since it is invoked with an integer (9842) and the font is a basefont,
Ghostscript correctly returns a "typecheck".

Probably lilypond needs to define a CIDFont so that integer glyphshow can be
used, but Ghostscript is operating correctly according to the PLRM.

It might be that othee Font experts (Chris or Ken) will want to weigh in on
this (particularly if I am off base), but I am closing this as INVALID since
it is clearly a lilypond (or whatever created the PS) bug.
Comment 9 Tom Yan 2017-07-29 22:01:05 UTC
Sounds plausible. So basically it created the font as a Type 42, and other part of the PostScript assume a Type 11 (Type 2 CIDFont).

One more question, apparently neither of these are right? Coz Noto CJK is obviously CFF, so it would only be right if it is created / embedded / accessed as instructed in section 5.8.1 of PLRM?
Comment 10 Ken Sharp 2017-07-30 03:39:11 UTC
(In reply to Tom Yan from comment #9)
> Sounds plausible.

This is what I was suspicious of, I have a vague memory of having looked at this for someone before.


> So basically it created the font as a Type 42, and other
> part of the PostScript assume a Type 11 (Type 2 CIDFont).

Essentially, yes. The working file, wqy.ps, creates the font as a regular Type 42 font. Although this limits the Encoding to 255 glyphs Lilypond abuses the glypshow operator to access glyphs directly by name, instead of using the Encoding. This is highly inefficient, but it does work.For example it makes for a seriously inefficient PDF file, because PDF does not have any equivalent to the glypshow operator, so you end up with lots of font definitions.

The faulty file, noto.ps, creates the font as a Font, in the same way as wqy.ps, so it could, and probably should, access it in exactly the same way. Instead it tries to find a CIDFont with that name, and then uses CIDs to access the glyphs for glyphshow, instead of glyph names.

Essentially the program needs to make its mind up; its currently schizophrenic, it creates a font, but tries to use a CIDFont. That's never going to work.

> One more question, apparently neither of these are right? Coz Noto CJK is
> obviously CFF, so it would only be right if it is created / embedded /
> accessed as instructed in section 5.8.1 of PLRM?

CFF fonts are only supported in certain versions of PostScript, I don't recall which version/supplement added support. (looks like it was added for language level 3)

Where they are supported, they can be Fonts or CIDFonts. CFF Fonts are font type 2, I'm having trouble proving it, but I believe CODFonts with CFF outlines are CIDFontType 0.

So no, you don't have to obey section 5.8.1, though if the PostScript program did, it would work, because it would use glyph names instead of CIDs.

Sorry, but this is not a Ghostscript bug and we can't fix it. FWIW I tried both files on Adobe Acrobat Distiller and another PostScript rip. The other rip threw an invalidfont error on noto.ps, but ran wqy.ps fine, just like Ghostscript; Adobe Acrobat threw an undefinedresource error on noto.ps and 'undefined; Offending Command:glyphshow' with wqy.ps.

So really Lilypond needs to do a *much* better job with its font support. You should probably raise a bug report with the maintainers.