Re: master bf0aeaa0d7a: Re-enable displaying `han' characters on Android

[Po Lu <luangruo@yahoo.com>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Po Lu <luangruo@yahoo.com>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 01 Aug 2024 16:16:58 +0800
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I don't understand what you are saying.  What is "my discovery"?  And
>> > why no font will ever match the font spec for 'han'? I've definitely
>> > seen fonts that support _all_ of the characters I added to
>> > script-representative-chars, so why wouldn't they be found?
>> 
>> I wasn't implying that no font will ever support one or more of these
>> characters, but that such fonts are sufficiently obscure that the
>> chances of a CJK font's being located by this value of
>> `script-representative-chars' is nil.
>
> I very much doubt that.  I see a few fonts on my Windows 11 system
> which support all of those.  I'd be very surprised to know that no
> such fonts are available on a modern GNU/Linux system.
>
> Can someone please check that?

I think I just did, but anyone is more than welcome to put the links I
posted into Firefox.

>> These characters may be han script, but they are not attested in CJK
>> documents in practice, except in contrived scenarios such as conversions
>> between incomplete character encodings.
>
> You keep saying that, but those are just your assertions.  These
> characters are there for a reason, and they should be supported as
> well as we can.  If worse comes to worst, we could split 'han' into
> two or more scripts, and have separate setup in our fontsets for them.
> Then each one could use its own representative characters.

I keep saying this because I read and write CJK characters in Emacs and
elsewhere, every day, without ever encountering the characters in
question, since, had I done so, the tofu would not have escaped my
notice.

> Not IME.

With respect, I think I write within compass when I say that no one has
expressed enough interest in these characters to persuade Noto
developers of their importance despite having nearly a decade:

  https://github.com/notofonts/noto-cjk/issues/13

and so these characters are likely to remain unavailable from a single
font on GNU/Linux systems for the foreseeable future.

>> > If there's no fonts installed that support those representative
>> > characters, and Emacs is capable of finding less capable fonts that
>> > support some of CJK (e.g., the BMP blocks), then why is that a
>> > problem?
>> 
>> I thought I explained that Emacs is _not_ capable of doing so on
>> Android.
>
> Then please design and implement a suitable solution for Android.  It
> is not right to punish other platforms for Android-specific issues.

No one is being punished or put at a disadvantage in any wise.
Previously, script-representative-chars was downright ignored, an
arrangement that existed till the Android port was installed and which
has been restored, and no more.

>> > The purpose of the change is to allow Emacs to find better fonts if
>> > they are installed, instead of ignoring them.  How is that a Bad
>> > Thing?
>> 
>> Because it renders the `han' script incapable of matching any fonts that
>> are installed in practice.
>
> Again, not IME.

Any GNU/Linux user is perfectly welcome to run:

  (dolist (char '(#x20000 #x2a700 #x2b740 #x2b820 #x2ceb0))
    (insert (format "%c" char)))

and reach the same conclusion I have.

>> > I still don't understand why this breaks Android, btw.  If Emacs
>> > employs the fallback font specs with :lang you show above, why don't
>> > they work for Android?
>> 
>> The problem is that QClang is not available on Android, because fonts do
>> not provide their design languages in one of the standard TrueType
>> tables its font backend groks, which deficiency prompted the addition of
>> the font spec in question in:
>
> OK, then it means we need to work around this, but without hampering
> other platforms.

Yes, that's the essence of the change installed in emacs-30.

>> The fact of the matter is that:
>> 
>> (let ((script-representative-chars
>>        '((han #x2e90 #x2f00 #x3010 #x3200 #x3300 #x3400
>> 	      #x31c0 #x4e10 #x5B57 #xfe30 #xf900
>> 	      #x1f210 #x20000 #x2a700 #x2b740 #x2b820 #x2ceb0 #x2f804))))
>>   (clear-font-cache)
>>   (find-font (font-spec :registry "iso10646-1" :script 'han
>>                         :type 'xfthb))) ;; or another ftfont backend.
>> 
>> returns no font on an up-to-date Fedora Workstation installation with a
>> wealth of multilingual fonts for CJK scripts, whereas:
>> 
>> (let ((script-representative-chars
>>        '((han #x2e90 #x2f00 #x3010 #x3200 #x3300 #x3400
>> 	      #x31c0 #x4e10 #x5B57 #xfe30 #xf900))))
>>   (clear-font-cache)
>>   (find-font (font-spec :registry "iso10646-1" :script 'han
>>                         :type 'xfthb)))
>> 
>> returns:
>> 
>> #<font-entity xfthb ADBO Noto\ Sans\ CJK\ HK nil iso10646-1 medium normal normal 0 nil nil 0>
>> 
>> which is more than adequate for editing CJK text in my language and
>> others.
>
> Not on MS-Windows: here, both of the above return
>
>   #<font-entity harfbuzz outline Malgun\ Gothic sans iso10646-1 bold normal normal 0 nil 0 nil>
>
> which is a lie in the latter case, since those additional characters
> are not supported by this font.  Given that this method is evidently
> unreliable, I don't think we should consider this a proof of your
> argument.

That's queer, and I suspect something is amiss in clear-font-cache.
Nevertheless, the results from Mozilla are essentially conclusive, as
far as I am concerned.