Re: master bf0aeaa0d7a: Re-enable displaying `han' characters on Android

[Po Lu <luangruo@yahoo.com>]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't understand what you are saying.  What is "my discovery"?  And
> why no font will ever match the font spec for 'han'? I've definitely
> seen fonts that support _all_ of the characters I added to
> script-representative-chars, so why wouldn't they be found?

I wasn't implying that no font will ever support one or more of these
characters, but that such fonts are sufficiently obscure that the
chances of a CJK font's being located by this value of
`script-representative-chars' is nil.

>> > The font Emacs sometimes selects due to those characters missing
>> > lacked support for important Han blocks because those blocks had no
>> > characters in script-representative-chars.
>> 
>> I didn't revert your change in whole, only characters beyond the BMP
>> that seldom appear in real Chinese writing; of the characters that were
>> deleted:
>> 
>>   #x1f210 #x20000 #x2a700 #x2b740 #x2b820 #x2ceb0 #x2f804
>
> I added them because when those sub-optimal fonts are selected, some
> of these characters appear as "tofu", which is ridiculous on a system
> that has fonts installed that cover all of them.

These characters may be han script, but they are not attested in CJK
documents in practice, except in contrived scenarios such as conversions
between incomplete character encodings.

> And "seldom" is in the eyes of the beholder, at least IME.  When one
> has text with these characters, the absolute frequency of their
> appearance is not very relevant; what _is_ relevant is the fact that
> the character cannot be shown by Emacs.

In practice, the outcome of this principle is that no font is detected
with which to display CJK documents featuring none of these characters,
very much against the expectations of CJK users.

> If there's no fonts installed that support those representative
> characters, and Emacs is capable of finding less capable fonts that
> support some of CJK (e.g., the BMP blocks), then why is that a
> problem?

I thought I explained that Emacs is _not_ capable of doing so on
Android.

> The purpose of the change is to allow Emacs to find better fonts if
> they are installed, instead of ignoring them.  How is that a Bad
> Thing?

Because it renders the `han' script incapable of matching any fonts that
are installed in practice.

> I still don't understand why this breaks Android, btw.  If Emacs
> employs the fallback font specs with :lang you show above, why don't
> they work for Android?

The problem is that QClang is not available on Android, because fonts do
not provide their design languages in one of the standard TrueType
tables its font backend groks, which deficiency prompted the addition of
the font spec in question in:

2023-02-16  Po Lu  <luangruo@yahoo.com>

	* doc/emacs/android.texi (Android Fonts):
	* doc/emacs/input.texi (On-Screen Keyboards):
	* doc/lispref/commands.texi (Misc Events): Update documentation.

	* java/org/gnu/emacs/EmacsInputConnection.java (setSelection): New
	function.
	* java/org/gnu/emacs/EmacsSurfaceView.java
	(reconfigureFrontBuffer): Make bitmap references weak references.

	* java/org/gnu/emacs/EmacsView.java (handleDirtyBitmap): Don't
	clear surfaceView bitmap.

	* lisp/comint.el (comint-mode): Set text-conversion-style to
	`action' so on screen keyboards' Return buttons send an actual key
	press event.

	* lisp/international/fontset.el (script-representative-chars)
	(setup-default-fontset): Improve detection of CJK fonts.

> The log message talked only about Android.  If users on GNU/Linux
> report problems caused by this change, with enough details to
> understand the problems, we can reconsider the change and modify it or
> even revert.  But I do need the details on those other platforms to
> think and discuss this intelligently.  This is all about details, it
> isn't an abstract or academic issue.  Choosing which characters to
> consider representative is frequently a judgment call based on
> practical considerations and practical problems with existing fonts.

The fact of the matter is that:

(let ((script-representative-chars
       '((han #x2e90 #x2f00 #x3010 #x3200 #x3300 #x3400
	      #x31c0 #x4e10 #x5B57 #xfe30 #xf900
	      #x1f210 #x20000 #x2a700 #x2b740 #x2b820 #x2ceb0 #x2f804))))
  (clear-font-cache)
  (find-font (font-spec :registry "iso10646-1" :script 'han
                        :type 'xfthb))) ;; or another ftfont backend.

returns no font on an up-to-date Fedora Workstation installation with a
wealth of multilingual fonts for CJK scripts, whereas:

(let ((script-representative-chars
       '((han #x2e90 #x2f00 #x3010 #x3200 #x3300 #x3400
	      #x31c0 #x4e10 #x5B57 #xfe30 #xf900))))
  (clear-font-cache)
  (find-font (font-spec :registry "iso10646-1" :script 'han
                        :type 'xfthb)))

returns:

#<font-entity xfthb ADBO Noto\ Sans\ CJK\ HK nil iso10646-1 medium normal normal 0 nil nil 0>

which is more than adequate for editing CJK text in my language and
others.