bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

For bug#24938, I have to set inhibit-compacting-font-caches to non-nil
[0] to fix the lagging issue. It fixes almost but not all the
problems. Lagging reappears when I opening a multi languages file
(e.g. M-x view-hello-file),  Emacs could become very very slow until
the file is completely opened.

This issue may need another fix.


In GNU Emacs 26.1 (build 1, x86_64-w64-mingw32)
 of 2018-05-30 built on CIRROCUMULUS
Repository revision: 07f8f9bc5a51f5aa94eb099f3e15fbe0c20ea1ea
Windowing system distributor 'Microsoft Corp.', version 10.0.17134

Configured using:
 'configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install 'CFLAGS=-O2 -static -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS THREADS LCMS2

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp936

Major mode: Fundamental

Minor modes in effect:
  erc-spelling-mode: t
  erc-netsplit-mode: t
  erc-services-mode: t
  erc-networks-mode: t
  erc-log-mode: t
  erc-stamp-mode: t
  erc-ring-mode: t
  tabbar-mwheel-mode: t
  tabbar-mode: t
  delete-selection-mode: t
  show-paren-mode: t
  display-time-mode: t
  auto-image-file-mode: t
  desktop-save-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  view-mode: t

Memory information:
((conses 16 689409 87539)
 (symbols 56 46936 1)
 (miscs 48 72 133)
 (strings 32 85086 25863)
 (string-bytes 1 1976295)
 (vectors 16 48121)
 (vector-slots 8 1152206 20738)
 (floats 8 237 89)
 (intervals 56 21696 8137)
 (buffers 992 16))

[0] https://lists.gnu.org/archive/html/bug-gnu-emacs/2016-11/msg00471.html

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Moses <moses.mason@gmail.com>
> Date: Sun, 15 Jul 2018 00:02:13 +0000
> 
> For bug#24938, I have to set inhibit-compacting-font-caches to non-nil
> [0] to fix the lagging issue. It fixes almost but not all the
> problems. Lagging reappears when I opening a multi languages file
> (e.g. M-x view-hello-file),  Emacs could become very very slow until
> the file is completely opened.

That lagging might have a valid reason: how many fonts do you have
installed?  What does (length (x-list-fonts "*")) return?

If you have many fonts, Emacs will search them all to find a suitable
one, and if some characters in HELLO don't have fonts, it will search
them several times.  That might take time.  If the lagging disappears
once you display the entire file, that is most probably the reason.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

You are right. I got 5906 installed fonts returned from (length
(x-list-fonts "*")). However, this still a problem, which is really
annoying when editing files that contain more than one language. Other
editor does not have the similar issue. I am not an expert, but as far
as I know, the font fallback link is already stored in the register
under key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows
NT\CurrentVersion\FontLink\SystemLink, why Emacs still needs to search
fonts multiple times?


On Sun, Jul 15, 2018 at 2:39 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Moses <moses.mason@gmail.com>
> > Date: Sun, 15 Jul 2018 00:02:13 +0000
> >
> > For bug#24938, I have to set inhibit-compacting-font-caches to non-nil
> > [0] to fix the lagging issue. It fixes almost but not all the
> > problems. Lagging reappears when I opening a multi languages file
> > (e.g. M-x view-hello-file),  Emacs could become very very slow until
> > the file is completely opened.
>
> That lagging might have a valid reason: how many fonts do you have
> installed?  What does (length (x-list-fonts "*")) return?
>
> If you have many fonts, Emacs will search them all to find a suitable
> one, and if some characters in HELLO don't have fonts, it will search
> them several times.  That might take time.  If the lagging disappears
> once you display the entire file, that is most probably the reason.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Moses <moses.mason@gmail.com>
> Date: Sun, 15 Jul 2018 05:50:26 +0000
> Cc: 32159@debbugs.gnu.org
> 
> You are right. I got 5906 installed fonts returned from (length
> (x-list-fonts "*")). However, this still a problem, which is really
> annoying when editing files that contain more than one language.

Do some of the characters in HELLO display as boxes with hex code
inside them, and if so, do you see slow-down when Emacs is about to
display those characters?  If the answer to both question is YES, I
can suggest a customization to try that might help you avoid the
slowdown (assuming you don't need to be able to display characters
from those scripts), but please tell what characters are involved in
this.

> Other editor does not have the similar issue.

AFAIK, no other editor supports such a large variety of fonts.

> I am not an expert, but as far
> as I know, the font fallback link is already stored in the register
> under key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows
> NT\CurrentVersion\FontLink\SystemLink, why Emacs still needs to search
> fonts multiple times?

I don't think that Registry setting is relevant to the issue at hand.
Emacs looks for fonts based on its internal database of features
required by each script.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

On Sun, Jul 15, 2018 at 2:41 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Moses <moses.mason@gmail.com>
> > Date: Sun, 15 Jul 2018 05:50:26 +0000
> > Cc: 32159@debbugs.gnu.org
> >
> > You are right. I got 5906 installed fonts returned from (length
> > (x-list-fonts "*")). However, this still a problem, which is really
> > annoying when editing files that contain more than one language.
>
> Do some of the characters in HELLO display as boxes with hex code
> inside them, and if so, do you see slow-down when Emacs is about to
> display those characters?  If the answer to both question is YES, I
> can suggest a customization to try that might help you avoid the
> slowdown (assuming you don't need to be able to display characters
> from those scripts), but please tell what characters are involved in
> this.

Nope...all characters in the HELLO file can be display correctly
without a problem, the lagging issue only appears while opening the
file. Once it already opened, there is no more slowdown.

> > Other editor does not have the similar issue.
>
> AFAIK, no other editor supports such a large variety of fonts.

I mean other editors do not have slowdown problem while opening such a
relatively small file. They are not as mighty as Emacs of course.
That's why I am using Emacs to process editing :)


> > I am not an expert, but as far
> > as I know, the font fallback link is already stored in the register
> > under key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows
> > NT\CurrentVersion\FontLink\SystemLink, why Emacs still needs to search
> > fonts multiple times?
>
> I don't think that Registry setting is relevant to the issue at hand.
> Emacs looks for fonts based on its internal database of features
> required by each script.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

On July 16, 2018 5:35:07 AM GMT+03:00, Moses <moses.mason@gmail.com> wrote:
> On Sun, Jul 15, 2018 at 2:41 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: Moses <moses.mason@gmail.com>
> > > Date: Sun, 15 Jul 2018 05:50:26 +0000
> > > Cc: 32159@debbugs.gnu.org
> > >
> > > You are right. I got 5906 installed fonts returned from (length
> > > (x-list-fonts "*")). However, this still a problem, which is
> really
> > > annoying when editing files that contain more than one language.
> >
> > Do some of the characters in HELLO display as boxes with hex code
> > inside them, and if so, do you see slow-down when Emacs is about to
> > display those characters?  If the answer to both question is YES, I
> > can suggest a customization to try that might help you avoid the
> > slowdown (assuming you don't need to be able to display characters
> > from those scripts), but please tell what characters are involved in
> > this.
> 
> Nope...all characters in the HELLO file can be display correctly
> without a problem, the lagging issue only appears while opening the
> file. Once it already opened, there is no more slowdown.

Okay, then what's your real-life use case?  I doubt you need 
to display HELLO very often, that was just an example, right?

> > > Other editor does not have the similar issue.
> >
> > AFAIK, no other editor supports such a large variety of fonts.
> 
> I mean other editors do not have slowdown problem while opening such a
> relatively small file. They are not as mighty as Emacs of course.

That's why they don't slow down: their job is easier.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

On Mon, Jul 16, 2018 at 11:34 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > Nope...all characters in the HELLO file can be display correctly
> > without a problem, the lagging issue only appears while opening the
> > file. Once it already opened, there is no more slowdown.
>
> Okay, then what's your real-life use case?  I doubt you need
> to display HELLO very often, that was just an example, right?

Well, in my real-life case it is a little complicated. I often editing
files other than English, besides that, I use Emacs doing a lot of
things, such as Gnus to read news and Twitter-mode to read tweets.
Please assume all these jobs involve multiple languages. So if one
content makes Emacs slow down, then it seriously affects my daily
Emacs usage.

> That's why they don't slow down: their job is easier.

At least some optimize can be done, isn't it?

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Moses <moses.mason@gmail.com>
> Date: Tue, 17 Jul 2018 09:38:20 +0800
> Cc: 32159@debbugs.gnu.org
> 
> On Mon, Jul 16, 2018 at 11:34 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > > Nope...all characters in the HELLO file can be display correctly
> > > without a problem, the lagging issue only appears while opening the
> > > file. Once it already opened, there is no more slowdown.
> >
> > Okay, then what's your real-life use case?  I doubt you need
> > to display HELLO very often, that was just an example, right?
> 
> Well, in my real-life case it is a little complicated. I often editing
> files other than English, besides that, I use Emacs doing a lot of
> things, such as Gnus to read news and Twitter-mode to read tweets.
> Please assume all these jobs involve multiple languages. So if one
> content makes Emacs slow down, then it seriously affects my daily
> Emacs usage.

I can only suggest customizing your fontset in the way I show in my
previous message.

> > That's why they don't slow down: their job is easier.
> 
> At least some optimize can be done, isn't it?

What optimizations did you have in mind?

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Drew Adams]

> > Other editor does not have the similar issue.
> 
> AFAIK, no other editor supports such a large variety of fonts.

I'll probably get jumped on for chiming in here
ignorantly, but...

It's great to support a large variety of fonts.  But
if a user has lots of fonts installed then it seems
that such an advantage can become a debilitating
(i.e., severe) disadvantage.  This seems like a case
of letting a search for the perfect become the enemy
of the good.

It sounds like Emacs has a do-or-die approach, here,
trying its utmost to find a font that will fit the
bill - and using all installed fonts in its search.

It seems, at least naively, like it should be
possible for a user to control this, by toning down
Emacs's overdrive and enthusiasm in this.

Not that individual users should _have_ to do that
- I don't think they should have to.  But they
should be able to do so - and easily.

Emacs had better analyze what's going on, itself, no?
Doesn't (can't) Emacs have a better understanding of
this problem than a user?  It knows what it's doing,
why, and what the result is - what the problems are.
All a user experiences is a hang/slowdown, and all
they get when they report a bug is the answer that
they likely have too many fonts installed.

If such a font search is taking "too long" (as
determined by some setting perhaps, settable by users
but with a reasonable default value for the average)
then (1) searching should stop and (2) future searches
should be curtailed earlier.

IOW, Emacs should learn when to stop hanging a user
up.  (Having Emacs do that or not should of course
be a user choice - optional.)

Are particular fonts problematic, leading to more
slowdown/hang?  Or is it just the number of fonts
installed that is the problem?  If particular fonts,
then can't Emacs itself learn not to bother with
those fonts (for a given user)?  Why must Emacs
always try _all installed fonts_ each time?

Can't Emacs learn which fonts work for which chars,
so that it doesn't keep trying a given font when
trying to display a given char, after it's already
determined previously that that font doesn't support
that char?

I'm clearly no expert on fonts or Emacs's handling
of them.  So no doubt some of what I'm writing here
makes little sense or betrays misunderstanding.
Please take this only as one naive user's plaint.
This problem is a big one, even if only for the few
(?) users who have many fonts installed.

We (users and Emacs developers) should not be
satisfied by the conclusion that Emacs is incompatible
with having many fonts installed - especially since,
as Moses said, "Other editor does not have the similar
issue."  Even with "too many" fonts installed, Emacs
should be at least as good as other editors.

My question is essentially this: It seems like you
keep saying, and being content to say, that if too
may fonts are installed then Emacs can be slow / hang.

I trust that that's a fact now.  But why should that
be a reason for concluding that nothing can be done?
That's what seems to be happening, at least: the
problem remains, and the prescription seems to be
just "Don't do that.  Don't install so many fonts."
(Or don't use Emacs, I guess.)

To me, that's not a solution.  I have no idea whether
it might be the case that _no_ solution is possible.
But it seems like hands have been thrown up, we've
given up, and "Nothing to be done" is the response.

To those users who do get this "no other editor does
this" behavior - whether you think it's great or
not-so-great that Emacs does it, this is a severe
problem.  With no facts or real understanding of the
underlying code, it seems to this user like Emacs
could/should do better.  The status quo is not good,
for at least some users.

The line is apparently that Emacs does _better_ than
other editors because it tries harder to find a font
that can display a given char.  The fact seems to be
that, at least in some cases (e.g. many fonts
installed), Emacs, in effect, does _worse_ than other
editors.  Hence Moses saying, "Other editor does not
have the similar issue."

A user (I, for example) with many fonts installed
does not suffer dead-in-the-water behavior from
other editors (in the larger sense, i.e., any
text-input, editing, or search context).  I never
see such a problem, outside Emacs.  Never.

Can we not find some way to let users choose the
Emacs full-force-try approach or an approach used
by "other editors"?

Can we not find a way for Emacs itself, by default,
to use better judgment about how far to take its
font-search zeal, so that users do not even have to
make such a choice (but still give them the choice)?

I expect that, even if you agree that that would be
desirable (which I'm not sure you do), you might
say only: "patches welcome".

To which I'd say that this problem seems to have
_started_ at a certain point (relatively) recently,
but it's been with us for a while now, and no new
font expert has stepped forward to tackle it.

I expect that you, Eli, are likely our only hope
anytime soon of addressing this problem.  I hope
you might think more about it, realizing that
"If it hurts, don't do that" is not a reasonable
solution here.

Even if a user were prepared to uninstall a bunch
of fonts - just to make Emacs happy (usable), it's
not clear (to me at least) how s?he would go about
that.

Can Emacs not analyze the problem while it searches
desperately for a font, and so be able to report
about which fonts seem the most useless, least used,
and least useful for Emacs?  That would help a user
think about which fonts s?he might try removing.
Otherwise, how to know?

And beyond putting this burden on the user, can't
such an analysis by Emacs be used by Emacs itself,
to help it try to do the right thing by default -
have it try dropping this or that font from its
search?  IOW, can't Emacs learn about the set of
fonts installed, and not blindly try each one
everytime when trying to display a given char?

Why must the only "solution" be for a user to
uninstall fonts?  Why can't Emacs be prevented
from trying to use some of the installed fonts
for its search?

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> Date: Sun, 15 Jul 2018 09:50:03 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 32159@debbugs.gnu.org
> 
> It's great to support a large variety of fonts.  But
> if a user has lots of fonts installed then it seems
> that such an advantage can become a debilitating
> (i.e., severe) disadvantage.

Only if some characters Emacs needs to display are not supported by
any of those many fonts, and if the user didn't customize Emacs to
tell that he/she doesn't _want_ those characters to be displayed,
ever.  How reasonable is such a situation?  Not very reasonable from
my POV.

> This seems like a case of letting a search for the perfect become
> the enemy of the good.

What is "good" in this context, and how do we define it in practical
terms that can allow us to express it in code?

> It sounds like Emacs has a do-or-die approach, here,
> trying its utmost to find a font that will fit the
> bill - and using all installed fonts in its search.

No, it doesn't use all of the installed fonts, only those which
satisfy the requirements of the script whose character Emacs needs to
display.

> It seems, at least naively, like it should be
> possible for a user to control this, by toning down
> Emacs's overdrive and enthusiasm in this.

We have that way: the user should customize the fontset.

> Emacs had better analyze what's going on, itself, no?

And what exactly is "going on", may I ask?

Emacs enumerates the relevant fonts, then goes over them one by one,
determining as quickly as it can whether each font supports the
required features.  Each font that does is opened and looked up for
the character we need to display.  In this context, what do you mean
by "going on"?

> Doesn't (can't) Emacs have a better understanding of
> this problem than a user?

What else can Emacs do except try the fonts that might fit the bill?
What other understanding do you expect from Emacs?  Can Emacs
"understand" that no font is available for a given character without
trying to find such a font?

> IOW, Emacs should learn when to stop hanging a user
> up.

Which is when, exactly?  If you have some ideas, please let us hear
them, even if they are vague.

> Why must Emacs always try _all installed fonts_ each time?

It doesn't, not all of them.  Only those that might be relevant.

> Can't Emacs learn which fonts work for which chars,

It can, and it has.  See the default fontset -- that's the database
Emacs uses to guide the search.

> so that it doesn't keep trying a given font when
> trying to display a given char, after it's already
> determined previously that that font doesn't support
> that char?

Emacs never tries to search for a font that supports a given character
after it has done that once and failed.  But it will look again when
presented with another character, or if we clear the font caches.

> My question is essentially this: It seems like you
> keep saying, and being content to say, that if too
> may fonts are installed then Emacs can be slow / hang.

I'm saying that a system that has 5000 fonts installed and still has
some characters not covered is misconfigured.  Installing one or two
more fonts will probably fix the problem; if the user doesn't _want_
those characters to be displayed, ever, there's a way of customizing
Emacs to do that as well.  Doing none of those alternatives makes no
sense to me, if the slowdown annoys the user who is in this situation.

> I expect that, even if you agree that that would be
> desirable (which I'm not sure you do), you might
> say only: "patches welcome".

Patches to speed up Emacs are always welcome, that's a truism.  Sadly,
I've not seen patches in this particular are in a long, long, long,
LONG time.

> I expect that you, Eli, are likely our only hope
> anytime soon of addressing this problem.

I'm not your hope, I don't know enough about this stuff to redesign it
(assuming that there is a way of redesigning it and getting better
performance for the same level of support for various scripts).  All I
can is try to explain how to configure your system better using the
available features and tricks about which I do know.  That, and
applying some simple band-aid, like inhibit-compacting-font-caches,
from time to time, is all I can do, and am doing.

> Can Emacs not analyze the problem while it searches
> desperately for a font, and so be able to report
> about which fonts seem the most useless, least used,
> and least useful for Emacs?  That would help a user
> think about which fonts s?he might try removing.

From the little I know, there's no answer to that question, even if
you only ask about Emacs.  Of course, people install fonts for other
applications as well, and might not want to uninstall fonts that get
in Emacs's way.

No, the way to solve these problems is to either install a few more
fonts that complete/improve the coverage, and/or customize the
fontsets to make the font search more efficient.

> And beyond putting this burden on the user, can't
> such an analysis by Emacs be used by Emacs itself,
> to help it try to do the right thing by default -
> have it try dropping this or that font from its
> search?  IOW, can't Emacs learn about the set of
> fonts installed, and not blindly try each one
> everytime when trying to display a given char?

We don't have infrastructure for such analysis.  And I don't think we
have anyone on board with expertise to design and code it, even for a
single platform, let alone all of them.  "Patches welcome", of course.

> Why must the only "solution" be for a user to
> uninstall fonts?

See above: it's not the only one.  Not even the best, if you ask me.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[net june]

On 07/16/2018 02:53 AM, Eli Zaretskii wrote:
>> Date: Sun, 15 Jul 2018 09:50:03 -0700 (PDT)
>> From: Drew Adams <drew.adams@oracle.com>
>> Cc: 32159@debbugs.gnu.org
>>
>> It's great to support a large variety of fonts.  But
>> if a user has lots of fonts installed then it seems
>> that such an advantage can become a debilitating
>> (i.e., severe) disadvantage.
> 
> Only if some characters Emacs needs to display are not supported by
> any of those many fonts, and if the user didn't customize Emacs to
> tell that he/she doesn't _want_ those characters to be displayed,
> ever.  How reasonable is such a situation?  Not very reasonable from
> my POV.
> 

Is it possible to save the font search result to file (custom file) for 
next use? Then we just need to open the HELLO file once to cache the result.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: net june <netjune@outlook.com>
> Date: Mon, 16 Jul 2018 03:43:10 +0000
> 
> Is it possible to save the font search result to file (custom file) for 
> next use? Then we just need to open the HELLO file once to cache the result.

I tried to explain in my other mail the downsides of caching font
information.  If you use Emacs as intended, i.e. by having a single
session running at all times, the cache is inside Emacs anyway.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Drew Adams]

> Only if some characters Emacs needs to display are not supported by
> any of those many fonts, and if the user didn't customize Emacs to
> tell that he/she doesn't _want_ those characters to be displayed,
> ever.  How reasonable is such a situation?  Not very reasonable from
> my POV.

How is a user to know which characters are problematic, and
thus perhaps to tell Emacs that s?he doesn't want/need those
chars to be displayed?

How is a user to tell Emacs that s?he doesn't want/need this
or that char to be displayed?

> > It seems, at least naively, like it should be
> > possible for a user to control this, by toning down
> > Emacs's overdrive and enthusiasm in this.
> 
> We have that way: the user should customize the fontset.

How does a user do that?  How does a user know that that
is what needs to be done?  Just what is a user supposed to
do, when s?he determines that Emacs hangs (is slow) because
it tries to search fonts?

> And what exactly is "going on", may I ask?

You tell me.  As a user, the symptom is hanging/slowdown.

> > Doesn't (can't) Emacs have a better understanding of
> > this problem than a user?
> 
> What else can Emacs do except try the fonts that might fit the bill?
> What other understanding do you expect from Emacs?  Can Emacs
> "understand" that no font is available for a given character without
> trying to find such a font?

Why would Emacs need to do that over and over?  Can't it
record that failure, so it doesn't try again in a future
session?

> > Can't Emacs learn which fonts work for which chars,
> 
> It can, and it has.  See the default fontset -- that's the database
> Emacs uses to guide the search.

I don't know how to "see" it or what to do when I do see
it.  But a naive guess would be that the default font set
isn't sufficient, if Emacs keeps hanging.

> > so that it doesn't keep trying a given font when
> > trying to display a given char, after it's already
> > determined previously that that font doesn't support
> > that char?
> 
> Emacs never tries to search for a font that supports a given character
> after it has done that once and failed.

Even across Emacs sessions?  Does it remember that that
happened, so it doesn't try the same font for the same
char in a future session?  If so, great.  In that case,
I don't see why it keeps on hanging (slowdown).

> But it will look again when
> presented with another character, or if we clear the font caches.

Is clearing the font caches something a user does (without
knowing it)?  Am I the one doing that somehow?

Based on what you said in other threads, I've already set
`inhibit-compacting-font-caches' to nil, but that doesn't
seem to have fixed the problem.

> > My question is essentially this: It seems like you
> > keep saying, and being content to say, that if too
> > may fonts are installed then Emacs can be slow / hang.
> 
> I'm saying that a system that has 5000 fonts installed and still has
> some characters not covered is misconfigured. Installing one or two
> more fonts will probably fix the problem;

So it's about installing more fonts, not uninstalling
fonts because there are too many.  Good to know, I guess.

I'll gladly install one or two more fonts.  I just need
to know what they are and where to get them.  You've often
mentioned Symbola, but I installed that long ago, and it
doesn't seem to have made any difference.

> if the user doesn't _want_ those characters to be displayed, ever,

Dunno what the chars are.  But yes, if some char is
causing Emacs to hang, in spite of the many fonts I
have installed, then at this point I'd say that it
should be ignored: no attempt made to display it.

Is there no way to just tell Emacs that, as a general
rule: don't bother trying to display any chars that
you've previously tried and failed to display?  Above,
you seem to say that Emacs already does that, but it's
not clear whether you meant in the same Emacs session
or persistently.

If it already does that across Emacs sessions then I
don't understand why the problem persists.  It seems
like it would have already learned by now that there
is no sense wasting time trying to (again) look for
a font to display such a char.

> there's a way of customizing Emacs to do that as well.

What way is that?

> Doing none of those alternatives makes no
> sense to me, if the slowdown annoys the user who is in this situation.

Those alternatives are not clear to me.  Perhaps you
could post some instructions somewhere, for what to do
if Emacs hangs because it tries to find a suitable font?

> > I expect that, even if you agree that that would be
> > desirable (which I'm not sure you do), you might
> > say only: "patches welcome".
> 
> Patches to speed up Emacs are always welcome, that's a truism.  Sadly,
> I've not seen patches in this particular are in a long, long, long,
> LONG time.

Yes, that's too bad.  But I didn't see the problem in
older releases.  At some point (relatively recently, but
I don't recall which release it was - perhaps you do),
the problem began.

> > I expect that you, Eli, are likely our only hope
> > anytime soon of addressing this problem.
> 
> I'm not your hope, I don't know enough about this stuff to redesign it
> (assuming that there is a way of redesigning it and getting better
> performance for the same level of support for various scripts).  All I
> can is try to explain how to configure your system better using the
> available features and tricks about which I do know.

OK; please do that.  It's not clear to me what a user
should try.

> That, and applying some simple band-aid, like
> inhibit-compacting-font-caches, from time to time, is
> all I can do, and am doing.

OK; thanks for that.  I did set that particular variable
to nil when you first mentioned it.

> > Can Emacs not analyze the problem while it searches
> > desperately for a font, and so be able to report
> > about which fonts seem the most useless, least used,
> > and least useful for Emacs?  That would help a user
> > think about which fonts s?he might try removing.
> 
> From the little I know, there's no answer to that question, even if
> you only ask about Emacs.  Of course, people install fonts for other
> applications as well, and might not want to uninstall fonts that get
> in Emacs's way.

That's true.  But some users, such as myself, may have
installed fonts that they don't really need and not
know exactly which fonts they might need for this or
that application.

> No, the way to solve these problems is to either install a few more
> fonts that complete/improve the coverage, and/or customize the
> fontsets to make the font search more efficient.

I'd like to know more about each of those possibilities.
I know nothing about the second (customizing fontsets).
Wrt the first, I already installed font Symbola.  I
thought you had indicated that that font should take
care of most chars Emacs itself tends to present to
users (e.g. in `Hello').  But installing Symbola didn't
seem to help.

> > And beyond putting this burden on the user, can't
> > such an analysis by Emacs be used by Emacs itself,
> > to help it try to do the right thing by default -
> > have it try dropping this or that font from its
> > search?  IOW, can't Emacs learn about the set of
> > fonts installed, and not blindly try each one
> > everytime when trying to display a given char?
> 
> We don't have infrastructure for such analysis.  And I don't think we
> have anyone on board with expertise to design and code it, even for a
> single platform, let alone all of them.  "Patches welcome", of course.

Do you have an idea why Emacs, but not other applications,
seems to have this problem?  You indicated in your answer
earlier that it's because Emacs does much more than others.

If that's the reason, is there a way for a user to dial
back some of that much-more that Emacs does?  Is there a way
for a user to get, in Emacs, the inferior-but-not-hanging
behavior of other applications?

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> Date: Sun, 15 Jul 2018 12:43:50 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: moses.mason@gmail.com, 32159@debbugs.gnu.org
> 
> How is a user to know which characters are problematic, and
> thus perhaps to tell Emacs that s?he doesn't want/need those
> chars to be displayed?

In most cases, those characters are displayed as boxes with hex code
inside, to mean no font was found for them.  If that's not the case,
only trial and error (display text with and without portions of the
buffer) will uncover them.

> How is a user to tell Emacs that s?he doesn't want/need this
> or that char to be displayed?

"C-u C-x =" displays the answer to that question, and much more.

Whether the user does or doesn't want a given character displayed is
up to the user: only users know which scripts they want to be legible
in their usage of Emacs.

> > We have that way: the user should customize the fontset.

> How does a user do that?

See the 2 manuals, where they document 'set-fontset-font'.  There's a
lot of info there, with examples.

> How does a user know that that is what needs to be done?

They are supposed to read the manuals, and this and similar
discussions.

> Just what is a user supposed to do, when s?he determines that Emacs
> hangs (is slow) because it tries to search fonts?

See above.

> > And what exactly is "going on", may I ask?
> 
> You tell me.  As a user, the symptom is hanging/slowdown.

That's Emacs searching for a suitable font.  Nothing else is going on.

> > > Doesn't (can't) Emacs have a better understanding of
> > > this problem than a user?
> > 
> > What else can Emacs do except try the fonts that might fit the bill?
> > What other understanding do you expect from Emacs?  Can Emacs
> > "understand" that no font is available for a given character without
> > trying to find such a font?
> 
> Why would Emacs need to do that over and over?  Can't it
> record that failure, so it doesn't try again in a future
> session?

It does record the failure, but only for this session.  Recording that
for future session (a.k.a. "caching") has its own problems: the cache
needs to be refreshed when fonts are installed or uninstalled, so it's
a mixed blessing.  The best way to "cache" the information is to
customize the fontset to record user preferences, as discussed here.

> > > Can't Emacs learn which fonts work for which chars,
> > 
> > It can, and it has.  See the default fontset -- that's the database
> > Emacs uses to guide the search.
> 
> I don't know how to "see" it

(Did you try 'apropos'?) "M-x describe-fontset RET RET" will show
that.  Be prepared for it to take a couple of dozen of seconds: it
goes through the entire range of Unicode characters.

> or what to do when I do see it.

See above and below.

> But a naive guess would be that the default font set isn't
> sufficient, if Emacs keeps hanging.

It's a starting point for further customization and optimization.

It is practically impossible to make Emacs OOTB fit every system out
there, because the fonts installed on each systems cannot be known in
advance.  Plus, some popular fonts are not free, and we don't want to
advertise/support them by including them in the default fontset.  The
defaults are supposed to cover the "usual" setups, and indeed IME
complaints about this are mostly for rare and exotic use cases.  Of
course, this particular case might just be one that will trigger some
changes in Emacs, I don't know yet.

> > > so that it doesn't keep trying a given font when
> > > trying to display a given char, after it's already
> > > determined previously that that font doesn't support
> > > that char?
> > 
> > Emacs never tries to search for a font that supports a given character
> > after it has done that once and failed.
> 
> Even across Emacs sessions?

No, only during the same session.

> > But it will look again when
> > presented with another character, or if we clear the font caches.
> 
> Is clearing the font caches something a user does (without
> knowing it)?

It's part of GC.  inhibit-compacting-font-caches disables it.

> Based on what you said in other threads, I've already set
> `inhibit-compacting-font-caches' to nil, but that doesn't
> seem to have fixed the problem.

It fixes a different problem.  It doesn't affect the search for a
suitable font when a given character needs to be displayed for the
first time.

> > I'm saying that a system that has 5000 fonts installed and still has
> > some characters not covered is misconfigured. Installing one or two
> > more fonts will probably fix the problem;
> 
> So it's about installing more fonts, not uninstalling
> fonts because there are too many.  Good to know, I guess.

It's a better solution, if you need to display the characters which
appear as boxes and/or those for which Emacs takes a long time to
search for a font.  An even better solution is to (also) customize
your fontset to specify fonts for specific scripts or character code
ranges, according to the fonts you have and/or whose appearance you
like.  That could bypass the search entirely in some cases.

> I'll gladly install one or two more fonts.  I just need
> to know what they are and where to get them.

Find the characters that display as boxes, then look for fonts that
support those characters, and install one of those fonts.

> > if the user doesn't _want_ those characters to be displayed, ever,
> 
> Dunno what the chars are.

See above.

> Is there no way to just tell Emacs that, as a general
> rule: don't bother trying to display any chars that
> you've previously tried and failed to display?

It already does that, in the same session.  See above about caching.

> > there's a way of customizing Emacs to do that as well.
> 
> What way is that?

See above and below.

> > Doing none of those alternatives makes no
> > sense to me, if the slowdown annoys the user who is in this situation.
> 
> Those alternatives are not clear to me.  Perhaps you
> could post some instructions somewhere, for what to do
> if Emacs hangs because it tries to find a suitable font?

Fontset customization is not just for "hanging", it's for any
situation where Emacs chooses a font that you don't like for some
reason, and you want it to choose another one.  It also helps to speed
up the search for fonts.  For example, the following setup speeds up
the initial display of HELLO by a factor of 5, from about 20 sec to
just 4:

  (set-fontset-font "fontset-default" 'gujarati '("Shruti" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'devanagari '("Mangal" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'kannada '("Tunga" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'malayalam '("Kartika" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'oriya '("Kalinga" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'sinhala '("Iskoola Pota" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'tamil '("Latha" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'telugu '("Gautami" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'tibetan '("Microsoft Himalaya" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'burmese nil)

The fonts mentioned above are those available OOTB on a Windows
machine I have here; yours may be different.  I collected them using
"C-u C-x =" for characters whose presence makes display of HELLO
significantly slower.  If you don't have a font available for some
script and/or don't want that script's characters displayed, you can
replace the last argument of 'set-fontset-font' with nil, like I did
for 'burmese'.  (You can also use ranges of codepoints instead of
script symbols; see the doc string of 'set-fontset-font'.)

The script of a character is also shown by "C-u C-x =".

> Yes, that's too bad.  But I didn't see the problem in
> older releases.  At some point (relatively recently, but
> I don't recall which release it was - perhaps you do),
> the problem began.

The only change was the one that is disabled by
'inhibit-compacting-font-caches'.  Other than that, it's probably the
fonts you installed.  If not, I don't know what could be the reason.

> > > Can Emacs not analyze the problem while it searches
> > > desperately for a font, and so be able to report
> > > about which fonts seem the most useless, least used,
> > > and least useful for Emacs?  That would help a user
> > > think about which fonts s?he might try removing.
> > 
> > From the little I know, there's no answer to that question, even if
> > you only ask about Emacs.  Of course, people install fonts for other
> > applications as well, and might not want to uninstall fonts that get
> > in Emacs's way.
>
> That's true.  But some users, such as myself, may have
> installed fonts that they don't really need and not
> know exactly which fonts they might need for this or
> that application.

I don't know how to solve this.  You will have to analyze your
font-related needs in some systematic way.  I find the BabelMap
program useful for studying the installed fonts, btw.

> > No, the way to solve these problems is to either install a few more
> > fonts that complete/improve the coverage, and/or customize the
> > fontsets to make the font search more efficient.
> 
> I'd like to know more about each of those possibilities.
> I know nothing about the second (customizing fontsets).

See above.

> Wrt the first, I already installed font Symbola.  I
> thought you had indicated that that font should take
> care of most chars Emacs itself tends to present to
> users (e.g. in `Hello').  But installing Symbola didn't
> seem to help.

Symbola may not help for HELLO, but that's not an interesting use
case, usually.  Symbola does help in many real-life use cases, I see
that every day on my systems.

> Do you have an idea why Emacs, but not other applications,
> seems to have this problem?  You indicated in your answer
> earlier that it's because Emacs does much more than others.

Yes.  Emacs optimizes the fonts it chooses for the scripts of the
characters it needs to display.

> If that's the reason, is there a way for a user to dial
> back some of that much-more that Emacs does?

Yes, by customizing the fontset; see above,

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

On Mon, Jul 16, 2018 at 10:22 PM Eli Zaretskii <eliz@gnu.org> wrote:
> Fontset customization is not just for "hanging", it's for any
> situation where Emacs chooses a font that you don't like for some
> reason, and you want it to choose another one.  It also helps to speed
> up the search for fonts.  For example, the following setup speeds up
> the initial display of HELLO by a factor of 5, from about 20 sec to
> just 4:
>
>   (set-fontset-font "fontset-default" 'gujarati '("Shruti" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'devanagari '("Mangal" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'kannada '("Tunga" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'malayalam '("Kartika" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'oriya '("Kalinga" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'sinhala '("Iskoola Pota" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'tamil '("Latha" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'telugu '("Gautami" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'tibetan '("Microsoft Himalaya" . "iso10646-1"))
>   (set-fontset-font "fontset-default" 'burmese nil)
>
> The fonts mentioned above are those available OOTB on a Windows
> machine I have here; yours may be different.  I collected them using
> "C-u C-x =" for characters whose presence makes display of HELLO
> significantly slower.  If you don't have a font available for some
> script and/or don't want that script's characters displayed, you can
> replace the last argument of 'set-fontset-font' with nil, like I did
> for 'burmese'.  (You can also use ranges of codepoints instead of
> script symbols; see the doc string of 'set-fontset-font'.)
>
> The script of a character is also shown by "C-u C-x =".

How to get the correct second argument (script name?) and also the
"iso10646-1" (encoding name?)? I do not find it in C-u C-x = and the
manual is not very clear about that. Anyway, why Emacs can not
generate a "suggestion" of set-fontset-font in the first place and
tell user to add it into .emacs? That would be very helpful.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Moses <moses.mason@gmail.com>
> Date: Tue, 17 Jul 2018 10:13:05 +0800
> Cc: drew.adams@oracle.com, 32159@debbugs.gnu.org
> 
> >   (set-fontset-font "fontset-default" 'gujarati '("Shruti" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'devanagari '("Mangal" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'kannada '("Tunga" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'malayalam '("Kartika" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'oriya '("Kalinga" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'sinhala '("Iskoola Pota" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'tamil '("Latha" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'telugu '("Gautami" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'tibetan '("Microsoft Himalaya" . "iso10646-1"))
> >   (set-fontset-font "fontset-default" 'burmese nil)
> >
> > The fonts mentioned above are those available OOTB on a Windows
> > machine I have here; yours may be different.  I collected them using
> > "C-u C-x =" for characters whose presence makes display of HELLO
> > significantly slower.  If you don't have a font available for some
> > script and/or don't want that script's characters displayed, you can
> > replace the last argument of 'set-fontset-font' with nil, like I did
> > for 'burmese'.  (You can also use ranges of codepoints instead of
> > script symbols; see the doc string of 'set-fontset-font'.)
> >
> > The script of a character is also shown by "C-u C-x =".
> 
> How to get the correct second argument (script name?)

It is shown by "C-u C-x =".

> and also the "iso10646-1" (encoding name?)?

Always use iso10646-1, it makes no sense to use anything else with
these scripts.

> I do not find it in C-u C-x = and the manual is not very clear about
> that. Anyway, why Emacs can not generate a "suggestion" of
> set-fontset-font in the first place and tell user to add it into
> .emacs?

It does, in a way: look at the output of "C-u C-x =".  Besides, how is
Emacs to know that what happens on your system now is a permanent
setup?

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

On Tue, Jul 17, 2018 at 10:40 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > How to get the correct second argument (script name?)
> It is shown by "C-u C-x =".

OK. I see it now. The info is somewhat not obvious to me.

> > and also the "iso10646-1" (encoding name?)?
> Always use iso10646-1, it makes no sense to use anything else with
> these scripts.

How could a user know that? This should at least put into the manual.

> > I do not find it in C-u C-x = and the manual is not very clear about
> > that. Anyway, why Emacs can not generate a "suggestion" of
> > set-fontset-font in the first place and tell user to add it into
> > .emacs?
> It does, in a way: look at the output of "C-u C-x =".  Besides, how is
> Emacs to know that what happens on your system now is a permanent
> setup?

It needn't be a permanent setup. If Emacs loading time becomes longer
(for example, > 5 sec.), it should prompt users to change their
fontset and give suggestion settings again.
BTW, I still confused why Emacs needs to scan fonts to determine which
character belongs to which font. On the other hand, browsers like
Firefox, Chromium could display every script fonts very fast without
issue.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Moses <moses.mason@gmail.com>
> Date: Tue, 17 Jul 2018 12:22:27 +0800
> Cc: drew.adams@oracle.com, 32159@debbugs.gnu.org
> 
> > > and also the "iso10646-1" (encoding name?)?
> > Always use iso10646-1, it makes no sense to use anything else with
> > these scripts.
> 
> How could a user know that? This should at least put into the manual.

You can just drop the encoding, i.e. use

  (set-fontset-font "fontset-default" 'gujarati "Shruti")

The encoding part is just a precaution, and is nowadays only important
for some, hopefully rare, CJK use cases.  I used it in my example for
completeness' sake.

> > > I do not find it in C-u C-x = and the manual is not very clear about
> > > that. Anyway, why Emacs can not generate a "suggestion" of
> > > set-fontset-font in the first place and tell user to add it into
> > > .emacs?
> > It does, in a way: look at the output of "C-u C-x =".  Besides, how is
> > Emacs to know that what happens on your system now is a permanent
> > setup?
> 
> It needn't be a permanent setup.

Sorry, I'm not following: what you put on your ~/.emacs _is_ permanent
setup, in the sense that it will be in effect as long as you don't
change it.  It won't react to changes in the fonts you install, for
one thing.  If, for example, you are just trying some font, and didn't
yet decide whether you like it or not, suggesting that font for your
fontset customization will just confuse if not worse.

> If Emacs loading time becomes longer
> (for example, > 5 sec.), it should prompt users to change their
> fontset and give suggestion settings again.

First, the 5 sec figure is hard to decide on, because it depends on
many factors, like user impatience, the CPU speed, etc.

More importantly, if Emacs stops short of finding a font, it will not
have any useful suggestion for the actual setup of the fontset,
because it will not know what font to suggest.

But I think we are getting ahead of ourselves.  There could be a place
for user-level customization commands based on set-fontset-font
(although I envision non-trivial UI issues, due to the power and
flexibility of fontsets), but before we discuss that, we should
establish that customizing fontsets indeed fixes your real-life
problems.  That doing so for some fonts makes HELLO display faster
doesn't yet mean your real-life use cases are solved by doing the
same.

So please see if the problems that prompted your bug report are indeed
solved by customizing your fontset, including determining which
scripts/ranges of character codepoints needed such customization, and
let's defer the discussion of more user-friendly facilities to after
that.

> BTW, I still confused why Emacs needs to scan fonts to determine which
> character belongs to which font.

Not "belongs", but "can be displayed by".

And I don't think I understand your confusion: how else can any
program determine which font supports a given character, except by
checking the candidate fonts?

> On the other hand, browsers like Firefox, Chromium could display
> every script fonts very fast without issue.

I don't know how the browsers look for fonts.  If someone does, maybe
they can describe that, and we might then be able to learn something
useful.

Just for the record: I didn't write the code in Emacs that implements
font search, and have no real experience or expertise in that area, I
just know one or two things (literally) about what our code does.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> Date: Tue, 17 Jul 2018 18:05:36 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 32159@debbugs.gnu.org
> 
> > On the other hand, browsers like Firefox, Chromium could display
> > every script fonts very fast without issue.
> 
> I don't know how the browsers look for fonts.  If someone does, maybe
> they can describe that, and we might then be able to learn something
> useful.

I've dug a little, and it looks like Firefox simply hard-codes an
explicit font name or a short list of explicit font names for every
"language" (what we call "script").  The defaults change whenever
Firefox developers decide that a better font is available on the
particular OS OOTB, or when a new version of the OS adds fonts.  To
see the defaults, type "about:config" into the address bar, and look
at the font.name-list.* variables.  The user can customize this (via
Options, hidden under "Fonts & Colors -> Advanced") by specifying a
single font for each script+style combination (where "style" is one of
Serif, Sans-Serif, and Monospaced).

Chrome seems to do something very similar, except that its defaults
are less sophisticated (many scripts don't have fonts defined and use
the global default font).

The advantage of this method is that it avoids font search in the
cases where by default there's only one font per script (slightly less
than 50% of the scripts in my version of Firefox), and makes the
search shorter when there are several candidate scripts in the
list.  The disadvantages are:

 . When the user installs a font that doesn't come with the OS OOTB,
   the only way to start using that font is to customize the defaults
   via Options.  Likewise, when a non-standard font is uninstalled
   that was previously customized to be used by Firefox.

 . The defaults explicitly name fonts installed on the OS, so for
   non-free systems, such as Windows, and maybe also for free systems,
   they name non-free fonts.  That is something we try very hard to
   avoid in our defaults, AFAIU.

Emacs defaults usually do not specify fonts explicitly (with a few
exceptions), but instead specify features that a font must have in
order to be considered for characters belonging to a script.  This
allows the user significant more flexibility, since which font will be
used is determined by the installed fonts, and does not require
customization after installed fonts change.  But the price is a
potentially longer search for a font, especially when a lot of fonts
are available on the system.

We could do something similar to what the browsers do, if we want, at
least on Free systems, but Someone™ would have to do the footwork of
looking at enough boxes and collecting free fonts that are available
OOTB there.  This could at least be an optional feature, for those who
don't intend changing their installed fonts too much, or not at all.
For non-free systems, the corresponding settings could be perhaps on
the wiki (and again, Someone™ should do the research and publish the
results).

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Drew Adams]

> I've dug a little, and it looks like...

I just want to thank you for doing that.  And for providing
a clear description of what you found out.  Kudos.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

On Tue, Jul 17, 2018 at 11:05 PM Eli Zaretskii <eliz@gnu.org> wrote:
> More importantly, if Emacs stops short of finding a font, it will not
> have any useful suggestion for the actual setup of the fontset,
> because it will not know what font to suggest.

Maybe Emacs can prompt the user to find/download the fonts that missing?


> But I think we are getting ahead of ourselves.  There could be a place
> for user-level customization commands based on set-fontset-font
> (although I envision non-trivial UI issues, due to the power and
> flexibility of fontsets), but before we discuss that, we should
> establish that customizing fontsets indeed fixes your real-life
> problems.  That doing so for some fonts makes HELLO display faster
> doesn't yet mean your real-life use cases are solved by doing the
> same.
>
> So please see if the problems that prompted your bug report are indeed
> solved by customizing your fontset, including determining which
> scripts/ranges of character codepoints needed such customization, and
> let's defer the discussion of more user-friendly facilities to after
> that.

After many failed tries. I finally make Emacs a little faster after
customized the fontset. I have to find the font name and the script
name for each script. It a painful process. Maybe it is not Emacs'
fault, but I hope the customizing process could be automatically done.

The HELLO file opening still slow though.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Moses <moses.mason@gmail.com>
> Date: Sun, 22 Jul 2018 05:45:46 +0000
> Cc: drew.adams@oracle.com, 32159@debbugs.gnu.org
> 
> On Tue, Jul 17, 2018 at 11:05 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > More importantly, if Emacs stops short of finding a font, it will not
> > have any useful suggestion for the actual setup of the fontset,
> > because it will not know what font to suggest.
> 
> Maybe Emacs can prompt the user to find/download the fonts that missing?

We could prompt, but without telling the names of the required fonts,
how useful would be such a prompt?  All we know is a script, and maybe
some other attributes.  How many users will know what fonts to look
for and download?

Also, in quite a few cases the problem is not that a font is not
available, the problem is that there are too many fonts to try before
Emacs gets to one that can display a character.

> > So please see if the problems that prompted your bug report are indeed
> > solved by customizing your fontset, including determining which
> > scripts/ranges of character codepoints needed such customization, and
> > let's defer the discussion of more user-friendly facilities to after
> > that.
> 
> After many failed tries. I finally make Emacs a little faster after
> customized the fontset.

Can you try to make this a little more quantitative?  How long did it
take before and how long after the fontset customizations?  And why
did you need many failed trials, what was the problem that prevented
you from succeeding on the first trial?

> I have to find the font name and the script name for each script. It
> a painful process.

Which scripts are the ones for which you needed to customize the
fontset?  I mean in your real-life use cases, not for displaying
HELLO.

> The HELLO file opening still slow though.

You mean, customizing the fontset didn't help with HELLO?  Is it
possible that you customized the fontset for scripts other than those
which slow down HELLO?

Thanks.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

On Sun, Jul 22, 2018 at 2:06 PM Eli Zaretskii <eliz@gnu.org> wrote:
> Can you try to make this a little more quantitative?  How long did it
> take before and how long after the fontset customizations?

Gnus was usually stopped responding and lagging several seconds when
scrolling before the fontset customization. It is much better now.

> And why
> did you need many failed trials, what was the problem that prevented
> you from succeeding on the first trial?

I need to pair every fonts name and script name for every
script/language but I do not know the font name for some
scripts/languages. So I leave Emacs to find the correct font and use
M-x descript-char on these language's character to find the font name.
When there are many languages, it takes some time. And I also found I
made some mistake before, so I need to delete the incorrect one and
repeat these step for that specific script/language again.

> Which scripts are the ones for which you needed to customize the
> fontset?  I mean in your real-life use cases, not for displaying
> HELLO.

This is my font setting, the Linux part is not complete, because
currently, I am on my Windows box.

;; font
(setq font-encoding-alist
(append '(("gb2312.1980" . chinese-gbk)
   ("iso8859-1$" . iso-8859-1)
   ("iso8859-2$" . iso-8859-2)
   ("iso8859-3$" . iso-8859-3)
   ("iso8859-4$" . iso-8859-4)
   ("iso8859-5$" . iso-8859-5)
   ("iso8859-6$" . iso-8859-6)
   ("iso8859-7$" . iso-8859-7)
   ("iso8859-8$" . iso-8859-8)
   ("iso8859-9$" . iso-8859-9)
   ("iso8859-10$" . iso-8859-10)
   ("iso8859-11$" . iso-8859-11)
   ("iso8859-13$" . iso-8859-13)
   ("iso8859-14$" . iso-8859-14)
   ("iso8859-15$" . iso-8859-15)
   ("iso8859-16$" . iso-8859-16)
   ("ascii-0$" . ascii)
   ("gb2312.1980" . chinese-gb2312)
   ("gbk" . chinese-gbk)
   ("gb18030" unicode)
   ("jisx0208.1978" . japanese-jisx0208-1978)
   ("jisx0208" . japanese-jisx0208)
   ("jisx0201" . jisx0201)
   ("jisx0212" . japanese-jisx0212)
   ("ksx1001" . korean-ksc5601)
   ("ksc5601.1987" . korean-ksc5601)
   ("cns11643.1992.*1" . chinese-cns11643-1)
   ("cns11643.1992.*2" . chinese-cns11643-2)
   ("cns11643.1992.*3" . chinese-cns11643-3)
   ("cns11643.1992.*4" . chinese-cns11643-4)
   ("cns11643.1992.*5" . chinese-cns11643-5)
   ("cns11643.1992.*6" . chinese-cns11643-6)
   ("cns11643.1992.*7" . chinese-cns11643-7)
   ("cns11643.92p1-0" . chinese-cns11643-1)
   ("cns11643.92p2-0" . chinese-cns11643-2)
   ("cns11643.92p3-0" . chinese-cns11643-3)
   ("cns11643.92p4-0" . chinese-cns11643-4)
   ("cns11643.92p5-0" . chinese-cns11643-5)
   ("cns11643.92p6-0" . chinese-cns11643-6)
   ("cns11643.92p7-0" . chinese-cns11643-7)
   ("big5" . big5)
   ("viscii" . viscii)
   ("tis620" . tis620-2533)
   ("microsoft-cp1251" . windows-1251)
   ("koi8-r" . koi8-r)
   ("jisx0213.2000-1" . japanese-jisx0213-1)
   ("jisx0213.2000-2" . japanese-jisx0213-2)
   ("jisx0213.2004-1" . japanese-jisx0213\.2004-1)
   ("iso10646-1$" unicode-bmp)
   ("iso10646.indian-1" unicode-bmp)
   ("unicode-bmp" unicode-bmp)
   ("abobe-symbol" . symbol)
   ("sisheng_cwnn" . chinese-sisheng)
   ("mulearabic-0" . arabic-digit)
   ("mulearabic-1" . arabic-1-column)
   ("mulearabic-2" . arabic-2-column)
   ("muleipa" . ipa)
   ("ethiopic-unicode" unicode-bmp . ethiopic)
   ("is13194-devanagari" . indian-is13194)
   ("Devanagari-CDAC" . devanagari-cdac)
   ("Sanskrit-CDAC" . sanskrit-cdac)
   ("Bengali-CDAC" . bengali-cdac)
   ("Assamese-CDAC" . assamese-cdac)
   ("Punjabi-CDAC" . punjabi-cdac)
   ("Gujarati-CDAC" . gujarati-cdac)
   ("Oriya-CDAC" . oriya-cdac)
   ("Tamil-CDAC" . tamil-cdac)
   ("Telugu-CDAC" . telugu-cdac)
   ("Kannada-CDAC" . kannada-cdac)
   ("Malayalam-CDAC" . malayalam-cdac)
   ("Devanagari-Akruti" . devanagari-akruti)
   ("Bengali-Akruti" . bengali-akruti)
   ("Punjabi-Akruti" . punjabi-akruti)
   ("Gujarati-Akruti" . gujarati-akruti)
   ("Oriya-Akruti" . oriya-akruti)
   ("Tamil-Akruti" . tamil-akruti)
   ("Telugu-Akruti" . telugu-akruti)
   ("Kannada-Akruti" . kannada-akruti)
   ("Malayalam-Akruti" . malayalam-akruti)
   ("muleindian-2" . indian-2-column)
   ("muleindian-1" . indian-1-column)
   ("mulelao-1" . mule-lao)
   ("muletibetan-2" . tibetan)
   ("muletibetan-0" . tibetan)
   ("muletibetan-1" . tibetan-1-column))
font-encoding-alist)
)

(defun my--set-font (&optional frame)
  (with-selected-frame (or frame (selected-frame))
    (if (string-equal system-type "windows-nt")
;; Windows
(progn
  (set-face-attribute 'default nil :font "consolas")
  (dolist (charset '(han cjk-misc chinese-gbk))
    (set-fontset-font "fontset-default" charset (font-spec :family
"Microsoft Jhenghei")))
    (set-fontset-font "fontset-default" 'kana '("Meiryo" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'hangul '("Malgun Gothic" .
"iso10646-1"))
    (set-fontset-font "fontset-default" 'arabic '("Adobe Naskh Medium"
. "iso10646-1"))
    (set-fontset-font "fontset-default" 'cyrillic '("Consolas" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'gujarati '("Nirmala UI" .
"iso10646-1"))
    (set-fontset-font "fontset-default" 'devanagari '("Adobe
Devanagari" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'kannada '("Nirmala UI" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'malayalam '("Nirmala UI" .
"iso10646-1"))
    (set-fontset-font "fontset-default" 'oriya '("Nirmala UI" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'sinhala '("Nirmala UI" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'tamil '("Nirmala UI" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'telugu '("Nirmala UI" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'tibetan '("Microsoft
Himalaya" . "iso10646-1"))
    (set-fontset-font "fontset-default" 'burmese '("Myanmar Text" .
"iso10646-1"))


  (setq use-default-font-for-symbols nil)

  (set-fontset-font "fontset-default" 'unicode "Segoe UI Symbol" nil 'append)
  (set-fontset-font "fontset-default" '(#x1F600 . #x1F64F) "Segoe UI
Symbol") ; Emoji
  (set-fontset-font "fontset-default" '(#xE000 . #xF8FF) "STIX")) ;
Private Use Areas
      ;; Linux
      (set-face-attribute
       'default nil :font "Inconsolata 14")
      (dolist (charset '(kana han cjk-misc bopomofo))
(set-fontset-font "fontset-default"
  charset (font-spec :name "Hiragino Sans GB")))))

  (set-fontset-font "fontset-default" 'egyptian '("Noto Sans Egyptian
Hieroglyphs" . "iso10646-1"))
  (set-fontset-font "fontset-default" 'javanese '("Javanese Text" .
"iso10646-1"))
  )

(my--set-font)

(add-hook 'after-make-frame-functions 'my--set-font)

(setq inhibit-compacting-font-caches t)

;; font size
(set-face-attribute 'default nil :height 120)




> You mean, customizing the fontset didn't help with HELLO?  Is it
> possible that you customized the fontset for scripts other than those
> which slow down HELLO?

As you can see, some fontset settings were taken from your previous
post. I only modified the font name to adjust my machine, other parts
is my own. I believe this setup covers all the characters range in the
HELLO file?


> Thanks.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Moses <moses.mason@gmail.com>
> Date: Sun, 22 Jul 2018 14:52:54 +0000
> Cc: drew.adams@oracle.com, 32159@debbugs.gnu.org
> 
> On Sun, Jul 22, 2018 at 2:06 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > Can you try to make this a little more quantitative?  How long did it
> > take before and how long after the fontset customizations?
> 
> Gnus was usually stopped responding and lagging several seconds when
> scrolling before the fontset customization. It is much better now.

OK, so it did help.

> This is my font setting, the Linux part is not complete, because
> currently, I am on my Windows box.
> 
> ;; font
> (setq font-encoding-alist
> (append '(("gb2312.1980" . chinese-gbk)
>    ("iso8859-1$" . iso-8859-1)

You don't need to copy the entire font-encoding-alist, you just need
to replace the gb2312.1980 association there.

> > You mean, customizing the fontset didn't help with HELLO?  Is it
> > possible that you customized the fontset for scripts other than those
> > which slow down HELLO?
> 
> As you can see, some fontset settings were taken from your previous
> post. I only modified the font name to adjust my machine, other parts
> is my own. I believe this setup covers all the characters range in the
> HELLO file?

The setup I posted only covered a few scripts that were slowing down
things on my system.  It doesn't cover all of the scripts, as I didn't
need that.

So what else needs to be done for this bug report?  Do we still have
issues that need to be resolved?

Thanks.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Moses]

On Sun, Jul 22, 2018 at 3:14 PM Eli Zaretskii <eliz@gnu.org> wrote:
> The setup I posted only covered a few scripts that were slowing down
> things on my system.  It doesn't cover all of the scripts, as I didn't
> need that.
>
> So what else needs to be done for this bug report?  Do we still have
> issues that need to be resolved?

Maybe the fontset customization needs more user-friendly and
automatically. It is not obvious which script is slowing down Emacs.

Thank you for your help.

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Moses <moses.mason@gmail.com>
> Date: Sun, 22 Jul 2018 15:41:58 +0000
> Cc: drew.adams@oracle.com, 32159@debbugs.gnu.org
> 
> On Sun, Jul 22, 2018 at 3:14 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > The setup I posted only covered a few scripts that were slowing down
> > things on my system.  It doesn't cover all of the scripts, as I didn't
> > need that.
> >
> > So what else needs to be done for this bug report?  Do we still have
> > issues that need to be resolved?
> 
> Maybe the fontset customization needs more user-friendly and
> automatically. It is not obvious which script is slowing down Emacs.

For that, we'd need a machinery to find out which fonts are the cause
of slowdown.  Does anyone have any ideas, or better yet, patches?

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Andy Moreton]

On Tue 17 Jul 2018, Moses wrote:

> On Tue, Jul 17, 2018 at 10:40 AM Eli Zaretskii <eliz@gnu.org> wrote:
>> > How to get the correct second argument (script name?)
>> It is shown by "C-u C-x =".
>
> OK. I see it now. The info is somewhat not obvious to me.
>
>> > and also the "iso10646-1" (encoding name?)?
>> Always use iso10646-1, it makes no sense to use anything else with
>> these scripts.

As a followup to the list Eli suggested, this works for me on Windows
10:

  (set-frame-font "DejaVu Sans Mono-9" nil t)
  (dolist (fontspec-targets
           '(;; Latin Extended Additional block (not supported by DejaVu Sans Mono)
             (("Liberation Mono"    . "iso10646-1") (#x1e00 . #x1eff))
             ;; Other scripts
             (("DejaVu Math Tex Gyre" . "iso10646-1") mathematical)
             (("Segoe UI Symbol"    . "iso10646-1") braille symbol)
             (("Myanmar Text"       . "iso10646-1") burmese)
             (("Ebrima"             . "iso10646-1") ethiopic)
             (("Microsoft Himalaya" . "iso10646-1") tibetan)
             (("Leelawadee UI"      . "iso10646-1") khmer thai)
             (("Gadugi"             . "iso10646-1") canadian-aboriginal cherokee)
             (("Nirmala UI"         . "iso10646-1") bengali devanagari gujarati
              kannada malayalam oriya sinhala tamil telugu)))
    (dolist (target (cdr fontspec-targets))
      (set-fontset-font "fontset-default" target (car fontspec-targets))))

That reduces the delay in showing the HELLO file from approx 20secs to
approx 3secs on my box.

    AndyM

bug#32159: 26.1; inhibit-compacting-font-caches does not fix all fonts lagging issue

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Tue, 17 Jul 2018 11:24:07 +0100
> 
> As a followup to the list Eli suggested, this works for me on Windows
> 10:
> 
>   (set-frame-font "DejaVu Sans Mono-9" nil t)
>   (dolist (fontspec-targets
>            '(;; Latin Extended Additional block (not supported by DejaVu Sans Mono)
>              (("Liberation Mono"    . "iso10646-1") (#x1e00 . #x1eff))
>              ;; Other scripts
>              (("DejaVu Math Tex Gyre" . "iso10646-1") mathematical)
>              (("Segoe UI Symbol"    . "iso10646-1") braille symbol)
>              (("Myanmar Text"       . "iso10646-1") burmese)
>              (("Ebrima"             . "iso10646-1") ethiopic)
>              (("Microsoft Himalaya" . "iso10646-1") tibetan)
>              (("Leelawadee UI"      . "iso10646-1") khmer thai)
>              (("Gadugi"             . "iso10646-1") canadian-aboriginal cherokee)
>              (("Nirmala UI"         . "iso10646-1") bengali devanagari gujarati
>               kannada malayalam oriya sinhala tamil telugu)))
>     (dolist (target (cdr fontspec-targets))
>       (set-fontset-font "fontset-default" target (car fontspec-targets))))
> 
> That reduces the delay in showing the HELLO file from approx 20secs to
> approx 3secs on my box.

Some of these fonts (DejaVu Math, at least) don't come with Windows
OOTB.  More importantly, most of the others are not free, which is why
we don't want to mention them in our default fontset.