Re: Supporting stylistic sets

[Eli Zaretskii <eliz@gnu.org>]

> From: Nicolas Ouellet-payeur <nicolaso@google.com>
> Date: Fri, 23 Sep 2022 15:20:54 -0400
> Cc: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>, 
> 	emacs-devel@gnu.org
> 
> In my case specific: because the 'cv31' variant of my font changes the
> shape of parentheses, and I like that shape better. I would rather set
> it everywhere, even the minibuffer, mode-line, header-line.

You could do something similar to what jit-lock does: place a text
property or an overlay on each parenthesis that is going to be
displayed.

> > IMO, it is impractical to shape everything via a shaping engine
> > without completely redesigning how we handle character compositions in
> > the display engine, because what we have now was not designed to be
> > used for all the text we display.
> 
> That's one of my big worries, as well. That's why I'm anxious to share
> a proof-of-concept ASAP, and see if it's worth pursuing.
> 
> Initially it *was* super janky, because I did the naïve thing and passed
> strings through hbfont_shape() without caching the result. Each
> redisplay would pass the strings to HarfBuzz again, and create a bunch
> of new Lisp vectors... Then the GC would go crazy, and cause a *lot* of
> jank.
> 
> I then tried to put everything into `gstring_hash_table' in composite.c.
> That made things better, but the hashtable would keep growing in size
> all the time. Basically leaking memory on purpose...
> 
> My latest attempt is to tweak `region_cache' so it can store arbitrary
> Lisp_Objects, rather than just 1s or 0s for known-ness of regions. Then
> I stick the lgstrings into that cache. Different frames have different
> face properties, so there's one region_cache per (frame,buffer) pair.
> This works, but right now it's incomplete and crashes all the time like
> I said.
> 
> It's not *noticeably* jankier so far, and I even used the
> `gstring_hash_table' version as my main editor for a while. But maybe my
> computer is just fast enough, or my use-cases are too simple. It's
> *definitely* way too slow for very long lines. For example, a JSON file
> that's a single line and multiple megabytes. We'd need to fall back to
> the existing code in cases like that.

To measure the slowdown, run the benchmark of scrolling through a
large file, like xdisp.c, one line at a time, and compare with the
current code.

I think one other source of slowdown is that the glyph production is
interrupted too frequently.  This is something unrelated to caching
the compositions.

> > You can have ligatures today without passing everything through the
> > shaping engine.
> 
> Yes, but you need to configure those manually. Here, the advantage of
> passing them to HarfBuzz is that it can read those ligatures from the
> font file without any extra configuration. i.e., "Support ligatures out
> of the box" as described in etc/TODO. That's not going to change the
> world, but if we can make it work it'll be neat.

But what do you do if the user wants to disable ligation, or control
which ligatures should and shouldn't happen?  The fact that a font
that supports ligatures is used in some buffer (or even the default
face) doesn't yet mean the user wants these ligatures everywhere.  As
a simple example, consider stuff like := or **-, which is sometimes
shown on the mode line -- we don't want ligation there, do we?  If you
leave it to the font to decide when to ligate, Emacs users lose
control of that.  That might be OK in the lies of VSCode, but not in
Emacs.

> > And the main difficulty with such a redesign is to do it in a way that
> > still allows easy customization of composition rules from Lisp.
> 
> A quick test shows that `prettify-symbols-mode' still works, but I'm
> sure there's subtle bugs hiding underneath.

prettify-symbols-mode uses static compositions, not automatic
compositions, so it's little wonder prettify-symbols-mode isn't
affected: it uses completely different code paths.

I was talking about the way we control automatic compositions through
composition-function-table.  This gives us all the freedom and
convenience to control compositions, but it comes with a price: we
need to call into Lisp (which then calls back into C) to produce
glyphs, and we need to match the text against regexps to know what
sequences can and cannot be composed.