Re: Need `truncate-string-to-pixel-width` and `glyph-pixel-width` functions in C

[Eli Zaretskii <eliz@gnu.org>]

> From: Jimmy Yuen Ho Wong <wyuenho@gmail.com>
> Date: Thu, 24 Oct 2024 22:11:36 +0100
> Cc: emacs-devel@gnu.org
> 
>  Why would you need to truncate? and which part needs to be truncated?
> 
>  My suggestion is to align the <Type>s part so that it and the
>  following pseudo-scrollbar end exactly at the window edge.  With this
>  arrangement, you shouldn't need to truncate at least the <Type>s part.
>  This can be dome by using :align-to which counts from 'right', which
>  is the right edge of the text area. 
> 
>  Actually, I don't understand why you'd need to truncate anything,
>  since the dimensions of the child frame can be calculated such that
>  the longest completion candidate will still fit without truncation.
>  Or what am I missing?
> 
> The :align-to of the padding between candidate and annotation can only be calculated by adding the prefix
> column width, calculated from the longest prefix width, the candidate width, calculated the same way, and the
> suffix column width minus the pixel width of the suffix.

We are mis-communicating.  My suggestion is _not_ to compute the
:align-to from the width of the prefix+candidate+suffix.  My
suggestion is to calculate the width of <Type>+scroll-bar, then use
:align-to relative to the right edge of the window, like this:

  :align-to (- right (N))

where N is the pixel-width of <Type>+scroll-bar

> The reason for this is that `string-pixel-width` will be
> called on all of these concatenated strings, and the maximum of which will be given as the width to a function
> to render the child frame. This also answers your next question, yes the pop up width is already calculated
> dynamically, but against a max width, this is what I mean by "confined", hence the need for truncation before
> blitzing the strings to a child frame. :align-to (- right suffix-width) will lead `string-pixel-width` to return a
> ridiculously large number that, funnily enough, is calculated from the current window's text area width, instead
> of the child frame's window, of which the size is unknown at this time.

I don't understand this.  What is "suffix-width" in the above text,
and in what units is it measured?  And why the size of the window is
unknown?

> As to the reason for a max width, let's just say there exists language servers such as Rust Analyzer, that will
> return the signature of a function as the completion annotation, which can be quite long. Nobody wants their
> completion pop up box taking up half the screen.

Long strings can be truncated with one of the known methods.  Those
methods will not give you pixel accuracy, you don't need that if you
use :align-to relative to the right edge of the window.

> Of course, one can also adjusts the child frame parameters to let it deal with min/max size, margins and allow
> the user to resize, but the beautifully emulated scroll bar will have to be sacrificed because either I delete it
> from Corfu or the window will start truncating from the scroll bar when the user resizes the window to be
> smaller than the content width.

I don't understand this, either.  Why cannot the calculation of the
frame parameters take the scroll-bar into account, so as to avoid its
deletion/truncation?

>  You assumed something I didn't say and didn't have in mind.  There
>  should be no need to use truncate-lines.
> 
> I didn't assume anything. It's just a logical conclusion. Do you have it in mind now?

Your logic applied to my suggestions will only be correct if you
understand my suggestions completely.  Which I don't think happens
yet, because we still seem to be mis-communicating.

>  This is not about refusal.  It isn't like what you are asking is easy
> 
>  to do, and the "Emacs devs" are just a lazy bunch.  I looked at the
>  code involved, and I currently have no idea how to implement what you
>  have in mind so that it works in all the cases, those which you
>  mentioned and those you didn't, and works in a way that will be useful
>  for Lisp programs.  You may think it's easy, but it isn't.  Some
>  things are even conceptually not simple, for example whether it is
>  okay or not to break a string between U+1F3C2 and the following
>  U+1F3FF, and if so, how to discover that subject to how the current
>  character-composition machinery in Emacs was designed and implemented.
> 
> You already have `string-glyph-split` which works correctly, (a job well done BTW, even more comformant
> than Chrome and Webkit). There are only 3 things you need to do for a hypothetical `glyph-pixel-width`:

You originally asked for truncate-string-to-pixel-width which would
avoid splitting grapheme clusters.  What is glyph-pixel-width, and
what should it do?

> 1. With no faces, find the width of a character glyph from the default font or whatever `char-displayable-p`
> returns. This is easy, I can already do this from Elisp reasonably fast by going straight to the terminal using
> `composite-get-gstring`.

This is incomplete and inaccurate, because it ignores some important
complications.  First, you forget images.  You also ignore the
problematic aspects of getting width from composite-get-gstring.  And
there are other complications, for example with TAB characters.

> 2. With faces, resolve the font from the faces, go to 1. Still no need to deal with windows. Can be done
> correctly in Elisp, but incredibly slowly due to anon faces, face lists and having to go through the recursive
> `face-attribute-merged-with`. The logic in `face-attribute-merged-with` can be duplicated and modified to deal
> with anon faces, and run iteratively, but still a few orders of magnitude slower than it needs to be.

Faces cannot be resolved without a frame (because each frame can have
a different definition of the same face), thus a window is required.
A window is also required because overlays (which can define faces)
can be window-specific.  In general you also need a buffer, because
there's buffer-face-mode.  And knowing the font is far from enough,
because of text shaping (what we call "character composition"),
without which we cannot reliably know what glyph(s) from the font to
use for the metrics.

IOW, your proposed idea will work for the simple cases, but falls
apart for a general-purpose API which will rightfully be expected to
work in any situation and use case supported by Emacs.

> 3. Display properties is when you start to need a window, as a first try, you can delegate this back to the
> current slow path via `window-text-pixel-size`.

This is only a small part of the problem.

> I'm not saying emacs-devs are lazy, I'm just saying after 14 years, Emacs finally can split strings with emojis
> correctly, can we go one step further and give Emacs users the ability to layout in pixel precision in a more
> performant manner please?

I explained why this is hard to implement correctly, at least as long
as we are talking about truncate-string-to-pixel-width which should
avoid splitting grapheme clusters.  string-glyph-split was relatively
easy because we were lucky to already have everything exposed to Lisp.
We are very far from that point with this request.  The existing
high-level C APIs, which we use in window-text-pixel-size and its ilk,
are inappropriate, because they are unable to reliably find a buffer
or string position that corresponds to a given pixel-width.  AFAIU, to
implement what you asked for, we'd need a whole new set of C APIs,
which will have to be implemented from scratch using the low-level
functions which move one display element at a time, and pay attention
to all the Emacs display complications: faces, character compositions,
display properties, overlays, images, xwidgets, word-wrap,
line-truncation, etc.  Sure, the elements of all this exist, so "we
have the technology", but the problem is the higher-level logic that
would combine those elements into a coherent whole and produce results
that Lisp programs expect.

IOW, this is not just "go one step further", this is a huge step that
will require a lot of coding and debugging.  Of course, if someone
sees a way to do it easier, let's see the code or at least an idea of
such a code.  The ideas you presented are too simplistic to fit the
bill, unfortunately.