bug#47712: 27.1; Provide `string-display-width` function, which takes properties into account, `substring-width`

[Eli Zaretskii <eliz@gnu.org>]

> Cc: 47712@debbugs.gnu.org
> From: Daniel Mendler <mail@daniel-mendler.de>
> Date: Mon, 12 Apr 2021 10:45:45 +0200
> 
> On 4/12/21 4:26 AM, Eli Zaretskii wrote:
> > We already have window-text-pixel-size; doesn't it fit the bill?  If
> > not, why not?
> 
> `window-text-pixel-size` computes the size of a portion of the window, 
> it does not take a string argument.

That is easy to work around, right?  Just insert the string into a
temporary buffer and say with-current-buffer (and/or
with-selected-window, if needed).

> The function `string-width` instead takes a string argument
> returning the number of columns needed to display this string. This
> function is needed to compute widths of strings which are not yet
> displayed.

But you compute the width because you are going to display that string
soon enough, right?  Or did I misunderstand the purpose?

In general, calculating a string's width out of display context is
meaningless in Emacs.  More about that below.

> `string-display-width` is a proposed function which takes properties 
> into account as `window-text-pixel-size` does, but without going through 
> the display system first.

That's exactly my point: by not using the display code, you allow up
front inaccurate results.  There's no practical way to implement this
in Lisp without yielding inaccurate and even grossly incorrect results
in some cases (see below), so any such implementation will be limited
from the get-go, in ways that are hard to even document precisely, let
alone use reliably.  Thus, users of such an implementation are bound
to be unpleasantly surprised if they try using it in a situation
different from yours.

> Instead it computes the `string-width` of a flattened string, where
> the invisible parts have been removed and the displayed parts have
> been replaced.

Yes, I understood what the code does.  But it only handles some of the
possible use cases, and will fail in others.  For example, it ignores
faces (which could change the metrics of characters); it assumes that
all the characters of the string can be displayed by the buffer's
fonts (because "tofu" has very different dimensions); it uses
char-width to measure a character's width (which is only accurate for
text-mode frames); it doesn't handle display properties that aren't
strings, such as (space :align-to N); etc.

> Orgmode uses its incarnation of `string-display-width`, 
> `org-string-width`, for formatting its tables (see org-table.el). The 
> width computation is performed before displaying the strings in the window.

What you propose might be good enough for org-table, but it falls
short of what is required from a general-purpose core feature: such a
feature needs to support, or at least be able to support, all the
possible display-related features that could affect the string's
width.  Your proposed implementation cannot support all of those
features even in principle, because some of the display features
mentioned above aren't accessible directly from Lisp.  And if we try
implementing this in C, we will end up with window-text-pixel-size,
because that's why it was written in the first place.  It was written
to use the display code and requires a context of a window, because
otherwise the problem you are trying to solve cannot be solved:
features like fonts, faces, invisibility specs, etc. -- all of those
depend on windows or buffers or frames, so trying to estimate a
string's width without that context will produce incorrect results.

So I urge you to try to use window-text-pixel-size for org-table and
elsewhere, because that's what that function is for.  If it lacks some
feature or has bugs, we will improve it.

Thanks.