Re: Implementing image support for kitty terminal

[Jose A Ortega Ruiz <jao@gnu.org>]

On Fri, Sep 09 2022, Eli Zaretskii wrote:

[...]

>> for instance, displaying an image would amount (logically) to
>> displaying a bunch of lines of text, all of the same length, with
>> the only peculiarity that maybe the cells on the borders wouldn't
>> look completely filled.
>
> That won't work, because Emacs must know the screen layout; you cannot
> lie to it with impunity.  For example, with your idea above, if point
> is in the first line of a window, and you type C-n, and the next line
> displays an image which takes more than one character cell's height,
> then what will be the line number, as it's known to Emacs, when point
> moves to that second line?  If you tell Emacs that there are 10 lines
> of text in-between, instead of a single tall line, Emacs will think
> the next line is line 11, whereas from the user POV it's line 2.

Say that, in that line, i have an image, which emacs knows about because
there's, say, a text property with 4 rows x 5 columns as the image
extent (which we derive from knowing the image size and the terminal
cell size, in pixels).  For movement and display purpose, we tell Emacs
to interpret that as the block of pure text

xxxx
xxxx
xxxx
xxxx
xxxx

and emacs, initially, displays that, as a normal tty would, with point
at the beginning of the block.  Now, it computes how many lines are
visible after any required scrolling, and then asks kitty to draw, on
the visible rectangle, the corresponding part of the image.

All movements and positioning would happen as if all that is being
displayed are blocks of text.  Of course that means that line numbers
won't treat images as spawning just one line, but several.  I find that
actually an advantage from the user's point of view, because the images
are taking more than one line in the glass, so to me it's very natural
that emacs tells me that, at the beginning of the image, line is no. 2,
and, at the end, line is no. 20.  and images could be scrolled line by
line in that schema. These terminals use the GPU for rendering when
available, so my expectation is that frequent redraws of the displayed
image sections will be fast enough, but i could well be wrong.

> A lot of Emacs features depend on Emacs knowing exactly what's on the
> screen, so you cannot easily lie to it about that.

Yes.  I am proposing we tell emacs that what's on the screen is some
lines of xs-text and making that, actually, true in every sense except
that we also tell kitty to display those rows differently at the very
end, when emacs is done with its text display duties.

jao
-- 
Whenever you commend, add your reasons for doing so; it is this which
distinguishes the approbation of a man of sense from the flattery of
sycophants and admiration of fools. -Richard Steele (1672-1729)