Re: Implementing image support for kitty terminal

[Eli Zaretskii <eliz@gnu.org>]

> From: Tomas Hlavaty <tom@logand.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 07 Sep 2022 20:56:09 +0200
> 
> On Wed 07 Sep 2022 at 21:11, Eli Zaretskii <eliz@gnu.org> wrote:
> > In particular, the current model of image display
> > in the Emacs display engine is that an image is basically considered a
> > single very large character, from the screen layout POV.
> 
> Are you talking about GUI emacs?

Yes.  That's the only kind of frame where we currently support
images.

> Is that not different in tty emacs?

Not in principle, no.  The only difference -- and that is important
for this discussion -- is that on TTY frames we assume that all the
screen lines have the same height.  Which contradicts the basic
requirement for displaying images.

> I thought that the tty emacs has no concept of image (so far?).

It indeed doesn't support image display, but the layout level, which
is what I'm talking about, doesn't care _how_ to display stuff, it
only cares about the dimensions on the screen.

> For example, the tty based w3m web browser optionally supports images.
> It reserves space w*h chars on the tty and then uses external program
> to draw the image in the right place.

I have no doubt that this is possible.  We are not talking about how
to display images on text-mode screens _in_principle_, we are talking
about how to add that to the existing Emacs TTY display with minimum
cost and complexity.

> I guess tty emacs would also need to reserve a w*h chars rectangle on
> the tty and then send the right escape code to fill that gap with the
> image.

The first part is the main problem to solve.  Currently, TTY display
cannot do that, it's outside its basic design, which assumes that each
screen line has the same height, no display element can be split
between screen lines (the latter design assumption is common to the
GUI display).

> First, tty emacs would need to understand images instead of throwing
> error.  At least their dimensions.  Maybe using some library or even in
> elisp, for example like this
> https://logand.com/sw/emacs-framebuffer/file/emacs-framebuffer.el.html#l286
> I have not figured out, how to actually plug it into upstream emacs, as
> this seems to require a lot of changes to C code.
> 
> Second, tty emacs would need to somehow reserve rectangle for the image.
> This might be lots of work I guess.
> 
> Third, there would have to be a piece of code which actually draws the
> image.  In case of kitty terminal, the image path and coordinates would
> be sent to the tty using the kitty terminal protocol.  In case of Linux
> console, the image path and coordinates would be sent to an external
> program like w3mimgdisplay.

The second part is the problem I'm talking about.  The other two are
much simpler to add, I think.