From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jose A Ortega Ruiz Newsgroups: gmane.emacs.devel Subject: Re: Implementing image support for kitty terminal Date: Wed, 07 Sep 2022 19:49:34 +0100 Message-ID: <87fsh3yq81.fsf@mail.jao.io> References: <87v8pz18wf.fsf@mail.jao.io> <83o7vrgimc.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="22812"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Sep 07 20:53:42 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oW0BB-0005m9-PK for ged-emacs-devel@m.gmane-mx.org; Wed, 07 Sep 2022 20:53:42 +0200 Original-Received: from localhost ([::1]:35850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oW0BA-00087A-Lv for ged-emacs-devel@m.gmane-mx.org; Wed, 07 Sep 2022 14:53:40 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:42550) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oW07F-0005Zm-OF for emacs-devel@gnu.org; Wed, 07 Sep 2022 14:49:37 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:38924) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oW07F-0007zQ-BV; Wed, 07 Sep 2022 14:49:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=snW5F+1CrR6ppSuD45KIPZ74j4QwkxXpequ75ZyC+zM=; b=fNTrh1CYDeXKlZsdu5US MkkaWCZ7TC58A29hvis9kl9c1TZarTXpjjnuDmP99RavfYT3iwRWagLg2JsUxmJ7re43ZMTdSXQcx dqsmvsLJI0SdW2h5qc1xr5No6rE8NyOFkAEIXBYdla+mjQqnft3AqDeRhpXFFHZLtTZke13qH8Uv9 KxUGvNQpN5VyPR8qatMzKbbRyzYygD62R8a2msljOq+dY1z/t6VxM3zRxWIy6O4DqaBoT3+/+S41z qslbiEm81965RGMQV7BsQQntd5jIpftdfT0egoiUAwjWZFzgHoDNnijJyZn699WRzG2F8JVh0F7BG CWf6v8Hy5IEREg==; Original-Received: from cpc103048-sgyl39-2-0-cust502.18-2.cable.virginm.net ([92.233.85.247]:40318 helo=rivendell.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oW07F-0001KM-0p; Wed, 07 Sep 2022 14:49:37 -0400 Original-Received: from localhost (rivendell.localdomain [local]) by rivendell.localdomain (OpenSMTPD) with ESMTPA id 06d5f3f1; Wed, 7 Sep 2022 18:49:34 +0000 (UTC) In-Reply-To: <83o7vrgimc.fsf@gnu.org> X-Attribution: jao X-Clacks-Overhead: GNU Terry Pratchett X-URL: X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:294868 Archived-At: On Wed, Sep 07 2022, Eli Zaretskii wrote: >> From: Jose Antonio Ortega Ruiz >> Date: Wed, 07 Sep 2022 16:50:08 +0100 >> >> The kitty terminal emulator (which runs under X11 and wayland) offers a >> simple protocol for displaying images, fully described at >> . >> >> In a nutshell, it accepts an escape sequence that make it enter "graphic >> mode", followed by either encoded image data or a path to an image file >> or a shared memory location to display. Among several other niceties, >> the protocol allows drawing to rectangles specified in cell units. As a >> simple example, the sequence: >> >> _Gf=100,t=f,c=50,r=100;\ >> >> would make kitty draw the image in file.png rescaling it to 50 columns >> and 100 rows. By default, the current cursor position is used, but it's >> also possible to specify pixel offsets and sizes. >> >> At first sight, it looks as if adding support for this protocol to >> emacs's tty terminal (when kitty, or the capability (it seems other >> terminals support the same protocol) is detected) shouldn't be too >> complex, and with that, perhaps, provide direct support for the >> elisp-level image- API for these terminals (so that, for instance, >> doc-view or pdf-tools or displaying images in eww buffers would work out >> of the box). Am i wrong? > > It's hard to tell, because you haven't described your ideas of > implementing that. In particular, the current model of image display > in the Emacs display engine is that an image is basically considered a > single very large character, from the screen layout POV. Would it be possible to keep that model and only alter it when the display actually draws the character in the underlying terminal? For an image-enhanced tty that'd mean that when we are going to write this "single very large character" (whose data can be either an array of bytes or simply a path in the filesystem, stored in an associated text property perhaps) what we do is write a ESC character sequence starting with _G, as the one shown above, and specifying not only the data to draw, but also its size in rows and columns. Then, according to the kitty API promises, the image will be rendered in that rectangle for us. I am assuming here that the display engine (for a tty) knows somehow that it's printing a very lage "character" that is going to spawn several rows and columns, and also that we can always tell how wide and tall the window around point is when we display this "character". We could then decide to tell kitty to either scale the image if it doesn't fit or display only part of it. That's done by passing x,y,w,h in the escape sequence instead or r,c... something like _Gf=100,t=f,x=0,y=0,w=100,h=20;\ As you can see, the API offered is quite powerful and would do a lot of work for us. It's even able to cache images and reuse them: according to their documentation, every transmitted image can be displayed an arbitrary number of times on the screen, in different locations, using different parts of the source image, as needed. As the above ideas most probably demonstrate, i don't know anything concrete about how emacs works at this level, so apologies in advance if i'm not making sense. Thanks, jao -- They are ill discoverers that think there is no land, when they can see nothing but sea. -Francis Bacon, philosopher (1561-1626)