From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tomas Hlavaty Newsgroups: gmane.emacs.devel Subject: Re: Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets) Date: Thu, 03 Dec 2020 22:02:47 +0100 Message-ID: <87360my554.fsf@logand.com> References: <87mtyw9c7f.fsf@logand.com> <83r1o7eyya.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="20107"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Dec 03 22:03:37 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkvlI-00055J-Lt for ged-emacs-devel@m.gmane-mx.org; Thu, 03 Dec 2020 22:03:36 +0100 Original-Received: from localhost ([::1]:39348 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kkvlH-0004iu-On for ged-emacs-devel@m.gmane-mx.org; Thu, 03 Dec 2020 16:03:35 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:47596) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kkvkl-0004HI-CH for emacs-devel@gnu.org; Thu, 03 Dec 2020 16:03:03 -0500 Original-Received: from logand.com ([37.48.87.44]:38016) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kkvki-000442-VR; Thu, 03 Dec 2020 16:03:02 -0500 Original-Received: by logand.com (Postfix, from userid 1001) id 8034119F5A3; Thu, 3 Dec 2020 22:02:50 +0100 (CET) X-Mailer: emacs 26.3 (via feedmail 11-beta-1 I) In-Reply-To: <83r1o7eyya.fsf@gnu.org> Received-SPF: pass client-ip=37.48.87.44; envelope-from=tom@logand.com; helo=logand.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:260246 Archived-At: On Thu 03 Dec 2020 at 16:39, Eli Zaretskii wrote: >> At the moment, in emacs-framebuffer, I just call it to display an image >> using call-process-region. It could be optimized with pipe, but this >> has low priority for emacs-framebuffer as it is fast enough for my >> needs. > > IMNSHO, neither call-process-region nor pipes are fast enough to be an > integral part of the Emacs redisplay: they are too slow. Especially > if you want to be able to display more than one image at the same > time. To see how the performance is inadequate, turn on flyspell-mode > and scroll through a large text buffer, preferably with lots of > mis-spelled words (or some text full of special terminology the > general-purpose dictionary won't know about). The effect in your case > will be very similar. > >> I am not exatly sure, what the concern is here. >> >> The images do not travel via pipe. Only meta-data like filenames, >> coordinates and handles. > > Yes, I understood that much. But that doesn't matter: communicating > with a spell-checker in the flyspell-mode case also sends just a small > number of words in each direction, and yet the slow-down is > considerable, sometimes downright annoying. Is it an appropriate comparison? I am sure it is possible to find many examples where Emacs slows down to crawl even without external process, e.g. a buffer with long lines. Does it mean I should dismiss Emacs for editing text? I do not think my use-case is comparable to "scrolling through a large text buffer, preferably with lots of mis-spelled words (or some text full of special terminology the general-purpose dictionary won't know about)". I do not intent to scroll through a large buffer with lots images. Even if I did, there is no evidence that it would suffer the problems in the above example. w3m manages just fine. My use case is something like image-mode or doc-view or pdf-tools. emacs-framebuffer already does that for many image and document types and I have no problems with speed. In any case, ability to display images and documents on the console beats no ability at all. >> What exactly needs to be done in the middle of redisplay that is >> relevant to drawing images? Determining image width and height? > > Sending the image spec and receiving the dimensions and other image > attributes, yes. And I presume that the sub-process needs some time > to calculate the answer (because at least some queries require to load > and process an image). This is a non-issue. Images usually do not change those attributes quickly, if at all. So this can usually be cached or even calculated just once. This is what I do in emacs-framebuffer. >> > You are supposed to submit data in one of the known formats. Why is >> > that a problem? >> >> Because they do not work on the console. >> >> And the reason they do not work on the console is because there is >> hardcoded dependency on image drawing libraries which do not work on the >> console. > > No, that's not the main reason. See below. > >> > On the console you currently cannot have images in Emacs, period. >> >> Exactly. And I want this to change. There is no reason for such >> restriction. >> >> > This is not because of image libraries, this is because the text-mode >> > display simply cannot currently support such display elements. >> >> This is not an issue at all. Capabilities of text-mode display are >> irrelevant. w3mimgdisplay is using framebuffer. I do not know how to >> better communicate that. > > I was talking about the Emacs text-mode display code, sorry for being > unclear. The Emacs display code which handles the console is not > capable of supporting images, or any display element whose dimensions > are not the same as character cells on a console. The assumption that > each display element takes exactly one character cell on display is > implicitly present in many places in the Emacs display code that > handles TTYs. We cannot even display a larger font on a console. > (There is a special handling of wide characters that take two columns > on display -- but still the same height as other characters -- but > that's all.) > > To be able to display images on a console within an Emacs text-mode > frame, you need to be able to tell the Emacs display code that a > certain area on display is "taken". But since images will generally > be larger than a character, in both dimensions, you need the Emacs > text-mode display code to support display elements of arbitrary > dimensions, and that is currently not possible. Adding that would > need serious changes in the text-mode parts of the Emacs display > engine code, and that is almost entirely in C. > > This is the main obstacle we'd need to overcome if Emacs will ever be > able to display images on TTYs. All the rest is much easier. I understand that. I do not aim to change that. Again, my use-case is like that of image-mode, doc-view and pdf-tools. There is a file, say /tmp/a.png, I open the file in a buffer and the buffer will show the png image. Or there is a file, say /tmp/a.pdf, I open the file in a buffer and the buffer will show the pdf document. This use-case does not need to deal with the issues outlined above. It just needs to know size and position of the buffer, width and height of characters and screen etc. emacs-framebuffer already does that and it is not that difficult, thanks to the hints I already got from you:-) >> > (You never said you were talking about a console.) We currently don't >> > expose console cursor control to Lisp. Should be easy to add that, if >> > needed. >> >> I see. >> >> cursor-type does not say it does not work on the console. >> >> I think not all values are doable and relevant. t and nil should be >> enough. >> >> Shell I open a bug report for this? > > A feature request, yes. ok, thank you. >> > I don't think we have such a hook, no. But if we had it, using it >> > like you want would cause annoying flickering, because you'd >> > constantly redraw the image at very high frequency. >> >> I do not think it would cause flickering on the console. The only thing >> that normally "flickers" is the blinking cursor. Emacs does not draw >> anything if nothing changed. > > Emacs doesn't draw anything because it knows what's on the glass and > what _should_ be on the glass, and compares the two to decide what > should be redrawn. But you want to draw on parts of the screen behind > Emacs's back, so it won't know whether something changed. And you > OTOH have no way of knowing where Emacs have redrawn something during > the last redisplay cycle. So you will have to redraw your images each > time Emacs finishes a redisplay cycle, regardless of whether it > changed anything on the glass in the area of an image. These > redisplay cycles happen at very high frequency -- usually, Emacs > enters redisplay, quickly decides what needs to be redrawn, and then > does whatever has to be done and exits redisplay. It tries very hard > not to redraw the parts that haven't changed, and that is why there's > no flickering. But I don't see how you could prevent the flickering > of those add-on images, because without knowing which parts of the > screen changed, you will have to redraw them very frequently. This is not an issue. You can try and see it easily. Start emacs on the console. Start shell in Emacs: M-x shell Execute: $ cat /dev/urandom >/dev/fb0 Observe what Emacs draws. I can see blinking cursor. Part of the *shell* buffer was drawn and line and column indicators on the mode-line. Everything else remains unchanged. Now ssh from a different computer and run $ cat /dev/urandom >/dev/fb0 Observe what Emacs draws. I can see blinking cursor. Everything else on the screen remains unchanged. After some time, clock on the mode-line is drawn. So if Emacs does not change anything, it does not draw anything on the console and I do not need to redraw the image to the framebuffer. The only time I need to redraw the image to the framebuffer is when emacs changed and drawn something. But this is a single event, not a periodic thing. Because there is no periodic redraw, there is no flickering. Unfortunately, there is no hook to run after this single event. >> >> Is there such hook? If not, would it be difficult to add it? >> > >> > It won't be hard to add it, but I'm not sure we want to add such a >> > hook, see above. >> >> That is a shame. If there was a hook to detect when Emacs has drawn >> something on the console, I could plug image redrawing there. >> >> If I wanted to add it, where in the code should I look? > > In update_frame and its subroutines. But my recommendation is not use > such a kludgey approach. Thanks, I'll have a look.