From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Emacs design and architecture. How about copy-on-write? Date: Wed, 20 Sep 2023 17:02:49 +0300 Message-ID: <83led1aqnq.fsf@gnu.org> References: <83edizjn0v.fsf@gnu.org> <0518f65b-1dd1-6923-8497-da4d3aeac631@gutov.dev> <87sf7fc7kd.fsf@dataswamp.org> <834jjuk68t.fsf@gnu.org> <87cyyhc7uu.fsf@dataswamp.org> <83ttrsg9nx.fsf@gnu.org> <83h6nrg4eg.fsf@gnu.org> <83v8c7elan.fsf@gnu.org> <877conk5ny.fsf@localhost> <83ttrreeu0.fsf@gnu.org> <87bkdzeas1.fsf@localhost> <83cyyfe5l8.fsf@gnu.org> <87led2o0nb.fsf@localhost> <83ttrqcpfb.fsf@gnu.org> <877comnv4a.fsf@localhost> <83fs3ackrq.fsf@gnu.org> <87ttrp89bx.fsf@localhost> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18213"; mail-complaints-to="usenet@ciao.gmane.io" Cc: acm@muc.de, incal@dataswamp.org, emacs-devel@gnu.org To: Ihor Radchenko Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Sep 20 16:03:35 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qixnj-0004Wc-2A for ged-emacs-devel@m.gmane-mx.org; Wed, 20 Sep 2023 16:03:35 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qixn7-0003Fn-5o; Wed, 20 Sep 2023 10:02:57 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qixn3-0003F1-Nh for emacs-devel@gnu.org; Wed, 20 Sep 2023 10:02:53 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qixn1-0002qT-St; Wed, 20 Sep 2023 10:02:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=X08OsDNqeSSaZrDw4MS9xTUbDhXDs9yKH7Yzo6LIQoo=; b=GIlwjEO6VARz xh7mjW7njn4iTZke7Mj1nMhHu4/R8W1gmQCYnU/p4zvQBhNkdPFPx0y3oNzay7WC9WTP338ZE314p m8yoX+m+G2xH8VY3Rpf/Spaz6kHufujQUIuH76htut6jNT486aC/tlIBzRZqrE7sWYQvsH+El34+z fzw0CIi8XuLzWX7rTDapBS7tNKrsr/A3uLnFRXvYogrC5NCTLrwhIzlKMxnvjqX7KYT+c+HS6IX0Q MnGRNIKcT2W2XlGsaAWXID/wFhtRbOPS+MfrXhZnyz2ozNeCnpFTaUII6zaZCukjm3GZIFO5vfbnP eE91sljjDPfFrVPb1A+L2A==; In-Reply-To: <87ttrp89bx.fsf@localhost> (message from Ihor Radchenko on Wed, 20 Sep 2023 09:47:46 +0000) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:310837 Archived-At: > From: Ihor Radchenko > Cc: acm@muc.de, incal@dataswamp.org, emacs-devel@gnu.org > Date: Wed, 20 Sep 2023 09:47:46 +0000 > > Eli Zaretskii writes: > > >> (I suggest this because I feel that xdisp is a rabbit hole we may sink > >> in instead of doing something more productive) > > > > I'm actually of the opposite opinion. I think trying to parallelize > > redisplay is a lower-hanging fruit, and if successful, it could bring > > non-trivial gains to Emacs even if the rest remains single-threaded. > > Without having Elisp async threads, what you suggest is only possible > for the non-Elisp parts of the redisplay. The display engine is written in C, not in Lisp. What to do with the few calls into Lisp we do in redisplay is an open question, but being able to call Lisp from redisplay and being able to run several async Lisp threads concurrently are two very different problems (although if someone comes up with a solution for the latter, that would probably solve the former as well). > May you list which C-level parts of the redisplay are known to be slow? I already did, in response to Dmitry's identical question. > AFAIU, long-lines performance is largely problematic when running Elisp. No, the slow parts are pure C. > (Also, isn't it solved already?) Not solved, but "bypassed". The solution does have its disadvantages, although we found them to be a much lesser evil than the original problem. > Another one you mentioned is displaying many frames. But isn't it > optimized by the part of the code that delays redisplay of frames that > are not visible? Or does Stefan (or anyone) tend to have so many > _visible_ frames? The latter. > >> I am particularly worried about scenarios when window geometry changes > >> by asynchronous threads. Or, say, face definitions. Imagine that it > >> happens at point when we are already drawing the now obsolete geometry > >> onto glass? > > > > xdisp.c has solutions for these two (and other similar) situations. > > The problems have nothing to do with parallelism, they happen today > > because we call Lisp at certain places in the display engine. > > Yes, but now we know exactly in which segments of the code the geometry > might change. And check for it in strategic places. No, we don't. We check the indications of this when a window or a frame has been processed, that's all. > In the case of asynchronous threads, the geometry may change any time > (when another threads happens to run something that affects geometry), > which is not accounted for by the current code. The geometry cannot change by itself, because redisplay doesn't change it. It just detects the situation where the geometry must change, and computes the variables that would determine the new geometry. Then it triggers resizing and recomputation of the geometry before retrying redisplay. The trigger to resize can be delivered to all redisplaying threads, which will make them stop what they are doing, and then retry after resize. > >> > We tried that already, with the existing Lisp threads. One reason why > >> > those are almost never used is that the display issue was left > >> > unresolved. > >> > >> That might be one reason, but not the only reason. And certainly not the > >> most important reason for the use cases where I did try to use threads > >> myself. > > > > There's more than one reason, that's true. But it doesn't change the > > fact that all those problems have to be resolved before we have > > reasonably usable threads. > > I disagree. Yes, _all_ the problems are to be solved eventually. But > solving _some_ of the problems will already be an improvement. IMHO, it > is not all-or-nothing; we can split the problem into several > sub-problems and solve them one by one. You've lost context, and effectively changed the subject, so now this sub-thread is not useful anymore. To recap, I was trying to explain why the display aspects of any concurrency cannot be disregarded if we want to have a workable solution that can be used in practice. > >> As we discussed previously, a number of Elisp programs can benefit from > >> async threads even when redisplay is not asynchronous - network queries > >> (gnus), text parsing (org-mode, slow LSP server communication, xml > >> processing). > > > > You forget that almost every Lisp program in Emacs either displays > > something or affects stuff that affects redisplay. It's easy to > > forget, I know, because in Emacs redisplay "just happens" by some > > magic. > > I don't forget that. However, we do not have to make each and every > thread _fully_ asynchronous. It is often enough to process the > performance bottlenecks asynchronously. After that processing, we can go > ahead and present the results synchronously to user. This assumes the display is always in the end. But that is a false assumption in Emacs: there are progress messages, there are prompts and requests for confirmation, there are display updates to show the stuff processed so far, etc. > Let me provide a more concrete example. > Consider something as simple as grepping across project files. > The process of grepping involves: (1) opening and searching many files; > (2) presenting results to the user. Stage (1) does not really involve > user interaction or redisplay and can greatly benefit from asynchronous > threads - we can simply search multiple files at the same time. Then, > even if stage (2) has to be synchronous, it does not matter - stage (1) > is what takes most of the time and makes the user wait. You describe an Emacs without async subprocesses -- this is how it works on MSDOS, for example. On more capable systems we display in parallel with running Grep. > > I'm not talking about async redisplay, I'm talking about the ability > > of non-main threads to display something or do something that would > > cause redisplay change the stuff on the glass. > > Then, how will it be possible without first having async Elisp threads? Async Lisp threads is a much harder problem to solve. If we solve it, this one will also be solved; but the opposite is not true.