From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Concurrency via isolated process/thread Date: Sun, 09 Jul 2023 10:01:57 +0300 Message-ID: <83bkglbmai.fsf@gnu.org> References: <871qhnr4ty.fsf@localhost> <871qhmo5nv.fsf@yahoo.com> <87bkgq8p5t.fsf@localhost> <831qhmjwk0.fsf@gnu.org> <875y6y8nlr.fsf@localhost> <87h6qhnalc.fsf@yahoo.com> <87ilax71wo.fsf@localhost> <831qhli14t.fsf@gnu.org> <87wmzdxewc.fsf@localhost> <83r0plgjeo.fsf@gnu.org> <87o7kpxapo.fsf@localhost> <83mt09gcaf.fsf@gnu.org> <87wmzbc3af.fsf@localhost> <87edljhmjq.fsf@yahoo.com> <87fs5zbuwn.fsf@localhost> <871qhjgrku.fsf@yahoo.com> <831qhieumz.fsf@gnu.org> <87lefqg7yl.fsf@yahoo.com> <83h6qed8kw.fsf@gnu.org> <87h6qefwj0.fsf@yahoo.com> <83bkgmcx0q.fsf@gnu.org> <87a5w5gbs4.fsf@yahoo.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="39792"; mail-complaints-to="usenet@ciao.gmane.io" Cc: yantar92@posteo.net, emacs-devel@gnu.org To: Po Lu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Jul 09 09:02:45 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qIORR-000AA7-2O for ged-emacs-devel@m.gmane-mx.org; Sun, 09 Jul 2023 09:02:45 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qIOQb-0000iv-Nr; Sun, 09 Jul 2023 03:01:53 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qIOQZ-0000iN-MG for emacs-devel@gnu.org; Sun, 09 Jul 2023 03:01:51 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qIOQY-0006oS-83; Sun, 09 Jul 2023 03:01:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=NLcMOrhWbyZSZNlBLYKBbJogijkkMYEgBMehlRFxS/Y=; b=cYcOSjHsdzpq InCzNjemodnFlFh5p6hnc5woT5rn7VTJ13Ny1nitO5iG0eA+cf//3sEqNVoLujrVx/YaNDuiC4jEv jZNURg95fFP7se0cCwAYbN7acITOSzAn1oA2El9disTLtu4n0XltLMPdMfPW18XcvvEBDupwBuajq pjKHO0fu7widAH6SfiYN6QPPGTmdqD2HPr1BHkSZL+lzjuFXIFCs18uzuhejnOI82WkAkeNyPZKxu Gl0lhsEKVDoO/zFVSxPF0aDNWzu2UQzhPz4Tpxn+XxsQhyPNwzKpiJhDBvxT/0KQM5f1hcOoYhuid GBQGJKBoZvbisgsJqWX3fw==; Original-Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qIOQX-0003dw-N6; Sun, 09 Jul 2023 03:01:50 -0400 In-Reply-To: <87a5w5gbs4.fsf@yahoo.com> (message from Po Lu on Sun, 09 Jul 2023 08:37:47 +0800) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:307642 Archived-At: > From: Po Lu > Cc: yantar92@posteo.net, emacs-devel@gnu.org > Date: Sun, 09 Jul 2023 08:37:47 +0800 > > Eli Zaretskii writes: > > > IOW, I think you have a very narrow idea of "number and text crunching > > tasks" that could benefit from threads. For example, one of the > > frequent requests is to run the part of Gnus that fetches email and > > articles in a separate thread -- if this is okay for "number and text > > crunching tasks", then it is likely to prompt users. > > Prompting for options should take place before the thread is started, or > after the data is retrieved and about to be displayed. That is of course not always possible, or even desirable. For example, some prompts are caused by specific aspects of the processing, which may or may not happen, depending on the data. Prompting for options unrelated to the actual processing would mean annoying users unnecessarily. Etc. etc. > > We are mis-communicating. My point is that it is almost impossible to > > take an existing non-trivial Lisp program and let it run from a > > non-main thread without bumping into this issue. Your responses > > indicate that you agree with me: such Lisp programs need to be written > > from scratch under the assumption that they will run from a non-main > > thread. > > I agree with this completely. From my POV, such requirements are > reasonable and not very different from the requirements for doing so in > other GUI toolkits and programs. So basically, what you have in mind is a way of accruing a body of special-purpose Lisp programs written specifically for running from non-main threads. Which means reusing existing code or packages not written to these specifications will be impossible, and we will in effect have two completely separate flavors of Emacs Lisp programs. It would mean, in particular, that many functions in simple.el, subr.el, and other similar infrastructure packages will need to have specialized variants suitable for running in non-main threads. And all this will be possible if -- and it's a large "if" -- the necessary support on the C level will be written and prove reliable. If this is the plan, it might be possible, at least in principle, but is it really what is on people's mind when they dream about "more concurrent Emacs"? I doubt that. > > How is this compatible with the goal of having threads in Emacs, which > > are to allow running Lisp code with less hair than the existing timers > > or emacs-async? > > I thought the concern was one of efficiency, and not ease of use: > process IO is slow compared to sharing the same VM space, and > subprocesses also utilize more memory. Memory is cheap these days, and "slow IO" is still a gain when it allows us to use more than a single CPU execution unit at the same time. So yes, efficiency is desirable, but ease of use is also important. What's more, I don't think anyone really wants to have to write Lisp programs in a completely different way when we want them to run from threads. > >> > Fixed how? > >> > >> By replacing `sit-for' with `sleep-for' (and in general avoiding > >> functions that call redisplay or GUI functions.) > > > > So programs running in non-main threads will be unable to do stuff > > like show progress etc.? That's not very encouraging, to say the > > least. > > They should be able to run code from the main thread's command loop, via > mechanisms such as `unread-command-events'. Those mechanisms only work when Emacs is idle, which is bad for features like progress reporting. Doing this right requires to redesign how redisplay kicks in, and probably have several different kinds of redisplay, not one. > > This basically means rewriting most of Emacs. Because most APIs and > > subroutines we use in every Lisp program were not "specifically > > written for running outside the main thread". So we'll need special > > variants of all of those to do those simple jobs. > > I wasn't thinking about Lisp functions used for text editing tasks, but > rather raw data crunching functions. Most of these are already > side-effect free, and some are even truly reentrant. I think none of them are side-effect free, if you look closely. They access buffer text, they move point, they temporarily change the selected window and the current buffer, the access and many times modify the obarray, etc. etc.