From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: HaiJun Zhang Newsgroups: gmane.emacs.bugs Subject: bug#38807: [Feature request]: Support lisp workers like web workers. Date: Wed, 1 Jan 2020 10:59:31 +0800 Message-ID: References: <83o8vpn8g1.fsf@gnu.org> <87mub9u0ld.fsf@gmx.de> <831rslmxih.fsf@gnu.org> <83lfqslafm.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="5e0c0b19_5451cf49_4379" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="20776"; mail-complaints-to="usenet@blaine.gmane.org" Cc: 38807@debbugs.gnu.org, michael.albinus@gmx.de To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Jan 01 04:00:15 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1imUF4-0005F1-LS for geb-bug-gnu-emacs@m.gmane.org; Wed, 01 Jan 2020 04:00:15 +0100 Original-Received: from localhost ([::1]:47866 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1imUF2-0000Eq-E0 for geb-bug-gnu-emacs@m.gmane.org; Tue, 31 Dec 2019 22:00:12 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:57081) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1imUEu-0000Eg-M7 for bug-gnu-emacs@gnu.org; Tue, 31 Dec 2019 22:00:06 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1imUEt-0003Cc-0o for bug-gnu-emacs@gnu.org; Tue, 31 Dec 2019 22:00:04 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:57633) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1imUEs-0003C2-Sl for bug-gnu-emacs@gnu.org; Tue, 31 Dec 2019 22:00:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1imUEs-0005EI-Pz for bug-gnu-emacs@gnu.org; Tue, 31 Dec 2019 22:00:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: HaiJun Zhang Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 01 Jan 2020 03:00:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38807 X-GNU-PR-Package: emacs Original-Received: via spool by 38807-submit@debbugs.gnu.org id=B38807.157784759220054 (code B ref 38807); Wed, 01 Jan 2020 03:00:02 +0000 Original-Received: (at 38807) by debbugs.gnu.org; 1 Jan 2020 02:59:52 +0000 Original-Received: from localhost ([127.0.0.1]:35373 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1imUEh-0005DO-SA for submit@debbugs.gnu.org; Tue, 31 Dec 2019 21:59:52 -0500 Original-Received: from mail-oln040092255103.outbound.protection.outlook.com ([40.92.255.103]:63228 helo=APC01-HK2-obe.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1imUEf-0005D9-3n for 38807@debbugs.gnu.org; Tue, 31 Dec 2019 21:59:51 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H+Sud/cROJF7JrQAz8bskZ6y2Pe29yuGIotQR6+MGlpxeP3mWaQxSeEXIWiNVP/FqqoFYwz4BlSvq8jao7lwr8agq6oNmw69ALs2nT6AfVNTvFMA06Slr9OSGOd4WIGuxxbb4e4vRybVR97+6egiB21Jt4kb2aG1GfaVJWu548QQdvQ0UyIldWCfxoHYOStXhwHGlO4dneYL/e8LBlO/ZYd8cscdQrdOCW+A9Kz7tb0irrqHkZRIzK4U9pdGiOmbmSVv9NQatSQVV9k/817OpxOT2Rtofz3J++Q2PYTdWLMqSwLpWBNR63/coGEjwVvABeNG7A4L37kkWDNTyMsidg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F0faxDl4LjYCHBppZS1a12DzVftmLe4lVgo5650XrTs=; b=YV04KTvehHmnCVpI0n//3PhNQdLuXgxUd0xVWC9kh1RyJJ/yzVTKo0r1kh80+jnwHjUaoV1l/Y/wUdN3kBqs97NElTtbn+FffO7tCawpEqq7wwb+kxKNXE7Hz6aeBanGBUtkFPwcoMXnwVBZg7aQXAFUefva4D+iMmY5QZ6UiboQ5dZ8AlmCylpiQ9l0bfW1MGTlPsauodezNC5/VlTsDwZ9tzFf4aI/xRfjQTv74fwk8r40YOAs2WPx7J45zOuxlbEPTxOWp0vISwMJrmpd4L0s/CX1JT4fJIYln32O1K07jwPJ2AtTtF9DzfTHCu1ljCR6M0fGAQtWEgcOgjr4nQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=outlook.com; dmarc=pass action=none header.from=outlook.com; dkim=pass header.d=outlook.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F0faxDl4LjYCHBppZS1a12DzVftmLe4lVgo5650XrTs=; b=gsfz0swMKVebpfdkjhASP8RxqcqPNvpE5FyFIew2tAlwuwEPEL0mdQQvgp2TsGqei/K9xjl7Z3E3gdam2QsCJhR/NRi9WAlzggx9SO3TZsWl+Kfato1dM4Giz4/67DRcMgdENW+UGwZ3ygdBoSEdIXU0/FGT3cS9iOP7YI4A70cqWkC1SS3N6Pk4p3vx0FoI650PMEZJwG2DheJsErJM8/aCQbgyIVjLPDL8YLxRW34VKrzWqbbXuqTqMOLjTpT7Nfxr7yiucn0oJhRzW2sSr/ZZMXgGUgIUpwr7/axEGufyysSCshjI57qTpWiroNBFSNsGHZTyIyveP2WlLHBNpw== Original-Received: from HK2APC01FT007.eop-APC01.prod.protection.outlook.com (10.152.248.58) by HK2APC01HT042.eop-APC01.prod.protection.outlook.com (10.152.249.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.11; Wed, 1 Jan 2020 02:59:40 +0000 Original-Received: from SG2PR03MB3611.apcprd03.prod.outlook.com (10.152.248.60) by HK2APC01FT007.mail.protection.outlook.com (10.152.248.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.11 via Frontend Transport; Wed, 1 Jan 2020 02:59:40 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:82A621800856D6A9AB04A0F71F784EE55A270B30FF0A1812FBA258D05404F738; UpperCasedChecksum:BCB8A226DDB832ECB37EF2EFBFEF78D6EFC065F46C6F0600311CD379B811B560; SizeAsReceived:9003; Count:48 Original-Received: from SG2PR03MB3611.apcprd03.prod.outlook.com ([fe80::65f4:d979:9d3d:1829]) by SG2PR03MB3611.apcprd03.prod.outlook.com ([fe80::65f4:d979:9d3d:1829%7]) with mapi id 15.20.2602.010; Wed, 1 Jan 2020 02:59:40 +0000 In-Reply-To: <83lfqslafm.fsf@gnu.org> X-Readdle-Message-ID: ea3f0cf0-d0b1-4680-a9e5-6d9384946289@Spark X-ClientProxiedBy: HK0PR03CA0105.apcprd03.prod.outlook.com (2603:1096:203:b0::21) To SG2PR03MB3611.apcprd03.prod.outlook.com (2603:1096:4:17::11) X-Microsoft-Original-Message-ID: Original-Received: from [192.168.1.103] (1.193.170.223) by HK0PR03CA0105.apcprd03.prod.outlook.com (2603:1096:203:b0::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.11 via Frontend Transport; Wed, 1 Jan 2020 02:59:39 +0000 X-Readdle-Message-ID: ea3f0cf0-d0b1-4680-a9e5-6d9384946289@Spark X-Microsoft-Original-Message-ID: X-TMN: [ACP+Qv8Vr0XiZuraZGopINkXphW+0ilv] X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 48 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: b57cb36b-eb63-4cd6-4f8d-08d78e66a522 X-MS-TrafficTypeDiagnostic: HK2APC01HT042: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +Pa6EpZTtnzGixivUbi8n9WWNioPaxA7UEk0QkO4UkD9NNmGoyEkkpynSLUOJ0f+fNz/1cYG6qhLDqnQhemLHZBBYLaCFi0S9ewyg7zCyidj5wzPm4PDg9N/jD8so4FGLw1LKdMzZuCU5FS+S292D0i3jjV2UGUabupBdYb37+WIxZV/TLNh+ZG36tkvO2JyyZuanFt46YRWN+Pwy3yyQu9IUuVimLx67jb56mxAnsk= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b57cb36b-eb63-4cd6-4f8d-08d78e66a522 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jan 2020 02:59:40.8449 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HK2APC01HT042 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:174000 Archived-At: --5e0c0b19_5451cf49_4379 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline =E5=9C=A8 2020=E5=B9=B41=E6=9C=881=E6=97=A5 +0800 AM12:36=EF=BC=8CEli Zar= etskii =EF=BC=8C=E5=86=99=E9=81=93=EF=BC=9A > The Lisp interpreter already sort-of =22queues=22 display changes, beca= use > it only changes Lisp data structures, and then the display engine > references those data structures at display time. So those data > structures serve as a kind-of =22queue=22, since redisplay runs when Em= acs > is idle (i.e., the last Lisp command completed its job). > > But the problem is in the other direction: it's not that Lisp somehow > =22drives=22 the UI, it's that the UI frequently calls into Lisp as par= t > of its job. The simplest example is mode-line format: it includes > references to variables, and can also include :eval forms that call > the interpreter. > It is much clearer to me now. We can keep them all as the UI thread. > Your idea in fact means to have several isolated Lisp machines in the > same process. But how can we do something like that without a very > radical redesign of Emacs, when so many things in Emacs are implicitly > part of the global state=3F Buffers, global variables, windows, > frames--all those are global resources, and every thread will want to > access them. Emacs was not designed to allow that. > All those global resources are not accessible by workers. Workers can onl= y access network, file system and other non-global resources. They only d= o the following things: 1. retrieve content from network, parse the data, and send the result(lis= p data) to the UI thread to present it 2. communicate with subprocesses, parse the data from subprocesses, and s= end the result to the UI thread 3. do file indexing and send the index result to the UI thread 4. do other heavy work like mathematicl calculation and deep learning, se= nd the result to the UI thread Let the UI thread do as less work as possible. > Your idea can be implemented with two processes, though. And there is > already a package called emacs-async which does that: > > https://github.com/jwiegley/emacs-async > I knew this package. It is a good idea. Modern web browsers also use sepa= rate processes to render page content(one or more pages per process). The= y design specific IPC to let the worker processes to communicate with the= main(UI) process. They have good performance and good user responsivenes= s. I think emacs is much like web browsers. Both render large text content a= nd care user responsiveness. We can learn from them. > The disadvantage is that it is cumbersome to share data between the > two instances of Emacs, and large amounts of data will make that > inefficient. We may design an IPC for their=C2=A0communication. --5e0c0b19_5451cf49_4379 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
=E5=9C=A8 2020=E5=B9=B41=E6=9C=881=E6=97= =A5 +0800 AM12:36=EF=BC=8CEli Zaretskii <eliz@gnu.org>=EF=BC=8C= =E5=86=99=E9=81=93=EF=BC=9A
The Lisp interpreter a= lready sort-of "queues" display changes, because
it only changes Lisp data structures, and then the display engine
references those data structures at display time. So those data
structures serve as a kind-of "queue", since redisplay runs when = Emacs
is idle (i.e., the last Lisp command completed its job).

But the problem is in the other direction: it's not that Lisp somehow
"drives" the UI, it's that the UI frequently calls into Lisp as p= art
of its job. The simplest example is mode-line format: it includes
references to variables, and can also include :eval forms that call
the interpreter.


It is much clearer to me now. We can keep them all as the UI thread.

Your idea in fact mean= s to have several isolated Lisp machines in the
same process. But how can we do something like that without a very
radical redesign of Emacs, when so many things in Emacs are implicitly
part of the global state? Buffers, global variables, windows,
frames--all those are global resources, and every thread will want to
access them. Emacs was not designed to allow that.


All those global resources are not accessible by workers.= Workers can only access network, file system and other non-global resource= s. They only do the following things:
1. retrieve content from network, parse the data, and sen= d the result(lisp data) to the UI thread to present it
2. communicate with subprocesses, parse the data from sub= processes, and send the result to the UI thread
3. do file indexing and send the index result to the UI t= hread
4. do other heavy work like mathematicl calculation and d= eep learning, send the result to the UI thread

Let the UI thread do as less work as possible.


Your idea can be imple= mented with two processes, though. And there is
already a package called emacs-async which does that:

https://github.com/jwiegley/emacs-async


I knew this package. It is a good idea. Modern web browsers also use separa= te processes to render page content(one or more pages per process). They de= sign specific IPC to let the worker processes to communicate with the main(= UI) process. They have good performance and good user responsiveness. =

I think emacs is much like web browsers. Both render larg= e text content and care user responsiveness. We can learn from them.

The disadvantage is th= at it is cumbersome to share data between the
two instances of Emacs, and large amounts of data will make that
inefficient. 

We may design an IPC for their communication.

--5e0c0b19_5451cf49_4379--