From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Vladimir Sedach Newsgroups: gmane.emacs.devel Subject: Re: Elisp containers Date: Sat, 08 Sep 2018 12:25:00 -0700 Message-ID: <87d0tnvlkj.fsf@exinda.orion.oneofus.la> References: <871sa5xdz2.fsf@tromey.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1536434649 25935 195.159.176.226 (8 Sep 2018 19:24:09 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 8 Sep 2018 19:24:09 +0000 (UTC) User-Agent: mu4e 1.1.0; emacs 25.3.1 Cc: Stefan Monnier , emacs-devel@gnu.org To: Tom Tromey Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Sep 08 21:24:04 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fyipv-0006dt-M3 for ged-emacs-devel@m.gmane.org; Sat, 08 Sep 2018 21:24:04 +0200 Original-Received: from localhost ([::1]:45048 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fyis2-00025R-73 for ged-emacs-devel@m.gmane.org; Sat, 08 Sep 2018 15:26:14 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:56727) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fyirO-0001km-Pm for emacs-devel@gnu.org; Sat, 08 Sep 2018 15:25:35 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fyirM-0007EM-31 for emacs-devel@gnu.org; Sat, 08 Sep 2018 15:25:34 -0400 Original-Received: from forward104j.mail.yandex.net ([5.45.198.247]:56106) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fyirL-0006zI-Ho for emacs-devel@gnu.org; Sat, 08 Sep 2018 15:25:31 -0400 Original-Received: from mxback10j.mail.yandex.net (mxback10j.mail.yandex.net [IPv6:2a02:6b8:0:1619::113]) by forward104j.mail.yandex.net (Yandex) with ESMTP id F168046338; Sat, 8 Sep 2018 22:25:27 +0300 (MSK) Original-Received: from smtp4o.mail.yandex.net (smtp4o.mail.yandex.net [2a02:6b8:0:1a2d::28]) by mxback10j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id KC3RfMiDyZ-PRkuIdaD; Sat, 08 Sep 2018 22:25:27 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oneofus.la; s=mail; t=1536434727; bh=+KRd25wbaWFNqc6sYG6tLz+L/KxGUN/SyBlH4dkoYnI=; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID; b=LccLzBcy/KRqSpgzlFempvZtzh3kybjgp9sEfWBg3GKpIioe/VnIE/IApgiH17wnn VUgG85e3Wky6G6V1ln9sCB6B/2gUc2NyhsNl5fFQaUHXAu7mF48eJp/cZwWvWexiUq APn+Dwy893M0BUgXB2cMVGCYLqSpBVEErHv7z9GM= Original-Received: by smtp4o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id aPG4gWV2fd-PPf8JZar; Sat, 08 Sep 2018 22:25:25 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oneofus.la; s=mail; t=1536434726; bh=+KRd25wbaWFNqc6sYG6tLz+L/KxGUN/SyBlH4dkoYnI=; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID; b=NgUmO7rA4FCeKMCrheHxRJ+ShZUWx+9xSKGcvDlXo1n4ZehMezLWKqke04Vsfgw5E 0RMIG2LJFUrx61FQBRLIgGU9Ihe7SFeyUlD9rcDullqbbPR9OajLJcCitsiMpsSzXj ulJATx3aKmvlKf/ldybR1wKy08x0DLlvXe/DgXwc= Authentication-Results: smtp4o.mail.yandex.net; dkim=pass header.i=@oneofus.la In-reply-to: <871sa5xdz2.fsf@tromey.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 5.45.198.247 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:229513 Archived-At: > I think the basic idea is to avoid data races while also avoiding the > overhead of a separate process. So the design should keep things in > process (duh) and also reject shared mutable state; instead objects must > be explicitly sent between threads. (If you lift this latter > requirement then you really just have what we have now.) This problem boils down to "what is the right way to virtualize a single address space Lisp system?" > So, first, what environment should a new thread inherit? > > If it is a clean environment -- say a copy of the dumped state -- then > that means that user changes won't affect threads. And, threading code > will need to know exactly what things must be copied into the new > container. > > This seemed unfortunate to me, not to mention hard on thread users, > because it's difficult in an open environment to know what may be > required. (You can see echoes of this problem in the mhtml mode, where > it has to use a regexp on symbol names, eww, to capture locals.) > However, if it is not a clean environment, then thread creation will be > very expensive. You will have to copy much of the heap into the new > thread. (Perhaps the copying could be done lazily at the expense of > slowing down access to global variables.) One way to solve the problems of "which environment to start from?" and expensive copying is to implement environments as a fully persistent (AKA purely functional) table. This will let you do roughly the equivalent of fork(2) in a single address space quickly (with the real cost amortized over the whole system). You can also use this for parent/child fork parallelism where the children inherit the parent's dynamic bindings. Having single address space share-nothing threads IMO defeats the main benefits of a single address space, which are: eliminating data duplication, and zero copy message passing. It is also a lot more work to implement than using multiple Emacs OS processes. Virtualizing the environment with a shared heap is an IMO interesting idea that has not been well explored yet, but it requires a very different approach to Lisp system functions (they will all have to be based around immutable data structures), and more to the point is not relevant to "Elisp containers." > One idea I had to reduce the cost here was to have a special case: when > a thread dies, let thread-join simply transfer the data from that thread > to its own heap. Cilk has a very well thought out model of lightweight thread forking/joining using cactus stacks (an idea derived from Interlisp's spaghetti stack), where child threads return their data to the parent. > Finally, this approach greatly reduces debuggability and the ability to > mess around. Emacs would no longer be an open system -- threads would > have private data and there would be no way to access that. The debugger could point to the environment of the thread from which it was invoked. The same could be done for REPLs. So the whole tool set would have to be changed to accommodate this virtualization. Multiple Emacs processes have the same problem, but there are already working solutions that can be adapted for async.el. Something like the SWANK protocol can be used to provide debugging and a REPL for multiple Emacs processes, and not just on the same machine but remotely as well. The nice thing is that the Emacs processes already show up in the process list, and would get their own names in SWANK, which would make managing them as easy as managing containers. Vladimir