From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: "Perry E. Metzger" Newsgroups: gmane.emacs.devel Subject: Re: Concurrency, again Date: Thu, 13 Oct 2016 09:27:01 -0400 Message-ID: <20161013092701.77461800@jabberwock.cb.piermont.com> References: <87wq97i78i.fsf@earlgrey.lan> <86k2dk77w6.fsf@molnjunk.nocrew.org> <9D64B8EA-DB52-413D-AE6A-264416C391F3@iotcl.com> <83int1g0s5.fsf@gnu.org> <83twckekqq.fsf@gnu.org> <83mvi9a3mh.fsf@gnu.org> <20161012165911.58437154@jabberwock.cb.piermont.com> <20161012173314.799d1dc5@jabberwock.cb.piermont.com> <8360owaj2s.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1476365253 7806 195.159.176.226 (13 Oct 2016 13:27:33 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 13 Oct 2016 13:27:33 +0000 (UTC) Cc: fiskomaten@gmail.com, monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Oct 13 15:27:29 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bug2h-0001NY-OS for ged-emacs-devel@m.gmane.org; Thu, 13 Oct 2016 15:27:27 +0200 Original-Received: from localhost ([::1]:40504 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bug2g-0008I5-AY for ged-emacs-devel@m.gmane.org; Thu, 13 Oct 2016 09:27:26 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44732) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bug2S-0008Eo-VH for emacs-devel@gnu.org; Thu, 13 Oct 2016 09:27:17 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bug2O-0003aC-HM for emacs-devel@gnu.org; Thu, 13 Oct 2016 09:27:12 -0400 Original-Received: from hacklheber.piermont.com ([166.84.7.14]:40153) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bug2I-0003TO-JS; Thu, 13 Oct 2016 09:27:02 -0400 Original-Received: from snark.cb.piermont.com (localhost [127.0.0.1]) by hacklheber.piermont.com (Postfix) with ESMTP id 18BAD7C2; Thu, 13 Oct 2016 09:27:01 -0400 (EDT) Original-Received: from jabberwock.cb.piermont.com (jabberwock.cb.piermont.com [10.160.2.107]) by snark.cb.piermont.com (Postfix) with ESMTP id A99512DE015; Thu, 13 Oct 2016 09:27:01 -0400 (EDT) In-Reply-To: <8360owaj2s.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 166.84.7.14 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208205 Archived-At: On Thu, 13 Oct 2016 08:55:55 +0300 Eli Zaretskii wrote: > > Concurrency is a giant tarpit of subtleties. :( > > And Emacs is another giant tarpit. Here's another possibility, which I'll admit is even more half baked (at best) and probably even more excessively ambitious: architect something entirely new, allow concurrency only for new code, maintain the old elisp interpreter as single thread only for backward compatibility, and subsystems that want concurrency can slowly migrate in to the new part of the system over a period of years. But, allowing ourselves to ponder the possibility for a minute, this could solve several problems: 1) There's a perceived desire to move to a new extension language. 2) There's a perceived desire for concurrency, and doing concurrency well (such that one doesn't end up with data corruption issues here and there) is hard to retrofit. 3) There's a lot of technical debt in emacs, and a chance to incrementally move to a new architecture would be a chance to correct a lot of it. So the notion (and again, don't take me too seriously, this is very half baked) would be that one would have a second extension language with a new design and its own execution engine running alongside the execution engine for the existing elisp, with relatively minimal interaction between them to avoid concurrency issues. Over a period of many years, the codebase would be converted over bit by bit. The new language should presumably be a lisp that isn't *too* far from elisp, and a particularly ambitious goal might be to build semi-automated tools for converting over straightforward code without much human intervention. However, all this depends on something I don't know offhand, which is what the right design for such a new, native-concurrent lisp would be, or even if a "correct" design is obvious, or even if all of this is possible. Again, I'm doing the wrong thing here, which is writing a note long before I've thought about the problem in depth. I do know that systems that avoid shared memory and do only message passing seem to be a lot easier to work with than pthreads style designs where the programmer has to worry that any global state is potentially in play. I also know that in the not that distant future processors with hundreds or thousands of cores are going to be common, so there will only be more demand with time for the ability to do real concurrency for performance. Solving the parallel execution issue may be a matter of importance if (in the future) people are going to do things like using Emacs as a primary web browser and development environment. But again, I won't pretend I know how to solve all of this. Perry -- Perry E. Metzger perry@piermont.com