From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: Martin Stjernholm Newsgroups: gmane.emacs.devel Subject: Re: Are there plans for a multi-threaded Emacs? Date: Mon, 08 Dec 2003 23:02:14 +0100 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: <5bn0a37yu1.fsf@lister.roxen.com> References: <87oevbes4h.fsf@emacswiki.org> <20031117040607.C6C5D79B72@server2.messagingengine.com> <87ekvpx18d.fsf@emptyhost.emptydomain.de> <4nad6cikxy.fsf@holmes.bwh.harvard.edu> <4nllpt3hr3.fsf@lockgroove.bwh.harvard.edu> <5bad69zd43.fsf@lister.roxen.com> <4noeuon378.fsf@lockgroove.bwh.harvard.edu> <4ny8tsgxy6.fsf@lockgroove.bwh.harvard.edu> <4nhe0ggv0u.fsf@lockgroove.bwh.harvard.edu> <4nk75bwjaf.fsf@lockgroove.bwh.harvard.edu> <4nsmjv8d32.fsf@collins.bwh.harvard.edu> NNTP-Posting-Host: deer.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1070921136 6954 80.91.224.253 (8 Dec 2003 22:05:36 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Mon, 8 Dec 2003 22:05:36 +0000 (UTC) Cc: Ted Zlatanov , emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Mon Dec 08 23:05:30 2003 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1ATTVS-0005GX-00 for ; Mon, 08 Dec 2003 23:05:30 +0100 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1ATTVS-0001VZ-00 for ; Mon, 08 Dec 2003 23:05:30 +0100 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.24) id 1ATURF-0001s7-VY for emacs-devel@quimby.gnus.org; Mon, 08 Dec 2003 18:05:13 -0500 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1ATUR8-0001qo-90 for emacs-devel@gnu.org; Mon, 08 Dec 2003 18:05:06 -0500 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1ATUQV-0001ZN-6a for emacs-devel@gnu.org; Mon, 08 Dec 2003 18:04:58 -0500 Original-Received: from [194.52.182.190] (helo=mail.roxen.com) by monty-python.gnu.org with esmtp (Exim 4.24) id 1ATUPy-0001PE-Gf; Mon, 08 Dec 2003 18:03:54 -0500 Original-Received: by mail.roxen.com (Postfix, from userid 52) id 4AAD19AC4; Mon, 8 Dec 2003 23:02:22 +0100 (MET) Original-Received: from lister.roxen.com (lister.roxen.com [194.52.182.147]) by mail.roxen.com (Postfix) with ESMTP id 84EA59A85; Mon, 8 Dec 2003 23:02:14 +0100 (MET) Original-Received: from mast by lister.roxen.com with local (Exim 3.36 #1 (Debian)) id 1ATTSI-0007mS-00; Mon, 08 Dec 2003 23:02:14 +0100 Original-To: David Kastrup In-Reply-To: (David Kastrup's message of "08 Dec 2003 18:09:07 +0100") User-Agent: Gnus/5.090016 (Oort Gnus v0.16) Emacs/20.7 (gnu/linux) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Emacs development discussions. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:18571 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:18571 David Kastrup wrote: > /.../ If I execute some keystrokes that will mark a region and then > press the delete key before the operation that will mark the region > actually has completed, I won't be happy if it instead deletes what > happens to be at the time marked by something else. /.../ That sounds like the first bug that would be fixed after a nonblocking command loop has been implemented. It's hardly an argument against such a thing in general. There are still many cases where the responsiveness can be improved. Threads (or one-shot continuations, if you like) often makes it easier to accomplish that, especially when it comes to fixing old applications which are written with blocking operations (since changing them to be event driven is more or less turning them inside-out). As for keyboard input, I don't think one ever wants to parallelize it, not even when loading a large file or starting Gnus. It could however be nice to put an executing command "in the background", just like you can press C-z bg RET in a shell if you find that a command takes too long. That'd be a really neat feature, actually. Things that should run in the background by default are instead those that are trigged by non-user activities, such as timers running out, network traffic, and display redraws. If a thread or continuation facitility is implemented, I have some ideas for the elisp interface to make it comparatively easy to adapt existing code to a reasonable degree of parallelism (not to be confused with true multi-cpu parallelism). It's basically the idea about buffer local threads discussed earlier in this discussion thread, but outlined in more detail: o There are two kinds of locks: One for everything and one for each buffer's local data.(*) o There is a form similar to `interactive' that declares a function to be buffer local. o Events that preferably should run in the background (i.e. anything but input events from the user, afaics) check if the function being called is declared as buffer local. o Functions declared as buffer local are allowed to run until they access some buffer local data, and the lock for that buffer is then taken automatically. o The global lock is taken for other functions. That implies taking the buffer locks for all buffers. The functions will run just like today. o If only a buffer lock is taken, the function and all code it calls are allowed to change the buffer local data. Any attempt to change global data will give an error, but reading it is fine. o Whenever a buffer local function waits in blocking I/O or is preempted (cooperative multitasking isn't necessary), a function local to another buffer can run. This is the improvement. Given that almost all activities in Emacs are correlated to a single buffer, this should be a common situation. The important points with this model are that it is free from all possible deadlocks and races, and that it gives good errors which makes it relatively easy to adapt existing code. To improve the parallelism of a certain piece of code, simply declare the callback as buffer local and then run until the global write error occurs. Investigate the error and remove the global change. Repeat. The byte compiler can already track call trees. With a bit of work it could probably be extended to check for global writes from buffer local functions at compile time. The locking model can of course be refined further. Primitives should be available to let the elisp programmer explicitly disable this safety net and implement custom locking instead, at least to the extent that several threads holds the same buffer lock (allowing it for the global lock is more questionable). *) Note that these are elisp level locks; on the C level there can still be a single "everything" lock without sacrificing responsiveness, as has been discussed earlier. That would probably avoid considerable internal complexity since things aren't as well separated into buffer contexts there as they appear to be on the elisp level.