From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: joakim@verona.se Newsgroups: gmane.emacs.devel Subject: Re: Multithreading, again and again Date: Wed, 28 Sep 2011 09:50:27 +0200 Message-ID: References: <4E82C377.4040007@yandex.ru> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1317196259 20324 80.91.229.12 (28 Sep 2011 07:50:59 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 28 Sep 2011 07:50:59 +0000 (UTC) Cc: emacs-devel@gnu.org To: Dmitry Antipov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Sep 28 09:50:51 2011 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1R8oun-0001bT-U1 for ged-emacs-devel@m.gmane.org; Wed, 28 Sep 2011 09:50:50 +0200 Original-Received: from localhost ([::1]:34229 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1R8oun-0006zv-8H for ged-emacs-devel@m.gmane.org; Wed, 28 Sep 2011 03:50:49 -0400 Original-Received: from eggs.gnu.org ([140.186.70.92]:44875) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1R8oug-0006zm-Gt for emacs-devel@gnu.org; Wed, 28 Sep 2011 03:50:46 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1R8ouZ-0007RN-Vk for emacs-devel@gnu.org; Wed, 28 Sep 2011 03:50:42 -0400 Original-Received: from iwfs.imcode.com ([82.115.149.64]:39258 helo=gate.verona.se) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1R8ouZ-0007QK-Gr for emacs-devel@gnu.org; Wed, 28 Sep 2011 03:50:35 -0400 Original-Received: from chopper.vpn.verona.se (IDENT:1005@localhost [127.0.0.1]) by gate.verona.se (8.13.4/8.11.4) with ESMTP id p8S7oRNa022865; Wed, 28 Sep 2011 09:50:27 +0200 In-Reply-To: <4E82C377.4040007@yandex.ru> (Dmitry Antipov's message of "Wed, 28 Sep 2011 10:49:27 +0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.90 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4-2.6 X-Received-From: 82.115.149.64 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:144433 Archived-At: Dmitry Antipov writes: > Hello all, > > this is an attempt to refresh the most important ideas everyone has about > threads support in Emacs. Thanks in advance to all who spent their time > thinking on answers. If someone thinks that I miss something important, > notices and references to previous discussions around this topic are highly > appreciated. > > 0. What we have, and where we go. > What's the status of git://gitorious.org/emacs-mt/emacs-mt.git and > http://bzr.savannah.gnu.org/r/emacs/concurrency? Whether there are > plans for further development? > > 1. To thread or not to thread? > Is there a consensus whether it's worth doing at all? If I don't miss > something important from previous discussions (referenced from > http://www.emacswiki.org/emacs/NoThreading, what a title), everyone > agrees that some kind of threading is useful - but maybe someone has > the votum separatum? > > 2. Hide them all! > Should the threads be visible (may be created and managed) from Lisp, > or they can just do some work orthogonal to it? If latter, which tasks may > be implicitly threaded? Redisplay (one thread per frame)? Buffer/socket > I/O? GC? Is it feasible to implement full implicit concurrency? For the > rough example, with the Lisp engine as the pool of threads, where (eval > FORM) just queues the FORM and then ask the pool to find a free thread > and evaluate it? > > 3. Too big, too global. > How to split/share/synchronize huge global state between threads? > What should be thread-local? (I feel (current-buffer) should be > global and max-lisp-eval-depth should be thread-local, but I suspect > that there are much more controversial cases). Is it reasonable to > have become-thread-local-on-write globals? > > 4. Say "thread"! > What should be done in C to implement thread management in Lisp? > I feel basically > > (let ((th (make-thread FUNCTION ARGS))) ...) > > should create thread object TH which represents running thread executing > Lisp function FUNCTION with ARGS (like funcall, but within separate thread), > and such thread object may be examined with stuff like (thread-running-p TH), > canceled with (thread-cancel TH), asked for FUNCTION's return value with > (thread-return TH), and so. But is this nothing more than adding some > builtins? Will it require some more substantial language changes? What > about old controversial topic on lexical vs. dynamic binding? > > 5. You and I. > What synchronization primitives should be implemented? Atomic variables? > Mutexes/semaphores? Barriers? Higher-level not-so-primitives like > thread pools or multithreaded queues? Should some basic operations, > like setq, be always atomic? Is it reasonable/feasible/useful to have > special form like (with-atomic BODY...), where all BODY forms are evaluated > atomically with respect to all other threads? > > 6. Under the cover. > What other low-level changes are required for thread support? > Basic structure of Lisp objects? Multiple stacks scanning for GC? > Per-thread heaps for small objects? > > 7. Look around. > Has someone an experience with development of multithreaded code for > commercial Common Lisp implementations, most probably LispWorks and/or > Allegro CL? If yes, can someone briefly explain pros and cons of their > approach to multithreading? > > Dmitry IMHO an interesting option is replacing the current Elisp implementation with GNU Guile. The Guile-Emacs project is working on this. Guile implements threads. That doesn't say anything about what the semantics for threads should be in Elisp however. Guile-Emacs is IMHO the right path for several reasons. -- Joakim Verona