Re: Multithreading, again and again

[joakim@verona.se]

Dmitry Antipov <dmantipov@yandex.ru> writes:

> Hello all,
>
> this is an attempt to refresh the most important ideas everyone has about
> threads support in Emacs. Thanks in advance to all who spent their time
> thinking on answers. If someone thinks that I miss something important,
> notices and references to previous discussions around this topic are highly
> appreciated.
>
> 0. What we have, and where we go.
>    What's the status of git://gitorious.org/emacs-mt/emacs-mt.git and
>    http://bzr.savannah.gnu.org/r/emacs/concurrency? Whether there are
>    plans for further development?
>
> 1. To thread or not to thread?
>    Is there a consensus whether it's worth doing at all? If I don't miss
>    something important from previous discussions (referenced from
>    http://www.emacswiki.org/emacs/NoThreading, what a title), everyone
>    agrees that some kind of threading is useful - but maybe someone has
>    the votum separatum?
>
> 2. Hide them all!
>    Should the threads be visible (may be created and managed) from Lisp,
>    or they can just do some work orthogonal to it? If latter, which tasks may
>    be implicitly threaded? Redisplay (one thread per frame)? Buffer/socket
>    I/O? GC? Is it feasible to implement full implicit concurrency? For the
>    rough example, with the Lisp engine as the pool of threads, where (eval
>    FORM) just queues the FORM and then ask the pool to find a free thread
>    and evaluate it?
>
> 3. Too big, too global.
>    How to split/share/synchronize huge global state between threads?
>    What should be thread-local? (I feel (current-buffer) should be
>    global and max-lisp-eval-depth should be thread-local, but I suspect
>    that there are much more controversial cases). Is it reasonable to
>    have become-thread-local-on-write globals?
>
> 4. Say "thread"!
>    What should be done in C to implement thread management in Lisp?
>    I feel basically
>
>      (let ((th (make-thread FUNCTION ARGS))) ...)
>
>    should create thread object TH which represents running thread executing
>    Lisp function FUNCTION with ARGS (like funcall, but within separate thread),
>    and such thread object may be examined with stuff like (thread-running-p TH),
>    canceled with (thread-cancel TH), asked for FUNCTION's return value with
>    (thread-return TH), and so. But is this nothing more than adding some
>    builtins? Will it require some more substantial language changes? What
>    about old controversial topic on lexical vs. dynamic binding?
>
> 5. You and I.
>    What synchronization primitives should be implemented? Atomic variables?
>    Mutexes/semaphores? Barriers? Higher-level not-so-primitives like
>    thread pools or multithreaded queues? Should some basic operations,
>    like setq, be always atomic? Is it reasonable/feasible/useful to have
>    special form like (with-atomic BODY...), where all BODY forms are evaluated
>    atomically with respect to all other threads?
>
> 6. Under the cover.
>    What other low-level changes are required for thread support?
>    Basic structure of Lisp objects? Multiple stacks scanning for GC?
>    Per-thread heaps for small objects?
>
> 7. Look around.
>    Has someone an experience with development of multithreaded code for
>    commercial Common Lisp implementations, most probably LispWorks and/or
>    Allegro CL? If yes, can someone briefly explain pros and cons of their
>    approach to multithreading?
>
> Dmitry

IMHO an interesting option is replacing the current Elisp implementation
with GNU Guile. The Guile-Emacs project is working on this. Guile
implements threads. That doesn't say anything about what the semantics
for threads should be in Elisp however. Guile-Emacs is IMHO the right
path for several reasons. 

-- 
Joakim Verona