Re: Elisp containers

[Tom Tromey <tom@tromey.com>]

>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

Stefan> If someone feels like they have too much time on their hands, I think
Stefan> a great feature to develop would be Elisp containers.

Stefan> This would be like running Elisp in a separate process, except that it's
Stefan> not a separate process, so communication between two containers can be
Stefan> very efficient (e.g. you can send a buffer from one container to the
Stefan> other as efficiently as you can send an integer).

I considered this when working on threads.  I thought I'd write to
discuss the problems I had with it; not to discourage anybody, but in
case somebody has thoughts on how to improve things.

I think the basic idea is to avoid data races while also avoiding the
overhead of a separate process.  So the design should keep things in
process (duh) and also reject shared mutable state; instead objects must
be explicitly sent between threads.  (If you lift this latter
requirement then you really just have what we have now.)


So, first, what environment should a new thread inherit?

If it is a clean environment -- say a copy of the dumped state -- then
that means that user changes won't affect threads.  And, threading code
will need to know exactly what things must be copied into the new
container.

This seemed unfortunate to me, not to mention hard on thread users,
because it's difficult in an open environment to know what may be
required.  (You can see echoes of this problem in the mhtml mode, where
it has to use a regexp on symbol names, eww, to capture locals.)

However, if it is not a clean environment, then thread creation will be
very expensive.  You will have to copy much of the heap into the new
thread.  (Perhaps the copying could be done lazily at the expense of
slowing down access to global variables.)


Second, sending a large object to another thread will also be slow.  The
buffer *itself* is not too bad, but buffers have properties and
buffer-local bindings, and these must all be deep copied.

One idea I had to reduce the cost here was to have a special case: when
a thread dies, let thread-join simply transfer the data from that thread
to its own heap.


Finally, this approach greatly reduces debuggability and the ability to
mess around.  Emacs would no longer be an open system -- threads would
have private data and there would be no way to access that.

Tom