Re: Are there plans for a multi-threaded Emacs?

[Martin Stjernholm <mast@lysator.liu.se>]

Ted Zlatanov <tzz@lifelogs.com> wrote:

>> o  To be able to run several threads simultaneously in multicpu or
>>    hyperthreaded systems.
>> o  To get OS level preemption between threads.
>
> Those two benefits make a big difference.

The last does not since it's quite feasible to simulate, as I've
outlined. (I know that from practical experience with this threading
model in a language environment fairly similar to elisp.)

> Today's Pentiums have hyper-threading technology, so even a single
> processor can act as two.

True. Hyperthreading is becoming more commonplace.

> The best cooperative multitasking can't make use of multiple
> processors as well as premptive multithreading.

I don't see how preemption per se can improve parallell execution.
Fine grained locking can, but it's hard to avoid making it too fine
grained so that the locking itself becomes more time consuming than
the speedup from the improved parallellism it achieves.

> Cooperative multitasking definitely has benefits - time-critical
> applications for instance, but I don't think that applies to Emacs.

One thing that does apply to Emacs, and very heavily too, is that
cooperative task switching makes it a lot easier to cope with the
threading issues at the application level - it becomes an opt-in
feature instead of an opt-out, i.e. code that doesn't need or want to
bother with thread safety remains safe. This is important even if the
considerable transitional work to thread safe existing code is
disregarded. Thread locking is still a complex issue that easily leads
to bugs (and often very tricky ones too), so it's a big advantage to
not have to bother about it in the vast majority of code.

> /.../ Converting a single-threaded application to multithreaded,
> especially one as complex as Emacs, is a daunting task.  I think it's
> worthwhile.

I gather then that one important goal for you with the thread support
is to make Emacs utilize true parallell execution better, i.e. it's
for performance reasons and not only to let elisp developers take
advantage of the threading paradigm.

I don't regard that performance gain as significant, at least not
compared to the amount of work involved. Emacs is still an interactive
application, and as such it essentially only responds to user actions.
Granted, there are some elisp packages that occasionally can be very
heavily cpu bound, but they are afterall only a few exceptions. The
only case when true parallell execution would improve things are when
two or more such tasks are run concurrently. I regard that as an
extremely rare situation, but you would probably disagree?

Anyway, a way to cope with such heavy tasks could be to make it easy
to fork a separate interpreter with its own data set. It would then
only take the main interpreter lock explicitly and rarely to
communicate its results and request new data, or perhaps even be
restricted to communication through a pipe.