Re: Are there plans for a multi-threaded Emacs?

[Martin Stjernholm <mast@lysator.liu.se>]

David Kastrup <dak@gnu.org> wrote:

> The whole proposal was geared towards parallel execution of
> computation-intensive tasks (like gnus summarizing and scoring) with
> the editing foreground task.

This discussion thread has been both about how threading should be
solved and what should be achieved with the solution. True parallel
execution is not the only discussed goal. In my opinion that goal is
fairly unimportant - the big win is instead to be able to use the
thread paradigm which in some cases is more convenient than the
alternative, i.e. callback driven programming.

As for the problem with Gnus, a threading model with a global lock is
a worthwhile improvement: It would allow that cpu intensive process to
be interrupted regularly to let Emacs respond to user actions. The big
win here is imho that Emacs remains responsive; it doesn't bother me
at all that the few milliseconds of work to achieve that is added to
the total time of the score buffer generation instead of being carried
out on another cpu.

The issue of preemption at the elisp level only affects what kind of
changes need to be done in Gnus to achieve these regular interrupts:
If there is no preemption, regular yield calls have to be added at
points where the state is consistent enough. Otherwise proper locking
has to be implemented.

> Cooperative yielding can't make use of hyperthreading.

Yes it can. Seems to me you are confusing cooperative yielding and
preemption with the use of a single global lock. A single global lock
prohibits both parallel execution and underlying preemption.
Preemption/cooperative yielding doesn't say anything about the kind of
locking being used.

> Tasks that require cooperative yielding must not share any resources
> with any other task in between synchronization points (yield points,
> whatever).

Depends on what you mean with sharing a resource. To be precise, they
can't leave any shared resource in an inconsistent state across a
yield point, and they can't assume anything about the state of any
shared resource afterwards.

Of course, even in a cooperative environment mutexes are useful to
lock out other threads from a resource that potentially is shared but
that the current thread want to retain exclusive access to.

> And even with cooperative multitasking you need separate stacks for
> several resources, for every task, /.../

Of course. That's one of two things a thread consists of (the other is
a promise of future cpu time).

>> Another reason to have only a single interpreter lock and no
>> preemption. It's hardly so expensive that it would take significant
>> time compared to blocking operations, is it?
>
> I don't see any usefulness at all.  Certainly every nontrivial task
> will need to use the interpreter?  So where is the gain?

Blocking operations, like waiting on a mutex or reading from a file or
socket, can leave the interpreter free to do other things. All
blocking operations can be fixed to not need direct access to the
interpreter data. In many cases it's such blocking operations that
make Emacs unresponsive. The alternative to threads is to rewrite all
such code to use callbacks (and in some cases the OS doesn't even
provide the necessary interface for callbacks, e.g. gethostbyname(3)).