On Mon, Jul 17, 2023 at 10:43:26PM +0200, Hugo Thunnissen wrote:
> 
> On 7/5/23 04:31, Eli Zaretskii wrote:
> > No, because we already handle sub-process output in a way that doesn't
> > require true concurrency.
> > 
> Not to hijack this thread and this is only tangentially related to the the
> message I'm responding to, but asynchronous IO like Emacs has for sub
> processes, network connections and the like are, in my opinion, more
> important features than true concurrency. Asynchronous IO is also what can
> make current lisp threads useful. I think that most of the time when people
> want to reach for concurrency in an Emacs lisp program it is to retrieve,
> process, store or emit data in the background [...]

I couldn't agree more. I'd add: true parallelism is a whole order of
magnitude more complex than just avoiding blocking operations. You
want the former when you need to harness more than one CPU in your
process. As you do, I don't think we need that in Emacs just yet.

But I think that there's a lot of the latter on the table for Emacs
until we reach a point of diminishing returns (and, btw, tackling the
easier part prepares the code for the more difficult one).

I've been watching the Java story from the sidelines for a long time
time (there was a huge push towards true parallelism, because Sun
back then was envisioning massively parallel processors). They tried
with "classical" interlocking. It was a world of pain. Their attempt
to develop a parallel GUI toolkit (it's tempting: each widget lives
a life of its own and iteracts with each other and the user is a
compelling idea) failed miserably and they had to showe the whole
thing into one thread after all (Po Lu is right).

There are other approaches which feel more manageable (communicating
processes over channels, à la Erlang, Concurrent ML). There is a
very nice series of blog posts by Andy Wingo, Guile's maintainer, on
that topic (ex. [1], [2]). But Emacs's architecture is just too far
away from that for it to be a viable option, methinks.

The other extreme is to design lockless algorithms from the ground
up (cf. the ref to Paul McKenney I posted elsewhere) as it is
being done in the Linux kernel.

Still I think those approaches aren't (yet?) for Emacs. Of course,
it makes a lot of sense to "draw a future picture" to have an idea
of what direction one wants to take...

Cheers

[1] https://wingolog.org/archives/2016/09/20/concurrent-ml-versus-go
[2] https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml

-- 
t