Re: Concurrency, again

[Eli Zaretskii <eliz@gnu.org>]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Wed, 12 Oct 2016 09:42:01 -0700
> Cc: emacs-devel@gnu.org
> 
> Look at what happened with curly quotes: seemingly simple, it got merged into
> master "as an experiment". Then witness how many little issues kept popping up
> everywhere, how much debate on this list, all for something I'm sure many
> thought was totally benign. Now we're talking about doing the same with
> something as open-ended as threading?

Introduction of a significant new feature such as concurrency should
be expected to destabilize Emacs.  It is not a minor feature like
quoting.

OTOH, quotes are pervasive in Emacs; most of the issues we had as
fallout were due to that pervasiveness, something that was not
appreciated at first.  By contrast, concurrency is NOT pervasive: it
only happens if a Lisp application requests that explicitly.  As long
as we don't convert standard Emacs features to use concurrency, it
will not happen in day-to-day operations.

We've had a much tougher decision when bidirectional editing was added
to Emacs.  Unlike concurrency, bidi affects every aspect of Emacs,
because it changed how the display engine works in fundamental ways.
Stefan and Chong, who made that decision, had all the reasons to get
their feet cold, but they made that decision anyway.  I think the risk
was much higher then than it is now.  Given the level of expertise Tom
Tromey has in both Emacs and concurrency, I see no reason to be afraid
now that something terrible will happen that couldn't be fixed as part
of the normal development process.

And if worse comes to worst, and we discover at some point that
concurrency gives us too much trouble for its worth, we can always
remove it from Emacs.

Once again, the concurrency branch is not yet ready for a merge.
Before it is ready, someone should fix the few remaining issues, and
perhaps try using it in a real-life application (which will most
probably expose some more issues).  Unless someone volunteers to do
that, there's no "danger" the concurrency branch will ever be merged.

> What could be done to gain experience, as Eli suggests, is maintain a parallel
> fork, the way Firefox is doing with their e10s support. That is, build a
> version of Emacs with threading enabled and, for a couple of years, release it
> in parallel with the main Emacs. Then pick one candidate package, say Gnus,
> and have it demonstrate that threading makes everyone's life better, without
> making things worse for the rest of us.

I don't think this is practical, what with the resources we have.

> If that's too much work to do, my guess is that it's too much work to maintain
> threading with the contributors we now have. My prediction is that it will
> increase our workload, since a new class of bugs will appear that are much
> harder to reproduce and reason about than the sorts of issues we face now.

I urge you to have a look at the sources on the concurrency branch.
There's no threading, in the usual sense of that word: at any given
time, only a single thread is running.  It's not at all what you seem
to be imagining.  Think about idle timers that run without your having
to do anything to start them: that's what we have there, in a
nutshell, plus a few niceties.