Re: Emacs design and architecture. How about copy-on-write?

[Eli Zaretskii <eliz@gnu.org>]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: acm@muc.de, incal@dataswamp.org, emacs-devel@gnu.org
> Date: Tue, 19 Sep 2023 13:35:49 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > What is the global redisplay lock?
> >> 
> >> I imagine that the implementation can involve global redisplay lock that
> >> `redisplay' (or its internal functions) need to acquire before running.
> >
> > I thought you were describing the current implementation (where
> > there's no lock).  If you are describing some hypothetical future
> > implementation, I don't see how we could usefully discuss this without
> > much more details of that hypothetical design.
> 
> This is very simple - I suggest to not touch xdisp.c as much as
> possible, just make sure that it is interlocked. And focus on async
> support in Elisp code. Once there is async support in Elisp code, we can
> move further and look into xdisp.
> 
> (I suggest this because I feel that xdisp is a rabbit hole we may sink
> in instead of doing something more productive)

I'm actually of the opposite opinion.  I think trying to parallelize
redisplay is a lower-hanging fruit, and if successful, it could bring
non-trivial gains to Emacs even if the rest remains single-threaded.

> >> Then, we had a misunderstanding. My point is exactly that xdisp.c relies
> >> on global state _a lot_. And it will be non-trivial to change it.
> >
> > It doesn't _rely_ on the state, not on the level of the code.  It
> > _accesses_ the state as every other Lisp program does, since that's
> > how Emacs accesses it everywhere.  But there are no hidden assumptions
> > that the variables used by xdisp.c are global.  If each one of them
> > will become buffer-local or thread-local, nothing significant will
> > change in the xdisp.c code.  All xdisp.c needs is to be able to access
> > the value when needed.  And since each window is processed separately,
> > there's no assumption that something observed when displaying window
> > W1 will necessarily hold when displaying window W2.
> 
> I am particularly worried about scenarios when window geometry changes
> by asynchronous threads. Or, say, face definitions. Imagine that it
> happens at point when we are already drawing the now obsolete geometry
> onto glass?

xdisp.c has solutions for these two (and other similar) situations.
The problems have nothing to do with parallelism, they happen today
because we call Lisp at certain places in the display engine.

> >> In my mind, it is easier to first solve the problem with generic Elisp
> >> async threads and only then try to look into async redisplay.
> >
> > We tried that already, with the existing Lisp threads.  One reason why
> > those are almost never used is that the display issue was left
> > unresolved.
> 
> That might be one reason, but not the only reason. And certainly not the
> most important reason for the use cases where I did try to use threads
> myself.

There's more than one reason, that's true.  But it doesn't change the
fact that all those problems have to be resolved before we have
reasonably usable threads.

> > ... Any real-life Lisp programs that tries to use threads
> > bumps into this issue sooner or later.
> 
> This is not true. Some Lisp programs? Yes. "Any"? No.

Only toy Lisp programs can get away.

> As we discussed previously, a number of Elisp programs can benefit from
> async threads even when redisplay is not asynchronous - network queries
> (gnus), text parsing (org-mode, slow LSP server communication, xml
> processing).

You forget that almost every Lisp program in Emacs either displays
something or affects stuff that affects redisplay.  It's easy to
forget, I know, because in Emacs redisplay "just happens" by some
magic.

> To be clear, it would indeed be nice to have async redisplay.

I'm not talking about async redisplay, I'm talking about the ability
of non-main threads to display something or do something that would
cause redisplay change the stuff on the glass.