Re: Gap buffer problem?

[Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: "Pip Cet via \"Emacs development discussions.\"" <emacs-devel@gnu.org>,
>>  Óscar Fuentes <ofv@wanadoo.es>
>> Date: Wed, 11 Dec 2024 16:33:18 +0100
>>
>> Pip Cet <pipcet@protonmail.com> writes:
>>
>> > This may be a very stupid idea, but why not use a separate process?
>>
>> Not stupid at all. I thought about something similar in a different
>> context, namely if one could decouple the GUI part of Emacs from the
>> rest.
>
> If it can be done by two processes, it can also be done by two threads
> in the same process.  Right?

AFAIU: No, not right.

I may have misunderstood, but if the idea is to preserve a consistent
state of all Lisp data and buffer text for redisplay to use, the easiest
way to ensure that consistency is fork().  The other ways, such as
copying all heap objects that might be used by redisplay (and adjusting
all internal pointers in such heap objects to point to the copy rather
than the original data), probably will end up either being a lot slower
or being very specific to the system we're running on.

I know that implementing fork() on Windows is very slow, and I don't
know about a comparable snapshotting mechanism for Windows.

To be honest, though, I'm a bit disappointed that GNU/Linux appears to
make fork() take significant time that is proportional to the size of
the mapped address space, even if it's never COW-faulted in.  I'm pretty
sure that could be avoided (and I hope the Linux kernel avoids doing it
for swapped-out memory, not that anyone still does that).

Concurrent access to Lisp data from several threads requires a locking
mechanism (fine-grained or coarse) for all such data, and possibly
requires rewriting addresses, which means no "ambiguous" references
whatsoever. That's a lot harder than using MPS, which generously allows
for ambiguous references.

It's possible we could have gotten away with concurrent access by the
redisplay machinery if we inhibited GC while the redisplay thread was
busy inspecting our data, but inhibiting MPS GC is a lot harder and
shouldn't be done for ordinary operations.

Oh, and of course mmap() breaks fork()'s snapshotting magic.

The reason I said this depends on a new GC is a bit subtle, by the way:
the old GC does best if we sacrifice a lot of memory and only run it
rarely, which we can usually get away with because RAM is cheap.  With a
fork()-based approach, memory usage comes with a performance penalty for
every fork(), so we need to reduce both memory usage and GC time, which
we can't do with non-incremental GC.

The last reason it's difficult is that MPS isn't optimized for
multi-thread settings: in an ideal world, "scanning" a memory area would
use a secondary mapping of the memory, known only to the scanning code,
so other threads could continue running while an area is being scanned.
With MPS, there is only one mapping, so we need to stop all other
threads while one thread un-mprotect()s a memory area to scan it.

Unless MPS breaks POSIX threads in some spectacular way, fork() should
still work, though.

Pip