Re: Gap buffer problem?

[Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> Date: Wed, 11 Dec 2024 17:41:29 +0000
>> From: Pip Cet <pipcet@protonmail.com>
>> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, emacs-devel@gnu.org, ofv@wanadoo.es
>>
>> "Eli Zaretskii" <eliz@gnu.org> writes:
>>
>> > If it can be done by two processes, it can also be done by two threads
>> > in the same process.  Right?
>>
>> AFAIU: No, not right.
>>
>> I may have misunderstood, but if the idea is to preserve a consistent
>> state of all Lisp data and buffer text for redisplay to use, the easiest
>> way to ensure that consistency is fork().  The other ways, such as
>> copying all heap objects that might be used by redisplay (and adjusting
>> all internal pointers in such heap objects to point to the copy rather
>> than the original data), probably will end up either being a lot slower
>> or being very specific to the system we're running on.
>
> How do you do the same in a forked process?  The glyph matrices are
> not allocated once, they are reallocated constantly.  Are you going to
> fork each time?

Not necessarily "each time" (meaning once per frame/keystroke), but
quite frequently, yes.

> And if you are, how is it different from copying
> stuff lazily within the same process, exactly like the OS does with
> forked processes?

It is very different indeed:

Copying within a process involves changing the (virtual) addresses that
the copied data is at (unless you use an architecture-specific
implementation of TLS). The beauty of fork() is that the virtual
addresses stay the same, so we don't need to adjust any pointers, which
we cannot do because there are ambiguous references to Lisp data.

IOW, no, you can't lazily create two copies of Lisp data in the same
process. You have to do so eagerly, adjusting any and all pointers (and
only those) in the Lisp data before the new data is read for the first
time (because what you read might be a pointer, and then it needs to be
adjusted). With fork(), you only have to make the copy when the data is
being written to, by either process.

(Of course you can just access all memory through some sort of API that
translates addresses for you, but that would effectively mean we'd be
running Emacs on a virtual machine and simulate fork() on it).

> Anyway, the fact that redisplay calls Lisp and Lisp calls back into
> redisplay all but kills this idea.  Gerd's document has also other
> gotchas.  We didn't just give up easily back when we discussed that.

I don't see why the redisplay process would not be able to call Lisp;
it's a full Emacs process (with a single thread), except it doesn't have
an FD or socket for the window system, and has an extra pipe to
communicate with the parent process instead.

It's true that the side effects of the called Lisp code won't be visible
to the next redisplay process, but such side effects are perilous
anyway, and avoiding them would seem to me to be a feature, not a bug.

However, if such side effects are desired, we can use IPC to execute
Lisp in the main process (some effort) or simply send a "this redisplay
needs to happen synchronously" message to the main process, which would
kill the current redisplay process and perform a synchronous redisplay
(as not all operating systems support fork() reliably, we'll have to
retain the ability to redisplay synchronously, either way).

But, to be perfectly honest, I'm not sure redisplay is slowing me down
the way traditional GC is.

Pip