Re: Some experience with the igc branch

[Eli Zaretskii <eliz@gnu.org>]

> Date: Fri, 27 Dec 2024 14:34:22 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: gerd.moellmann@gmail.com, ofv@wanadoo.es, emacs-devel@gnu.org, eller.helmut@gmail.com, acorallo@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> > OK, but still, since you wrote the code to implement it, I guess you
> > have at least some initial design ideas?  I hoped you could describe
> > those ideas, so we could better understand what you have in mind, and
> > provide a more useful feedback about possible problems, if any, with
> > those ideas.
> 
> The idea is that the main thread, after initialization, never calls into
> MPS itself.

Thanks.  I will ask some questions below to understand better what you
suggest.

> Instead, we create an allocation thread, reacting to messages from the
> main thread.
> 
> The allocation thread never actually does anything in parallel with the
> main thread: its purpose is to provide a separate stack, not
> parallelization.

Why is it important to have a separate stack when MPS allocates
memory?

> All redirected MPS calls wait synchronously for the allocation thread to
> respond.
> 
> This includes the MPS SIGSEGV handler, which calls into MPS, so it must
> be directed to another thread.

MPS SIGSEGV handler is invoked when the Lisp machine touches objects
which were relocated by MPS, right?  What exactly does the allocation
thread do when that happens?

> All this makes the previously fast allocation path very slow, and we
> need a workaround for that:
> 
> We ensure that we allocate at least 1MB (magic number here) at a time,
> then split the area into MPS objects when we need to.  The assumption
> that we can split MPS allocations is significant but justifiable,
> because MPS will be in the same state after two successful back-to-back
> allocations and a single allocation combining the two.

This seems to rely on some knowledge of MPS internals?

But more worrisome: what about "sudden" needs for more that 1MB of
memory?  For example, C-w in a large buffer needs to allocate a Lisp
string for the killed text.

> 1. there is no other thread which might trigger a memory barrier (the
> allocation thread doesn't)

So the allocation thread doesn't GC?  If so, who does?

If the allocation thread does GC, then how can you ensure it doesn't
trigger a barrier?

> 3. we don't allocate memory

Why can't GC happen when we don't allocate memory?

> 4. we don't trigger memory barriers

Same question here.