bug#46900: 28.0.50; GC uses excessive amounts of stack

bug#46900: 28.0.50; GC uses excessive amounts of stack

[Pip Cet]

This is a long-standing issue.

Currently, GC is written to recurse when marking objects. That means
for every level of recursion that we see, a new function stack frame
is being created on the C stack; on this x86_64-pc-linux-gnu system
running Debian GNU/Linux, in an optimized build, that means 80 bytes
of stack space are reserved for each level of recursion.

That number could be much closer to 8, which is the number of bytes of
relevant data in those 80 bytes of stack space.

My proposal is to continue allocating the mark queue on the stack (not
the heap), but to do so in larger chunks, allocating enough space for
N objects and recursing, in the C sense, only when all N object slots
have been used.

In addition to saving stack space, this should also help, at least,
older systems with limited return address prediction stacks.

IMHO, it would also help, not hinder, debugging of GC internals, as
more context would be available in the function frame. Indeed, the
patch I will send once this has a bug number keeps all of the large
"soft" stack frames in a linked list so we can easily walk them.

In further patches, I'd like to change things so we no longer recurse
through symbols so enthusiastically, building up tens of thousands of
objects on the mark queue.

In the interest of full disclosure, I have several ulterior motives
for this: the WebAssembly implementation I'm using doesn't allow deep
nesting of calls; I'd like to look into parallelizing GC for which
this might be useful; and I suspect native-comp will benefit more from
further work in this area, too.

bug#46900: 28.0.50; GC uses excessive amounts of stack

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Wed, 3 Mar 2021 15:44:33 +0000
> 
> Currently, GC is written to recurse when marking objects. That means
> for every level of recursion that we see, a new function stack frame
> is being created on the C stack; on this x86_64-pc-linux-gnu system
> running Debian GNU/Linux, in an optimized build, that means 80 bytes
> of stack space are reserved for each level of recursion.
> 
> That number could be much closer to 8, which is the number of bytes of
> relevant data in those 80 bytes of stack space.
> 
> My proposal is to continue allocating the mark queue on the stack (not
> the heap), but to do so in larger chunks, allocating enough space for
> N objects and recursing, in the C sense, only when all N object slots
> have been used.
> 
> In addition to saving stack space, this should also help, at least,
> older systems with limited return address prediction stacks.

FWIW, I'm only interested in changes to GC if they bring significant
gains, such as replacing the mark & sweep algorithm with something
more modern and powerful.  Otherwise, experience shows that messing
with GC for smaller gains is not worth it: it destabilizes Emacs for
no good reason, since users rarely if ever see any significant
improvements.

> In further patches, I'd like to change things so we no longer recurse
> through symbols so enthusiastically, building up tens of thousands of
> objects on the mark queue.

I'd like to hear and understand the long-term plan before I agree to
any such changes.  And in general, I'd like to see the plan
implemented in large steps, not small changes.  We have bad experience
with the latter: over the years there were a couple of attempts to do
it that led nowhere, and we are left with their leftovers that don't
make much sense on their own, and just make the sources harder to read
and understand, let alone change.

So please: no small changes in GC, thank you.

> In the interest of full disclosure, I have several ulterior motives
> for this: the WebAssembly implementation I'm using doesn't allow deep
> nesting of calls; I'd like to look into parallelizing GC for which
> this might be useful; and I suspect native-comp will benefit more from
> further work in this area, too.

If the goal is to make GC parallel, let's first discuss the idea for
how to do it, and (assuming we agree to go that way) let's start with
it, not present it as some long-term goal.

Thanks.

bug#46900: 28.0.50; GC uses excessive amounts of stack

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Could you please tell me what you're doing in Emacs with webassembly?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#46900: 28.0.50; GC uses excessive amounts of stack

[Pip Cet]

On Thu, Mar 4, 2021 at 5:32 AM Richard Stallman <rms@gnu.org> wrote:
> Could you please tell me what you're doing in Emacs with webassembly?

Running it locally, in batch mode, with the SpiderMonkey shell (not a
web browser). There's no "Web" involved, at present. I'm vaguely
considering running Emacs on the non-free hand-me-down Android phone I
have, which I appear to be unable to install real software on, but my
interest in environments which aren't clearly free is somewhat
limited.

It did help shake out a couple of bugs, though. Some of them, unlike
this one, may even be fixed.

Pip