[MPS test] Is there some customization options for `scratch/igc` feature?

[MPS test] Is there some customization options for `scratch/igc` feature?

[Eval Exec]

Hello,

I'm helping to test `scratch/igc` branch (commit id: 42731228d24) .

At start, my cursor is in buffer's top line, I long press [down] key
and don't release, expect the cursor will move to bottom line. I feel
lag, and buffer window doesn't update until I release [down] key.
Then I see there is some message in `*Message*`:
```
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
(many...)
```

What's the `capacity`? how can I config the `capacity`? I think the
lag is related to the `capacity`
I tried `(customize-group 'mps) or `(customze-group 'igc)` but look
like igc feature doesn't provide customization options.

Re: [MPS test] Is there some customization options for `scratch/igc` feature?

[Pip Cet]

"Eval Exec" <execvy@gmail.com> writes:

> I'm helping to test `scratch/igc` branch (commit id: 42731228d24) .

Thanks again! Any news on the X toolkit problem, or can I push that
patch?

> At start, my cursor is in buffer's top line, I long press [down] key
> and don't release, expect the cursor will move to bottom line. I feel
> lag, and buffer window doesn't update until I release [down] key.

I can reproduce this behavior, kind of, if I build with CFLAGS="-O0 -g3
-ggdb", my usual set; in an optimized build, things seem to work out
more like I would have expected.

> Then I see there is some message in `*Message*`:
> ```
> Garbage collecting...
> Generation 0 of a chain has reached capacity: start a minor collection.
> Garbage collecting...
> Generation 0 of a chain has reached capacity: start a minor collection.
> Garbage collecting...
> Generation 0 of a chain has reached capacity: start a minor collection.
> Garbage collecting...
> Generation 0 of a chain has reached capacity: start a minor collection.
> Garbage collecting...
> (many...)
> ```

What that means is that many garbage collections happen, which may be
unexpected or not depending on how many "many" is...

> What's the `capacity`?

All allocations in MPS happen in a generation; once the total size of
objects allocated in that generation reaches the "capacity", a minor GC
is triggered, which may take a long time during some of which the
generation is larger than its capacity.  So it's a bit of a misnomer.

> how can I config the `capacity`?

Right now, you have to go to igc.c and edit the line which reads:

  mps_gen_param_s gens[] = { { 128000, 0.8 }, { 5 * 128000, 0.4 } };

That means that there are two explicitly-specified generations: the
first has capacity ~128 MB (128000 times 1024 bytes), the second ~640 MB
(5 * 128000 * 1024 bytes). There's also an implicit last generation
which is "unlimited" in size and holds objects which have survived
collection twice.

The 0.8 and 0.4 values specify a mortality prediction for each
generation: this value is updated by MPS, so it's not that relevant.

It'd be nice to be able to change these values at run time, but MPS
doesn't seem to offer an API to change them then: we'd have to create
new pools with the new generation chain, and stop allocating in the
existing old pools.

The other problem is that the values are needed very early during Emacs
initialization, so the usual customization options aren't available.

On my branch, I've implemented a command line option which is parsed
early and specifies the sizes of the generation chain, so I can
experiment with them, but I haven't really gotten around to that and had
a fairly embarrassing bug in my code until just now which would have
rendered such experiments useless anyway.

Anyway, the patch is at:
https://codeberg.org/pipcet/emacs/commit/8e28cde297fe0b1a4f8a333e9abee17d76588a7c

> I think the
> lag is related to the `capacity`

I don't know at which point we suspend redisplay to handle further key
presses, I must confess. Can you see whether the behavior is different
for (while t (forward-line 1) (redisplay t)) ?

Pip

Re: [MPS test] Is there some customization options for `scratch/igc` feature?

[Eval Exec]

> Thanks again! Any news on the X toolkit problem, or can I push that
> patch?
Yes, thanks for your work, the patch is great. Emacs hasn't crash
anymore after I applied the patch.

> I don't know at which point we suspend redisplay to handle further key
> presses, I must confess. Can you see whether the behavior is different
> for (while t (forward-line 1) (redisplay t)) ?
After profiling for CPU and memory, I've concluded that the lag isn't
related to scratch or incremental garbage collection (IGC); it's
actually connected to flycheck-mode. When I disabled flycheck-mode,
the lag disappeared. I eventually discovered that flycheck was
creating thousands of overlays at the very top line, which caused the
lag during (forward-line).

On Fri, Aug 9, 2024 at 12:26 AM Pip Cet <pipcet@protonmail.com> wrote:
>
> "Eval Exec" <execvy@gmail.com> writes:
>
> > I'm helping to test `scratch/igc` branch (commit id: 42731228d24) .
>
> Thanks again! Any news on the X toolkit problem, or can I push that
> patch?
>
> > At start, my cursor is in buffer's top line, I long press [down] key
> > and don't release, expect the cursor will move to bottom line. I feel
> > lag, and buffer window doesn't update until I release [down] key.
>
> I can reproduce this behavior, kind of, if I build with CFLAGS="-O0 -g3
> -ggdb", my usual set; in an optimized build, things seem to work out
> more like I would have expected.
>
> > Then I see there is some message in `*Message*`:
> > ```
> > Garbage collecting...
> > Generation 0 of a chain has reached capacity: start a minor collection.
> > Garbage collecting...
> > Generation 0 of a chain has reached capacity: start a minor collection.
> > Garbage collecting...
> > Generation 0 of a chain has reached capacity: start a minor collection.
> > Garbage collecting...
> > Generation 0 of a chain has reached capacity: start a minor collection.
> > Garbage collecting...
> > (many...)
> > ```
>
> What that means is that many garbage collections happen, which may be
> unexpected or not depending on how many "many" is...
>
> > What's the `capacity`?
>
> All allocations in MPS happen in a generation; once the total size of
> objects allocated in that generation reaches the "capacity", a minor GC
> is triggered, which may take a long time during some of which the
> generation is larger than its capacity.  So it's a bit of a misnomer.
>
> > how can I config the `capacity`?
>
> Right now, you have to go to igc.c and edit the line which reads:
>
>   mps_gen_param_s gens[] = { { 128000, 0.8 }, { 5 * 128000, 0.4 } };
>
> That means that there are two explicitly-specified generations: the
> first has capacity ~128 MB (128000 times 1024 bytes), the second ~640 MB
> (5 * 128000 * 1024 bytes). There's also an implicit last generation
> which is "unlimited" in size and holds objects which have survived
> collection twice.
>
> The 0.8 and 0.4 values specify a mortality prediction for each
> generation: this value is updated by MPS, so it's not that relevant.
>
> It'd be nice to be able to change these values at run time, but MPS
> doesn't seem to offer an API to change them then: we'd have to create
> new pools with the new generation chain, and stop allocating in the
> existing old pools.
>
> The other problem is that the values are needed very early during Emacs
> initialization, so the usual customization options aren't available.
>
> On my branch, I've implemented a command line option which is parsed
> early and specifies the sizes of the generation chain, so I can
> experiment with them, but I haven't really gotten around to that and had
> a fairly embarrassing bug in my code until just now which would have
> rendered such experiments useless anyway.
>
> Anyway, the patch is at:
> https://codeberg.org/pipcet/emacs/commit/8e28cde297fe0b1a4f8a333e9abee17d76588a7c
>
> > I think the
> > lag is related to the `capacity`
>
> I don't know at which point we suspend redisplay to handle further key
> presses, I must confess. Can you see whether the behavior is different
> for (while t (forward-line 1) (redisplay t)) ?
>
> Pip
>