Re: MPS: Loaded pdump

[Helmut Eller <eller.helmut@gmail.com>]

On Thu, May 09 2024, Gerd Möllmann wrote:

> I feel more and more that the handling of the loaded pdump with MPS as
> it is now is not sustainable, and would like ask you for your ideas.
>
> What we do now is make the hot part of the dump an ambig root. I don't
> remember the exact numbers, but I think that's about 18 Mb of root. This
> has at least these problems, from my POV:
>
> - It is very large, and every time MPS scans roots, and that is all the
>   time, the world is stopped until it has finished. That's not good for
>   pause times.

Maybe it would be better with the MPS_RM_PROT option.  Do you know which
of the telemetry events could be used to measure this?

I recorded telemetry for the nbody benchmark.  It seems that there are
16 GC flips.  From one TraceFlipBegin event to the next TraceFlipEnd
takes about 32 million cycles (I think the timestamps are cycles, as
returned by rtdsc).  If we assume the clock frequency is 2.5 GHz that
would be about 13 milliseconds per GC flip.  But I don't know what the
events actually mean and whether this includes scanning the pdump.

> This has of course also consequences:
>
> - copying 18 Mb of hot objects + 12 Mb or so of leaf objects to MPS
>   could be slow. No idea if it is. That could impact startup time (not
>   important to me at all, but people have different preferences).

It would certainly be interesting to know how long it takes.

> - copying the graph requires that the copying functions know the layout
>   of Lisp objects so that the functions can exchange references in the
>   old graph to the corresponding ones in the new graph. I'm getting
>   exhausted already from thinking of writing such functions, and we
>   don't have C++ templates to help.

Do we need to know the layout or can we get away with just knowing the
size and the relocation information that the pdump already has?

> - AFAIK, but see admin/igc.org, there is no good way of allocating
>   objects in an old generation, so they will maybe take some time to
>   wander to an older generation.

There is this ramp allocation pattern but it's not exactly what we need.

> Enough rambling.
>
> Ideas, opinions, ...?

Does Open Dylan use MPS in some way to dump/load a large amount of
state?