Re: MPS: User GC customizations

["Gerd Möllmann" <gerd.moellmann@gmail.com>]

Pip Cet <pipcet@protonmail.com> writes:

> I think we can just set flags for "called MPS" and "in a scan
> function" and look at them in the SIGPROF handler to distinguish the
> four cases?

Not sure. What if MPS calls these in its own thread? I guess that
wouldn't be so interesting for Ihor.

> My suspicion is that most problems are going to be due to large
> objects creating large segments which we have to scan completely, but
> that's a (micro-?) optimization for another day.
>
> Would it be okay to keep track of the largest object/pvec of each type
> in igc.el/Figc_info? I've got a patch here which does that, and I
> think the number is at least as interesting as the average :-)

Good idea! That first weill showing the number of objects and average
size in igc-stats.

>
>> > > What this variable does is give MPS notice that the client is currently
>> > > idle and it might be a good time to do some work.
>> > 
>> > Without understanding what effect setting this variable has on the Emacs
>> > responsiveness, it does not seem very useful. (Exactly because MPS is
>> > concurrent)
>> 
>> That was kind of my point. We can use idle time for using this variable
>> and see what effect it has, especially in interactive use. Sorry if that
>> wasn't clear. I'm personally trying 0.1 (100ms) here at the moment.
>
> I think it might very well have an effect, but being able to tune the
> MPS is important, too, both for debugging and to improve performance.
> And the documentation seems to be quite minimal.
>
>> > > > For example, my recent measurement of building agendas displayed 30% of
>> > > > the time spend in GC. (whatever this means in the context of our handling
>> > > > of SIGPRF)
>> > > 
>> > > Exactly, whqt does it mean? And if we don't know, why is it an example
>> > > for anything?
>> > 
>> > AFAIU, on master, SIGPROF handled while our vanilla GC is running
>> > will record it. In contrast, on scratch/igc, SIGPROF will put all the
>> > time when igc_busy_p () is non-nil into "GC".
>> 
>> Right. And I wonder if that simply is because MPS is doing stuff in its
>> own thread.
>
> That (or another thread calling MPS to make an allocation) would definitely show up as a false positive.
>
>> > And igc_busy_p is not only non-nil when MPS is pausing Emacs to do its
>> > job, but also during object allocation. So, on master, profiler "GC"
>> > field records real GC pauses, while on scratch/igc "GC" field is GC
>> > pauses + new object allocation.
>> 
>> The docs say
>> 
>> -- C Function: *note mps_bool_t: 129. mps_arena_busy (mps_arena_t
>> arena)
>> 
>> Return true if an *note arena: 16. is part of the way through
>> execution of an operation, false otherwise.
>> 
>> ‘arena’ is the arena.
>> 
>> Note: This function is intended to assist with debugging fatal
>> errors in the *note client program: d0. It is not expected to
>> be needed in normal use. If you find yourself wanting to use
>> this function other than in the use case described below,
>> there may be a better way to meet your requirements: please
>> *note contact us: d8.
>> 
>> What "partly through an operation" means is anyone's guess at this
>> point. Someone would have to consult the sources. The docs don't say
>> what you are suggesting, from my POV.
>
> IIRC it just checks whether the arena lock is held, whenever that
> might be.

Good to know, thanks! That's what I was suspecting.

>
>> > My figure of 30% says that igc_busy_p () is for 30% of CPU time, which
>> > is a significant number. But it is not very useful unless we get some
>> > idea about which part of it is memory allocation and which part of it is
>> > MPS pausing all Emacs threads.
>> > 
>> > Ideally, we should also have some way to get the allocation times on
>> > master. Then, we can compare them directly.
>> 
>> Maybe it would be interesting what the meansurements look like on macOS,
>> where the check for igc_busy are not needed in the SIGPROF handler.
>
> How sure are we of that, by the way? My understanding is there are two
> ways signals can interfere with one another: SIGSEGV -> SIGPROF ->
> SIGSEGV (which wouldn't happen on macOS) and alloc -> SIGPROF ->
> SIGSEGV, which might.

I'm not an expert and so on, so just talking about my understanding:

Darwin is a kind of Mach descendant, and as such it uses Mach exceptions
for hardware faults. Among those is EXC_BAD_ACCESS which one gets for
protection faults. Exceptions are handled by installing a "port"
which runs in its own thread and which the OS invokes when the exception
occurs.

Exception handling is "synchronous", which I think means the OS suspends
the thread which caused the exception, invokes the port for doing what
it wants, and when done let's the original thread continue, if the port
wants that.

Signals on Darwin are something different and used for non-hardware
cases.