Re: MPS: dangling markers

["Gerd Möllmann" <gerd.moellmann@gmail.com>]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> - perf points to (1) itree lookup; (2) bytepos<->charpos, which implies
>>   that things may be related to the number of allocated markers and
>>   overlays.
>> - I observe things slowing down as the number of "PVEC_MARKER" in the
>>   output of `igc-info' grows.
>> - Forcing (igc--collect) returns speed to normal
> [...]
>> AFAIK, at least overlays, buffers, and markers are used in C code within
>> object lists/trees are often traversed in full. If objects in these
>> lists are not regularly garbage-collected, we may end up in situation
>> when dead objects significantly impact performance.
>
> AFAIK, markers are the only one that can be reachable (i.e. we may
> spend time looking at them) yet GC-able, because the buffers' `markers`
> slot contains a linked-list of all markers that's treated specially by
> the GC (basically, the list is "weak" so markers get removed from this
> list during GC if the marker is reachable only from the list).

Exactly.

> [ Of course, similar things can occur with other objects via our weak
>   hash table.  And there are a few other "background cleanups" we do in
>   GC (such as resizing gaps or truncating undo logs) which could be
>   impacted as well.  ]

(FWIW, in igc, I'm trying to do these things incrementally when idle,
Weaj hash tables are currently not implemented. I can't bring me to do
that.)

> So I'm not surprised by your point (2) above, but I don't have an
> explanation for your point (1), OTOH.

Would be interesting to so some profiler output showing in which context
these itree operations happen.

> BTW, a lof of C and ELisp code is careful to delete markers after their
> use, so I suspect that a large proportion of the "dead" markers that can
> accumulate in the list of markers are those created by the chars<->bytes
> conversion code (as a memoization cache), and we should be able to
> decouple their collection from the GC by simply adding a bit that says
> this is just a cached result of a previous chars<->bytes conversion,
> rather than a true marker object, and then we could have a "background
> task" that flushes them as needed.

That might be a way, yes. In igc I could try to do that in on_idle,
incrementally, or even in another thread (not the Emacs ones).