Re: MPS: assertion failed: header_type (h) != IGC_OBJ_FWD

["Gerd Möllmann" <gerd.moellmann@gmail.com>]

Pip Cet <pipcet@protonmail.com> writes:

> On Sunday, July 14th, 2024 at 04:12, Gerd Möllmann <gerd.moellmann@gmail.com> wrote:
>> I'm seeing this assertion sometimes in an Emacs built with
>> --enable-checking=igc_debug,igc_check_fwd,w here sometines means it can
>> take days of using/running Emacs, or it can take a couple of hours.
>> This is macOS 14, arm64. I'm linking with -lmps-debug.
>
> That means lldb and no core dumps, right? (I've had to work with lldb
> to make the Android port work with mps, and it's decidedly Not My
> GDB).

Yes, no GDB for macOS/arm64. I mean LLDB is okay for the usual
debugging, but in this case hm. But I don't know if GDB would do better,
either.

(One can produce core dumps, BTW, but it's complicated, requires signing
and entitlements, and is not really helpful debugging-wise, and the
dumps are several GB large.)

What I did is enter a loop with sleep(3) instead of asserting, so that
one can attach to the process. That works, but with optimizations I
can't see much.

>> The assertion means that we likely have a reference somewhere that
>> isn't traced. Because it isn't traced, the reference isn't changed to
>> point to the new location when the object being references is copied
>> to a new address in memory. Instead, it points to kind of a tombstone
>> that is left behind when the object is moved.
>
> My approach would be to try to capture it in a debugger, then follow
> the forwarding pointer and find out what kind of object the pointer
> should be referring to.

Exactly. If it's reproducible, one can also remember the hash, and stop
when objects with that hash are allocated, to see where they are stored.
That worked pretty well in the past, but in this case led to nothing.

> I'm thinking, though, about how to increase pressure to flush out such
> bugs. Here are some ideas:
>
> 1. scan xmalloc'd memory for pointers that refer to MPS-managed objects.
> 2. reduce generation sizes and increase the number of generations, making it more likely objects will be copied.
> 3. trigger GC regularly while allocating objects
> 4. Hack MPS to do something.
> 5. Keep a log of forwarded objects and their old/new pointers
>
(2) and (3) I've tried but no new findings, with (3) as far as it went
witout rendering Emacs unbearable interactively.

Also tried making specpdl and byte stack ambiguous roots, in their
entirety. Slowness but still asserted once, so I guess I can exclude
these two, with some probability.

> (1) seems the most complete approach but relies on unusual pointer
> representations to reduce the number of false positives (and even
> then, they might be pinned objects and then the pointer is okay...).
>
> (2) is easy to do, but impacts performance. (3) is easy to do, but
> impacts performance a lot. I'd prefer avoiding (4), and while (5) is
> doable it's probably unnecessary: you can just read the forwarding
> pointer if it's still there.
>
>> Alas, I haven't been able to debug this. One problem is that I
>> can't reproduce it easily, the other is that is is either not
>> happening or happening much less often when building with -O0, and
>> without -O0 I can't see much here.
>
> I assume it's being accessed from Lisp or in an exception handler?
> Which optimization options are you using?

Just -O.

igc_check_fwd runs in places like XCONS and XSYMBOL etc. so it detects
the problem as soom as possible, on the Lisp side. I'd say most of the
time it's a cons that's the problem, and sometimes a symbol. Maybe it's
a cons containing a symbol, or something like that, where sometimes the
cons isn't copied yet, and the symbol is, or something. Hard to tell.

>> This is just to let people know of the problem. If you find a recipe
>> how to reproduce this, please let me know. Or better yet, debug it :-).
>
> Well, it's possible, but quite unlikely, that it is the
> handlerlist_sentinel thing or the Lisp_Mutex->name thing (both fixed).
> Much more likely it's another issue.

Let's see, I've just transferred you latest commits. Like I mentioned,
it can take days for the thing to surface.