Re: igc, macOS avoiding signals

[Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> From: Stefan Kangas <stefankangas@gmail.com>
>> Date: Thu, 2 Jan 2025 02:37:12 -0600
>> Cc: eller.helmut@gmail.com, pipcet@protonmail.com, emacs-devel@gnu.org
>>
>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>>
>> >> Mercy: I have a lot of other Emacs-related stuff on my plate, as you
>> >> are well aware.  Just keeping up with this discussion is already hard
>> >> for me.  Please help me by reaching out to the MPS folks about this
>> >> issue.
>> >
>> > I think it should be something "official". Maybe Stefan Kangas could
>> > contact them, or Richard.
>>
>> I'm happy to reach out to them in official capacity, but I'm not really
>> close enough to the code to be able to usefully discuss the issue with
>> them.  So I think it might be best to put some or all of you in Cc.
>>
>> Before we do anything though, are we sure that it is faster to ask them
>> to do things for us, instead of, say, just sending a patch?  I'm not
>> sure how confident people are with hacking MPS, but if we are still
>> seriously entertaining the idea of a fork then maybe we should be, to
>> some extent.
>
> We don't know the answer, AFAIU.  We could tell them that if they
> prefer a patch, we can send one.

I see no requirement to modify MPS, so far.

The current solution should work fine for Emacs once the special-casing
of fast symbol handlers, which Helmut is working on IIUC, is in place.

>> If we do decide to contact them, I'm afraid that I don't have sufficient
>> context to accurately describe the proposed callback.  I would need to
>> ask someone to summarize the idea in sufficient detail so that we can
>> start a conversation.
>
> The problem is that evidently (at least on Posix platforms), if a
> program that uses MPS runs application code from a SIGPROF or a
> SIGALRM or a SIGCHLD signal handler can trigger a recursive access to
> the MPS arena,

Let's be specific here: it's about accessing MPS memory, not about
allocating memory.  In particular, we're fully aware that thread APs
cannot be used from signal handlers if they're also used by the main
thread.

> which causes a fatal signal if that happens while MPS
> holds the arena lock.

I don't know what "fatal signal" is supposed ot mean in this case, to
the MPS folks.

If there is a memory barrier in place, the result will be a deadlock
situation, which may or may not be detected.  In the first case, there
is an assertion violation, in some builds (I believe the "rash" build
will simply continue and corrupt memory).  In the second case, there's a
user-visible deadlock.

If there is no memory barrier in place (for example, while the segment
in question is being scanned), but the arena lock is held, silent data
corruption will occur, because we'll read an invalid intermediate state
of the memory.

IOW, the abort() situation is the best of three possible outcomes, not
the only one we want to avoid.  (I say this because I sometimes get the
impression that "make the MPS arena lock recursive" is considered as a
possible "solution"; it's not, for various reasons).

> So we want to ask for a callback when MPS is
> about to lock the arena, and another callback immediately after it
> releases the lock.

That's the first time I hear about the "about to lock the arena"
callback.  Wouldn't hurt, of course, but it's also a new idea.

"Immediately", of course, may be misleading: if another thread is
waiting for the lock, the lock will not be available until that thread
is done with it.

There's a third lock operation, of course: _trylock, used for
(invasively) checking whether a mutex is currently available.  Do we
want a pair of callbacks for that, or a third callback, or nothing at
all?

> With that, we could defer the application code of
> these signal handlers until after the arena is free to be accessed
> again.

We could block the appropriate signals before (in some cases, quite a
while before) we take the arena lock and unblock them after we release
it.  That's not obviously the best solution, but it's the only one this
change would enable, AFAICS.

Running signal handler code from C without ever blocking signals at the
OS level is much more complicated, and I'm not convinced it would work
even with atomic types.

This would also require the OS to continue guaranteeing full POSIX
signal semantics while a "SIGSEGV" exception is being handled, since
that involves taking the arena lock.

> Alternatively, if MPS already has a solution for such applications
> that use signals, we'd like to hear what they suggest.

That's a useful question to ask, of course.  My understanding is that
MPS is configured mostly at build time, and your idea would amount to
creating a replacement for lockix.c and lockw3.c which allows blocking
signals.

> As background, you can point them to this discussion:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2024-06/msg00568.html

I think the polite thing to do would be to agree on a short but accurate
summary of what it is we want, explaining why it would be helpful (it
may simplify things but isn't required for correctness).

Pip