igc, macOS avoiding signals

igc, macOS avoiding signals

[Gerd Möllmann]

This is about commit

ceec5ace134081b64dbf46c4fb5702ef5209c5fd
Avoid MPS being interrupted by signals

I've been running with this for some days now, and must report that
Emacs feels a _bit_ different here in interactive use, maybe one could
say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
very recent master++).

After reverting the commit, it's feeling smoother again.

Just saying. FWIW.

Maybe it's a point for making things conditional on OS, don't know.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> This is about commit
>
> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
> Avoid MPS being interrupted by signals
>
> I've been running with this for some days now, and must report that
> Emacs feels a _bit_ different here in interactive use, maybe one could
> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
> very recent master++).

I think we should quantify that.  Set a watchpoint on
igc_global->signals_pending, check how often we even set that, and how
often we call igc_maybe_quit, particularly if we were previously idle.
Maybe it's sufficient to call it again from the idle handler.

> After reverting the commit, it's feeling smoother again.

Entirely possible.  Let's measure it.

> Maybe it's a point for making things conditional on OS, don't know.

If we establish it's not necessary on macOS, sure.  If it slows things
down that's kind of a hint that it might be necessary, though.

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> This is about commit
>>
>> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
>> Avoid MPS being interrupted by signals
>>
>> I've been running with this for some days now, and must report that
>> Emacs feels a _bit_ different here in interactive use, maybe one could
>> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
>> very recent master++).
>
> I think we should quantify that.  Set a watchpoint on
> igc_global->signals_pending, check how often we even set that, and how
> often we call igc_maybe_quit, particularly if we were previously idle.
> Maybe it's sufficient to call it again from the idle handler.
>
>> After reverting the commit, it's feeling smoother again.
>
> Entirely possible.  Let's measure it.

I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
else using macOS can do that. I just wanted to make you aware of this.

Re: igc, macOS avoiding signals

[Sean Devlin]

> Pip Cet <pipcet@protonmail.com> writes:
> 
> > Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> >
> >> This is about commit
> >>
> >> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
> >> Avoid MPS being interrupted by signals
> >>
> >> I've been running with this for some days now, and must report that
> >> Emacs feels a _bit_ different here in interactive use, maybe one could
> >> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
> >> very recent master++).
> >
> > I think we should quantify that.  Set a watchpoint on
> > igc_global->signals_pending, check how often we even set that, and how
> > often we call igc_maybe_quit, particularly if we were previously idle.
> > Maybe it's sufficient to call it again from the idle handler.
> >
> >> After reverting the commit, it's feeling smoother again.
> >
> > Entirely possible.  Let's measure it.
> 
> I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
> else using macOS can do that. I just wanted to make you aware of this.

I can take a stab at this. I’ve been running the scratch/igc branch on macOS using the NS build.

To be clear, I should make sure to include the above commit in my build? Or should I not include it?

When I take my measurements, are there any kinds of tasks I should try to perform? What durations of time should I try to measure?

Cheers.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Sean Devlin <spd@toadstyle.org> writes:

>> Pip Cet <pipcet@protonmail.com> writes:
>> 
>> > Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>> >
>> >> This is about commit
>> >>
>> >> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
>> >> Avoid MPS being interrupted by signals
>> >>
>> >> I've been running with this for some days now, and must report that
>> >> Emacs feels a _bit_ different here in interactive use, maybe one could
>> >> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
>> >> very recent master++).
>> >
>> > I think we should quantify that.  Set a watchpoint on
>> > igc_global->signals_pending, check how often we even set that, and how
>> > often we call igc_maybe_quit, particularly if we were previously idle.
>> > Maybe it's sufficient to call it again from the idle handler.
>> >
>> >> After reverting the commit, it's feeling smoother again.
>> >
>> > Entirely possible.  Let's measure it.
>> 
>> I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
>> else using macOS can do that. I just wanted to make you aware of this.
>
> I can take a stab at this. I’ve been running the scratch/igc branch on
> macOS using the NS build.
>
> To be clear, I should make sure to include the above commit in my
> build? Or should I not include it?

Just let in in. I reverted it only in my Emacs.

> When I take my measurements, are there any kinds of tasks I should try
> to perform? What durations of time should I try to measure?
>
> Cheers.

That's something for Pip to tell.

Re: igc, macOS avoiding signals

[Óscar Fuentes]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

>>> After reverting the commit, it's feeling smoother again.
>>
>> Entirely possible.  Let's measure it.
>
> I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
> else using macOS can do that. I just wanted to make you aware of this.

Is it difficult to implement a way to measure some relevant metrics?

Something related to UI responsiveness would be great. For instance,
record the time from each interactive command start to command end (or,
better, until Emacs is idle again, to account for commands accumulating
on the queue). Then we can perform some statistical analysis on that
info.

I'm afraid that if we start discussing personal perceptions we will
devote a lot of time trying to fine-adjust parameters.

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: pipcet@protonmail.com,  emacs-devel@gnu.org
> Date: Sat, 28 Dec 2024 14:28:57 +0100
> 
> Sean Devlin <spd@toadstyle.org> writes:
> 
> > To be clear, I should make sure to include the above commit in my
> > build? Or should I not include it?
> 
> Just let in in. I reverted it only in my Emacs.

Can you show that MPS runs GC from a non-main thread?  Like, if you
set a breakpoint in ArenaEnter, does it ever break on a non-main
thread?

I've just reviewed all the backtraces we had in our MPS discussions,
and all I see there is MPS being triggered either from igc_on_idle or
from alloc_impl, but every single time it happens on the main thread,
the one where I see the main function that enters recursive-edit.

So maybe Pip is right, and MPS always runs in the main (Lisp) thread,
even on macOS?  Can you catch it on a non-main thread?

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> So maybe Pip is right, and MPS always runs in the main (Lisp) thread,
> even on macOS?  Can you catch it on a non-main thread?

It's well possible that I misunderstand what the MPS guide says about it
being concurrent (see my reply to Pip), and that the thread I see here
is something else.

If you don't see an additional thread on Linux, just don't listen to me
and do what you think is TRT. I don't know anything about MPS internals.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Sean Devlin" <spd@toadstyle.org> writes:

>> Pip Cet <pipcet@protonmail.com> writes:
>>
>> > Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>> >
>> >> This is about commit
>> >>
>> >> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
>> >> Avoid MPS being interrupted by signals
>> >>
>> >> I've been running with this for some days now, and must report that
>> >> Emacs feels a _bit_ different here in interactive use, maybe one could
>> >> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
>> >> very recent master++).
>> >
>> > I think we should quantify that.  Set a watchpoint on
>> > igc_global->signals_pending, check how often we even set that, and how
>> > often we call igc_maybe_quit, particularly if we were previously idle.
>> > Maybe it's sufficient to call it again from the idle handler.
>> >
>> >> After reverting the commit, it's feeling smoother again.
>> >
>> > Entirely possible.  Let's measure it.
>>
>> I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
>> else using macOS can do that. I just wanted to make you aware of this.
>
> I can take a stab at this. I’ve been running the scratch/igc branch on macOS using the NS build.

Thanks!

> To be clear, I should make sure to include the above commit in my build? Or should I not include it?

I think we probably need to put instrumentation in the source code, so
we gain some idea of how long signals are delayed for when we mark them
pending.

(I'm also noticing that igc_maybe_quit isn't called as much as I thought
it would be.  Maybe we need to call it explicitly when Emacs becomes
idle?)

I'll try to come up with a patch.

Pip

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

> I'll try to come up with a patch.

This should provide some data (on stderr) about which signals we delay,
and for how long (the "delayed" messages).  It also includes some
information on additional points at which we can detect whether signals
are pending (the "delaying" messages); it's probably safe to run them at
that point, but the code might need some changes because other signals
(or even the signal in question) might be legitimately blocked when we
reach that point.

If the "delaying" messages indicate acceptable (initial) delays, we
might get away with simply calling gc_maybe_quit more often.  If they
don't, further fixes will be necessary, or we need to find more such
points.

On POSIX systems where we can spare an additional signal, we can run a
separate thread to ask us to retry running signal handlers when the
arena lock might be available again.

Or we could move to a separate thread for slow-path allocations.

diff --git a/src/igc.c b/src/igc.c
index 4d9aec90340..1402b9d10a5 100644
--- a/src/igc.c
+++ b/src/igc.c
@@ -811,6 +811,7 @@ IGC_DEFINE_LIST (igc_thread);
   /* Per-signal flags.  These are `int' to reduce the chance of
    * corruption when accessed non-atomically.  */
   int pending_signals[64];
+  struct timespec pending_since[64];
   /* The real signal mask we want to restore after handling pending
    * signals.  */
   sigset_t signal_mask;
@@ -3833,6 +3834,7 @@ alloc_impl (size_t size, enum igc_obj_type type, mps_ap_t ap)
     case IGC_STATE_INITIAL:
       emacs_abort ();
     }
+  gc_maybe_dont_quit ();
   return p;
 }
 
@@ -4924,8 +4926,9 @@ gc_signal_handler_can_run (int sig)
   if (igc_busy_p ())
     {
       sigset_t sigs;
-      global_igc->signals_pending = 1;
+      clock_gettime (CLOCK_REALTIME, &global_igc->pending_since[sig]);
       global_igc->pending_signals[sig] = 1;
+      global_igc->signals_pending = 1;
       sigemptyset (&sigs);
       sigaddset (&sigs, sig);
       pthread_sigmask (SIG_BLOCK, &sigs, NULL);
@@ -4946,6 +4949,12 @@ gc_maybe_quit (void)
       for (int i = 0; i < ARRAYELTS (global_igc->pending_signals); i++)
 	if (global_igc->pending_signals[i])
 	  {
+	    struct timespec ts;
+	    clock_gettime (CLOCK_REALTIME, &ts);
+	    long long nsec = ts.tv_nsec - global_igc->pending_since[i].tv_nsec;
+	    long long sec = ts.tv_sec - global_igc->pending_since[i].tv_sec;
+	    nsec += 1000000000 * sec;
+	    fprintf (stderr, "delayed %d for %f sec\n", i, nsec * 1.0e-9);
 	    global_igc->pending_signals[i] = 0;
 	    raise (i);
 	  }
@@ -4953,6 +4962,23 @@ gc_maybe_quit (void)
     }
 }
 
+void gc_maybe_dont_quit (void)
+{
+  if (global_igc->signals_pending)
+    {
+      for (int i = 0; i < ARRAYELTS (global_igc->pending_signals); i++)
+	if (global_igc->pending_signals[i])
+	  {
+	    struct timespec ts;
+	    clock_gettime (CLOCK_REALTIME, &ts);
+	    long long nsec = ts.tv_nsec - global_igc->pending_since[i].tv_nsec;
+	    long long sec = ts.tv_sec - global_igc->pending_since[i].tv_sec;
+	    nsec += 1000000000 * sec;
+	    fprintf (stderr, "delaying %d for %f sec\n", i, nsec * 1.0e-9);
+	  }
+    }
+}
+
 DEFUN ("igc--add-extra-dependency", Figc__add_extra_dependency,
        Sigc__add_extra_dependency, 3, 3, 0,
        doc: /* Add an extra DEPENDENCY to object OBJ, associate it with KEY.
diff --git a/src/keyboard.c b/src/keyboard.c
index e875e98fde6..906595f3be9 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -8203,6 +8203,7 @@ unblock_input_to (int level)
   interrupt_input_blocked = level;
   if (level == 0)
     {
+      gc_maybe_dont_quit ();
       if (pending_signals && !fatal_error_in_progress)
 	process_pending_signals ();
     }
diff --git a/src/lisp.h b/src/lisp.h
index 48585c2d8a1..fb7f3847a5d 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -47,12 +47,17 @@ #define EMACS_LISP_H
 #ifdef HAVE_MPS
 union gc_header;
 extern void gc_maybe_quit (void);
+extern void gc_maybe_dont_quit (void);
 extern bool gc_signal_handler_can_run (int);
 #else
 INLINE void gc_maybe_quit (void)
 {
 }
 
+INLINE void gc_maybe_dont_quit (void)
+{
+}
+
 INLINE bool gc_signal_handler_can_run (int sig)
 {
   return true;

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 15:12:23 +0000
> Cc: gerd.moellmann@gmail.com, emacs-devel@gnu.org
> From:  Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>
> 
> I think we probably need to put instrumentation in the source code, so
> we gain some idea of how long signals are delayed for when we mark them
> pending.

What do we expect to learn from this, except the timing of the OS
scheduler?

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> Date: Sat, 28 Dec 2024 15:12:23 +0000
>> Cc: gerd.moellmann@gmail.com, emacs-devel@gnu.org
>> From:  Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>
>>
>> I think we probably need to put instrumentation in the source code, so
>> we gain some idea of how long signals are delayed for when we mark them
>> pending.
>
> What do we expect to learn from this,

It tests the current code, which does this:

When a signal arrives, and we can't handle it because we might have
interrupted MPS, we mark the signal as pending in the igc structure.  At
some point later, we check the igc structure for pending signals,
reraise them, and unmask them.

Gerd's experience suggests that the "some point later" happens too late.
This patch gives us measurements.

It's unrelated to the OS scheduler, AFAICS.

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 18:40:30 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> >> Date: Sat, 28 Dec 2024 15:12:23 +0000
> >> Cc: gerd.moellmann@gmail.com, emacs-devel@gnu.org
> >> From:  Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>
> >>
> >> I think we probably need to put instrumentation in the source code, so
> >> we gain some idea of how long signals are delayed for when we mark them
> >> pending.
> >
> > What do we expect to learn from this,
> 
> It tests the current code, which does this:
> 
> When a signal arrives, and we can't handle it because we might have
> interrupted MPS, we mark the signal as pending in the igc structure.  At
> some point later, we check the igc structure for pending signals,
> reraise them, and unmask them.
> 
> Gerd's experience suggests that the "some point later" happens too late.
> This patch gives us measurements.
> 
> It's unrelated to the OS scheduler, AFAICS.

Ah, okay.  I note that if we'd block signals when calling MPS and
unblock on exit, then these delays couldn't have happened, AFAIU.

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 20:50:22 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> > > What do we expect to learn from this,
> > 
> > It tests the current code, which does this:
> > 
> > When a signal arrives, and we can't handle it because we might have
> > interrupted MPS, we mark the signal as pending in the igc structure.  At
> > some point later, we check the igc structure for pending signals,
> > reraise them, and unmask them.
> > 
> > Gerd's experience suggests that the "some point later" happens too late.
> > This patch gives us measurements.
> > 
> > It's unrelated to the OS scheduler, AFAICS.
> 
> Ah, okay.  I note that if we'd block signals when calling MPS and
> unblock on exit, then these delays couldn't have happened, AFAIU.

But OTOH, if this delaying of a signal affects responsiveness, then
all we need to do is exempt SIGSEGV from being delayed, right?  This
signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
but there's no reason to delay SIGSEGV.

And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
does this delaying affect responsiveness?

Or what am I missing?

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

> Ah, okay.  I note that if we'd block signals when calling MPS and
> unblock on exit, then these delays couldn't have happened, AFAIU.

We can't do so for SIGSEGV calling into MPS, so this wouldn't fix all
cases.  Blocking signals around mps_alloc slows down (make-list 1000000
nil) by a factor of about 5 (on current GNU/Linux; possibly
significantly more on other operating systems).

But, yes, if we're willing to give up on unmodified MPS, blocking
signals in the slow path only might work.  We'd need to check
finalizable objects, though, because we need to call into MPS for every
finalizable object.  That's a problem this approach shares with the
allocation thread approach, by the way.  In both cases, registering
objects for finalization doesn't need to happen at allocation time: if
we save a reference to the object somewhere MPS sees it, the objects
won't be collected, so they won't be finalized.

Pip

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> Date: Sat, 28 Dec 2024 20:50:22 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
>>
>> > > What do we expect to learn from this,
>> >
>> > It tests the current code, which does this:
>> >
>> > When a signal arrives, and we can't handle it because we might have
>> > interrupted MPS, we mark the signal as pending in the igc structure.  At
>> > some point later, we check the igc structure for pending signals,
>> > reraise them, and unmask them.
>> >
>> > Gerd's experience suggests that the "some point later" happens too late.
>> > This patch gives us measurements.
>> >
>> > It's unrelated to the OS scheduler, AFAICS.
>>
>> Ah, okay.  I note that if we'd block signals when calling MPS and
>> unblock on exit, then these delays couldn't have happened, AFAIU.
>
> But OTOH, if this delaying of a signal affects responsiveness, then
> all we need to do is exempt SIGSEGV from being delayed, right?  This
> signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
> but there's no reason to delay SIGSEGV.

SIGSEGV is never delayed in any proposal I'm aware of.  I don't see how
it could be, to be honest, but maybe I'm missing something there.

> And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
> does this delaying affect responsiveness?

Possibly SIGPOLL.

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 19:15:26 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> > Ah, okay.  I note that if we'd block signals when calling MPS and
> > unblock on exit, then these delays couldn't have happened, AFAIU.
> 
> We can't do so for SIGSEGV calling into MPS, so this wouldn't fix all
> cases.

We don't need to block SIGSEGV (or any other fatal signal).  We only
need to block SIGPROF, SIGALRM and SIGCHLD.

> Blocking signals around mps_alloc slows down (make-list 1000000
> nil) by a factor of about 5 (on current GNU/Linux; possibly
> significantly more on other operating systems).

No, I think only GNU/Linux is affected.

You are saying that sigblock is very expensive?  Can you measure it?

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 19:20:40 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> > But OTOH, if this delaying of a signal affects responsiveness, then
> > all we need to do is exempt SIGSEGV from being delayed, right?  This
> > signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
> > but there's no reason to delay SIGSEGV.
> 
> SIGSEGV is never delayed in any proposal I'm aware of.

The call to gc_signal_handler_can_run is inside
deliver_process_signal.  Are you saying that deliver_process_signal is
not called for SIGSEGV?

> > And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
> > does this delaying affect responsiveness?
> 
> Possibly SIGPOLL.

We don't need to block SIGPOLL, either.  Its handler is safe, the same
as SIGIO.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> Date: Sat, 28 Dec 2024 19:20:40 +0000
>> From: Pip Cet <pipcet@protonmail.com>
>> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
>>
>> "Eli Zaretskii" <eliz@gnu.org> writes:
>>
>> > But OTOH, if this delaying of a signal affects responsiveness, then
>> > all we need to do is exempt SIGSEGV from being delayed, right?  This
>> > signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
>> > but there's no reason to delay SIGSEGV.
>>
>> SIGSEGV is never delayed in any proposal I'm aware of.
>
> The call to gc_signal_handler_can_run is inside
> deliver_process_signal.  Are you saying that deliver_process_signal is
> not called for SIGSEGV?

MPS installs its own SIGSEGV handler which doesn't go through
deliver_process_signal.  Only if it fails, the Emacs handler which does
go through deliver_process_signal is restored for the final SIGSEGV
which will then terminate Emacs.

>> > And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
>> > does this delaying affect responsiveness?
>>
>> Possibly SIGPOLL.
>
> We don't need to block SIGPOLL, either.  Its handler is safe, the same
> as SIGIO.

Thanks!  That's good to know, and that's why we pass the signal number
to gc_signal_handler_can_run.

Pip

Re: igc, macOS avoiding signals

[Sean Devlin]

Hi Pip,

> On Dec 29, 2024, at 1:29 AM, Pip Cet <pipcet@protonmail.com> wrote:
> 
>> I'll try to come up with a patch.
> 
> This should provide some data (on stderr) about which signals we delay,
> and for how long (the "delayed" messages).  It also includes some
> information on additional points at which we can detect whether signals
> are pending (the "delaying" messages); it's probably safe to run them at
> that point, but the code might need some changes because other signals
> (or even the signal in question) might be legitimately blocked when we
> reach that point.
> 
> If the "delaying" messages indicate acceptable (initial) delays, we
> might get away with simply calling gc_maybe_quit more often.  If they
> don't, further fixes will be necessary, or we need to find more such
> points.
> 
> On POSIX systems where we can spare an additional signal, we can run a
> separate thread to ask us to retry running signal handlers when the
> arena lock might be available again.
> 
> Or we could move to a separate thread for slow-path allocations.
> 

I’ve built Emacs with your patch. After running Emacs -Q for a few minutes, I can confirm I see a few log statements:

delaying 20 for 0.066594 sec
delaying 20 for 0.066612 sec
delayed 20 for 0.066614 sec

Please let me know if there are any particular tasks you’d like me to try, or if I should just collect the logs in the background during general usage.

Cheers.

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 20:54:03 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> >> Date: Sat, 28 Dec 2024 19:20:40 +0000
> >> From: Pip Cet <pipcet@protonmail.com>
> >> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> >>
> >> "Eli Zaretskii" <eliz@gnu.org> writes:
> >>
> >> > But OTOH, if this delaying of a signal affects responsiveness, then
> >> > all we need to do is exempt SIGSEGV from being delayed, right?  This
> >> > signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
> >> > but there's no reason to delay SIGSEGV.
> >>
> >> SIGSEGV is never delayed in any proposal I'm aware of.
> >
> > The call to gc_signal_handler_can_run is inside
> > deliver_process_signal.  Are you saying that deliver_process_signal is
> > not called for SIGSEGV?
> 
> MPS installs its own SIGSEGV handler which doesn't go through
> deliver_process_signal.  Only if it fails, the Emacs handler which does
> go through deliver_process_signal is restored for the final SIGSEGV
> which will then terminate Emacs.

Then I ask once again: how can we explain what Gerd reports about
responsiveness if your changes, which only affect
deliver_process_signal, cannot affect MPS?

> >> > And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
> >> > does this delaying affect responsiveness?
> >>
> >> Possibly SIGPOLL.
> >
> > We don't need to block SIGPOLL, either.  Its handler is safe, the same
> > as SIGIO.
> 
> Thanks!  That's good to know, and that's why we pass the signal number
> to gc_signal_handler_can_run.

But currently gc_signal_handler_can_run does nothing with the signal
number except recording that it happened.  IMO, it should return
'true' immediately unless the signal is SIGPROF, SIGCHLD, or SIGALRM.

Re: igc, macOS avoiding signals

[Helmut Eller]

On Sat, Dec 28 2024, Óscar Fuentes wrote:
[...]
> Something related to UI responsiveness would be great. For instance,
> record the time from each interactive command start to command end (or,
> better, until Emacs is idle again, to account for commands accumulating
> on the queue). Then we can perform some statistical analysis on that
> info.
>
> I'm afraid that if we start discussing personal perceptions we will
> devote a lot of time trying to fine-adjust parameters.

I couldn't agree more. 

What would you think about using one of those tracing frameworks, like
LTTng[*]?  Are those any good?

It would be nice, if those tracing points could be used in normal
everyday sessions.  E.g. to see how memory usage evolved over the last
15 minutes or to see if there was a particularly slow regexp search.

Helmut

[*] https://lttng.org/

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Sean Devlin" <spd@toadstyle.org> writes:

> Hi Pip,
>
>> On Dec 29, 2024, at 1:29 AM, Pip Cet <pipcet@protonmail.com> wrote:
>>
>>> I'll try to come up with a patch.
>>
>> This should provide some data (on stderr) about which signals we delay,
>> and for how long (the "delayed" messages).  It also includes some
>> information on additional points at which we can detect whether signals
>> are pending (the "delaying" messages); it's probably safe to run them at
>> that point, but the code might need some changes because other signals
>> (or even the signal in question) might be legitimately blocked when we
>> reach that point.
>>
>> If the "delaying" messages indicate acceptable (initial) delays, we
>> might get away with simply calling gc_maybe_quit more often.  If they
>> don't, further fixes will be necessary, or we need to find more such
>> points.
>>
>> On POSIX systems where we can spare an additional signal, we can run a
>> separate thread to ask us to retry running signal handlers when the
>> arena lock might be available again.
>>
>> Or we could move to a separate thread for slow-path allocations.
>>
>
> I’ve built Emacs with your patch. After running Emacs -Q for a few minutes, I can confirm I see a few log statements:

Can you try setting igc-step-interval to a small float value, like 0.05
?  As long as it's just a few messages, I don't think it'd cause
significant problems, but maybe enabling the background work would do
something.

> Please let me know if there are any particular tasks you’d like me to try, or if I should just collect the logs in the background during general usage.

Repeatedly hitting "s" in an M-x igc-stats buffer should cause more
messages, but that uses IGC in an atypical fashion, so I'm not sure
that's actually useful data...

Thanks!

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> Repeatedly hitting "s" in an M-x igc-stats buffer should cause more
> messages, but that uses IGC in an atypical fashion, so I'm not sure
> that's actually useful data...

Maybe not running with -Q but a normal config would help? I'm using all
sorts of packages to generate garbage. Eglot, flymake, Corfu, Jinx,
Vertico, Marginalia ...

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Pip Cet <pipcet@protonmail.com> writes:
>
>> Repeatedly hitting "s" in an M-x igc-stats buffer should cause more
>> messages, but that uses IGC in an atypical fashion, so I'm not sure
>> that's actually useful data...
>
> Maybe not running with -Q but a normal config would help? I'm using all
> sorts of packages to generate garbage. Eglot, flymake, Corfu, Jinx,
> Vertico, Marginalia ...

Speaking of running with a "normal" config: something about my
configuration makes buffer_step (the balance_intervals call, in
particular) take forever, to the point the mps build becomes unusable.
The buffer in question, when I caught it, is an M-x shell buffer of size
8 MB, so I don't understand why it's taking so long.

Still investigating, but skipping the buffer_step seems to help.

Pip

Re: igc, macOS avoiding signals

[Sean Devlin]

[-- Attachment #1: Type: text/plain, Size: 2260 bytes --]



> On Dec 29, 2024, at 9:22 PM, Pip Cet <pipcet@protonmail.com> wrote:
> 
> "Sean Devlin" <spd@toadstyle.org <mailto:spd@toadstyle.org>> writes:
> 
>> Hi Pip,
>> 
>>> On Dec 29, 2024, at 1:29 AM, Pip Cet <pipcet@protonmail.com> wrote:
>>> 
>>>> I'll try to come up with a patch.
>>> 
>>> This should provide some data (on stderr) about which signals we delay,
>>> and for how long (the "delayed" messages).  It also includes some
>>> information on additional points at which we can detect whether signals
>>> are pending (the "delaying" messages); it's probably safe to run them at
>>> that point, but the code might need some changes because other signals
>>> (or even the signal in question) might be legitimately blocked when we
>>> reach that point.
>>> 
>>> If the "delaying" messages indicate acceptable (initial) delays, we
>>> might get away with simply calling gc_maybe_quit more often.  If they
>>> don't, further fixes will be necessary, or we need to find more such
>>> points.
>>> 
>>> On POSIX systems where we can spare an additional signal, we can run a
>>> separate thread to ask us to retry running signal handlers when the
>>> arena lock might be available again.
>>> 
>>> Or we could move to a separate thread for slow-path allocations.
>>> 
>> 
>> I’ve built Emacs with your patch. After running Emacs -Q for a few minutes, I can confirm I see a few log statements:
> 
> Can you try setting igc-step-interval to a small float value, like 0.05
> ?  As long as it's just a few messages, I don't think it'd cause
> significant problems, but maybe enabling the background work would do
> something.

Sounds good, will do.

> 
>> Please let me know if there are any particular tasks you’d like me to try, or if I should just collect the logs in the background during general usage.
> 
> Repeatedly hitting "s" in an M-x igc-stats buffer should cause more
> messages, but that uses IGC in an atypical fashion, so I'm not sure
> that's actually useful data...

Well, I can definitely make it log a bunch of messages either by running the profiler or by spawning subprocesses. Is that helpful by itself, or do I also need to generate garbage for collection?

> 
> Thanks!
> 
> Pip


[-- Attachment #2: Type: text/html, Size: 12799 bytes --]

Re: igc, macOS avoiding signals

[Sean Devlin]

> On Dec 30, 2024, at 12:01 AM, Gerd Möllmann <gerd.moellmann@gmail.com> wrote:
> 
> Pip Cet <pipcet@protonmail.com> writes:
> 
>> Repeatedly hitting "s" in an M-x igc-stats buffer should cause more
>> messages, but that uses IGC in an atypical fashion, so I'm not sure
>> that's actually useful data...
> 
> Maybe not running with -Q but a normal config would help? I'm using all
> sorts of packages to generate garbage. Eglot, flymake, Corfu, Jinx,
> Vertico, Marginalia …


Sounds good, I’ll use my normal configuration.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> Speaking of running with a "normal" config: something about my
> configuration makes buffer_step (the balance_intervals call, in
> particular) take forever, to the point the mps build becomes unusable.
> The buffer in question, when I caught it, is an M-x shell buffer of size
> 8 MB, so I don't understand why it's taking so long.
>
> Still investigating, but skipping the buffer_step seems to help.

balance_intervals means text properties. The only candidate I see in
comint/shell is ANSI escapes. That could be turned on/off with M-x
ansi-color-for-comint-mode-xy. Only as a workaround, and maybe to check
if it's that.

What I do in buffer_step in idle time is basically one step of what the
old GC does in sweep_buffers.

My expectation was that balancing a tree couldn't take long, and that
this is not called often enough to be a problem if were expensive. Both
wrong, as usual.

Not calling balance_intervals is, BTW, not a catastrophic problem. if
one does anything leading to a graft_intervals_into_buffer, w</r hich is
called in a lot of places in editfns.c and insdel.c, that balances the
tree. And if not, the tree might become slower for lookup (redisplay),
but it still works.

<rant> It's BTW well possible that I myself put that balancing into
sweep_buffers because of redisplay, I seem to remember that. The
interval tree has always been a source of fun. I hope, some day, some
kind soul will eradicate it like the GCPROs. </rant>

In any case, what's a solution?

Right now I'm tending to put the balance_intervals in an if so that one
can turn it on/off with a Lisp variable. Default would be to not to balance,
because I think the problems with degenerated interval trees in
redisplay where rare, and I don't remember problems outside of
redisplay. But that was an awful long time ago, OTOH.

That would give us more time to think about a possible strategy to solve
this.

WDYT?

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Sean Devlin <spd@toadstyle.org> writes:

>> On Dec 30, 2024, at 12:01 AM, Gerd Möllmann <gerd.moellmann@gmail.com> wrote:
>> 
>> Pip Cet <pipcet@protonmail.com> writes:
>> 
>>> Repeatedly hitting "s" in an M-x igc-stats buffer should cause more
>>> messages, but that uses IGC in an atypical fashion, so I'm not sure
>>> that's actually useful data...
>> 
>> Maybe not running with -Q but a normal config would help? I'm using all
>> sorts of packages to generate garbage. Eglot, flymake, Corfu, Jinx,
>> Vertico, Marginalia …
>
>
> Sounds good, I’ll use my normal configuration.

Thanks!

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> So maybe Pip is right, and MPS always runs in the main (Lisp) thread,
>> even on macOS?  Can you catch it on a non-main thread?
>
> It's well possible that I misunderstand what the MPS guide says about it
> being concurrent (see my reply to Pip), and that the thread I see here
> is something else.
>
> If you don't see an additional thread on Linux, just don't listen to me
> and do what you think is TRT. I don't know anything about MPS internals.

I've investigated this a bit using LLDB. Starting Emacs and attaching to
it, I see 3 threads.

  (lldb) thread list
  Process 55210 stopped
  * thread #1: tid = 0x8b6558, 0x0000000190be51a8 libsystem_kernel.dylib`__pselect + 8, queue 
    thread #2: tid = 0x8b655c, 0x0000000190bdef54 libsystem_kernel.dylib`mach_msg2_trap + 8
    thread #3: tid = 0x8b65df, 0x0000000190be0ba4 libsystem_kernel.dylib`__workq_kernreturn + 8

Thread 1 is Emacs main thread.

  (lldb) thread backtrace
  * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x0000000190be51a8 libsystem_kernel.dylib`__pselect + 8
    frame #1: 0x0000000190be5080 libsystem_kernel.dylib`pselect$DARWIN_EXTSN + 64
      frame #2: 0x0000000100775948 emacs`really_call_select(arg=0x000000016f851990) at thread.c:620:16 [opt]
      frame #3: 0x00000001007758bc emacs`thread_select [inlined] flush_stack_ca
      frame #4: 0x00000001007758ac emacs`thread_select(func=<unavailable>, max_
      frame #5: 0x0000000100744e78 emacs`wait_reading_process_output(time_limit=<unavailable>, n

Thread 2 is MPS' port, for EXC_BAD_ACCESS

  (lldb) thread backtrace
  * thread #2
    frame #0: 0x0000000190bdef54 libsystem_kernel.dylib`mach_msg2_trap + 8
    frame #1: 0x0000000190bf169c libsystem_kernel.dylib`mach_msg2_internal + 232
    frame #2: 0x0000000190be7af8 libsystem_kernel.dylib`mach_msg_overwrite + 480
    frame #3: 0x0000000190bdf29c libsystem_kernel.dylib`mach_msg + 24
      frame #4: 0x000000010080ae20 emacs`protCatchThread [inlined] protCatchOne at protxc.c:207:8 [opt]
      frame #5: 0x000000010080adf0 emacs`protCatchThread(p=<unavailable>) at protxc.c:284:5 [opt]
    frame #6: 0x0000000190c202e4 libsystem_pthread.dylib`_pthread_start + 136

The protxc.c is from MPS.

Thread 3 is something I can't explain.

  (lldb) thread backtrace
  * thread #3
    frame #0: 0x0000000190be0ba4 libsystem_kernel.dylib`__workq_kernreturn + 8

The __workq_kernreturn should indicate a thread that it is in the
process of finishing, but I have no idea what that could have been.

Anyway, it definitely seems to be the case that MPS is _not_ running GCs
concurrently, unless it would do things that I find highly unlikely.

I find that a bit, let's say, disappointing, TBH :-(.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>> So maybe Pip is right, and MPS always runs in the main (Lisp) thread,
>>> even on macOS?  Can you catch it on a non-main thread?
>>
>> It's well possible that I misunderstand what the MPS guide says about it
>> being concurrent (see my reply to Pip), and that the thread I see here
>> is something else.
>>
>> If you don't see an additional thread on Linux, just don't listen to me
>> and do what you think is TRT. I don't know anything about MPS internals.
>
> I've investigated this a bit using LLDB. Starting Emacs and attaching to
> it, I see 3 threads.
>
>   (lldb) thread list
>   Process 55210 stopped
>   * thread #1: tid = 0x8b6558, 0x0000000190be51a8 libsystem_kernel.dylib`__pselect + 8, queue 
>     thread #2: tid = 0x8b655c, 0x0000000190bdef54 libsystem_kernel.dylib`mach_msg2_trap + 8
>     thread #3: tid = 0x8b65df, 0x0000000190be0ba4 libsystem_kernel.dylib`__workq_kernreturn + 8
>
> Thread 1 is Emacs main thread.
>
>   (lldb) thread backtrace
>   * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
>     frame #0: 0x0000000190be51a8 libsystem_kernel.dylib`__pselect + 8
>     frame #1: 0x0000000190be5080 libsystem_kernel.dylib`pselect$DARWIN_EXTSN + 64
>       frame #2: 0x0000000100775948 emacs`really_call_select(arg=0x000000016f851990) at thread.c:620:16 [opt]
>       frame #3: 0x00000001007758bc emacs`thread_select [inlined] flush_stack_ca
>       frame #4: 0x00000001007758ac emacs`thread_select(func=<unavailable>, max_
>       frame #5: 0x0000000100744e78 emacs`wait_reading_process_output(time_limit=<unavailable>, n
>
> Thread 2 is MPS' port, for EXC_BAD_ACCESS
>
>   (lldb) thread backtrace
>   * thread #2
>     frame #0: 0x0000000190bdef54 libsystem_kernel.dylib`mach_msg2_trap + 8
>     frame #1: 0x0000000190bf169c libsystem_kernel.dylib`mach_msg2_internal + 232
>     frame #2: 0x0000000190be7af8 libsystem_kernel.dylib`mach_msg_overwrite + 480
>     frame #3: 0x0000000190bdf29c libsystem_kernel.dylib`mach_msg + 24
>       frame #4: 0x000000010080ae20 emacs`protCatchThread [inlined] protCatchOne at protxc.c:207:8 [opt]
>       frame #5: 0x000000010080adf0 emacs`protCatchThread(p=<unavailable>) at protxc.c:284:5 [opt]
>     frame #6: 0x0000000190c202e4 libsystem_pthread.dylib`_pthread_start + 136
>
> The protxc.c is from MPS.
>
> Thread 3 is something I can't explain.
>
>   (lldb) thread backtrace
>   * thread #3
>     frame #0: 0x0000000190be0ba4 libsystem_kernel.dylib`__workq_kernreturn + 8
>
> The __workq_kernreturn should indicate a thread that it is in the
> process of finishing, but I have no idea what that could have been.
>
> Anyway, it definitely seems to be the case that MPS is _not_ running GCs
> concurrently, unless it would do things that I find highly unlikely.
>
> I find that a bit, let's say, disappointing, TBH :-(.

And now, git grep in MPS, reveals

Concurrent collection
.....................

_`.improv.concurrent`: The MPS currently does not collect
concurrently, however the only thing that makes it not-concurrent is a

:-(

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Gerd Möllmann wrote:
> Anyway, it definitely seems to be the case that MPS is _not_ running GCs
> concurrently, unless it would do things that I find highly unlikely.
>
> I find that a bit, let's say, disappointing, TBH :-(.

Richard Brooksby thinks[*] that MPS could be concurrent with software
barriers.  Feel like going down that road? :-)

Helmut

[*] https://memory-pool-system.readthedocs.io/en/latest/design/shield.html#concurrent-collection

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>> Anyway, it definitely seems to be the case that MPS is _not_ running GCs
>> concurrently, unless it would do things that I find highly unlikely.
>>
>> I find that a bit, let's say, disappointing, TBH :-(.
>
> Richard Brooksby thinks[*] that MPS could be concurrent with software
> barriers.  Feel like going down that road? :-)
>
> Helmut
>
> [*] https://memory-pool-system.readthedocs.io/en/latest/design/shield.html#concurrent-collection

Yep, found that too, with git grep.

Still grumpy.

I'm afraid Modifying MPS is not my thing, But What about using something
more modern like Oilpan (aka cppgc) from V8? Can be used as a lib, is
concurrent for real. That would also be a perfect time to lift Emacs to
C++.

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Gerd Möllmann wrote:

> I'm afraid Modifying MPS is not my thing, But What about using something
> more modern like Oilpan (aka cppgc) from V8? Can be used as a lib, is
> concurrent for real.

Ideally, Emacs would have an abstract GC interface so that different
implementations could be plugged in.

> That would also be a perfect time to lift Emacs to
> C++.

I'd rather see Emacs move to Rust.  Anyway, neither option seems
realistic.

Helmut

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>
>> I'm afraid Modifying MPS is not my thing, But What about using something
>> more modern like Oilpan (aka cppgc) from V8? Can be used as a lib, is
>> concurrent for real.
>
> Ideally, Emacs would have an abstract GC interface so that different
> implementations could be plugged in.

That would indeed be nice to have.

>> That would also be a perfect time to lift Emacs to
>> C++.
>
> I'd rather see Emacs move to Rust.  Anyway, neither option seems
> realistic.

In mainline... (pondering to put a smiley).

But something else: Given what I now believe, I think I want to
understand better (a bit) why everything appears to work just fine on
macOS, with signals. Could you perhaps check if I'm off? MacOS only.

In normal operation, there are only ever 2 threads running. An Emacs
thread is interrupted by a signal and lands in a signal handler, the
MPS port thread keeps running.

In the signal handler, hitting barriers is handled by the MPS port
thread. Consistency of Emacs's state is a problem the signal handler has
to deal with, consistency of MPS' GC data is a problem that hopefully
MPS handles, and it seems to work.

I think I understand that, except when the Emacs thread is interrupted
while in MPS code, which happens for allocation points running out of
memory and mps_arena_step (idle time).

Do you agree so far? If yes, I'd bite the bullet and look at the MPS
code for macOS how that is done, if it's done.

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Gerd Möllmann wrote:

[...]
> But something else: Given what I now believe, I think I want to
> understand better (a bit) why everything appears to work just fine on
> macOS, with signals. Could you perhaps check if I'm off? MacOS only.
>
> In normal operation, there are only ever 2 threads running. An Emacs
> thread is interrupted by a signal and lands in a signal handler, the
> MPS port thread keeps running.
>
> In the signal handler, hitting barriers is handled by the MPS port
> thread. Consistency of Emacs's state is a problem the signal handler has
> to deal with,

Agreed.

> consistency of MPS' GC data is a problem that hopefully
> MPS handles, and it seems to work.

My interpretation of this design document[*], is that MPS's arena lock
protects most of MPS entry points.  There are a few (e.g. mps_reserve
and mp_ld_add) that don't claim the arena lock and for those it's the
burden of the client to call them in a thread safe way.  For us this
probably means: don't call them in a signal handler.

The main entry point that we want to call in the signal handler is the
SEGFAULT handler (not sure how this works on MacOS).  The fault handler
claims the non-recursive arena lock.  So, in the signal handler we
should not hold the lock while hitting a barrier.

> I think I understand that, except when the Emacs thread is interrupted
> while in MPS code, which happens for allocation points running out of
> memory and mps_arena_step (idle time).

Hmm, is that sentence incomplete?  I don't quite understand it.

> Do you agree so far? If yes, I'd bite the bullet and look at the MPS
> code for macOS how that is done, if it's done.

Yes, mostly.

Helmut

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Helmut Eller wrote:

> My interpretation of this design document[*], is that MPS's arena lock

That one:

https://memory-pool-system.readthedocs.io/en/latest/design/thread-safety.html

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

>> In the signal handler, hitting barriers is handled by the MPS port
>> thread. Consistency of Emacs's state is a problem the signal handler has
>> to deal with,
>
> Agreed.
>
>> consistency of MPS' GC data is a problem that hopefully
>> MPS handles, and it seems to work.
>
> My interpretation of this design document[*], is that MPS's arena lock
> protects most of MPS entry points.  There are a few (e.g. mps_reserve
> and mp_ld_add) that don't claim the arena lock and for those it's the
> burden of the client to call them in a thread safe way.  For us this
> probably means: don't call them in a signal handler.
>
> The main entry point that we want to call in the signal handler is the
> SEGFAULT handler (not sure how this works on MacOS).  The fault handler
> claims the non-recursive arena lock.  So, in the signal handler we
> should not hold the lock while hitting a barrier.

Okay, that I think I understand then. The "only" difference between
macOS and Linux is that on macOS no SEGV handler is involved. Hitting
the barrier on macOS means that the EXC_BAD_ACCESS port thread, which
was waiting for Mach message, receives a message from the OS and starts
working.

>> I think I understand that, except when the Emacs thread is interrupted
>> while in MPS code, which happens for allocation points running out of
>> memory and mps_arena_step (idle time).
>
> Hmm, is that sentence incomplete?  I don't quite understand it.

What I meant is that I imagine a signal interrupts the Emacs thread at a
point where we are "in MPS". AreaEnter/Leave I think I understand, it's
some pthread_mutex_t, I think, from other mails. A problematic "in MPS"
could then be "while the Emacs thread owns the mutex". The places where
I imagine that mutex could be owned are mps_arena_step (I call it Emacs
is idle) and mps_commit (in alloc_impl, when the allocation point used
runs out of memory). Maybe other places, but mainly these two.

And the question on macOS for me would be if the port thread tries to
qcquire the same mutex, or how the heck that works. Or IOW, if there is
a problem, why I've never seen it happening in all that time I'm using
igc. I find that difficult to understand. But it may be just a
statistical phenomenon. Maybe filling up an APs memory is so fast so
that the probability of a signal hitting while owning the mutex is close
to zero, or something. 

>
>> Do you agree so far? If yes, I'd bite the bullet and look at the MPS
>> code for macOS how that is done, if it's done.
>
> Yes, mostly.
>
> Helmut

Thanks for your help! I'll post something when I think I have something.

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Gerd Möllmann wrote:

>> My interpretation of this design document[*], is that MPS's arena lock
>> protects most of MPS entry points.  There are a few (e.g. mps_reserve
>> and mp_ld_add) that don't claim the arena lock and for those it's the
>> burden of the client to call them in a thread safe way.  For us this
>> probably means: don't call them in a signal handler.
>>
>> The main entry point that we want to call in the signal handler is the
>> SEGFAULT handler (not sure how this works on MacOS).  The fault handler
>> claims the non-recursive arena lock.  So, in the signal handler we
>> should not hold the lock while hitting a barrier.
>
> Okay, that I think I understand then. The "only" difference between
> macOS and Linux is that on macOS no SEGV handler is involved. Hitting
> the barrier on macOS means that the EXC_BAD_ACCESS port thread, which
> was waiting for Mach message, receives a message from the OS and starts
> working.

I guess that both, macOS and Linux version, will end up in ArenaAccess.
Perhaps on macOS in the other thread.

>>> I think I understand that, except when the Emacs thread is interrupted
>>> while in MPS code, which happens for allocation points running out of
>>> memory and mps_arena_step (idle time).
>>
>> Hmm, is that sentence incomplete?  I don't quite understand it.
>
> What I meant is that I imagine a signal interrupts the Emacs thread at a
> point where we are "in MPS". AreaEnter/Leave I think I understand, it's
> some pthread_mutex_t, I think, from other mails. A problematic "in MPS"
> could then be "while the Emacs thread owns the mutex". The places where
> I imagine that mutex could be owned are mps_arena_step (I call it Emacs
> is idle) and mps_commit (in alloc_impl, when the allocation point used
> runs out of memory). Maybe other places, but mainly these two.

Yes.

> And the question on macOS for me would be if the port thread tries to
> qcquire the same mutex, or how the heck that works. Or IOW, if there is
> a problem, why I've never seen it happening in all that time I'm using
> igc.

Maybe you could set a breakpoint in AreaAccess to find out which thread
removes the barriers.

> I find that difficult to understand. But it may be just a
> statistical phenomenon. Maybe filling up an APs memory is so fast so
> that the probability of a signal hitting while owning the mutex is close
> to zero, or something. 

Very few of Emacs' signal handlers actually touch a barrier.  I've also
not seen any reproducable receipes for the "signal issues" that the igc
branch supposedly has.

Helmut

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> Anyway, it definitely seems to be the case that MPS is _not_ running GCs
> concurrently, unless it would do things that I find highly unlikely.
>
> I find that a bit, let's say, disappointing, TBH :-(.

Well, I think that MPS is bring-your-own-thread concurrent.  I'm not
sure the current MPS can usefully be concurrent, because of the thread
suspension thing, but we can run it in another thread if we want to.

(If it turns out that using a separate thread isn't an advantage, we
should look at reducing the size of our roots (or protecting them)
rather than giving up on the idea entirely.  As long as other threads
freely move references to and from global roots, detecting unreachable
objects is hard.)

Pip

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Helmut Eller" <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>> Anyway, it definitely seems to be the case that MPS is _not_ running GCs
>> concurrently, unless it would do things that I find highly unlikely.
>>
>> I find that a bit, let's say, disappointing, TBH :-(.
>
> Richard Brooksby thinks[*] that MPS could be concurrent with software
> barriers.  Feel like going down that road? :-)

I saw that, but it's from 2008, so I'm not sure whether things changed
after that.

Note that for typical Emacs usage, I'd look into making the
stop-the-world phase of GC interruptible rather than nonexistent.  MPS
has a lot of code to deal with failed scans, so we could find one of
those code paths that fails non-catastrophically and fail it.

That we didn't make the old GC interruptible still seems like a mistake
to me, but for igc it means that we'll be able to pass that off as a new
feature.  Kind of like purespace, the old code we compare again always
had a hand tied behind its back.  (Helmut very impressively demonstrated
that for purespace, so this assumes he doesn't get around to
implementing interruptible mark-and-sweep GC before breakfast).

Pip

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Helmut Eller <eller.helmut@gmail.com> writes:
>
>> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>>> Anyway, it definitely seems to be the case that MPS is _not_ running GCs
>>> concurrently, unless it would do things that I find highly unlikely.
>>>
>>> I find that a bit, let's say, disappointing, TBH :-(.
>>
>> Richard Brooksby thinks[*] that MPS could be concurrent with software
>> barriers.  Feel like going down that road? :-)
>>
>> Helmut
>>
>> [*] https://memory-pool-system.readthedocs.io/en/latest/design/shield.html#concurrent-collection
>
> Yep, found that too, with git grep.
>
> Still grumpy.
>
> I'm afraid Modifying MPS is not my thing, But What about using something
> more modern like Oilpan (aka cppgc) from V8? Can be used as a lib, is
> concurrent for real. That would also be a perfect time to lift Emacs to
> C++.

Ultimately, I'm still surprised there isn't that much work on precise GC
with compiler support, allawing us to mark the C stack precisely by
emitting DWARF tables indicating precisely which registers or stack
locations references live in at the time of an interruption.
(Identifying them is the easy part.  Emitting only code which allows you
to move such references asynchronously is the hard part).

But if you're doing all that, why not go all the way and implement
resumable exceptions?

(I wasn't going to mention it, but there's always my experiment with the
SpiderMonkey garbage collector.  Lots of bitrot, and I never got things
working quite the way I wanted to (no ambiguous references)).

Pip

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Helmut Eller <eller.helmut@gmail.com> writes:
>
>> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>>
>>> I'm afraid Modifying MPS is not my thing, But What about using something
>>> more modern like Oilpan (aka cppgc) from V8? Can be used as a lib, is
>>> concurrent for real.
>>
>> Ideally, Emacs would have an abstract GC interface so that different
>> implementations could be plugged in.
>
> That would indeed be nice to have.
>
>>> That would also be a perfect time to lift Emacs to
>>> C++.
>>
>> I'd rather see Emacs move to Rust.  Anyway, neither option seems
>> realistic.
>
> In mainline... (pondering to put a smiley).

I'll throw Zig in and run away quickly.

> But something else: Given what I now believe, I think I want to
> understand better (a bit) why everything appears to work just fine on
> macOS, with signals. Could you perhaps check if I'm off? MacOS only.

Do we know that?  I think macOS doesn't use signals as heavily as other
platforms do, and I don't know how SIGPROF is handled on that platform,
but I would not be surprised if that or SIGALRM require the signal
checking thing on macOS, too.

The macOS thing is equivalent to blocking signals in the SIGSEGV
handler.  I still think that's what MPS should have done.

> In normal operation, there are only ever 2 threads running. An Emacs
> thread is interrupted by a signal and lands in a signal handler, the
> MPS port thread keeps running.
>
> In the signal handler, hitting barriers is handled by the MPS port
> thread. Consistency of Emacs's state is a problem the signal handler has
> to deal with, consistency of MPS' GC data is a problem that hopefully
> MPS handles, and it seems to work.
>
> I think I understand that, except when the Emacs thread is interrupted
> while in MPS code, which happens for allocation points running out of
> memory and mps_arena_step (idle time).

My assumption is that if the signal handler is allowed to run in that
case, and tries to access MPS-managed memory, we deadlock.  It might not
be a detectable deadlock causing a crash, as it would be on POSIX, but
that makes things worse, not better.

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>
>>> My interpretation of this design document[*], is that MPS's arena lock
>>> protects most of MPS entry points.  There are a few (e.g. mps_reserve
>>> and mp_ld_add) that don't claim the arena lock and for those it's the
>>> burden of the client to call them in a thread safe way.  For us this
>>> probably means: don't call them in a signal handler.
>>>
>>> The main entry point that we want to call in the signal handler is the
>>> SEGFAULT handler (not sure how this works on MacOS).  The fault handler
>>> claims the non-recursive arena lock.  So, in the signal handler we
>>> should not hold the lock while hitting a barrier.
>>
>> Okay, that I think I understand then. The "only" difference between
>> macOS and Linux is that on macOS no SEGV handler is involved. Hitting
>> the barrier on macOS means that the EXC_BAD_ACCESS port thread, which
>> was waiting for Mach message, receives a message from the OS and starts
>> working.
>
> I guess that both, macOS and Linux version, will end up in ArenaAccess.
> Perhaps on macOS in the other thread.
>
>>>> I think I understand that, except when the Emacs thread is interrupted
>>>> while in MPS code, which happens for allocation points running out of
>>>> memory and mps_arena_step (idle time).
>>>
>>> Hmm, is that sentence incomplete?  I don't quite understand it.
>>
>> What I meant is that I imagine a signal interrupts the Emacs thread at a
>> point where we are "in MPS". AreaEnter/Leave I think I understand, it's
>> some pthread_mutex_t, I think, from other mails. A problematic "in MPS"
>> could then be "while the Emacs thread owns the mutex". The places where
>> I imagine that mutex could be owned are mps_arena_step (I call it Emacs
>> is idle) and mps_commit (in alloc_impl, when the allocation point used
>> runs out of memory). Maybe other places, but mainly these two.
>
> Yes.
>
>> And the question on macOS for me would be if the port thread tries to
>> qcquire the same mutex, or how the heck that works. Or IOW, if there is
>> a problem, why I've never seen it happening in all that time I'm using
>> igc.
>
> Maybe you could set a breakpoint in AreaAccess to find out which thread
> removes the barriers.
>
>> I find that difficult to understand. But it may be just a
>> statistical phenomenon. Maybe filling up an APs memory is so fast so
>> that the probability of a signal hitting while owning the mutex is close
>> to zero, or something.
>
> Very few of Emacs' signal handlers actually touch a barrier.  I've also
> not seen any reproducable receipes for the "signal issues" that the igc
> branch supposedly has.
>
> Helmut

I've got something, using Eglot/clangd.

Executive summary: If a signal interrupts the Emacs thread, and we are
"inside MPS", meaning the Emacs threads owns an arena's mutex, the macOS
port thread can try to acquire the same mutex and won't get because the
Emacs thread that owns it is stopped by the signal.

It goes like this:

  ** mps_commit

  mps_commit
  -> mps_ap_trip
     -> ArenaEnter

  ** ArenaEnter

  ArenaEnter
  -> ArenaEnterLock non-recursive
    -> LockClaim arena lock
      -> pthread_mutex_lock.
    -> ShieldEnter
      -> shieldQueueReset
         Sets some members of a Shield, .inside = true

  ** MPS port thread

  protCatchThread
  -> mach_msg waiting
  -> protCatchOne thread involved has been stopped by OS
    -> ArenaAccess
      -> arenaClaimRingLock
         -> LockClaimGlobal
           -> LockClaim globalLock (static LockStruct)
      -> RING_FOR uninteresting macrology
      -> ArenaEnter for the 1 arena we have

  ** Global lock (uninteresting here)

  LockInitGlobal
  -> LockInit globalLock
  -> LockInit globalLockRecLock
     -> pthread_mutex_init with no interesting attrs

Sp, I'd say

- It's possible to freeze on macOS because of hitting a barrier, or
  allocating Lisp objects. I don't believe that I would have overlooked
  something preventing it.

- It's apparently very unlikely to happen, for me at least it
  never happened in, don't know, half a year or so of daily usage.
  Maybe one could make it happen when profiling for long enough.

- Only signal handlers are affected, and what you said about signal
  handlers using Lisp and so on.

Hm.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Helmut Eller wrote:
>
>> My interpretation of this design document[*], is that MPS's arena lock
>
> That one:
>
> https://memory-pool-system.readthedocs.io/en/latest/design/thread-safety.html

Thanks.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>> Anyway, it definitely seems to be the case that MPS is _not_ running GCs
>> concurrently, unless it would do things that I find highly unlikely.
>>
>> I find that a bit, let's say, disappointing, TBH :-(.
>
> Well, I think that MPS is bring-your-own-thread concurrent.  I'm not
> sure the current MPS can usefully be concurrent, because of the thread
> suspension thing, but we can run it in another thread if we want to.
>
> (If it turns out that using a separate thread isn't an advantage, we
> should look at reducing the size of our roots (or protecting them)
> rather than giving up on the idea entirely.  As long as other threads
> freely move references to and from global roots, detecting unreachable
> objects is hard.)
>
> Pip

If they had said something, maybe I'd not tried to use MPS in Emacs. Or
maybe I would who knows.

Anyway, I must say that Emacs with igc feels a lot better and is as
stable. I don't think I'll ever go back :-).

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>> Anyway, it definitely seems to be the case that MPS is _not_ running GCs
>> concurrently, unless it would do things that I find highly unlikely.
>>
>> I find that a bit, let's say, disappointing, TBH :-(.
>
> Well, I think that MPS is bring-your-own-thread concurrent.  I'm not
> sure the current MPS can usefully be concurrent, because of the thread
> suspension thing, but we can run it in another thread if we want to.
>
> (If it turns out that using a separate thread isn't an advantage, we
> should look at reducing the size of our roots (or protecting them)
> rather than giving up on the idea entirely.  As long as other threads
> freely move references to and from global roots, detecting unreachable
> objects is hard.)

(BTW, just remembering. MPS has root protection in it's API, but it's
not implemented.)

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> (I wasn't going to mention it, but there's always my experiment with the
> SpiderMonkey garbage collector.  Lots of bitrot, and I never got things
> working quite the way I wanted to (no ambiguous references)).

I've noticed that, and also that Daniel C. did something. Maybe others
too. The MPS thing is actually just a coincidence. I wasn't trying to
deliberately ignore all previous effort to improve Emacs' GC. It was
that I wasn't interested in improving Emacs' GC. I was only trying to
learn something about MPS, and Emacs was only a test bed. I guess
sometimes things happen in weird ways :-).

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

>> In mainline... (pondering to put a smiley).
>
> I'll throw Zig in and run away quickly.

(Zig would be fine, weren't it for the future plans of its inventor,
which irritated me enough that I didn't try it in earnest.)

>
>> But something else: Given what I now believe, I think I want to
>> understand better (a bit) why everything appears to work just fine on
>> macOS, with signals. Could you perhaps check if I'm off? MacOS only.
>
> Do we know that?  

Well, that it "appears to..." I know for fact :-). It never happened for
me. But please see my other mails.

> I think macOS doesn't use signals as heavily as other platforms do,
> and I don't know how SIGPROF is handled on that platform, but I would
> not be surprised if that or SIGALRM require the signal checking thing
> on macOS, too.

SIGPROF and SIGLARM are signals in macOS. Only the hardware faults are
not, like EXC_BAD_ACCESS and the float exceptions. Others I don't
remember ATM.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Helmut Eller" <eller.helmut@gmail.com> writes:
>> I find that difficult to understand. But it may be just a
>> statistical phenomenon. Maybe filling up an APs memory is so fast so
>> that the probability of a signal hitting while owning the mutex is close
>> to zero, or something.
>
> Very few of Emacs' signal handlers actually touch a barrier.  I've also

Indeed.  These crashes are rare in typical usage, which doesn't mean we
should delay fixing them until Emacs is "unstable enough".  It already
is, IMHO, because we take that approach too frequently.

> not seen any reproducable receipes for the "signal issues" that the igc
> branch supposedly has.

Removing the SIGPROF protection code should allow Ihor's recipe to crash
again.  And, anyway, there's no reason for the "supposedly": we know MPS
can't possibly deal with the situation because we've seen the code, so
we should fix it rather than ignoring it because it's rare in typical
usage.

If there's any evidence this cannot happen, I haven't seen it.

GC code in general contains many rarely-exercised code paths.  This is
no exception.

Pip

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Pip Cet <pipcet@protonmail.com> writes:

> "Helmut Eller" <eller.helmut@gmail.com> writes:
>>> I find that difficult to understand. But it may be just a
>>> statistical phenomenon. Maybe filling up an APs memory is so fast so
>>> that the probability of a signal hitting while owning the mutex is close
>>> to zero, or something.
>>
>> Very few of Emacs' signal handlers actually touch a barrier.  I've also
>
> Indeed.  These crashes are rare in typical usage, which doesn't mean we
> should delay fixing them until Emacs is "unstable enough".  It already
> is, IMHO, because we take that approach too frequently.
>
>> not seen any reproducable receipes for the "signal issues" that the igc
>> branch supposedly has.
>
> Removing the SIGPROF protection code should allow Ihor's recipe to crash
> again.

Confirmed.  Here's the recipe (which, yes, you have already seen):

https://lists.gnu.org/archive/html/emacs-devel/2024-06/msg00560.html

Make igc_busy_p () return false (as we could do if the "supposed" signal
issue weren't real), immediate crash.

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

[-- Attachment #1: Type: text/plain, Size: 2406 bytes --]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Pip Cet <pipcet@protonmail.com> writes:
>
>> Speaking of running with a "normal" config: something about my
>> configuration makes buffer_step (the balance_intervals call, in
>> particular) take forever, to the point the mps build becomes unusable.
>> The buffer in question, when I caught it, is an M-x shell buffer of size
>> 8 MB, so I don't understand why it's taking so long.
>>
>> Still investigating, but skipping the buffer_step seems to help.
>
> balance_intervals means text properties. The only candidate I see in
> comint/shell is ANSI escapes. That could be turned on/off with M-x
> ansi-color-for-comint-mode-xy. Only as a workaround, and maybe to check
> if it's that.
>
> What I do in buffer_step in idle time is basically one step of what the
> old GC does in sweep_buffers.
>
> My expectation was that balancing a tree couldn't take long, and that
> this is not called often enough to be a problem if were expensive. Both
> wrong, as usual.
>
> Not calling balance_intervals is, BTW, not a catastrophic problem. if
> one does anything leading to a graft_intervals_into_buffer, w</r hich is
> called in a lot of places in editfns.c and insdel.c, that balances the
> tree. And if not, the tree might become slower for lookup (redisplay),
> but it still works.
>
> <rant> It's BTW well possible that I myself put that balancing into
> sweep_buffers because of redisplay, I seem to remember that. The
> interval tree has always been a source of fun. I hope, some day, some
> kind soul will eradicate it like the GCPROs. </rant>
>
> In any case, what's a solution?
>
> Right now I'm tending to put the balance_intervals in an if so that one
> can turn it on/off with a Lisp variable. Default would be to not to balance,
> because I think the problems with degenerated interval trees in
> redisplay where rare, and I don't remember problems outside of
> redisplay. But that was an awful long time ago, OTOH.
>
> That would give us more time to think about a possible strategy to solve
> this.
>
> WDYT?

Patch for that attached. I'm now running with that.

I tried to look at the history of intervals.c to see too which degree
the intervals tree is behaving better than decades ago (I think Stefan
Monnier said it's better), but I couldn't really determine that. Too
much has changed.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Make-balancing-buffer-intervals-optional.patch --]
[-- Type: text/x-patch, Size: 1315 bytes --]

From 0aa8b2f483da11bfd6a6397c56182b5877cb779e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gerd=20M=C3=B6llmann?= <gerd@gnu.org>
Date: Mon, 30 Dec 2024 13:41:40 +0100
Subject: [PATCH] Make balancing buffer intervals optional

* src/igc.c (buffer_step): Balance intervals only if
igc__balance_intervals is true.
(syms_of_igc): New DEFVAR_BOOL for igc__balance_intervals.
---
 src/igc.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/igc.c b/src/igc.c
index 39158b38f05..964723ce315 100644
--- a/src/igc.c
+++ b/src/igc.c
@@ -3697,7 +3697,8 @@ buffer_step (struct igc_buffer_it *it)
       buffer_it_next (it);
       struct buffer *b = XBUFFER (buf);
       compact_buffer (b);
-      b->text->intervals = balance_intervals (b->text->intervals);
+      if (igc__balance_intervals)
+	b->text->intervals = balance_intervals (b->text->intervals);
       return true;
     }
   return false;
@@ -5100,4 +5101,8 @@ syms_of_igc (void)
 don't do something when idle.  Negative values and values that are not numbers
 are handled as if they were the default value.  */);
   Vigc_step_interval = make_fixnum (0);
+
+  DEFVAR_BOOL ("igc--balance-intervals", igc__balance_intervals,
+     doc: /* Whether to balance buffer intervals when idle.  */);
+  igc__balance_intervals = false;
 }
-- 
2.47.1

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Pip Cet <pipcet@protonmail.com> writes:
>>
>>> Speaking of running with a "normal" config: something about my
>>> configuration makes buffer_step (the balance_intervals call, in
>>> particular) take forever, to the point the mps build becomes unusable.
>>> The buffer in question, when I caught it, is an M-x shell buffer of size
>>> 8 MB, so I don't understand why it's taking so long.
>>>
>>> Still investigating, but skipping the buffer_step seems to help.
>>
>> balance_intervals means text properties. The only candidate I see in
>> comint/shell is ANSI escapes. That could be turned on/off with M-x
>> ansi-color-for-comint-mode-xy. Only as a workaround, and maybe to check
>> if it's that.
>>
>> What I do in buffer_step in idle time is basically one step of what the
>> old GC does in sweep_buffers.
>>
>> My expectation was that balancing a tree couldn't take long, and that
>> this is not called often enough to be a problem if were expensive. Both
>> wrong, as usual.
>>
>> Not calling balance_intervals is, BTW, not a catastrophic problem. if
>> one does anything leading to a graft_intervals_into_buffer, w</r hich is
>> called in a lot of places in editfns.c and insdel.c, that balances the
>> tree. And if not, the tree might become slower for lookup (redisplay),
>> but it still works.
>>
>> <rant> It's BTW well possible that I myself put that balancing into
>> sweep_buffers because of redisplay, I seem to remember that. The
>> interval tree has always been a source of fun. I hope, some day, some
>> kind soul will eradicate it like the GCPROs. </rant>

(I have a crazy idea for that, too.  Code, too.  But it does away with
the gap buffer, which the regexp code assumes, so we end up creating a
shadow single-string buffer whenever we call into the regexp code, which
is, er, slow.)

>> In any case, what's a solution?
>>
>> Right now I'm tending to put the balance_intervals in an if so that one
>> can turn it on/off with a Lisp variable. Default would be to not to balance,
>> because I think the problems with degenerated interval trees in
>> redisplay where rare, and I don't remember problems outside of
>> redisplay. But that was an awful long time ago, OTOH.

I did implement a Lisp variable as well (defaulting it to on because I'm
more conservative than you are :-) ).  I still think it's more likely
it's (also) a bug elsewhere: balancing a tree for an 8 MB buffer should
not take long.

I am currently not calling compact_buffer when the variable is off.
Maybe that's something to look into, too.

In my current session, keypresses (with idle time in between, because I
can't outtype Emacs) become noticeably laggy if I set the variable to t,
but not when it's nil.

Further investigation needed, I think.  Unfortunately, that's going to
require some instrumentation code and then I have to restart my Emacs
session...

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> I did implement a Lisp variable as well (defaulting it to on because I'm
> more conservative than you are :-) ).  I still think it's more likely
> it's (also) a bug elsewhere: balancing a tree for an 8 MB buffer should
> not take long.

Could be, of course.

>
> I am currently not calling compact_buffer when the variable is off.
> Maybe that's something to look into, too.

I think compact_buffer must be done, otherwise the undo list isn't isn't
truncated. I've attached what I'm now using in my Emacs to another
mail.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> Pip Cet <pipcet@protonmail.com> writes:
>
>> "Helmut Eller" <eller.helmut@gmail.com> writes:
>>>> I find that difficult to understand. But it may be just a
>>>> statistical phenomenon. Maybe filling up an APs memory is so fast so
>>>> that the probability of a signal hitting while owning the mutex is close
>>>> to zero, or something.
>>>
>>> Very few of Emacs' signal handlers actually touch a barrier.  I've also
>>
>> Indeed.  These crashes are rare in typical usage, which doesn't mean we
>> should delay fixing them until Emacs is "unstable enough".  It already
>> is, IMHO, because we take that approach too frequently.
>>
>>> not seen any reproducable receipes for the "signal issues" that the igc
>>> branch supposedly has.
>>
>> Removing the SIGPROF protection code should allow Ihor's recipe to crash
>> again.
>
> Confirmed.  Here's the recipe (which, yes, you have already seen):
>
> https://lists.gnu.org/archive/html/emacs-devel/2024-06/msg00560.html
>
> Make igc_busy_p () return false (as we could do if the "supposed" signal
> issue weren't real), immediate crash.
>
> Pip

With

  modified   src/profiler.c
  @@ -347,7 +347,7 @@ record_backtrace (struct profiler_log *plog, EMACS_INT count)
   add_sample (struct profiler_log *plog, EMACS_INT count)
   {
   #ifdef HAVE_MPS
  -  if (igc_busy_p ())
  +  if (false)
   #else
     if (EQ (backtrace_top_function (), QAutomatic_GC)) /* bug#60237 */
   #endif

Result:

        1083  89% + command-execute
         106   8% + redisplay_internal (C function)
          13   1% + timer-event-handler
           1   0% + help--append-keystrokes-help
           1   0% + #<byte-code-function A81>
           0   0%   ...

🤷

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Pip Cet <pipcet@protonmail.com> writes:
>
>> Pip Cet <pipcet@protonmail.com> writes:
>>
>>> "Helmut Eller" <eller.helmut@gmail.com> writes:
>>>>> I find that difficult to understand. But it may be just a
>>>>> statistical phenomenon. Maybe filling up an APs memory is so fast so
>>>>> that the probability of a signal hitting while owning the mutex is close
>>>>> to zero, or something.
>>>>
>>>> Very few of Emacs' signal handlers actually touch a barrier.  I've also
>>>
>>> Indeed.  These crashes are rare in typical usage, which doesn't mean we
>>> should delay fixing them until Emacs is "unstable enough".  It already
>>> is, IMHO, because we take that approach too frequently.
>>>
>>>> not seen any reproducable receipes for the "signal issues" that the igc
>>>> branch supposedly has.
>>>
>>> Removing the SIGPROF protection code should allow Ihor's recipe to crash
>>> again.
>>
>> Confirmed.  Here's the recipe (which, yes, you have already seen):
>>
>> https://lists.gnu.org/archive/html/emacs-devel/2024-06/msg00560.html
>>
>> Make igc_busy_p () return false (as we could do if the "supposed" signal
>> issue weren't real), immediate crash.
>>
>> Pip
>
> With
>
>   modified   src/profiler.c
>   @@ -347,7 +347,7 @@ record_backtrace (struct profiler_log *plog, EMACS_INT count)
>    add_sample (struct profiler_log *plog, EMACS_INT count)
>    {
>    #ifdef HAVE_MPS
>   -  if (igc_busy_p ())
>   +  if (false)
>    #else
>      if (EQ (backtrace_top_function (), QAutomatic_GC)) /* bug#60237 */
>    #endif

This is after removing gc_signal_handler_can_run, right?

Even if, in addition, I block signals in the SIGSEGV handler, I see
crashes here, FWIW, but not quite every time that recipe is run.  It
seems to work even less reliably when rr is in use (and then I realized
I was running an optimized build so the trace was useless, sigh).

I'd still like to see at least one crash on macOS, but there's nothing
I'm aware of that would prevent such crashes, only make them (maybe
much) less likely.  For starters, fewer pages so fewer barriers :-)

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

>> With
>>
>>   modified   src/profiler.c
>>   @@ -347,7 +347,7 @@ record_backtrace (struct profiler_log *plog, EMACS_INT count)
>>    add_sample (struct profiler_log *plog, EMACS_INT count)
>>    {
>>    #ifdef HAVE_MPS
>>   -  if (igc_busy_p ())
>>   +  if (false)
>>    #else
>>      if (EQ (backtrace_top_function (), QAutomatic_GC)) /* bug#60237 */
>>    #endif
>
> This is after removing gc_signal_handler_can_run, right?

Right. And meanwhile it also survived the same parsing 100 times in a
loop with profiling on around it. 

> Even if, in addition, I block signals in the SIGSEGV handler, I see
> crashes here, FWIW, but not quite every time that recipe is run.  It
> seems to work even less reliably when rr is in use (and then I realized
> I was running an optimized build so the trace was useless, sigh).
>
> I'd still like to see at least one crash on macOS, but there's nothing
> I'm aware of that would prevent such crashes, only make them (maybe
> much) less likely.  For starters, fewer pages so fewer barriers :-)
>
> Pip

Me too, but I agree with you. It should eventually freeze, there's
nothing preventing it that I could point to.

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Mon, 30 Dec 2024 12:32:31 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, Eli Zaretskii <eliz@gnu.org>, spd@toadstyle.org, emacs-devel@gnu.org
> 
> Removing the SIGPROF protection code should allow Ihor's recipe to crash
> again.

At least some (if not all) Ihor's crashes were due to SIGCHLD, not
SIGPROF.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Pip Cet <pipcet@protonmail.com> writes:
>
>>> With
>>>
>>>   modified   src/profiler.c
>>>   @@ -347,7 +347,7 @@ record_backtrace (struct profiler_log *plog, EMACS_INT count)
>>>    add_sample (struct profiler_log *plog, EMACS_INT count)
>>>    {
>>>    #ifdef HAVE_MPS
>>>   -  if (igc_busy_p ())
>>>   +  if (false)
>>>    #else
>>>      if (EQ (backtrace_top_function (), QAutomatic_GC)) /* bug#60237 */
>>>    #endif
>>
>> This is after removing gc_signal_handler_can_run, right?
>
> Right. And meanwhile it also survived the same parsing 100 times in a
> loop with profiling on around it.

Yes, without the SIGSEGV-alloc-SIGSEGV case, it becomes less likely to
hit a memory barrier.  But the alloc-SIGPROF-SIGSEGV case is real, we've
seen backtraces for it.

>> Even if, in addition, I block signals in the SIGSEGV handler, I see
>> crashes here, FWIW, but not quite every time that recipe is run.  It
>> seems to work even less reliably when rr is in use (and then I realized
>> I was running an optimized build so the trace was useless, sigh).
>>
>> I'd still like to see at least one crash on macOS, but there's nothing
>> I'm aware of that would prevent such crashes, only make them (maybe
>> much) less likely.  For starters, fewer pages so fewer barriers :-)
>>
>> Pip
>
> Me too, but I agree with you. It should eventually freeze, there's
> nothing preventing it that I could point to.

Or produce invalid data, which seems more likely: if we're scanning the
segment, the memory barrier won't be in place, but the contents will be
invalid, pointing to objects which we already decided to move (IOW, if
my double-mapping idea were in place, it'd be easier to catch these
bugs).  A crash is better than derefing random pointers pointing to the
(new) young generation when the object has been moved.

(I'm running into separate rr bugs on both systems I'm testing this on,
so we'll just have to assume that's what behind those crashes, at least
sometimes).

Does adding mps_message_poll (global_igc->arena) in the signal handler
produce a crash/deadlock for you?  Last thing I'm asking you to try
today, I promise :-)

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> Does adding mps_message_poll (global_igc->arena) in the signal handler
> produce a crash/deadlock for you?  Last thing I'm asking you to try
> today, I promise :-)

:-). When I do this:

1 file changed, 1 insertion(+)
src/igc.c | 1 +

modified   src/igc.c
@@ -4966,6 +4966,7 @@ igc_alloc_dump (size_t nbytes)
 bool
 igc_busy_p (void)
 {
+  mps_message_poll (global_igc->arena);
   return mps_arena_busy (global_igc->arena);
 }
 

with no changes, except the 1 commit reverted, I get an assertion
a short time after M-x profiler-start

Fatal error 6: Aborted
Backtrace:
0   emacs                               0x0000000104fceca8 emacs_backtrace + 104
1   emacs                               0x000000010515b31c terminate_due_to_signal + 220
2   emacs                               0x00000001050c090c syms_of_igc + 0
3   emacs                               0x00000001051081a4 LockClaim + 144
4   emacs                               0x0000000105107f78 ArenaEnterLock + 96
5   emacs                               0x00000001050fa380 mps_message_poll + 24
6   emacs                               0x00000001050c008c igc_busy_p + 28
7   emacs                               0x00000001050b8a54 add_sample + 44
8   emacs                               0x0000000104fce6fc deliver_process_signal + 64
9   libsystem_platform.dylib            0x0000000190c56e04 _sigtramp + 56
10  libsystem_c.dylib                   0x0000000190ad9ba4 clock + 52
11  emacs                               0x00000001050f4c10 ArenaPoll + 164
12  emacs                               0x00000001050f5b68 mps_ap_fill + 160
13  emacs                               0x00000001050c0f24 alloc_impl + 128
14  emacs                               0x00000001050bf290 igc_add_marker + 136
15  emacs                               0x0000000104fe21cc set_marker_internal + 576
16  emacs                               0x0000000104fe2700 Fcopy_marker + 180
17  emacs                               0x0000000104f558b0 save_window_save + 748
18  emacs                               0x0000000104f555ac Fcurrent_window_configuration + 296
19  emacs                               0x0000000104fe3bfc Fread_from_minibuffer + 1044
20  minibuffer-1b0f548b-1c53371b.eln    0x0000000106a680f4
F636f6d706c6574696e672d726561642d64656661
...

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  spd@toadstyle.org,
>   pipcet@protonmail.com,  emacs-devel@gnu.org
> Date: Mon, 30 Dec 2024 12:53:33 +0100
> 
> Executive summary: If a signal interrupts the Emacs thread, and we are
> "inside MPS", meaning the Emacs threads owns an arena's mutex, the macOS
> port thread can try to acquire the same mutex and won't get because the
> Emacs thread that owns it is stopped by the signal.

Are you sure the "because the Emacs thread that owns it is stopped by
the signal" part is correct?  AFAIU, since the macOS port thread is a
different thread, it tries to take the arena lock, and is simply stuck
there, waiting for the main thread to release the lock.  IOW, this is
a simple mutex-based synchronization between two threads, that's all:
one thread takes the lock, the other must wait until the first one
releases it before it itself can take the lock.

Moreover, the main thread is not stopped, it runs the signal handler.
Right?

Or what am I missing?

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Pip Cet wrote:

> "Helmut Eller" <eller.helmut@gmail.com> writes:
>> Very few of Emacs' signal handlers actually touch a barrier.  I've also
>
> Indeed.  These crashes are rare in typical usage, which doesn't mean we
> should delay fixing them until Emacs is "unstable enough".  It already
> is, IMHO, because we take that approach too frequently.
>
>> not seen any reproducable receipes for the "signal issues" that the igc
>> branch supposedly has.
>
> Removing the SIGPROF protection code should allow Ihor's recipe to crash
> again.

Talking about SIGPROF protection code.  It appears to me now (again)
that, for the SIGPROF handler, this pseudo code

  if (mps_arena_busy (<arena>))
    plog->gc_count = saturated_add (plog->gc_count, count);
  else
    record_backtrace (plog, count);

is safe.  If we don't hold the lock, then mps_arena_busy returns false
and we can access memory.  We are safe even if another thread has
claimed the lock by the time that we reach record_backtrace: the SIGPROF
handler will just block until the lock is released.

Does somebody disagree?

Helmut

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  spd@toadstyle.org,
>>   pipcet@protonmail.com,  emacs-devel@gnu.org
>> Date: Mon, 30 Dec 2024 12:53:33 +0100
>> 
>> Executive summary: If a signal interrupts the Emacs thread, and we are
>> "inside MPS", meaning the Emacs threads owns an arena's mutex, the macOS
>> port thread can try to acquire the same mutex and won't get because the
>> Emacs thread that owns it is stopped by the signal.
>
> Are you sure the "because the Emacs thread that owns it is stopped by
> the signal" part is correct?  

It's not correct, sorry. When it lands in the port thread, the Emacs
thread has been suspended by the OS.

> AFAIU, since the macOS port thread is a different thread, it tries to
> take the arena lock, and is simply stuck there, waiting for the main
> thread to release the lock. IOW, this is a simple mutex-based
> synchronization between two threads, that's all: one thread takes the
> lock, the other must wait until the first one releases it before it
> itself can take the lock.

That's right. The port thread can't get the lock because it's owned by
the Emacs thread. And the Emacs thread is suspended by the OS.

>
> Moreover, the main thread is not stopped, it runs the signal handler.
> Right?

The Emacs thread is suspended by the OS to let the port thread handle the
situation. After the thread is suspended, the OS sends the port a
message. The port thread is waiting for such a message with mach_msg.
The port does its thing, and when it replies to the OS message
accordingly, the OS lets the Emacs thread continue. But that reply is
never sent because the port is stuck on the mutex.

Hope that's more correct.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  spd@toadstyle.org,
>>   pipcet@protonmail.com,  emacs-devel@gnu.org
>> Date: Mon, 30 Dec 2024 12:53:33 +0100
>>
>> Executive summary: If a signal interrupts the Emacs thread, and we are
>> "inside MPS", meaning the Emacs threads owns an arena's mutex, the macOS
>> port thread can try to acquire the same mutex and won't get because the
>> Emacs thread that owns it is stopped by the signal.
>
> Are you sure the "because the Emacs thread that owns it is stopped by
> the signal" part is correct?  AFAIU, since the macOS port thread is a
> different thread, it tries to take the arena lock, and is simply stuck
> there, waiting for the main thread to release the lock.  IOW, this is
> a simple mutex-based synchronization between two threads, that's all:
> one thread takes the lock, the other must wait until the first one
> releases it before it itself can take the lock.

My understanding is that the main thread cannot continue running while
it's waiting for a memory barrier to be removed.  What would it use for
the memory values it cannot access?

> Moreover, the main thread is not stopped, it runs the signal handler.

It is stopped, in the signal handler, waiting for the memory barrier to
be removed.

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,  Eli
>  Zaretskii <eliz@gnu.org>,
>   spd@toadstyle.org,  emacs-devel@gnu.org
> Date: Mon, 30 Dec 2024 15:59:08 +0100
> 
>   if (mps_arena_busy (<arena>))
>     plog->gc_count = saturated_add (plog->gc_count, count);
>   else
>     record_backtrace (plog, count);
> 
> is safe.  If we don't hold the lock, then mps_arena_busy returns false
> and we can access memory.  We are safe even if another thread has
> claimed the lock by the time that we reach record_backtrace: the SIGPROF
> handler will just block until the lock is released.

Which other thread could have claimed the lock?

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Eli Zaretskii wrote:

>> From: Helmut Eller <eller.helmut@gmail.com>
>> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,  Eli
>>  Zaretskii <eliz@gnu.org>,
>>   spd@toadstyle.org,  emacs-devel@gnu.org
>> Date: Mon, 30 Dec 2024 15:59:08 +0100
>> 
>>   if (mps_arena_busy (<arena>))
>>     plog->gc_count = saturated_add (plog->gc_count, count);
>>   else
>>     record_backtrace (plog, count);
>> 
>> is safe.  If we don't hold the lock, then mps_arena_busy returns false
>> and we can access memory.  We are safe even if another thread has
>> claimed the lock by the time that we reach record_backtrace: the SIGPROF
>> handler will just block until the lock is released.
>
> Which other thread could have claimed the lock?

Any other registered thread that we might have, e.g. the
signal_receiver_thread from Pip's proposal.

Helmut

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Helmut Eller" <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Pip Cet wrote:
>
>> "Helmut Eller" <eller.helmut@gmail.com> writes:
>>> Very few of Emacs' signal handlers actually touch a barrier.  I've also
>>
>> Indeed.  These crashes are rare in typical usage, which doesn't mean we
>> should delay fixing them until Emacs is "unstable enough".  It already
>> is, IMHO, because we take that approach too frequently.
>>
>>> not seen any reproducable receipes for the "signal issues" that the igc
>>> branch supposedly has.
>>
>> Removing the SIGPROF protection code should allow Ihor's recipe to crash
>> again.
>
> Talking about SIGPROF protection code.  It appears to me now (again)
> that, for the SIGPROF handler, this pseudo code
>
>   if (mps_arena_busy (<arena>))
>     plog->gc_count = saturated_add (plog->gc_count, count);
>   else
>     record_backtrace (plog, count);
>
> is safe.

Yes, it's safe, because it does have protection code.  The question was
the extent to which this protection code is required, and whether we can
find another way to deliver signals which doesn't require it.

IMHO, we now have three solutions that are still in the running (my
order of preference):

1. keep the current code and special-case some signals which are needed
for user responsiveness
2. use the signal serialization thread you proposed
3. use an allocation thread, but keep SIGSEGV on the main thread

The first two can be combined with blocking other signals in the SIGSEGV
handler (which would make all platforms behave the same and avoid
SIGSEGV-handler-SIGSEGV races).  The third requires it.

I'd like to take (3) out of the picture for now.  It's working here
(still forwarding SIGSEGV, but that's not the point), and performance
seems okay (better, for some unknown reason; maybe it's just chunk
size), but I couldn't make it behave reproducibly when run under rr, and
it makes a "GC is rare" assumption when it splits MPS objects.  I'd like
to be able to use rr, and "rare GC" assumptions mean that further
improvements to or even fine-tuning of MPS would have to happen
differently.

IOW, MPS needs hacking to make an allocation thread truly viable, mostly
to distinguish fast-path and slow-path allocations.  Not a major change,
I hope, but also out of scope for getting things to work with upstream
MPS, which remains my goal.

I think improving (1) is most likely to do that (but that would require
a shadow signal mask, most likely).

> If we don't hold the lock, then mps_arena_busy returns false
> and we can access memory.  We are safe even if another thread has
> claimed the lock by the time that we reach record_backtrace: the SIGPROF
> handler will just block until the lock is released.
>
> Does somebody disagree?

Not me.

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Pip Cet wrote:
>
>> "Helmut Eller" <eller.helmut@gmail.com> writes:
>>> Very few of Emacs' signal handlers actually touch a barrier.  I've also
>>
>> Indeed.  These crashes are rare in typical usage, which doesn't mean we
>> should delay fixing them until Emacs is "unstable enough".  It already
>> is, IMHO, because we take that approach too frequently.
>>
>>> not seen any reproducable receipes for the "signal issues" that the igc
>>> branch supposedly has.
>>
>> Removing the SIGPROF protection code should allow Ihor's recipe to crash
>> again.
>
> Talking about SIGPROF protection code.  It appears to me now (again)
> that, for the SIGPROF handler, this pseudo code
>
>   if (mps_arena_busy (<arena>))
>     plog->gc_count = saturated_add (plog->gc_count, count);
>   else
>     record_backtrace (plog, count);
>
> is safe.  If we don't hold the lock, then mps_arena_busy returns false
> and we can access memory.  We are safe even if another thread has
> claimed the lock by the time that we reach record_backtrace: the SIGPROF
> handler will just block until the lock is released.
>
> Does somebody disagree?
>
> Helmut

That's this:

mps_bool_t mps_arena_busy(mps_arena_t arena)
{
  /* Don't call ArenaEnter -- the purpose of this function is to
   * determine if the arena lock is held */
  AVER(TESTT(Arena, arena));
  return ArenaBusy(arena);
}

Bool ArenaBusy(Arena arena)
{
  return LockIsHeld(ArenaGlobals(arena)->lock);
}

Bool (LockIsHeld)(Lock lock)
{
  AVERT(Lock, lock);
  if (pthread_mutex_trylock(&lock->mut) == 0) {
    Bool claimed = lock->claims > 0;
    int res = pthread_mutex_unlock(&lock->mut);
    AVER(res == 0);
    return claimed;
  }
  return TRUE;
}

There might be a small window after pthread_mutex_trylock and being back
in the signal handler. Can anything happen in this window?

If no other Emacs threads are running, and the Emacs thread is in the
signal handler, we can trust the "false" from the mps_arena_busy.

If other threads were running (and I don't think that's currently the case),
the "false" also means that the Emacs thread in the signal handler
can't have the lock.

In the "true" case from mps_arena_busy, we're anyway not doing much.

So I think I agree.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

>> Talking about SIGPROF protection code.  It appears to me now (again)
>> that, for the SIGPROF handler, this pseudo code
>>
>>   if (mps_arena_busy (<arena>))
>>     plog->gc_count = saturated_add (plog->gc_count, count);
>>   else
>>     record_backtrace (plog, count);
>>
>> is safe.
>
> Yes, it's safe, because it does have protection code.  
> The question was the extent to which this protection code is required,
> and whether we can find another way to deliver signals which doesn't
> require it.
>

What do you mean by protection code? Is that the commit I reverted?

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Gerd Möllmann wrote:

> Bool (LockIsHeld)(Lock lock)
> {
>   AVERT(Lock, lock);
>   if (pthread_mutex_trylock(&lock->mut) == 0) {
>     Bool claimed = lock->claims > 0;
>     int res = pthread_mutex_unlock(&lock->mut);
>     AVER(res == 0);
>     return claimed;
>   }
>   return TRUE;
> }
>
> There might be a small window after pthread_mutex_trylock and being back
> in the signal handler. Can anything happen in this window?
>
> If no other Emacs threads are running, and the Emacs thread is in the
> signal handler, we can trust the "false" from the mps_arena_busy.

Theoretically, a signal handler could interrupt the Emacs thread and
lock the mutex without unlocking it.  That would be a very unusual
signal handler.  I hope no other surprises happen in signal handlers.

Helmut

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>
>> Bool (LockIsHeld)(Lock lock)
>> {
>>   AVERT(Lock, lock);
>>   if (pthread_mutex_trylock(&lock->mut) == 0) {
>>     Bool claimed = lock->claims > 0;
>>     int res = pthread_mutex_unlock(&lock->mut);
>>     AVER(res == 0);
>>     return claimed;
>>   }
>>   return TRUE;
>> }
>>
>> There might be a small window after pthread_mutex_trylock and being back
>> in the signal handler. Can anything happen in this window?
>>
>> If no other Emacs threads are running, and the Emacs thread is in the
>> signal handler, we can trust the "false" from the mps_arena_busy.
>
> Theoretically, a signal handler could interrupt the Emacs thread and
> lock the mutex without unlocking it.  That would be a very unusual
> signal handler.  I hope no other surprises happen in signal handlers.
>
> Helmut

Right, that one I forgot. A nested signal may interrupt the signal
handler and acquire the lock in its signal handler but not release it.
The effect would be that the original signal handler would see a false
from mps_arena_busy which would not the truth.

I'd call that a bug in the nested signal handler.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Helmut Eller" <eller.helmut@gmail.com> writes:

> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>
>> Bool (LockIsHeld)(Lock lock)
>> {
>>   AVERT(Lock, lock);
>>   if (pthread_mutex_trylock(&lock->mut) == 0) {
>>     Bool claimed = lock->claims > 0;
>>     int res = pthread_mutex_unlock(&lock->mut);
>>     AVER(res == 0);
>>     return claimed;
>>   }
>>   return TRUE;
>> }
>>
>> There might be a small window after pthread_mutex_trylock and being back
>> in the signal handler. Can anything happen in this window?
>>
>> If no other Emacs threads are running, and the Emacs thread is in the
>> signal handler, we can trust the "false" from the mps_arena_busy.
>
> Theoretically, a signal handler could interrupt the Emacs thread and
> lock the mutex without unlocking it.

I don't think that's a problem.  Here's why:

We'd have to call the POSIX police.  I believe it's a conscious POSIX
decision not to allow hand-over of locks from one thread/signal handler
(those can't even call _trylock) to another; this is relevant to the
priority inversion scenario (if we had a "background" GC thread running
at a lower priority (whatever that would mean?  E-core?  Different
power-performance prefs?  Throttled?), the main thread would have to
find a way to boost its priority (move it to a P-core, unthrottle,
whatever) if we're actually waiting for it to release the arena lock.
One way would be to take over its lock (easy) and stack (hard) while
retaining thread settings, but POSIX decided we don't want to do that.
Thank you, POSIX (in this case)).

On inhomogeneous systems (almost everything you can buy today, ESP32 to
server CPU), "priority inversion" can happen with just two threads,
since priority is no longer defined by access to a single or several
identical cores.

But anyway, POSIX prohibits it, glibc on GNU/Linux doesn't support it,
I'm not aware of any other systems making that useful, certainly not for
Emacs.

> That would be a very unusual signal handler.  I hope no other surprises happen in signal handlers.

longjmp-based green threads?  (MPS currently assumes a simple linear
stack, gcc can produce split-stack code, getting that combination to
work would be good; I can dig up the patch for enabling it for the old
GC).

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: Pip Cet <pipcet@protonmail.com>,  Eli Zaretskii <eliz@gnu.org>,
>   spd@toadstyle.org,  emacs-devel@gnu.org
> Date: Mon, 30 Dec 2024 17:57:02 +0100
> 
> On Mon, Dec 30 2024, Gerd Möllmann wrote:
> 
> > Bool (LockIsHeld)(Lock lock)
> > {
> >   AVERT(Lock, lock);
> >   if (pthread_mutex_trylock(&lock->mut) == 0) {
> >     Bool claimed = lock->claims > 0;
> >     int res = pthread_mutex_unlock(&lock->mut);
> >     AVER(res == 0);
> >     return claimed;
> >   }
> >   return TRUE;
> > }
> >
> > There might be a small window after pthread_mutex_trylock and being back
> > in the signal handler. Can anything happen in this window?
> >
> > If no other Emacs threads are running, and the Emacs thread is in the
> > signal handler, we can trust the "false" from the mps_arena_busy.
> 
> Theoretically, a signal handler could interrupt the Emacs thread and
> lock the mutex without unlocking it.  That would be a very unusual
> signal handler.  I hope no other surprises happen in signal handlers.

We should keep our signal handlers very simple and safe, that's true.
It is not very hard, and much of that work was already done, when we
stopped running complex stuff in SIGIO handler etc.

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Pip Cet wrote:

>> Theoretically, a signal handler could interrupt the Emacs thread and
>> lock the mutex without unlocking it.
>
> I don't think that's a problem.  Here's why:
>
> We'd have to call the POSIX police.  I believe it's a conscious POSIX
> decision not to allow hand-over of locks from one thread/signal handler
> (those can't even call _trylock) to another;

Do you mean, it is not allowed to call pthread_mutex_trylock in a signal
handler?

[...]
> But anyway, POSIX prohibits it, glibc on GNU/Linux doesn't support it,
> I'm not aware of any other systems making that useful, certainly not for
> Emacs.
>
>> That would be a very unusual signal handler.  I hope no other
>> surprises happen in signal handlers.
>
> longjmp-based green threads?  (MPS currently assumes a simple linear
> stack, gcc can produce split-stack code, getting that combination to
> work would be good; I can dig up the patch for enabling it for the old
> GC).

I think MPS would require a special thread module that can handle green
threads.  Probably nothing we have to worry about.

Helmut

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Helmut Eller <eller.helmut@gmail.com>
>> Cc: Pip Cet <pipcet@protonmail.com>,  Eli Zaretskii <eliz@gnu.org>,
>>   spd@toadstyle.org,  emacs-devel@gnu.org
>> Date: Mon, 30 Dec 2024 17:57:02 +0100
>> 
>> On Mon, Dec 30 2024, Gerd Möllmann wrote:
>> 
>> > Bool (LockIsHeld)(Lock lock)
>> > {
>> >   AVERT(Lock, lock);
>> >   if (pthread_mutex_trylock(&lock->mut) == 0) {
>> >     Bool claimed = lock->claims > 0;
>> >     int res = pthread_mutex_unlock(&lock->mut);
>> >     AVER(res == 0);
>> >     return claimed;
>> >   }
>> >   return TRUE;
>> > }
>> >
>> > There might be a small window after pthread_mutex_trylock and being back
>> > in the signal handler. Can anything happen in this window?
>> >
>> > If no other Emacs threads are running, and the Emacs thread is in the
>> > signal handler, we can trust the "false" from the mps_arena_busy.
>> 
>> Theoretically, a signal handler could interrupt the Emacs thread and
>> lock the mutex without unlocking it.  That would be a very unusual
>> signal handler.  I hope no other surprises happen in signal handlers.
>
> We should keep our signal handlers very simple and safe, that's true.
> It is not very hard, and much of that work was already done, when we
> stopped running complex stuff in SIGIO handler etc.

So, to summarize, everyone agrees with Helmut? 

Re: igc, macOS avoiding signals

[Helmut Eller]

On Mon, Dec 30 2024, Pip Cet wrote:

> IMHO, we now have three solutions that are still in the running (my
> order of preference):
>
> 1. keep the current code and special-case some signals which are needed
> for user responsiveness
> 2. use the signal serialization thread you proposed
> 3. use an allocation thread, but keep SIGSEGV on the main thread

I think this is missing:

4. add callbacks to ArenaEnter/ArenaLeave to block/unblock signals

Perhaps add 5. (or make it a variant of 1)

5. special-case some performance critical handlers and simplify all
   others so that they are obviously harmless.

   The SIGIO handler is an example for a harmless signal handler.
   handle_alarm_signal seems harmless too.

   handle_interrupt_signal is definitely not harmless, but may not be
   preformance critical.

   So far SIGPROF seems to be the only performance cortical handler.
   
Helmut

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Helmut Eller <eller.helmut@gmail.com>,  pipcet@protonmail.com,
>   spd@toadstyle.org,  emacs-devel@gnu.org
> Date: Mon, 30 Dec 2024 19:37:38 +0100
> 
> So, to summarize, everyone agrees with Helmut? 

That the SIGPROF handler in the form he described would be safe?  I
agree.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Helmut Eller <eller.helmut@gmail.com>,  pipcet@protonmail.com,
>>   spd@toadstyle.org,  emacs-devel@gnu.org
>> Date: Mon, 30 Dec 2024 19:37:38 +0100
>> 
>> So, to summarize, everyone agrees with Helmut? 
>
> That the SIGPROF handler in the form he described would be safe?  I
> agree.

What we have in scratch/igc:

static void
handle_profiler_signal (int signal)
{
  EMACS_INT count = 1;
#if defined HAVE_ITIMERSPEC && defined HAVE_TIMER_GETOVERRUN
  if (profiler_timer_ok)
    {
      int overruns = timer_getoverrun (profiler_timer);
      eassert (overruns >= 0);
      count += overruns;
    }
#endif
  add_sample (&cpu, count);
}

static void
add_sample (struct profiler_log *plog, EMACS_INT count)
{
#ifdef HAVE_MPS
  if (igc_busy_p ())
#else
  if (EQ (backtrace_top_function (), QAutomatic_GC)) /* bug#60237 */
#endif
    /* Special case the time-count inside GC because the hash-table
       code is not prepared to be used while the GC is running.
       More specifically it uses ASIZE at many places where it does
       not expect the ARRAY_MARK_FLAG to be set.  We could try and
       harden the hash-table code, but it doesn't seem worth the
       effort.  */
    plog->gc_count = saturated_add (plog->gc_count, count);
  else
    record_backtrace (plog, count);
}

bool
igc_busy_p (void)
{
  return mps_arena_busy (global_igc->arena);
}

Now the question is if that's what Helmut was describing.