Re: igc, macOS avoiding signals

Re: igc, macOS avoiding signals

[Sean Devlin]

> Pip Cet <pipcet@protonmail.com> writes:
> 
> > Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> >
> >> This is about commit
> >>
> >> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
> >> Avoid MPS being interrupted by signals
> >>
> >> I've been running with this for some days now, and must report that
> >> Emacs feels a _bit_ different here in interactive use, maybe one could
> >> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
> >> very recent master++).
> >
> > I think we should quantify that.  Set a watchpoint on
> > igc_global->signals_pending, check how often we even set that, and how
> > often we call igc_maybe_quit, particularly if we were previously idle.
> > Maybe it's sufficient to call it again from the idle handler.
> >
> >> After reverting the commit, it's feeling smoother again.
> >
> > Entirely possible.  Let's measure it.
> 
> I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
> else using macOS can do that. I just wanted to make you aware of this.

I can take a stab at this. I’ve been running the scratch/igc branch on macOS using the NS build.

To be clear, I should make sure to include the above commit in my build? Or should I not include it?

When I take my measurements, are there any kinds of tasks I should try to perform? What durations of time should I try to measure?

Cheers.

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Sean Devlin <spd@toadstyle.org> writes:

>> Pip Cet <pipcet@protonmail.com> writes:
>> 
>> > Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>> >
>> >> This is about commit
>> >>
>> >> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
>> >> Avoid MPS being interrupted by signals
>> >>
>> >> I've been running with this for some days now, and must report that
>> >> Emacs feels a _bit_ different here in interactive use, maybe one could
>> >> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
>> >> very recent master++).
>> >
>> > I think we should quantify that.  Set a watchpoint on
>> > igc_global->signals_pending, check how often we even set that, and how
>> > often we call igc_maybe_quit, particularly if we were previously idle.
>> > Maybe it's sufficient to call it again from the idle handler.
>> >
>> >> After reverting the commit, it's feeling smoother again.
>> >
>> > Entirely possible.  Let's measure it.
>> 
>> I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
>> else using macOS can do that. I just wanted to make you aware of this.
>
> I can take a stab at this. I’ve been running the scratch/igc branch on
> macOS using the NS build.
>
> To be clear, I should make sure to include the above commit in my
> build? Or should I not include it?

Just let in in. I reverted it only in my Emacs.

> When I take my measurements, are there any kinds of tasks I should try
> to perform? What durations of time should I try to measure?
>
> Cheers.

That's something for Pip to tell.

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: pipcet@protonmail.com,  emacs-devel@gnu.org
> Date: Sat, 28 Dec 2024 14:28:57 +0100
> 
> Sean Devlin <spd@toadstyle.org> writes:
> 
> > To be clear, I should make sure to include the above commit in my
> > build? Or should I not include it?
> 
> Just let in in. I reverted it only in my Emacs.

Can you show that MPS runs GC from a non-main thread?  Like, if you
set a breakpoint in ArenaEnter, does it ever break on a non-main
thread?

I've just reviewed all the backtraces we had in our MPS discussions,
and all I see there is MPS being triggered either from igc_on_idle or
from alloc_impl, but every single time it happens on the main thread,
the one where I see the main function that enters recursive-edit.

So maybe Pip is right, and MPS always runs in the main (Lisp) thread,
even on macOS?  Can you catch it on a non-main thread?

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> So maybe Pip is right, and MPS always runs in the main (Lisp) thread,
> even on macOS?  Can you catch it on a non-main thread?

It's well possible that I misunderstand what the MPS guide says about it
being concurrent (see my reply to Pip), and that the thread I see here
is something else.

If you don't see an additional thread on Linux, just don't listen to me
and do what you think is TRT. I don't know anything about MPS internals.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Sean Devlin" <spd@toadstyle.org> writes:

>> Pip Cet <pipcet@protonmail.com> writes:
>>
>> > Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>> >
>> >> This is about commit
>> >>
>> >> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
>> >> Avoid MPS being interrupted by signals
>> >>
>> >> I've been running with this for some days now, and must report that
>> >> Emacs feels a _bit_ different here in interactive use, maybe one could
>> >> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
>> >> very recent master++).
>> >
>> > I think we should quantify that.  Set a watchpoint on
>> > igc_global->signals_pending, check how often we even set that, and how
>> > often we call igc_maybe_quit, particularly if we were previously idle.
>> > Maybe it's sufficient to call it again from the idle handler.
>> >
>> >> After reverting the commit, it's feeling smoother again.
>> >
>> > Entirely possible.  Let's measure it.
>>
>> I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
>> else using macOS can do that. I just wanted to make you aware of this.
>
> I can take a stab at this. I’ve been running the scratch/igc branch on macOS using the NS build.

Thanks!

> To be clear, I should make sure to include the above commit in my build? Or should I not include it?

I think we probably need to put instrumentation in the source code, so
we gain some idea of how long signals are delayed for when we mark them
pending.

(I'm also noticing that igc_maybe_quit isn't called as much as I thought
it would be.  Maybe we need to call it explicitly when Emacs becomes
idle?)

I'll try to come up with a patch.

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 15:12:23 +0000
> Cc: gerd.moellmann@gmail.com, emacs-devel@gnu.org
> From:  Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>
> 
> I think we probably need to put instrumentation in the source code, so
> we gain some idea of how long signals are delayed for when we mark them
> pending.

What do we expect to learn from this, except the timing of the OS
scheduler?

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> Date: Sat, 28 Dec 2024 15:12:23 +0000
>> Cc: gerd.moellmann@gmail.com, emacs-devel@gnu.org
>> From:  Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>
>>
>> I think we probably need to put instrumentation in the source code, so
>> we gain some idea of how long signals are delayed for when we mark them
>> pending.
>
> What do we expect to learn from this,

It tests the current code, which does this:

When a signal arrives, and we can't handle it because we might have
interrupted MPS, we mark the signal as pending in the igc structure.  At
some point later, we check the igc structure for pending signals,
reraise them, and unmask them.

Gerd's experience suggests that the "some point later" happens too late.
This patch gives us measurements.

It's unrelated to the OS scheduler, AFAICS.

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 18:40:30 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> >> Date: Sat, 28 Dec 2024 15:12:23 +0000
> >> Cc: gerd.moellmann@gmail.com, emacs-devel@gnu.org
> >> From:  Pip Cet via "Emacs development discussions." <emacs-devel@gnu.org>
> >>
> >> I think we probably need to put instrumentation in the source code, so
> >> we gain some idea of how long signals are delayed for when we mark them
> >> pending.
> >
> > What do we expect to learn from this,
> 
> It tests the current code, which does this:
> 
> When a signal arrives, and we can't handle it because we might have
> interrupted MPS, we mark the signal as pending in the igc structure.  At
> some point later, we check the igc structure for pending signals,
> reraise them, and unmask them.
> 
> Gerd's experience suggests that the "some point later" happens too late.
> This patch gives us measurements.
> 
> It's unrelated to the OS scheduler, AFAICS.

Ah, okay.  I note that if we'd block signals when calling MPS and
unblock on exit, then these delays couldn't have happened, AFAIU.

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 20:50:22 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> > > What do we expect to learn from this,
> > 
> > It tests the current code, which does this:
> > 
> > When a signal arrives, and we can't handle it because we might have
> > interrupted MPS, we mark the signal as pending in the igc structure.  At
> > some point later, we check the igc structure for pending signals,
> > reraise them, and unmask them.
> > 
> > Gerd's experience suggests that the "some point later" happens too late.
> > This patch gives us measurements.
> > 
> > It's unrelated to the OS scheduler, AFAICS.
> 
> Ah, okay.  I note that if we'd block signals when calling MPS and
> unblock on exit, then these delays couldn't have happened, AFAIU.

But OTOH, if this delaying of a signal affects responsiveness, then
all we need to do is exempt SIGSEGV from being delayed, right?  This
signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
but there's no reason to delay SIGSEGV.

And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
does this delaying affect responsiveness?

Or what am I missing?

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> Date: Sat, 28 Dec 2024 20:50:22 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
>>
>> > > What do we expect to learn from this,
>> >
>> > It tests the current code, which does this:
>> >
>> > When a signal arrives, and we can't handle it because we might have
>> > interrupted MPS, we mark the signal as pending in the igc structure.  At
>> > some point later, we check the igc structure for pending signals,
>> > reraise them, and unmask them.
>> >
>> > Gerd's experience suggests that the "some point later" happens too late.
>> > This patch gives us measurements.
>> >
>> > It's unrelated to the OS scheduler, AFAICS.
>>
>> Ah, okay.  I note that if we'd block signals when calling MPS and
>> unblock on exit, then these delays couldn't have happened, AFAIU.
>
> But OTOH, if this delaying of a signal affects responsiveness, then
> all we need to do is exempt SIGSEGV from being delayed, right?  This
> signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
> but there's no reason to delay SIGSEGV.

SIGSEGV is never delayed in any proposal I'm aware of.  I don't see how
it could be, to be honest, but maybe I'm missing something there.

> And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
> does this delaying affect responsiveness?

Possibly SIGPOLL.

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 19:20:40 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> > But OTOH, if this delaying of a signal affects responsiveness, then
> > all we need to do is exempt SIGSEGV from being delayed, right?  This
> > signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
> > but there's no reason to delay SIGSEGV.
> 
> SIGSEGV is never delayed in any proposal I'm aware of.

The call to gc_signal_handler_can_run is inside
deliver_process_signal.  Are you saying that deliver_process_signal is
not called for SIGSEGV?

> > And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
> > does this delaying affect responsiveness?
> 
> Possibly SIGPOLL.

We don't need to block SIGPOLL, either.  Its handler is safe, the same
as SIGIO.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

>> Date: Sat, 28 Dec 2024 19:20:40 +0000
>> From: Pip Cet <pipcet@protonmail.com>
>> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
>>
>> "Eli Zaretskii" <eliz@gnu.org> writes:
>>
>> > But OTOH, if this delaying of a signal affects responsiveness, then
>> > all we need to do is exempt SIGSEGV from being delayed, right?  This
>> > signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
>> > but there's no reason to delay SIGSEGV.
>>
>> SIGSEGV is never delayed in any proposal I'm aware of.
>
> The call to gc_signal_handler_can_run is inside
> deliver_process_signal.  Are you saying that deliver_process_signal is
> not called for SIGSEGV?

MPS installs its own SIGSEGV handler which doesn't go through
deliver_process_signal.  Only if it fails, the Emacs handler which does
go through deliver_process_signal is restored for the final SIGSEGV
which will then terminate Emacs.

>> > And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
>> > does this delaying affect responsiveness?
>>
>> Possibly SIGPOLL.
>
> We don't need to block SIGPOLL, either.  Its handler is safe, the same
> as SIGIO.

Thanks!  That's good to know, and that's why we pass the signal number
to gc_signal_handler_can_run.

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 20:54:03 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> >> Date: Sat, 28 Dec 2024 19:20:40 +0000
> >> From: Pip Cet <pipcet@protonmail.com>
> >> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> >>
> >> "Eli Zaretskii" <eliz@gnu.org> writes:
> >>
> >> > But OTOH, if this delaying of a signal affects responsiveness, then
> >> > all we need to do is exempt SIGSEGV from being delayed, right?  This
> >> > signal-delay mechanism was invented for SIGPROF, SIGCHLD, and SIGALRM,
> >> > but there's no reason to delay SIGSEGV.
> >>
> >> SIGSEGV is never delayed in any proposal I'm aware of.
> >
> > The call to gc_signal_handler_can_run is inside
> > deliver_process_signal.  Are you saying that deliver_process_signal is
> > not called for SIGSEGV?
> 
> MPS installs its own SIGSEGV handler which doesn't go through
> deliver_process_signal.  Only if it fails, the Emacs handler which does
> go through deliver_process_signal is restored for the final SIGSEGV
> which will then terminate Emacs.

Then I ask once again: how can we explain what Gerd reports about
responsiveness if your changes, which only affect
deliver_process_signal, cannot affect MPS?

> >> > And AFAIU, on macOS there's no SIGSEGV anyway, is that right?  So why
> >> > does this delaying affect responsiveness?
> >>
> >> Possibly SIGPOLL.
> >
> > We don't need to block SIGPOLL, either.  Its handler is safe, the same
> > as SIGIO.
> 
> Thanks!  That's good to know, and that's why we pass the signal number
> to gc_signal_handler_can_run.

But currently gc_signal_handler_can_run does nothing with the signal
number except recording that it happened.  IMO, it should return
'true' immediately unless the signal is SIGPROF, SIGCHLD, or SIGALRM.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Eli Zaretskii" <eliz@gnu.org> writes:

> Ah, okay.  I note that if we'd block signals when calling MPS and
> unblock on exit, then these delays couldn't have happened, AFAIU.

We can't do so for SIGSEGV calling into MPS, so this wouldn't fix all
cases.  Blocking signals around mps_alloc slows down (make-list 1000000
nil) by a factor of about 5 (on current GNU/Linux; possibly
significantly more on other operating systems).

But, yes, if we're willing to give up on unmodified MPS, blocking
signals in the slow path only might work.  We'd need to check
finalizable objects, though, because we need to call into MPS for every
finalizable object.  That's a problem this approach shares with the
allocation thread approach, by the way.  In both cases, registering
objects for finalization doesn't need to happen at allocation time: if
we save a reference to the object somewhere MPS sees it, the objects
won't be collected, so they won't be finalized.

Pip

Re: igc, macOS avoiding signals

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 19:15:26 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: spd@toadstyle.org, gerd.moellmann@gmail.com, emacs-devel@gnu.org
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> > Ah, okay.  I note that if we'd block signals when calling MPS and
> > unblock on exit, then these delays couldn't have happened, AFAIU.
> 
> We can't do so for SIGSEGV calling into MPS, so this wouldn't fix all
> cases.

We don't need to block SIGSEGV (or any other fatal signal).  We only
need to block SIGPROF, SIGALRM and SIGCHLD.

> Blocking signals around mps_alloc slows down (make-list 1000000
> nil) by a factor of about 5 (on current GNU/Linux; possibly
> significantly more on other operating systems).

No, I think only GNU/Linux is affected.

You are saying that sigblock is very expensive?  Can you measure it?

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

> I'll try to come up with a patch.

This should provide some data (on stderr) about which signals we delay,
and for how long (the "delayed" messages).  It also includes some
information on additional points at which we can detect whether signals
are pending (the "delaying" messages); it's probably safe to run them at
that point, but the code might need some changes because other signals
(or even the signal in question) might be legitimately blocked when we
reach that point.

If the "delaying" messages indicate acceptable (initial) delays, we
might get away with simply calling gc_maybe_quit more often.  If they
don't, further fixes will be necessary, or we need to find more such
points.

On POSIX systems where we can spare an additional signal, we can run a
separate thread to ask us to retry running signal handlers when the
arena lock might be available again.

Or we could move to a separate thread for slow-path allocations.

diff --git a/src/igc.c b/src/igc.c
index 4d9aec90340..1402b9d10a5 100644
--- a/src/igc.c
+++ b/src/igc.c
@@ -811,6 +811,7 @@ IGC_DEFINE_LIST (igc_thread);
   /* Per-signal flags.  These are `int' to reduce the chance of
    * corruption when accessed non-atomically.  */
   int pending_signals[64];
+  struct timespec pending_since[64];
   /* The real signal mask we want to restore after handling pending
    * signals.  */
   sigset_t signal_mask;
@@ -3833,6 +3834,7 @@ alloc_impl (size_t size, enum igc_obj_type type, mps_ap_t ap)
     case IGC_STATE_INITIAL:
       emacs_abort ();
     }
+  gc_maybe_dont_quit ();
   return p;
 }
 
@@ -4924,8 +4926,9 @@ gc_signal_handler_can_run (int sig)
   if (igc_busy_p ())
     {
       sigset_t sigs;
-      global_igc->signals_pending = 1;
+      clock_gettime (CLOCK_REALTIME, &global_igc->pending_since[sig]);
       global_igc->pending_signals[sig] = 1;
+      global_igc->signals_pending = 1;
       sigemptyset (&sigs);
       sigaddset (&sigs, sig);
       pthread_sigmask (SIG_BLOCK, &sigs, NULL);
@@ -4946,6 +4949,12 @@ gc_maybe_quit (void)
       for (int i = 0; i < ARRAYELTS (global_igc->pending_signals); i++)
 	if (global_igc->pending_signals[i])
 	  {
+	    struct timespec ts;
+	    clock_gettime (CLOCK_REALTIME, &ts);
+	    long long nsec = ts.tv_nsec - global_igc->pending_since[i].tv_nsec;
+	    long long sec = ts.tv_sec - global_igc->pending_since[i].tv_sec;
+	    nsec += 1000000000 * sec;
+	    fprintf (stderr, "delayed %d for %f sec\n", i, nsec * 1.0e-9);
 	    global_igc->pending_signals[i] = 0;
 	    raise (i);
 	  }
@@ -4953,6 +4962,23 @@ gc_maybe_quit (void)
     }
 }
 
+void gc_maybe_dont_quit (void)
+{
+  if (global_igc->signals_pending)
+    {
+      for (int i = 0; i < ARRAYELTS (global_igc->pending_signals); i++)
+	if (global_igc->pending_signals[i])
+	  {
+	    struct timespec ts;
+	    clock_gettime (CLOCK_REALTIME, &ts);
+	    long long nsec = ts.tv_nsec - global_igc->pending_since[i].tv_nsec;
+	    long long sec = ts.tv_sec - global_igc->pending_since[i].tv_sec;
+	    nsec += 1000000000 * sec;
+	    fprintf (stderr, "delaying %d for %f sec\n", i, nsec * 1.0e-9);
+	  }
+    }
+}
+
 DEFUN ("igc--add-extra-dependency", Figc__add_extra_dependency,
        Sigc__add_extra_dependency, 3, 3, 0,
        doc: /* Add an extra DEPENDENCY to object OBJ, associate it with KEY.
diff --git a/src/keyboard.c b/src/keyboard.c
index e875e98fde6..906595f3be9 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -8203,6 +8203,7 @@ unblock_input_to (int level)
   interrupt_input_blocked = level;
   if (level == 0)
     {
+      gc_maybe_dont_quit ();
       if (pending_signals && !fatal_error_in_progress)
 	process_pending_signals ();
     }
diff --git a/src/lisp.h b/src/lisp.h
index 48585c2d8a1..fb7f3847a5d 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -47,12 +47,17 @@ #define EMACS_LISP_H
 #ifdef HAVE_MPS
 union gc_header;
 extern void gc_maybe_quit (void);
+extern void gc_maybe_dont_quit (void);
 extern bool gc_signal_handler_can_run (int);
 #else
 INLINE void gc_maybe_quit (void)
 {
 }
 
+INLINE void gc_maybe_dont_quit (void)
+{
+}
+
 INLINE bool gc_signal_handler_can_run (int sig)
 {
   return true;

Re: igc, macOS avoiding signals

[Sean Devlin]

Hi Pip,

> On Dec 29, 2024, at 1:29 AM, Pip Cet <pipcet@protonmail.com> wrote:
> 
>> I'll try to come up with a patch.
> 
> This should provide some data (on stderr) about which signals we delay,
> and for how long (the "delayed" messages).  It also includes some
> information on additional points at which we can detect whether signals
> are pending (the "delaying" messages); it's probably safe to run them at
> that point, but the code might need some changes because other signals
> (or even the signal in question) might be legitimately blocked when we
> reach that point.
> 
> If the "delaying" messages indicate acceptable (initial) delays, we
> might get away with simply calling gc_maybe_quit more often.  If they
> don't, further fixes will be necessary, or we need to find more such
> points.
> 
> On POSIX systems where we can spare an additional signal, we can run a
> separate thread to ask us to retry running signal handlers when the
> arena lock might be available again.
> 
> Or we could move to a separate thread for slow-path allocations.
> 

I’ve built Emacs with your patch. After running Emacs -Q for a few minutes, I can confirm I see a few log statements:

delaying 20 for 0.066594 sec
delaying 20 for 0.066612 sec
delayed 20 for 0.066614 sec

Please let me know if there are any particular tasks you’d like me to try, or if I should just collect the logs in the background during general usage.

Cheers.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

"Sean Devlin" <spd@toadstyle.org> writes:

> Hi Pip,
>
>> On Dec 29, 2024, at 1:29 AM, Pip Cet <pipcet@protonmail.com> wrote:
>>
>>> I'll try to come up with a patch.
>>
>> This should provide some data (on stderr) about which signals we delay,
>> and for how long (the "delayed" messages).  It also includes some
>> information on additional points at which we can detect whether signals
>> are pending (the "delaying" messages); it's probably safe to run them at
>> that point, but the code might need some changes because other signals
>> (or even the signal in question) might be legitimately blocked when we
>> reach that point.
>>
>> If the "delaying" messages indicate acceptable (initial) delays, we
>> might get away with simply calling gc_maybe_quit more often.  If they
>> don't, further fixes will be necessary, or we need to find more such
>> points.
>>
>> On POSIX systems where we can spare an additional signal, we can run a
>> separate thread to ask us to retry running signal handlers when the
>> arena lock might be available again.
>>
>> Or we could move to a separate thread for slow-path allocations.
>>
>
> I’ve built Emacs with your patch. After running Emacs -Q for a few minutes, I can confirm I see a few log statements:

Can you try setting igc-step-interval to a small float value, like 0.05
?  As long as it's just a few messages, I don't think it'd cause
significant problems, but maybe enabling the background work would do
something.

> Please let me know if there are any particular tasks you’d like me to try, or if I should just collect the logs in the background during general usage.

Repeatedly hitting "s" in an M-x igc-stats buffer should cause more
messages, but that uses IGC in an atypical fashion, so I'm not sure
that's actually useful data...

Thanks!

Pip

igc, macOS avoiding signals

[Gerd Möllmann]

This is about commit

ceec5ace134081b64dbf46c4fb5702ef5209c5fd
Avoid MPS being interrupted by signals

I've been running with this for some days now, and must report that
Emacs feels a _bit_ different here in interactive use, maybe one could
say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
very recent master++).

After reverting the commit, it's feeling smoother again.

Just saying. FWIW.

Maybe it's a point for making things conditional on OS, don't know.

Re: igc, macOS avoiding signals

[Pip Cet via Emacs development discussions.]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> This is about commit
>
> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
> Avoid MPS being interrupted by signals
>
> I've been running with this for some days now, and must report that
> Emacs feels a _bit_ different here in interactive use, maybe one could
> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
> very recent master++).

I think we should quantify that.  Set a watchpoint on
igc_global->signals_pending, check how often we even set that, and how
often we call igc_maybe_quit, particularly if we were previously idle.
Maybe it's sufficient to call it again from the idle handler.

> After reverting the commit, it's feeling smoother again.

Entirely possible.  Let's measure it.

> Maybe it's a point for making things conditional on OS, don't know.

If we establish it's not necessary on macOS, sure.  If it slows things
down that's kind of a hint that it might be necessary, though.

Pip

Re: igc, macOS avoiding signals

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> This is about commit
>>
>> ceec5ace134081b64dbf46c4fb5702ef5209c5fd
>> Avoid MPS being interrupted by signals
>>
>> I've been running with this for some days now, and must report that
>> Emacs feels a _bit_ different here in interactive use, maybe one could
>> say not as smooth. (macOS, --without-ns, in my fork of Emacs, which is
>> very recent master++).
>
> I think we should quantify that.  Set a watchpoint on
> igc_global->signals_pending, check how often we even set that, and how
> often we call igc_maybe_quit, particularly if we were previously idle.
> Maybe it's sufficient to call it again from the idle handler.
>
>> After reverting the commit, it's feeling smoother again.
>
> Entirely possible.  Let's measure it.

I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
else using macOS can do that. I just wanted to make you aware of this.

Re: igc, macOS avoiding signals

[Óscar Fuentes]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

>>> After reverting the commit, it's feeling smoother again.
>>
>> Entirely possible.  Let's measure it.
>
> I'm sorry, but I pass. It's too time-consuming for me. Maybe someone
> else using macOS can do that. I just wanted to make you aware of this.

Is it difficult to implement a way to measure some relevant metrics?

Something related to UI responsiveness would be great. For instance,
record the time from each interactive command start to command end (or,
better, until Emacs is idle again, to account for commands accumulating
on the queue). Then we can perform some statistical analysis on that
info.

I'm afraid that if we start discussing personal perceptions we will
devote a lot of time trying to fine-adjust parameters.

Re: igc, macOS avoiding signals

[Helmut Eller]

On Sat, Dec 28 2024, Óscar Fuentes wrote:
[...]
> Something related to UI responsiveness would be great. For instance,
> record the time from each interactive command start to command end (or,
> better, until Emacs is idle again, to account for commands accumulating
> on the queue). Then we can perform some statistical analysis on that
> info.
>
> I'm afraid that if we start discussing personal perceptions we will
> devote a lot of time trying to fine-adjust parameters.

I couldn't agree more. 

What would you think about using one of those tracing frameworks, like
LTTng[*]?  Are those any good?

It would be nice, if those tracing points could be used in normal
everyday sessions.  E.g. to see how memory usage evolved over the last
15 minutes or to see if there was a particularly slow regexp search.

Helmut

[*] https://lttng.org/