bug#72496: 31.0.50; macOS: freezes without beach ball

bug#72496: 31.0.50; macOS: freezes without beach ball

[Gerd Möllmann]

In GNU Emacs 31.0.50 (build 1, aarch64-apple-darwin23.6.0, NS
 appkit-2487.70 Version 14.6 (Build 23G80)) of 2024-08-05 built on
 pro2.fritz.box
Repository revision: c7d9cd722e5a7042a52c92f8497f903bfe9870b8

This is one of the problems on macOS that I'm experiencing quite
often. I have no idea what is causing this, and I haven't found a way
to make it reproducible. I guess I should file a bug report anyway.

Let me first try to describe briefly how NS GUI event handling works
in Emacs.

The whole story starts with get_input_pending which calls gobble_input
which calls a terminal's read_socket_hook, which is ns_read_socket in
macOS. ns_read_socket calls [NSApplication run] to process macOS GUI
events.

The problem starts with [NSApplication run] being an endless loop that
gets the next event, and dispatches it by calling the application's
sendEvent method. The only way to make the run loop terminate is by
calling [NSApplication stop]. This sets a flag that [NSApplicatoin
run] is supposed to check and then return, so that we eventually
return to ns_read_socket.

We call [NSApplicaton stop] in our [EmacsApp sendEvent] method. To get
there, we post special application-defined events to the application
which [NSApplication run] processes and dispatches via sendEvent which
calls stop and makes [NSApplication run] return to its caller. Note
that stop only sets a flag, so we need to process another event to
make run terminate. That's at least my understanding.

We are posting these events all over the place, not only before
ns_read_socket calls [NSApplication run]. And, to complicate matters,
whether or not ns_send_appdefined actually posts an event depends on a
global boolean variable. IOW, it's impenetrable.

(I'm also leaving out the generation of input_events for Emacs here,
which is another can of worms.)

Problem is that this not always works. More specifically, this code in
ns_read_socket_1

          /* Run and wait for events.  We must always send one NX_APPDEFINED event
             to ourself, otherwise [NXApp run] will never exit.  */
          send_appdefined = YES;
          ns_send_appdefined (-1);

          [NSApp run];

gets stuck in the GUI event loop, and the last line never returns. The
effect being that Emacs freezes without a beach ball of death. It
processes Cocoa events but Emacs never sees any input_events.

bug#72496: 31.0.50; macOS: freezes without beach ball

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Date: Tue, 06 Aug 2024 15:39:43 +0200
> 
> We call [NSApplicaton stop] in our [EmacsApp sendEvent] method. To get
> there, we post special application-defined events to the application
> which [NSApplication run] processes and dispatches via sendEvent which
> calls stop and makes [NSApplication run] return to its caller.

Is there any way to initiate posting that "special application-defined
event" when Emacs is stuck thusly?  For example, from some system
signal-like event, or from a debugger?  Then you at least would have a
fire escape.

bug#72496: 31.0.50; macOS: freezes without beach ball

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Date: Tue, 06 Aug 2024 15:39:43 +0200
>> 
>> We call [NSApplicaton stop] in our [EmacsApp sendEvent] method. To get
>> there, we post special application-defined events to the application
>> which [NSApplication run] processes and dispatches via sendEvent which
>> calls stop and makes [NSApplication run] return to its caller.
>
> Is there any way to initiate posting that "special application-defined
> event" when Emacs is stuck thusly?  For example, from some system
> signal-like event, or from a debugger?  Then you at least would have a
> fire escape.

Running under LLDB I can do something, with a little code change in
ns_send_appdefined.

  (lldb) expr ns_send_appdefined(-42)

where I let -42 force posting an event, ignoring the global variable I
mentioned, send_appdefined, that prevents the posting. Sometimes I have
to do that twice before Emacs receives key strokes again. And it seems
something is also broken afterwards, for example the cursor stops
blinking, tooltips don't work and such things. But at least one can save
buffers and exit.

I've tried this

  (when (fboundp 'ns-app-stop)
    (defun sigusr1-handler ()
      (interactive)
      (message "SIGUSR1 - stop event loop")
      (ns-app-stop))
    (keymap-set special-event-map "<sigusr1>" 'sigusr1-handler))

where ns-app-stop does such a ns_send_appdefined, but that didn't work
for a reason unknown to me. I also tried to do that in the signal
handler directly, but couldn't make it work either.

BTW, there are also freezes with beach ball, where the situation seems
to be reversed, i.e. for some reason [NSApplication run] seems to no
longer processing events, hence the beach ball. I haven't event a theory
what that could oossibly be caused by.

bug#72496: 31.0.50; macOS: freezes without beach ball

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: 72496@debbugs.gnu.org
> Date: Tue, 06 Aug 2024 18:36:39 +0200
> 
> I've tried this
> 
>   (when (fboundp 'ns-app-stop)
>     (defun sigusr1-handler ()
>       (interactive)
>       (message "SIGUSR1 - stop event loop")
>       (ns-app-stop))
>     (keymap-set special-event-map "<sigusr1>" 'sigusr1-handler))
> 
> where ns-app-stop does such a ns_send_appdefined, but that didn't work
> for a reason unknown to me.

Maybe the way SIGUSR1 is handled involves the same event queue that is
botched in this scenario?

bug#72496: 31.0.50; macOS: freezes without beach ball

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: 72496@debbugs.gnu.org
>> Date: Tue, 06 Aug 2024 18:36:39 +0200
>> 
>> I've tried this
>> 
>>   (when (fboundp 'ns-app-stop)
>>     (defun sigusr1-handler ()
>>       (interactive)
>>       (message "SIGUSR1 - stop event loop")
>>       (ns-app-stop))
>>     (keymap-set special-event-map "<sigusr1>" 'sigusr1-handler))
>> 
>> where ns-app-stop does such a ns_send_appdefined, but that didn't work
>> for a reason unknown to me.
>
> Maybe the way SIGUSR1 is handled involves the same event queue that is
> botched in this scenario?

Yes, that's quite likely. I had a little hope that a signal would maybe
handled in some special way, but apparently not.

bug#72496: 31.0.50; macOS: freezes without beach ball

[Gerd Möllmann]

[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>>> Cc: 72496@debbugs.gnu.org
>>> Date: Tue, 06 Aug 2024 18:36:39 +0200
>>> 
>>> I've tried this
>>> 
>>>   (when (fboundp 'ns-app-stop)
>>>     (defun sigusr1-handler ()
>>>       (interactive)
>>>       (message "SIGUSR1 - stop event loop")
>>>       (ns-app-stop))
>>>     (keymap-set special-event-map "<sigusr1>" 'sigusr1-handler))
>>> 
>>> where ns-app-stop does such a ns_send_appdefined, but that didn't work
>>> for a reason unknown to me.
>>
>> Maybe the way SIGUSR1 is handled involves the same event queue that is
>> botched in this scenario?
>
> Yes, that's quite likely. I had a little hope that a signal would maybe
> handled in some special way, but apparently not.

I'm now running locally with the attached change. This removes the
global variable that can prevent sending app-defined events when set
wrong. Instead, I'm using [NSApplication nextEventMatchingMask] to check
if an app-defined event has already been posted. This is infinelty less
dangerous.

Let's see if that is the problem.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nsterm.m patch --]
[-- Type: text/x-patch, Size: 3410 bytes --]

From f48aafd73c7d2694bf2c9e980e9624bde9ab3b3e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gerd=20M=C3=B6llmann?= <gerd@gnu.org>
Date: Thu, 8 Aug 2024 06:57:21 +0200
Subject: [PATCH] NS: Send application-defined event differently (bug#72496)

* src/nsterm.m (send_appdefined): Global variable removed, all uses
removed.
(ns_send_appdefined): Check if an application-defined event is already
queued with [NSApplicaton nextEventMatching]. Post an event if not.
---
 src/nsterm.m | 26 ++++++++++----------------
 1 file changed, 10 insertions(+), 16 deletions(-)

diff --git a/src/nsterm.m b/src/nsterm.m
index 5588425686b..594f7ba974b 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -309,7 +309,6 @@ - (unsigned long)unsignedLong
 #endif
 
 /* event loop */
-static BOOL send_appdefined = YES;
 #define NO_APPDEFINED_DATA (-8)
 static int last_appdefined_event_data = NO_APPDEFINED_DATA;
 static NSTimer *timed_entry = 0;
@@ -507,7 +506,6 @@ - (unsigned long)unsignedLong
   hold_event_q.q[hold_event_q.nr++] = *event;
   /* Make sure ns_read_socket is called, i.e. we have input.  */
   raise (SIGIO);
-  send_appdefined = YES;
 }
 
 static Lisp_Object
@@ -4689,13 +4687,17 @@ Function modeled after x_draw_glyph_string_box ().
   /* Only post this event if we haven't already posted one.  This will end
      the [NXApp run] main loop after having processed all events queued at
      this moment.  */
-  if (send_appdefined || value == -42)
+
+  NSEvent *app_defined_event =
+    [NSApp nextEventMatchingMask: NSEventMaskApplicationDefined
+		       untilDate: nil
+			  inMode: NSDefaultRunLoopMode
+			 dequeue: NO];
+
+  if (app_defined_event == nil)
     {
       NSEvent *nxev;
 
-      /* We only need one NX_APPDEFINED event to stop NXApp from running.  */
-      send_appdefined = NO;
-
       /* Don't need wakeup timer any more.  */
       if (timed_entry)
         {
@@ -4812,7 +4814,6 @@ Function modeled after x_draw_glyph_string_box ().
         {
           /* Run and wait for events.  We must always send one NX_APPDEFINED event
              to ourself, otherwise [NXApp run] will never exit.  */
-          send_appdefined = YES;
           ns_send_appdefined (-1);
 
           [NSApp run];
@@ -4907,7 +4908,6 @@ Function modeled after x_draw_glyph_string_box ().
   // outerpool = [[NSAutoreleasePool alloc] init];
 
 
-  send_appdefined = YES;
   if (nr > 0)
     {
       pthread_mutex_lock (&select_mutex);
@@ -4951,7 +4951,6 @@ Function modeled after x_draw_glyph_string_box ().
   else /* No timeout and no file descriptors, can this happen?  */
     {
       /* Send appdefined so we exit from the loop.  */
-      ns_send_appdefined (-1);
     }
 
   block_input ();
@@ -6051,10 +6050,6 @@ - (void)sendEvent: (NSEvent *)theEvent
           last_appdefined_event_data = [theEvent data1];
           [self stop: self];
         }
-      else
-        {
-          send_appdefined = YES;
-        }
     }
 
 
@@ -7727,7 +7722,7 @@ - (void)mouseMoved: (NSEvent *)e
                       help_echo_object, help_echo_pos);
     }
 
-  if ((*emacsframe)->mouse_moved && send_appdefined)
+  if ((*emacsframe)->mouse_moved)
     ns_send_appdefined (-1);
 }
 
@@ -8336,8 +8331,7 @@ - (void)windowDidExpose: sender
   SET_FRAME_VISIBLE (*emacsframe, 1);
   SET_FRAME_GARBAGED (*emacsframe);
 
-  if (send_appdefined)
-    ns_send_appdefined (-1);
+  ns_send_appdefined (-1);
 }
 
 
-- 
2.46.0

bug#72496: 31.0.50; macOS: freezes without beach ball

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>>>> Cc: 72496@debbugs.gnu.org
>>>> Date: Tue, 06 Aug 2024 18:36:39 +0200
>>>> 
>>>> I've tried this
>>>> 
>>>>   (when (fboundp 'ns-app-stop)
>>>>     (defun sigusr1-handler ()
>>>>       (interactive)
>>>>       (message "SIGUSR1 - stop event loop")
>>>>       (ns-app-stop))
>>>>     (keymap-set special-event-map "<sigusr1>" 'sigusr1-handler))
>>>> 
>>>> where ns-app-stop does such a ns_send_appdefined, but that didn't work
>>>> for a reason unknown to me.
>>>
>>> Maybe the way SIGUSR1 is handled involves the same event queue that is
>>> botched in this scenario?
>>
>> Yes, that's quite likely. I had a little hope that a signal would maybe
>> handled in some special way, but apparently not.
>
> I'm now running locally with the attached change. This removes the
> global variable that can prevent sending app-defined events when set
> wrong. Instead, I'm using [NSApplication nextEventMatchingMask] to check
> if an app-defined event has already been posted. This is infinelty less
> dangerous.
>
> Let's see if that is the problem.

I don't even know if this is related to my change to ns_send_appdefined, but
I got a freeze today while using child frames (vertico + posframe) that
I haven't seen before. Event the mouse behaved strangely on the whole
macOS until I killed the process. That was without beach ball.

So, whatever that was, my patch at least didn't help, if it wasn't
causing it.

I give up. I'll switch to using tty Emacs.