bug#70032: 30.0.50; thread-yield inconsistency with macOS GUI and other platforms

bug#70032: 30.0.50; thread-yield inconsistency with macOS GUI and other platforms

[Daniel Pettersson]

(make-thread (lambda ()
               (benchmark-progn
                 (dotimes (_ 100000)
                   (thread-yield))))
                   "benchmark-thread")

The thread will finish in less then one second on macOS '-nw' and Linux
(GUI and '-nw') (have not tested on Windows).

On macOS GUI "benchmark-thread" will almost certainly never finish.

The only way that I know to drive the thread forward is to generate
input events, so you need to generate at 100000 input events for it to
finish. With macOS GUI the Main thread does only seam to yield back to
the other thread after an mouse/keyboard event.

* src/nsterm.m: ns_select_1

  /* emacs -nw doesn't have an NSApp, so we're done.  */
  if (NSApp == nil)
    return thread_select (pselect, nfds, readfds, writefds, exceptfds,
			  timeout, sigmask);

  if (![NSThread isMainThread]
      || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
    thread_select (pselect, nfds, readfds, writefds,
		   exceptfds, timeout, sigmask);
  else
    {
      struct timespec t = {0, timeout->tv_nsec}; 
      thread_select (pselect, 0, NULL, NULL, NULL, &t, sigmask);
    }

I don't understand Emacs threads well enough but it seams that the
else block is the reason for the main thread not yielding to the
"benchmark-thread" is the Main thread will almost always pselect with
timeout {0, 0}.  But if timeout is not zeroed Emacs becomes extremely
unresponsive (but "benchmark-thread" will finish in the same amount of
time as running emacs with '-nw')

In GNU Emacs 30.0.50 (build 1, aarch64-apple-darwin23.1.0, NS
 appkit-2487.20 Version 14.1.1 (Build 23B81)) of 2023-12-20 built on
 Daniels-Air
Repository revision: 281be72422f42fcc84d43f50723a3e91b7d03cbc
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.1.1

Configured features:
ACL DBUS GLIB GNUTLS JSON LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS PDUMPER
PNG RSVG SQLITE3 THREADS TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date subr-x
cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/ns-win ns-win ucs-normalize mule-util term/common-win tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind kqueue cocoa ns
lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 38459 8478) (symbols 48 5148 0) (strings 32 13180 1700)
 (string-bytes 1 386239) (vectors 16 10512)
 (vector-slots 8 164949 10278) (floats 8 24 23) (intervals 56 242 3)
 (buffers 992 11))

bug#70032: 30.0.50; thread-yield inconsistency with macOS GUI and other platforms

[Daniel Pettersson]

Now that I am already being a nuisance, I might as well try to resurrect
this bug as well.  I have tried to create a patch for this issue, but
failed.  I do not understand the internals of emacs threads well enough,
I am happy to try to give it a go but I would require some pointers to
get me started.

bug#70032: 30.0.50; thread-yield inconsistency with macOS GUI and other platforms

[Gerd Möllmann]

Daniel Pettersson <daniel@dpettersson.net> writes:

> Now that I am already being a nuisance, I might as well try to resurrect
> this bug as well.  I have tried to create a patch for this issue, but
> failed.  I do not understand the internals of emacs threads well enough,
> I am happy to try to give it a go but I would require some pointers to
> get me started.

In bug#72496 I wrote something that I think might be helpful to
underwtand how this works in NS. I nowadays build Emacs --without-ns.

  In GNU Emacs 31.0.50 (build 1, aarch64-apple-darwin23.6.0, NS
   appkit-2487.70 Version 14.6 (Build 23G80)) of 2024-08-05 built on
   pro2.fritz.box
  Repository revision: c7d9cd722e5a7042a52c92f8497f903bfe9870b8

  This is one of the problems on macOS that I'm experiencing quite
  often. I have no idea what is causing this, and I haven't found a way
  to make it reproducible. I guess I should file a bug report anyway.

  Let me first try to describe briefly how NS GUI event handling works
  in Emacs.

  The whole story starts with get_input_pending which calls gobble_input
  which calls a terminal's read_socket_hook, which is ns_read_socket in
  macOS. ns_read_socket calls [NSApplication run] to process macOS GUI
  events.

  The problem starts with [NSApplication run] being an endless loop that
  gets the next event, and dispatches it by calling the application's
  sendEvent method. The only way to make the run loop terminate is by
  calling [NSApplication stop]. This sets a flag that [NSApplicatoin
  run] is supposed to check and then return, so that we eventually
  return to ns_read_socket.

  We call [NSApplicaton stop] in our [EmacsApp sendEvent] method. To get
  there, we post special application-defined events to the application
  which [NSApplication run] processes and dispatches via sendEvent which
  calls stop and makes [NSApplication run] return to its caller. Note
  that stop only sets a flag, so we need to process another event to
  make run terminate. That's at least my understanding.

  We are posting these events all over the place, not only before
  ns_read_socket calls [NSApplication run]. And, to complicate matters,
  whether or not ns_send_appdefined actually posts an event depends on a
  global boolean variable. IOW, it's impenetrable.

  (I'm also leaving out the generation of input_events for Emacs here,
  which is another can of worms.)

  Problem is that this not always works. More specifically, this code in
  ns_read_socket_1

            /* Run and wait for events.  We must always send one NX_APPDEFINED event
               to ourself, otherwise [NXApp run] will never exit.  */
            send_appdefined = YES;
            ns_send_appdefined (-1);

            [NSApp run];

  gets stuck in the GUI event loop, and the last line never returns. The
  effect being that Emacs freezes without a beach ball of death. It
  processes Cocoa events but Emacs never sees any input_events.