bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Stefan Kangas]

I have run into a bug with make-thread on macOS 15.2, running on an M2.

I can reproduce the issue consistently both on emacs-30 and master by
evaluating this in emacs -Q:

    (make-thread (lambda () (sleep-for 1)) "bug")

This leads to Emacs freezing up completely within a fraction of a
second.  I have time to move point once or maybe twice before it gets
non-responsive, let's say within a few tenths of a second.

Running under lldb, I see the following lines get printed over and over:

2025-01-02 05:47:20.776364+0100 emacs[78593:1366649] [General]
nextEventMatchingMask should only be called from the Main Thread!
2025-01-02 05:47:20.777212+0100 emacs[78593:1366649] [General] (
	0   CoreFoundation                      0x000000018e2c6e80
__exceptionPreprocess + 176
	1   libobjc.A.dylib                     0x000000018ddaecd8
objc_exception_throw + 88
	2   AppKit                              0x0000000192721c24
-[NSApplication(NSEventRouting)
_nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3072
	3   AppKit                              0x0000000191db0060
-[NSApplication run] + 480
	4   emacs                               0x0000000100403fec -[EmacsApp run] + 88
	5   emacs                               0x00000001004024b0 ns_select_1 + 1144
	6   emacs                               0x000000010040202c ns_select + 72
	7   emacs                               0x000000010036930c
wait_reading_process_output + 5264
	8   emacs                               0x000000010000b400 Fsleep_for + 372
	9   emacs                               0x00000001002bd5d8 eval_sub + 2164
	10  emacs                               0x00000001002bdd2c Fprogn + 100
	11  emacs                               0x00000001002c79dc
funcall_lambda + 1964
	12  emacs                               0x00000001002c6af4
funcall_general + 276
	13  emacs                               0x00000001002bedf8 Ffuncall + 464
	14  emacs                               0x00000001003bf944
invoke_thread_function + 48
	15  emacs                               0x00000001002c2a9c
internal_condition_case + 148
	16  emacs                               0x00000001003be36c run_thread + 340
	17  libsystem_pthread.dylib             0x000000018e16c2e4 _pthread_start + 136
	18  libsystem_pthread.dylib             0x000000018e1670fc thread_start + 8
)
2025-01-02 05:47:20.777270+0100 emacs[78593:1366649] [General]
nextEventMatchingMask should only be called from the Main Thread!
2025-01-02 05:47:20.778126+0100 emacs[78593:1366649] [General] (

When I kill the process in the lldb window with Ctrl+C, I can get the
following (this is on emacs-30):

[...]
2025-01-02 05:47:20.778199+0100 emacs[78593:1366649] [General]
nextEventMatchingMask should only be called from the Main Thread!
Process 78593 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x000000018e12dbbc libsystem_kernel.dylib`__psynch_mutexwait + 8
libsystem_kernel.dylib`:
->  0x18e12dbbc <+8>:  b.lo   0x18e12dbdc               ; <+40>
    0x18e12dbc0 <+12>: pacibsp
    0x18e12dbc4 <+16>: stp    x29, x30, [sp, #-0x10]!
    0x18e12dbc8 <+20>: mov    x29, sp
Target 0: (emacs) stopped.
(lldb) bt all
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x000000018e12dbbc libsystem_kernel.dylib`__psynch_mutexwait + 8
    frame #1: 0x000000018e1693f8
libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_wait + 84
    frame #2: 0x000000018e166dbc
libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_slow + 220
    frame #3: 0x00000001003bfb94
emacs`sys_mutex_lock(mutex=0x0000000100b7bd30) at systhread.c:140:15
    frame #4: 0x00000001003bcef8
emacs`acquire_global_lock(self=0x0000000100b074d0) at thread.c:160:3
    frame #5: 0x00000001003bdce0
emacs`really_call_select(arg=0x000000016fdfc5d8) at thread.c:636:7
    frame #6: 0x000000010026ee70
emacs`flush_stack_call_func1(func=(emacs`really_call_select at
thread.c:614), arg=0x000000016fdfc5d8) at alloc.c:5534:3
    frame #7: 0x00000001003bd35c
emacs`flush_stack_call_func(func=(emacs`really_call_select at
thread.c:614), arg=0x000000016fdfc5d8) at lisp.h:4509:3
    frame #8: 0x00000001003bdc14
emacs`thread_select(func=(libsystem_kernel.dylib`pselect$DARWIN_EXTSN),
max_fds=0, rfds=0x0000000000000000, wfds=0x0000000000000000,
efds=0x0000000000000000, timeout=0x000000016fdfc690,
sigmask=0x0000000000000000) at thread.c:656:3
    frame #9: 0x00000001004022e4 emacs`ns_select_1(nfds=0,
readfds=0x000000016fdfcf2c, writefds=0x000000016fdfceac,
exceptfds=0x0000000000000000, timeout=0x000000016fdfce80,
sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4883:7
    frame #10: 0x000000010040202c emacs`ns_select(nfds=0,
readfds=0x000000016fdfcf2c, writefds=0x000000016fdfceac,
exceptfds=0x0000000000000000, timeout=0x000000016fdfce80,
sigmask=0x0000000000000000) at nsterm.m:5006:10
    frame #11: 0x000000010036930c
emacs`wait_reading_process_output(time_limit=30, nsecs=0, read_kbd=-1,
do_display=true, wait_for_cell=(i = 0x0000000000000000),
wait_proc=0x0000000000000000, just_wait_proc=0) at process.c:5753:18
    frame #12: 0x000000010000b7b8 emacs`sit_for(timeout=(i =
0x000000000000007a), reading=true, display_option=1) at
dispnew.c:6335:7
    frame #13: 0x00000001001a5964 emacs`read_char(commandflag=1,
map=(i = 0x0000000138061a63), prev_event=(i = 0x0000000000000000),
used_mouse_menu=0x000000016fdfe367, end_time=0x0000000000000000) at
keyboard.c:2923:11
    frame #14: 0x00000001001a0ed0
emacs`read_key_sequence(keybuf=0x000000016fdfea38, prompt=(i =
0x0000000000000000), dont_downcase_last=false,
can_return_switch_frame=true, fix_current_buffer=true,
prevent_redisplay=false, disable_text_conversion_p=false) at
keyboard.c:10743:12
    frame #15: 0x000000010019ebac emacs`command_loop_1 at keyboard.c:1429:15
    frame #16: 0x00000001002c2a9c
emacs`internal_condition_case(bfun=(emacs`command_loop_1 at
keyboard.c:1324), handlers=(i = 0x0000000000000090),
hfun=(emacs`cmd_error at keyboard.c:970)) at eval.c:1613:25
    frame #17: 0x000000010019e560 emacs`command_loop_2(handlers=(i =
0x0000000000000090)) at keyboard.c:1168:11
    frame #18: 0x00000001002c1b64 emacs`internal_catch(tag=(i =
0x0000000000011220), func=(emacs`command_loop_2 at keyboard.c:1164),
arg=(i = 0x0000000000000090)) at eval.c:1292:25
    frame #19: 0x000000010019d4f8 emacs`command_loop at keyboard.c:1146:2
    frame #20: 0x000000010019d2a0 emacs`recursive_edit_1 at keyboard.c:754:9
    frame #21: 0x000000010019d884 emacs`Frecursive_edit at keyboard.c:837:3
    frame #22: 0x0000000100199c50 emacs`main(argc=2,
argv=0x000000016fdff4c8) at emacs.c:2635:3
    frame #23: 0x000000018ddec274 dyld`start + 2840
  thread #4
    frame #0: 0x000000018e1311a8 libsystem_kernel.dylib`__pselect + 8
    frame #1: 0x000000018e131080
libsystem_kernel.dylib`pselect$DARWIN_EXTSN + 64
    frame #2: 0x0000000100405c18 emacs`-[EmacsApp
fd_handler:](self=0x0000000156610fe0, _cmd="fd_handler:",
unused=0x0000000000000000) at nsterm.m:6440:20
    frame #3: 0x000000018f41cc24 Foundation`__NSThread__start__ + 724
    frame #4: 0x000000018e16c2e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #5, name = 'com.apple.NSEventThread'
    frame #0: 0x000000018e12af54 libsystem_kernel.dylib`mach_msg2_trap + 8
    frame #1: 0x000000018e13d604 libsystem_kernel.dylib`mach_msg2_internal + 80
    frame #2: 0x000000018e133af8 libsystem_kernel.dylib`mach_msg_overwrite + 480
    frame #3: 0x000000018e12b29c libsystem_kernel.dylib`mach_msg + 24
    frame #4: 0x000000018e254a3c CoreFoundation`__CFRunLoopServiceMachPort + 160
    frame #5: 0x000000018e25329c CoreFoundation`__CFRunLoopRun + 1212
    frame #6: 0x000000018e252724 CoreFoundation`CFRunLoopRunSpecific + 588
    frame #7: 0x0000000191ee1a20 AppKit`_NSEventThread + 148
    frame #8: 0x000000018e16c2e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #6
    frame #0: 0x000000018e1670e8 libsystem_pthread.dylib`start_wqthread
  thread #7, name = 'bug'
    frame #0: 0x000000018de3a2b0
dyld`dyld3::MachOLoaded::findClosestSymbol(unsigned long long, char
const**, unsigned long long*) const + 488
    frame #1: 0x000000018de1b13c dyld`dyld4::APIs::dladdr(void const*,
dl_info*) + 236
    frame #2: 0x000000018e012f00 libsystem_c.dylib`backtrace_symbols + 144
    frame #3: 0x000000018f4998c0 Foundation`-[_NSCallStackArray
descriptionWithLocale:indent:] + 144
    frame #4: 0x000000018f3e8c10 Foundation`_NS_os_log_callback + 276
    frame #5: 0x000000018debee60
libsystem_trace.dylib`_os_log_fmt_flatten_NSCF + 64
    frame #6: 0x000000018dec5830
libsystem_trace.dylib`_os_log_fmt_flatten_object_impl + 372
    frame #7: 0x000000018debc9c8
libsystem_trace.dylib`_os_log_impl_flatten_and_send + 2144
    frame #8: 0x000000018debc150 libsystem_trace.dylib`_os_log + 168
    frame #9: 0x000000018debc0a0 libsystem_trace.dylib`_os_log_impl + 28
    frame #10: 0x000000019209151c AppKit`-[NSApplication reportException:] + 624
    frame #11: 0x0000000191db0118 AppKit`-[NSApplication run] + 664
    frame #12: 0x0000000100403fec emacs`-[EmacsApp
run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
    frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
    frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
sigmask=0x0000000000000000) at nsterm.m:5006:10
    frame #15: 0x000000010036930c
emacs`wait_reading_process_output(time_limit=1, nsecs=0, read_kbd=0,
do_display=false, wait_for_cell=(i = 0x0000000000000000),
wait_proc=0x0000000000000000, just_wait_proc=0) at process.c:5753:18
    frame #16: 0x000000010000b400 emacs`Fsleep_for(seconds=(i =
0x0000000000000006), milliseconds=(i = 0x0000000000000000)) at
dispnew.c:6248:2
    frame #17: 0x00000001002bd5d8 emacs`eval_sub(form=(i =
0x0000000138061f43)) at eval.c:2601:15
    frame #18: 0x00000001002bdd2c emacs`Fprogn(body=(i =
0x0000000000000000)) at eval.c:439:13
    frame #19: 0x00000001002c79dc emacs`funcall_lambda(fun=(i =
0x000000015809908d), nargs=0, arg_vector=0x000000015693e868) at
eval.c:3350:15
    frame #20: 0x00000001002c6af4 emacs`funcall_general(fun=(i =
0x000000015809908d), numargs=0, args=0x000000015693e868) at
eval.c:3044:12
    frame #21: 0x00000001002bedf8 emacs`Ffuncall(nargs=1,
args=0x000000015693e860) at eval.c:3093:21
    frame #22: 0x00000001003bf944 emacs`invoke_thread_function at
thread.c:744:28
    frame #23: 0x00000001002c2a9c
emacs`internal_condition_case(bfun=(emacs`invoke_thread_function at
thread.c:741), handlers=(i = 0x0000000000000030),
hfun=(emacs`record_thread_error at thread.c:752)) at eval.c:1613:25
    frame #24: 0x00000001003be36c
emacs`run_thread(state=0x000000015693e840) at thread.c:815:3
    frame #25: 0x000000018e16c2e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #8, name = 'HIE: __ 6f45e584fd961eaa 2025-01-02 05:47:10.265'
    frame #0: 0x000000018e12af54 libsystem_kernel.dylib`mach_msg2_trap + 8
    frame #1: 0x000000018e13d604 libsystem_kernel.dylib`mach_msg2_internal + 80
    frame #2: 0x000000018e15a458 libsystem_kernel.dylib`thread_suspend + 108
    frame #3: 0x0000000194bce174
HIServices`SOME_OTHER_THREAD_SWALLOWED_AT_LEAST_ONE_EXCEPTION + 20
    frame #4: 0x000000018f41cc24 Foundation`__NSThread__start__ + 724
    frame #5: 0x000000018e16c2e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #9
    frame #0: 0x000000018e12cba4 libsystem_kernel.dylib`__workq_kernreturn + 8
(lldb)

In GNU Emacs 30.0.93 (build 1, aarch64-apple-darwin24.2.0, NS
 appkit-2575.30 Version 15.2 (Build 24C101)) of 2025-01-02 built on
 foo.local
Repository revision: 43a8f0de5ecbe8430917968af703f4a630567461
Repository branch: emacs-30
Windowing system distributor 'Apple', version 10.3.2575
System Description:  macOS 15.2

Configured using:
 'configure --without-dbus --with-native-compilation
 --enable-checking=yes --enable-check-lisp-object-type
 'CFLAGS=-I/opt/homebrew/opt/libmps/include -O0 -g3'
 LDFLAGS=-L/opt/homebrew/opt/libmps/lib'

Configured features:
ACL GNUTLS LCMS2 LIBXML2 MODULES NATIVE_COMP NOTIFY KQUEUE NS PDUMPER
PNG SQLITE3 THREADS TOOLKIT_SCROLL_BARS TREE_SITTER WEBP ZLIB

Important settings:
  value of $LC_CTYPE: UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr compile comint ansi-osc ansi-color ring comp-run
bytecomp byte-compile comp-common rx emacsbug message mailcap yank-media
puny dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived
epg rfc6068 epg-config gnus-util text-property-search time-date subr-x
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/ns-win ns-win ucs-normalize mule-util term/common-win tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads kqueue cocoa ns lcms2
multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 55821 9594) (symbols 48 6633 0) (strings 32 15516 4510)
 (string-bytes 1 404656) (vectors 16 11284)
 (vector-slots 8 125438 8014) (floats 8 21 14) (intervals 56 243 1)
 (buffers 992 11))

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Stefan Kangas <stefankangas@gmail.com> writes:

> I have run into a bug with make-thread on macOS 15.2, running on an M2.
>
> I can reproduce the issue consistently both on emacs-30 and master by
> evaluating this in emacs -Q:
>
>     (make-thread (lambda () (sleep-for 1)) "bug")
>
> This leads to Emacs freezing up completely within a fraction of a
> second.  I have time to move point once or maybe twice before it gets
> non-responsive, let's say within a few tenths of a second.

Curious question: Do you see a beach ball?

> When I kill the process in the lldb window with Ctrl+C, I can get the
> following (this is on emacs-30):
>
>     frame #12: 0x0000000100403fec emacs`-[EmacsApp
> run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
>     frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
> sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
>     frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
> sigmask=0x0000000000000000) at nsterm.m:5006:10

I'm asking because of bug#72496. The part of the backtrace above reminds
me a bit of what I trued to describe in that bug: EmacsApp.run never
returning to ns_select_1. Can you see if it ever returns?

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>> I have run into a bug with make-thread on macOS 15.2, running on an M2.
>>
>> I can reproduce the issue consistently both on emacs-30 and master by
>> evaluating this in emacs -Q:
>>
>>     (make-thread (lambda () (sleep-for 1)) "bug")
>>
>> This leads to Emacs freezing up completely within a fraction of a
>> second.  I have time to move point once or maybe twice before it gets
>> non-responsive, let's say within a few tenths of a second.
>
> Curious question: Do you see a beach ball?
>
>> When I kill the process in the lldb window with Ctrl+C, I can get the
>> following (this is on emacs-30):
>>
>>     frame #12: 0x0000000100403fec emacs`-[EmacsApp
>> run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
>>     frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
>> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>> sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
>>     frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
>> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>> sigmask=0x0000000000000000) at nsterm.m:5006:10
>
> I'm asking because of bug#72496. The part of the backtrace above reminds
> me a bit of what I trued to describe in that bug: EmacsApp.run never
> returning to ns_select_1. Can you see if it ever returns?

This is what Apple says about Cocoa thread-safety

https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/Multithreading/ThreadSafetySummary/ThreadSafetySummary.html#:~:text=Guidelines%20for%20using%20Cocoa%20from,are%20generally%20not%20thread%2Dsafe.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Stefan Kangas]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>> I have run into a bug with make-thread on macOS 15.2, running on an M2.
>>
>> I can reproduce the issue consistently both on emacs-30 and master by
>> evaluating this in emacs -Q:
>>
>>     (make-thread (lambda () (sleep-for 1)) "bug")
>>
>> This leads to Emacs freezing up completely within a fraction of a
>> second.  I have time to move point once or maybe twice before it gets
>> non-responsive, let's say within a few tenths of a second.
>
> Curious question: Do you see a beach ball?

Yes, I do see a beach ball.  It takes a second or so to show up, which I
guess is expected.

>> When I kill the process in the lldb window with Ctrl+C, I can get the
>> following (this is on emacs-30):
>>
>>     frame #12: 0x0000000100403fec emacs`-[EmacsApp
>> run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
>>     frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
>> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>> sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
>>     frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
>> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>> sigmask=0x0000000000000000) at nsterm.m:5006:10
>
> I'm asking because of bug#72496. The part of the backtrace above reminds
> me a bit of what I trued to describe in that bug: EmacsApp.run never
> returning to ns_select_1. Can you see if it ever returns?

After recompiling with NSTRACE_ENABLED, and based on reading at the
terminal output, it seems like it does not return to ns_select_1 after
the first line that reads

    nextEventMatchingMask should only be called from the Main Thread!

Is there a better way to see if it returns to that function?

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Stefan Kangas <stefankangas@gmail.com> writes:

>> Curious question: Do you see a beach ball?
>
> Yes, I do see a beach ball.  It takes a second or so to show up, which I
> guess is expected.

Then it's the other category of freezes I see with NS, and not
bug#72496. Which doesn't mean the cause might not be the same, in some
way.

>
>>> When I kill the process in the lldb window with Ctrl+C, I can get the
>>> following (this is on emacs-30):
>>>
>>>     frame #12: 0x0000000100403fec emacs`-[EmacsApp
>>> run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
>>>     frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
>>> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>>> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>>> sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
>>>     frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
>>> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>>> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>>> sigmask=0x0000000000000000) at nsterm.m:5006:10
>>
>> I'm asking because of bug#72496. The part of the backtrace above reminds
>> me a bit of what I trued to describe in that bug: EmacsApp.run never
>> returning to ns_select_1. Can you see if it ever returns?
>
> After recompiling with NSTRACE_ENABLED, and based on reading at the
> terminal output, it seems like it does not return to ns_select_1 after
> the first line that reads
>
>     nextEventMatchingMask should only be called from the Main Thread!
>
> Is there a better way to see if it returns to that function?

I think it doesn't matter for this bug. The curious thing about 72496 is
that EmacsApp.run apparently handles events so that no beach ball
appears, but Emacs doesn't get a chance to handle its events. It's as if
the Cocoa event handling decouples itself from Emacs. And that's because
it doesn't return to ns_select_1. Anyway, different construction site
apparently.

I have something in my notes which might or might not help:

  * Breakpoint for errors displayed by macOS on stderr

  Set a breakpoint on CGPostError to find sources of errors.

  #+begin_src sh
  To find out from where macOS prints stuff to stderr, like
  023-07-20 13:41:17.073449+0200 emacs[53072:1205906] [default]
      CGSWindowShmemCreateWithPort failed on port 0
  b _os_log_error_impl
  b _os_log_impl
  b _os_log_debug_impl
  b _os_log_fault_impl
  #+end_src

If on of these breakpoints is hit when the error is displayed, we could
perhaps see why ns_select_1 calls EmacsApp.run. In my understanding, it
shouldn't because Cocoa event handling should only be done in the main
thread.

If the breakpoints are not hit, we could instrument ns_select_1 to call
an empty function do_break on which we could set a breakpoint.

  if (![NSThread isMainThread])
    do_break ();
  [NSApp run];

Or something like that. The question would be what in the code above the
[NSApp run] is wrong so that we land here. Probably some if condition.

CC to Alan Third.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Wed, 1 Jan 2025 22:57:38 -0600
> 
> I have run into a bug with make-thread on macOS 15.2, running on an M2.
> 
> I can reproduce the issue consistently both on emacs-30 and master by
> evaluating this in emacs -Q:
> 
>     (make-thread (lambda () (sleep-for 1)) "bug")
> 
> This leads to Emacs freezing up completely within a fraction of a
> second.  I have time to move point once or maybe twice before it gets
> non-responsive, let's say within a few tenths of a second.

The above works as expected on MS-Windows and on GNU/Linux: after
about 1 sec the new thread exits, and Emacs works normally.
list-threads shows a single main thread running at that time.

> When I kill the process in the lldb window with Ctrl+C, I can get the
> following (this is on emacs-30):
> 
> [...]
> 2025-01-02 05:47:20.778199+0100 emacs[78593:1366649] [General]
> nextEventMatchingMask should only be called from the Main Thread!
> Process 78593 stopped
> * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
>     frame #0: 0x000000018e12dbbc libsystem_kernel.dylib`__psynch_mutexwait + 8
> libsystem_kernel.dylib`:
> ->  0x18e12dbbc <+8>:  b.lo   0x18e12dbdc               ; <+40>
>     0x18e12dbc0 <+12>: pacibsp
>     0x18e12dbc4 <+16>: stp    x29, x30, [sp, #-0x10]!
>     0x18e12dbc8 <+20>: mov    x29, sp
> Target 0: (emacs) stopped.
> (lldb) bt all
> * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
>   * frame #0: 0x000000018e12dbbc libsystem_kernel.dylib`__psynch_mutexwait + 8
>     frame #1: 0x000000018e1693f8
> libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_wait + 84
>     frame #2: 0x000000018e166dbc
> libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_slow + 220
>     frame #3: 0x00000001003bfb94
> emacs`sys_mutex_lock(mutex=0x0000000100b7bd30) at systhread.c:140:15
>     frame #4: 0x00000001003bcef8
> emacs`acquire_global_lock(self=0x0000000100b074d0) at thread.c:160:3
>     frame #5: 0x00000001003bdce0

This is the main thread waiting for the global lock, which is
expected.

>   thread #7, name = 'bug'
>     frame #0: 0x000000018de3a2b0
> dyld`dyld3::MachOLoaded::findClosestSymbol(unsigned long long, char
> const**, unsigned long long*) const + 488
>     frame #1: 0x000000018de1b13c dyld`dyld4::APIs::dladdr(void const*,
> dl_info*) + 236
>     frame #2: 0x000000018e012f00 libsystem_c.dylib`backtrace_symbols + 144
>     frame #3: 0x000000018f4998c0 Foundation`-[_NSCallStackArray
> descriptionWithLocale:indent:] + 144
>     frame #4: 0x000000018f3e8c10 Foundation`_NS_os_log_callback + 276
>     frame #5: 0x000000018debee60
> libsystem_trace.dylib`_os_log_fmt_flatten_NSCF + 64
>     frame #6: 0x000000018dec5830
> libsystem_trace.dylib`_os_log_fmt_flatten_object_impl + 372
>     frame #7: 0x000000018debc9c8
> libsystem_trace.dylib`_os_log_impl_flatten_and_send + 2144
>     frame #8: 0x000000018debc150 libsystem_trace.dylib`_os_log + 168
>     frame #9: 0x000000018debc0a0 libsystem_trace.dylib`_os_log_impl + 28
>     frame #10: 0x000000019209151c AppKit`-[NSApplication reportException:] + 624
>     frame #11: 0x0000000191db0118 AppKit`-[NSApplication run] + 664
>     frame #12: 0x0000000100403fec emacs`-[EmacsApp
> run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
>     frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
> sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
>     frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
> sigmask=0x0000000000000000) at nsterm.m:5006:10
>     frame #15: 0x000000010036930c
> emacs`wait_reading_process_output(time_limit=1, nsecs=0, read_kbd=0,
> do_display=false, wait_for_cell=(i = 0x0000000000000000),
> wait_proc=0x0000000000000000, just_wait_proc=0) at process.c:5753:18
>     frame #16: 0x000000010000b400 emacs`Fsleep_for(seconds=(i =
> 0x0000000000000006), milliseconds=(i = 0x0000000000000000)) at
> dispnew.c:6248:2

This is the new Lisp thread started by make-thread, but it is
somewhere in the bowels of macOS.  Is this what you see each time you
kill Emacs after it hangs?

Can someone describe how sleep-for works on macOS, i.e. what is
supposed to happen after ns_select_1 calls -[EmacsApp run], whatever
that is?  It sounds like something in that machinery conflicts with
how Lisp threads are implemented.

From the backtrace of the new Lisp thread, it looks like it finished
sleeping for 1 sec and then it proceeds to calling [NSApp run] -- is
this correct behavior for a non-main thread?  E.g., does that try to
run the main thread (which is parked inside sys_mutex_lock, waiting
for the global lock to become free)?

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> From the backtrace of the new Lisp thread, it looks like it finished
> sleeping for 1 sec and then it proceeds to calling [NSApp run]

I think that's the problem, see my other reply to Stefan.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> The above works as expected on MS-Windows and on GNU/Linux: after
> about 1 sec the new thread exits, and Emacs works normally.
> list-threads shows a single main thread running at that time.

Thanks for verifying that this is macOS specific.

>>   thread #7, name = 'bug'
>>     frame #0: 0x000000018de3a2b0
>> dyld`dyld3::MachOLoaded::findClosestSymbol(unsigned long long, char
>> const**, unsigned long long*) const + 488
>>     frame #1: 0x000000018de1b13c dyld`dyld4::APIs::dladdr(void const*,
>> dl_info*) + 236
>>     frame #2: 0x000000018e012f00 libsystem_c.dylib`backtrace_symbols + 144
>>     frame #3: 0x000000018f4998c0 Foundation`-[_NSCallStackArray
>> descriptionWithLocale:indent:] + 144
>>     frame #4: 0x000000018f3e8c10 Foundation`_NS_os_log_callback + 276
>>     frame #5: 0x000000018debee60
>> libsystem_trace.dylib`_os_log_fmt_flatten_NSCF + 64
>>     frame #6: 0x000000018dec5830
>> libsystem_trace.dylib`_os_log_fmt_flatten_object_impl + 372
>>     frame #7: 0x000000018debc9c8
>> libsystem_trace.dylib`_os_log_impl_flatten_and_send + 2144
>>     frame #8: 0x000000018debc150 libsystem_trace.dylib`_os_log + 168
>>     frame #9: 0x000000018debc0a0 libsystem_trace.dylib`_os_log_impl + 28
>>     frame #10: 0x000000019209151c AppKit`-[NSApplication reportException:] + 624
>>     frame #11: 0x0000000191db0118 AppKit`-[NSApplication run] + 664
>>     frame #12: 0x0000000100403fec emacs`-[EmacsApp
>> run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
>>     frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
>> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>> sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
>>     frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
>> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>> sigmask=0x0000000000000000) at nsterm.m:5006:10
>>     frame #15: 0x000000010036930c
>> emacs`wait_reading_process_output(time_limit=1, nsecs=0, read_kbd=0,
>> do_display=false, wait_for_cell=(i = 0x0000000000000000),
>> wait_proc=0x0000000000000000, just_wait_proc=0) at process.c:5753:18
>>     frame #16: 0x000000010000b400 emacs`Fsleep_for(seconds=(i =
>> 0x0000000000000006), milliseconds=(i = 0x0000000000000000)) at
>> dispnew.c:6248:2
>
> This is the new Lisp thread started by make-thread, but it is
> somewhere in the bowels of macOS.  Is this what you see each time you
> kill Emacs after it hangs?

Yes, the backtrace seems to be the same every time.  For example, I
tried again just now and got this over three attempts:

1. thread #18, name = 'bug'
    frame #0: 0x000000018de3a268
dyld`dyld3::MachOLoaded::findClosestSymbol(unsigned long long, char
const**, unsigned long long*) const + 416
    frame #1: 0x000000018de1b13c dyld`dyld4::APIs::dladdr(void const*,
dl_info*) + 236
    frame #2: 0x000000018e012f00 libsystem_c.dylib`backtrace_symbols + 144
    frame #3: 0x000000018f4998c0 Foundation`-[_NSCallStackArray
descriptionWithLocale:indent:] + 144

2. thread #7, name = 'bug'
    frame #0: 0x000000018de3a2b0
dyld`dyld3::MachOLoaded::findClosestSymbol(unsigned long long, char
const**, unsigned long long*) const + 488
    frame #1: 0x000000018de1b13c dyld`dyld4::APIs::dladdr(void const*,
dl_info*) + 236
    frame #2: 0x000000018e012f00 libsystem_c.dylib`backtrace_symbols + 144
    frame #3: 0x000000018f4998c0 Foundation`-[_NSCallStackArray
descriptionWithLocale:indent:] + 144

3. thread #9, name = 'bug'
    frame #0: 0x000000018de3a268
dyld`dyld3::MachOLoaded::findClosestSymbol(unsigned long long, char
const**, unsigned long long*) const + 416
    frame #1: 0x000000018de1b13c dyld`dyld4::APIs::dladdr(void const*,
dl_info*) + 236
    frame #2: 0x000000018e012f00 libsystem_c.dylib`backtrace_symbols + 144
    frame #3: 0x000000018f4998c0 Foundation`-[_NSCallStackArray
descriptionWithLocale:indent:] + 144

> Can someone describe how sleep-for works on macOS, i.e. what is
> supposed to happen after ns_select_1 calls -[EmacsApp run], whatever
> that is?  It sounds like something in that machinery conflicts with
> how Lisp threads are implemented.

Note that this is a minimized reproducer.  I first noticed the issue
after I upgraded the GNU ELPA package debbugs, which recently got
support for threads using `make-thread'.

In that package, the function `debbugs-gnu-show-reports' in
debbugs-gnu.el is called in a thread, and the backtrace is the same.
See debbugs-gnu.el:897.

I don't see `sleep-for' called directly there, but I didn't yet
investigate it very closely.  Maybe Michael (in Cc) knows more.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> Cc: 75275@debbugs.gnu.org
> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 2 Jan 2025 00:47:06 -0600
> 
> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> 
> > Stefan Kangas <stefankangas@gmail.com> writes:
> >
> >> I have run into a bug with make-thread on macOS 15.2, running on an M2.
> >>
> >> I can reproduce the issue consistently both on emacs-30 and master by
> >> evaluating this in emacs -Q:
> >>
> >>     (make-thread (lambda () (sleep-for 1)) "bug")
> >>
> >> This leads to Emacs freezing up completely within a fraction of a
> >> second.  I have time to move point once or maybe twice before it gets
> >> non-responsive, let's say within a few tenths of a second.
> >
> > Curious question: Do you see a beach ball?
> 
> Yes, I do see a beach ball.  It takes a second or so to show up, which I
> guess is expected.
> 
> >> When I kill the process in the lldb window with Ctrl+C, I can get the
> >> following (this is on emacs-30):
> >>
> >>     frame #12: 0x0000000100403fec emacs`-[EmacsApp
> >> run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
> >>     frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
> >> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
> >> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
> >> sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
> >>     frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
> >> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
> >> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
> >> sigmask=0x0000000000000000) at nsterm.m:5006:10
> >
> > I'm asking because of bug#72496. The part of the backtrace above reminds
> > me a bit of what I trued to describe in that bug: EmacsApp.run never
> > returning to ns_select_1. Can you see if it ever returns?
> 
> After recompiling with NSTRACE_ENABLED, and based on reading at the
> terminal output, it seems like it does not return to ns_select_1 after
> the first line that reads
> 
>     nextEventMatchingMask should only be called from the Main Thread!
> 
> Is there a better way to see if it returns to that function?

Who calls nextEventMatchingMask?  I don't see it in the backtraces you
posted.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 75275@debbugs.gnu.org
>> From: Stefan Kangas <stefankangas@gmail.com>
>> Date: Thu, 2 Jan 2025 00:47:06 -0600
>>
>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>>
>> > Stefan Kangas <stefankangas@gmail.com> writes:
>> >
>> >> I have run into a bug with make-thread on macOS 15.2, running on an M2.
>> >>
>> >> I can reproduce the issue consistently both on emacs-30 and master by
>> >> evaluating this in emacs -Q:
>> >>
>> >>     (make-thread (lambda () (sleep-for 1)) "bug")
>> >>
>> >> This leads to Emacs freezing up completely within a fraction of a
>> >> second.  I have time to move point once or maybe twice before it gets
>> >> non-responsive, let's say within a few tenths of a second.
>> >
>> > Curious question: Do you see a beach ball?
>>
>> Yes, I do see a beach ball.  It takes a second or so to show up, which I
>> guess is expected.
>>
>> >> When I kill the process in the lldb window with Ctrl+C, I can get the
>> >> following (this is on emacs-30):
>> >>
>> >>     frame #12: 0x0000000100403fec emacs`-[EmacsApp
>> >> run](self=0x0000000156610fe0, _cmd="run") at nsterm.m:5938:7
>> >>     frame #13: 0x00000001004024b0 emacs`ns_select_1(nfds=0,
>> >> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>> >> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>> >> sigmask=0x0000000000000000, run_loop_only=NO) at nsterm.m:4954:3
>> >>     frame #14: 0x000000010040202c emacs`ns_select(nfds=0,
>> >> readfds=0x00000001708c26bc, writefds=0x00000001708c263c,
>> >> exceptfds=0x0000000000000000, timeout=0x00000001708c2610,
>> >> sigmask=0x0000000000000000) at nsterm.m:5006:10
>> >
>> > I'm asking because of bug#72496. The part of the backtrace above reminds
>> > me a bit of what I trued to describe in that bug: EmacsApp.run never
>> > returning to ns_select_1. Can you see if it ever returns?
>>
>> After recompiling with NSTRACE_ENABLED, and based on reading at the
>> terminal output, it seems like it does not return to ns_select_1 after
>> the first line that reads
>>
>>     nextEventMatchingMask should only be called from the Main Thread!
>>
>> Is there a better way to see if it returns to that function?
>
> Who calls nextEventMatchingMask?  I don't see it in the backtraces you
> posted.

I believe the below two lines are relevant, but note that the function
in that case is actually called "_nextEventMatchingEventMask".  IOW, I
had assumed that "nextEventMatchingMask" is just what the error message
(perhaps confusingly) said:

2   AppKit 0x0000000192721c24 -[NSApplication(NSEventRouting)
_nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3072
3   AppKit 0x0000000191db0060 -[NSApplication run] + 480

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Stefan Kangas <stefankangas@gmail.com>,  Alan Third <alan@idiocy.org>,
>   75275@debbugs.gnu.org
> Date: Thu, 02 Jan 2025 08:30:25 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > From the backtrace of the new Lisp thread, it looks like it finished
> > sleeping for 1 sec and then it proceeds to calling [NSApp run]
> 
> I think that's the problem, see my other reply to Stefan.

So should we add a condition before calling [NSApp run] that we are in
the main thread?

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 2 Jan 2025 01:31:39 -0600
> Cc: 75275@debbugs.gnu.org, Michael Albinus <michael.albinus@gmx.de>
> 
> Note that this is a minimized reproducer.  I first noticed the issue
> after I upgraded the GNU ELPA package debbugs, which recently got
> support for threads using `make-thread'.
> 
> In that package, the function `debbugs-gnu-show-reports' in
> debbugs-gnu.el is called in a thread, and the backtrace is the same.
> See debbugs-gnu.el:897.
> 
> I don't see `sleep-for' called directly there, but I didn't yet
> investigate it very closely.  Maybe Michael (in Cc) knows more.

The call to sleep-for is just a trigger.  What you need to look for is
a call to ns_select_1, which calls [NSApp run].

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Stefan Kangas <stefankangas@gmail.com>,  Alan Third <alan@idiocy.org>,
>>   75275@debbugs.gnu.org
>> Date: Thu, 02 Jan 2025 08:30:25 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > From the backtrace of the new Lisp thread, it looks like it finished
>> > sleeping for 1 sec and then it proceeds to calling [NSApp run]
>> 
>> I think that's the problem, see my other reply to Stefan.
>
> So should we add a condition before calling [NSApp run] that we are in
> the main thread?

ATM, I don't understand how we land in that line in ns_select_1 if not
[NSThread isMainThread]. Maybe I need new glasses. I asked Stefan if he
can see something in LLDB.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>>> Cc: Stefan Kangas <stefankangas@gmail.com>,  Alan Third <alan@idiocy.org>,
>>>   75275@debbugs.gnu.org
>>> Date: Thu, 02 Jan 2025 08:30:25 +0100
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> > From the backtrace of the new Lisp thread, it looks like it finished
>>> > sleeping for 1 sec and then it proceeds to calling [NSApp run]
>>> 
>>> I think that's the problem, see my other reply to Stefan.
>>
>> So should we add a condition before calling [NSApp run] that we are in
>> the main thread?
>
> ATM, I don't understand how we land in that line in ns_select_1 if not
> [NSThread isMainThread]. Maybe I need new glasses. I asked Stefan if he
> can see something in LLDB.

It must something in here:

  if (![NSThread isMainThread]
      || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
    thread_select (pselect, nfds, readfds, writefds,
		   exceptfds, timeout, sigmask);

Should we return here?

  else
    {
      struct timespec t = {0, 0};
      thread_select (pselect, 0, NULL, NULL, NULL, &t, sigmask);
    }

  /* FIXME: This draining of outerpool causes a crash when a buffer
     running over tramp is displayed and the user tries to use the
     menus.  I believe some other autorelease pool's lifetime
     straddles this call causing a violation of autorelease pool
     nesting.  There's no good reason to keep these here since the
     pool will be drained some other time anyway, but removing them
     leaves the menus sometimes not opening until the user moves their
     mouse pointer, but that's better than a crash.

     There must be something about running external processes like
     tramp that interferes with the modal menu code.

     See bugs 24472, 37557, 37922.  */

  // [outerpool release];
  // outerpool = [[NSAutoreleasePool alloc] init];

What is the following about? What is select_mutex or, for example?

  send_appdefined = YES;
  if (nr > 0)
    {
      pthread_mutex_lock (&select_mutex);
      select_nfds = nfds;
      select_valid = 0;
      if (readfds)
        {
          select_readfds = *readfds;
          select_valid += SELECT_HAVE_READ;
        }
      if (writefds)
        {
          select_writefds = *writefds;
          select_valid += SELECT_HAVE_WRITE;
        }

      if (timeout)
        {
          select_timeout = *timeout;
          select_valid += SELECT_HAVE_TMO;
        }

      pthread_mutex_unlock (&select_mutex);

      /* Inform fd_handler that select should be called.  */
      c = 'g';
      emacs_write_sig (selfds[1], &c, 1);
    }
  else if (nr == 0 && timeout)
    {
      /* No file descriptor, just a timeout, no need to wake fd_handler.  */
      double time = timespectod (*timeout);
      timed_entry = [[NSTimer scheduledTimerWithTimeInterval: time
                                                      target: NSApp
                                                    selector:
                                  @selector (timeout_handler:)
                                                    userInfo: 0
                                                     repeats: NO]
                      retain];
    }
  else /* No timeout and no file descriptors, can this happen?  */
    {
      /* Send appdefined so we exit from the loop.  */
      ns_send_appdefined (-1);
    }

  block_input ();
  ns_init_events (&event);

  [NSApp run];

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>>> Cc: Stefan Kangas <stefankangas@gmail.com>,  Alan Third <alan@idiocy.org>,
>>>   75275@debbugs.gnu.org
>>> Date: Thu, 02 Jan 2025 08:30:25 +0100
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> > From the backtrace of the new Lisp thread, it looks like it finished
>>> > sleeping for 1 sec and then it proceeds to calling [NSApp run]
>>> 
>>> I think that's the problem, see my other reply to Stefan.
>>
>> So should we add a condition before calling [NSApp run] that we are in
>> the main thread?
>
> ATM, I don't understand how we land in that line in ns_select_1 if not
> [NSThread isMainThread]. Maybe I need new glasses. I asked Stefan if he
> can see something in LLDB.

Actually, consider that fast explained. I somehow hallucinated a return
in that here. But I still don't know if putting an if around the NSApp
run is sufficient or not. I don't understand the code, TBH.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: stefankangas@gmail.com,  alan@idiocy.org,  75275@debbugs.gnu.org
> Date: Thu, 02 Jan 2025 09:41:38 +0100
> 
> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> >> So should we add a condition before calling [NSApp run] that we are in
> >> the main thread?
> >
> > ATM, I don't understand how we land in that line in ns_select_1 if not
> > [NSThread isMainThread]. Maybe I need new glasses. I asked Stefan if he
> > can see something in LLDB.
> 
> It must something in here:
> 
>   if (![NSThread isMainThread]
>       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
>     thread_select (pselect, nfds, readfds, writefds,
> 		   exceptfds, timeout, sigmask);
> 
> Should we return here?

I don't know.  Is there anything in the following code that can be
relevant to a non-main thread?  Note that non-main threads can
legitimately call wait_reading_process_output, which calls ns_select.
For example, what happens if a non-main Lisp thread starts a
sub-process? we do expect to be able to read the output from that
sub-process.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: stefankangas@gmail.com,  alan@idiocy.org,  75275@debbugs.gnu.org
>> Date: Thu, 02 Jan 2025 09:41:38 +0100
>> 
>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>> 
>> > Eli Zaretskii <eliz@gnu.org> writes:
>> >
>> >> So should we add a condition before calling [NSApp run] that we are in
>> >> the main thread?
>> >
>> > ATM, I don't understand how we land in that line in ns_select_1 if not
>> > [NSThread isMainThread]. Maybe I need new glasses. I asked Stefan if he
>> > can see something in LLDB.
>> 
>> It must something in here:
>> 
>>   if (![NSThread isMainThread]
>>       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
>>     thread_select (pselect, nfds, readfds, writefds,
>> 		   exceptfds, timeout, sigmask);
>> 
>> Should we return here?
>
> I don't know.  Is there anything in the following code that can be
> relevant to a non-main thread?  Note that non-main threads can
> legitimately call wait_reading_process_output, which calls ns_select.
> For example, what happens if a non-main Lisp thread starts a
> sub-process? we do expect to be able to read the output from that
> sub-process.

Really hard to tell. Perhaps someone could try to follow what I write
below and tell if it makes sense? Everything in ns_select_1.

1. I think this code must run in a non-main thread:

  if (nr > 0)
    {
      pthread_mutex_lock (&select_mutex);
      ... set some variables ...
      /* Inform fd_handler that select should be called.  */
      c = 'g';
      emacs_write_sig (selfds[1], &c, 1);
    }

selfds is apparently some pipe, NS-specific. The function fd_handler is
called when writing to the pipe I assume. fd_handler is set up like
this

  [NSThread detachNewThreadSelector:@selector (fd_handler:)
                           toTarget:NSApp
                         withObject:nil];

Looks to me like it runs in a thread of its own. fd_handler then
pselects on the fd sets set in the if above. That looks like it is
relevant to reading process output. And that means we may _not_ return
from ns_select_1 early when ![NSThread isMainThread].

  else if (nr == 0 && timeout)
    {
      /* No file descriptor, just a timeout, no need to wake fd_handler.  */
      double time = timespectod (*timeout);
      timed_entry = [[NSTimer scheduledTimerWithTimeInterval: time
                                                      target: NSApp
                                                    selector:
                                  @selector (timeout_handler:)
                                                    userInfo: 0
                                                     repeats: NO]
                      retain];
    }

2. This code

  else if (nr == 0 && timeout)
    {
      /* No file descriptor, just a timeout, no need to wake fd_handler.  */
      double time = timespectod (*timeout);
      timed_entry = [[NSTimer scheduledTimerWithTimeInterval: time
                                                      target: NSApp
                                                    selector:
                                  @selector (timeout_handler:)
                                                    userInfo: 0
                                                     repeats: NO]
                      retain];
    }

  means basically only to send an app-defined event after a timeout. I
  interpret this as "leave the NS event loop to let Emacs do things
  after a timeout". Looks okay to me.

3. This

  else /* No timeout and no file descriptors, can this happen?  */
    {
      /* Send appdefined so we exit from the loop.  */
      ns_send_appdefined (-1);
    }

  is likely also okay because send_app_defined has code checking for
  being in the main thread.

4. The [NSApp run] follows, and it can under no circumstances be done
   in a mon-main thread. We should put that in an if for sure.

  if ([NSThread isMainThread]) [NSApp run];

5. The code below is another enigma.

  I can't figure out why that is done, and what
  last_appdefined_event_data is for. But since it is run today, I'd
  propose to just let it run. I don't see that it does immediate harm.

:-/

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Kangas <stefankangas@gmail.com> writes:

Hi Stefan,

> Note that this is a minimized reproducer.  I first noticed the issue
> after I upgraded the GNU ELPA package debbugs, which recently got
> support for threads using `make-thread'.
>
> In that package, the function `debbugs-gnu-show-reports' in
> debbugs-gnu.el is called in a thread, and the backtrace is the same.
> See debbugs-gnu.el:897.
>
> I don't see `sleep-for' called directly there, but I didn't yet
> investigate it very closely.  Maybe Michael (in Cc) knows more.

In debbugs-gnu.el, there are some invocations of sit-for. But they are
not related to the threads machinery in debbugs-gnu-show-reports.

Best regards, Michael.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

On Thu, Jan 02, 2025 at 11:04:11AM +0100, Gerd Möllmann wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> >> Cc: stefankangas@gmail.com,  alan@idiocy.org,  75275@debbugs.gnu.org
> >> Date: Thu, 02 Jan 2025 09:41:38 +0100
> >> 
> >> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> >> 
> >> > Eli Zaretskii <eliz@gnu.org> writes:
> >> >
> >> >> So should we add a condition before calling [NSApp run] that we are in
> >> >> the main thread?
> >> >
> >> > ATM, I don't understand how we land in that line in ns_select_1 if not
> >> > [NSThread isMainThread]. Maybe I need new glasses. I asked Stefan if he
> >> > can see something in LLDB.
> >> 
> >> It must something in here:
> >> 
> >>   if (![NSThread isMainThread]
> >>       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
> >>     thread_select (pselect, nfds, readfds, writefds,
> >> 		   exceptfds, timeout, sigmask);
> >> 
> >> Should we return here?

Yes. We used to be but it was removed:

9370a4763aacbb9278b5be9c92a2484e3652bc29

I don't know why it was removed, but I'd bet that, at the very least,
the isMainThread check should have moved with the 'NSApp == nil'
check.

> > I don't know.  Is there anything in the following code that can be
> > relevant to a non-main thread?  Note that non-main threads can
> > legitimately call wait_reading_process_output, which calls ns_select.
> > For example, what happens if a non-main Lisp thread starts a
> > sub-process? we do expect to be able to read the output from that
> > sub-process.

My take on how this works was that in a non-main thread ns_select
should just act like pselect, hence it used to literally just call
pselect and return. This would hopefully allow Emacs to handle IO
correctly without having to make the NS runloop code do things it
can't do.

FWIW, I still think the NS code in its current form is unsuitable for
multi-threaded use and must be rewritten.

> Really hard to tell. Perhaps someone could try to follow what I write
> below and tell if it makes sense? Everything in ns_select_1.
> 
> 1. I think this code must run in a non-main thread:
> 
>   if (nr > 0)
>     {
>       pthread_mutex_lock (&select_mutex);
>       ... set some variables ...
>       /* Inform fd_handler that select should be called.  */
>       c = 'g';
>       emacs_write_sig (selfds[1], &c, 1);
>     }
> 
> selfds is apparently some pipe, NS-specific. The function fd_handler is
> called when writing to the pipe I assume. fd_handler is set up like
> this
> 
>   [NSThread detachNewThreadSelector:@selector (fd_handler:)
>                            toTarget:NSApp
>                          withObject:nil];
> 
> Looks to me like it runs in a thread of its own. fd_handler then
> pselects on the fd sets set in the if above. That looks like it is
> relevant to reading process output. And that means we may _not_ return
> from ns_select_1 early when ![NSThread isMainThread].
> 
>   else if (nr == 0 && timeout)
>     {
>       /* No file descriptor, just a timeout, no need to wake fd_handler.  */
>       double time = timespectod (*timeout);
>       timed_entry = [[NSTimer scheduledTimerWithTimeInterval: time
>                                                       target: NSApp
>                                                     selector:
>                                   @selector (timeout_handler:)
>                                                     userInfo: 0
>                                                      repeats: NO]
>                       retain];
>     }

No, none of that needs to run when we're not in the main thread.
fd_handler run pselect in a separate thread because the NS main thread
has to run the ns main thread run loop to handle incoming IO from the
window system.

The NS run loop can emulate parts of pselect, but not the whole thing,
so we are required to run both the NS runloop and pselect
simultaneously, hence fd_handler. If we don't need to run the runloop,
i.e. we're in a non-main thread, then we can just run pselect directly
and ignore fd_handler.

> 2. This code
> 
>   else if (nr == 0 && timeout)
>     {
>       /* No file descriptor, just a timeout, no need to wake fd_handler.  */
>       double time = timespectod (*timeout);
>       timed_entry = [[NSTimer scheduledTimerWithTimeInterval: time
>                                                       target: NSApp
>                                                     selector:
>                                   @selector (timeout_handler:)
>                                                     userInfo: 0
>                                                      repeats: NO]
>                       retain];
>     }
> 
>   means basically only to send an app-defined event after a timeout. I
>   interpret this as "leave the NS event loop to let Emacs do things
>   after a timeout". Looks okay to me.

Correct. In more detail it sends an "App defined" event to the main
thread which signals to the run loop to stop itself.

> 3. This
> 
>   else /* No timeout and no file descriptors, can this happen?  */
>     {
>       /* Send appdefined so we exit from the loop.  */
>       ns_send_appdefined (-1);
>     }
> 
>   is likely also okay because send_app_defined has code checking for
>   being in the main thread.

This will send the app defined event to the main thread run loop. The
code in ns_send_appdefined actually instructs the main thread runloop
to send itself the event if called from a non-main thread.

> 4. The [NSApp run] follows, and it can under no circumstances be done
>    in a mon-main thread. We should put that in an if for sure.
> 
>   if ([NSThread isMainThread]) [NSApp run];

In this circumstance no. In *Step each thread has its own run loop and
event queue, so if you call [NSApp run] on a sub-thread it will look
at its own queue, which in this case likely has nothing on it, and it
will run forever. That's by design, that's how you're supposed to
write NS apps. We obviously don't want to do that.

> 5. The code below is another enigma.
> 
>   I can't figure out why that is done, and what
>   last_appdefined_event_data is for. But since it is run today, I'd
>   propose to just let it run. I don't see that it does immediate harm.

It's essentially just working out how the run loop was terminated (by
fd_handler, by some window system input, or by timeout) and creating
suitable return values by, for example, gathering the results of the
pselect run in fd_handler.

Basically, none of this needs to run in a sub-thread. We should be
able to just run pselect directly and return. Perhaps there's some
other edge case that prevents that, but I suspect it was just
overlooked. After all, nobody understands this code.
-- 
Alan Third

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Alan Third <alan@idiocy.org> writes:

> No, none of that needs to run when we're not in the main thread.
> fd_handler run pselect in a separate thread because the NS main thread
> has to run the ns main thread run loop to handle incoming IO from the
> window system.
>
> The NS run loop can emulate parts of pselect, but not the whole thing,
> so we are required to run both the NS runloop and pselect
> simultaneously, hence fd_handler. If we don't need to run the runloop,
> i.e. we're in a non-main thread, then we can just run pselect directly
> and ignore fd_handler.

So IIUC, you'd write this in ns_select_1

  if (![NSThread isMainThread]
      || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
    thread_select (pselect, nfds, readfds, writefds,
		   exceptfds, timeout, sigmask);

as return "return thread_select(...)"?

One strange thing about threads having their own event queue is that the
message Stefan sees comes from the NSApplicaiton::run in ns_select_1. Be
that at it may.

(BTW, I've given up on NS completely, meanwhile. Always building
--without-ns.)

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

On Thu, Jan 02, 2025 at 02:05:44PM +0100, Gerd Möllmann wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > No, none of that needs to run when we're not in the main thread.
> > fd_handler run pselect in a separate thread because the NS main thread
> > has to run the ns main thread run loop to handle incoming IO from the
> > window system.
> >
> > The NS run loop can emulate parts of pselect, but not the whole thing,
> > so we are required to run both the NS runloop and pselect
> > simultaneously, hence fd_handler. If we don't need to run the runloop,
> > i.e. we're in a non-main thread, then we can just run pselect directly
> > and ignore fd_handler.
> 
> So IIUC, you'd write this in ns_select_1
> 
>   if (![NSThread isMainThread]
>       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
>     thread_select (pselect, nfds, readfds, writefds,
> 		   exceptfds, timeout, sigmask);
> 
> as return "return thread_select(...)"?

I don't know. The code that in Emacs 29 looked like:

  if (NSApp == nil
      || ![NSThread isMainThread]
      || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
    return thread_select (pselect, nfds, readfds, writefds,
			  exceptfds, timeout, sigmask);
  else
    {
      struct timespec t = {0, 0};
      thread_select (pselect, 0, NULL, NULL, NULL, &t, sigmask);
    }

Now looks like:

  /* emacs -nw doesn't have an NSApp, so we're done.  */
  if (NSApp == nil)
    return thread_select (pselect, nfds, readfds, writefds, exceptfds,
			  timeout, sigmask);

  if (![NSThread isMainThread]
      || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
    thread_select (pselect, nfds, readfds, writefds,
		   exceptfds, timeout, sigmask);
  else
    {
      struct timespec t = {0, 0};
      thread_select (pselect, 0, NULL, NULL, NULL, &t, sigmask);
    }

and I don't know why. The change (9370a4763aa) has no bug report
listed and I'm pretty sure I wasn't asked about it, so I have to
assume Po Lu had some reason for the main thread and timeout checks to
result in falling-through to the rest of the function.

> One strange thing about threads having their own event queue is that the
> message Stefan sees comes from the NSApplicaiton::run in ns_select_1. Be
> that at it may.

Indeed, ns_select_1 is being run in a sub-thread and it therefore runs
a thread-specific run queue which has no events in it. It requires an
event to be sent to it so it will stop but none ever is because we
only ever send events to the main thread's event queue.

-- 
Alan Third

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Alan Third <alan@idiocy.org> writes:

> On Thu, Jan 02, 2025 at 02:05:44PM +0100, Gerd Möllmann wrote:
>> Alan Third <alan@idiocy.org> writes:
>> 
>> > No, none of that needs to run when we're not in the main thread.
>> > fd_handler run pselect in a separate thread because the NS main thread
>> > has to run the ns main thread run loop to handle incoming IO from the
>> > window system.
>> >
>> > The NS run loop can emulate parts of pselect, but not the whole thing,
>> > so we are required to run both the NS runloop and pselect
>> > simultaneously, hence fd_handler. If we don't need to run the runloop,
>> > i.e. we're in a non-main thread, then we can just run pselect directly
>> > and ignore fd_handler.
>> 
>> So IIUC, you'd write this in ns_select_1
>> 
>>   if (![NSThread isMainThread]
>>       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
>>     thread_select (pselect, nfds, readfds, writefds,
>> 		   exceptfds, timeout, sigmask);
>> 
>> as return "return thread_select(...)"?
>
> I don't know. The code that in Emacs 29 looked like:
>
>   if (NSApp == nil
>       || ![NSThread isMainThread]
>       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
>     return thread_select (pselect, nfds, readfds, writefds,
> 			  exceptfds, timeout, sigmask);
>   else
>     {
>       struct timespec t = {0, 0};
>       thread_select (pselect, 0, NULL, NULL, NULL, &t, sigmask);
>     }
>
> Now looks like:
>
>   /* emacs -nw doesn't have an NSApp, so we're done.  */
>   if (NSApp == nil)
>     return thread_select (pselect, nfds, readfds, writefds, exceptfds,
> 			  timeout, sigmask);
>
>   if (![NSThread isMainThread]
>       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
>     thread_select (pselect, nfds, readfds, writefds,
> 		   exceptfds, timeout, sigmask);
>   else
>     {
>       struct timespec t = {0, 0};
>       thread_select (pselect, 0, NULL, NULL, NULL, &t, sigmask);
>     }
>
> and I don't know why. The change (9370a4763aa) has no bug report
> listed and I'm pretty sure I wasn't asked about it, so I have to
> assume Po Lu had some reason for the main thread and timeout checks to
> result in falling-through to the rest of the function.

OK, let's see if Po Lu remembers.

>> One strange thing about threads having their own event queue is that the
>> message Stefan sees comes from the NSApplicaiton::run in ns_select_1. Be
>> that at it may.
>
> Indeed, ns_select_1 is being run in a sub-thread and it therefore runs
> a thread-specific run queue which has no events in it. It requires an
> event to be sent to it so it will stop but none ever is because we
> only ever send events to the main thread's event queue.

What I meant was the the NSApplication::run in ns_select_1 in Stefan's
case seems to call something complaining about being called in the wrong
thread. I guess it would also complain if there were an event in the
queue. I don't know. This whole code gets on my nerves :-).

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

On Thu, Jan 02, 2025 at 03:03:54PM +0100, Gerd Möllmann wrote:
> >> One strange thing about threads having their own event queue is that the
> >> message Stefan sees comes from the NSApplicaiton::run in ns_select_1. Be
> >> that at it may.
> >
> > Indeed, ns_select_1 is being run in a sub-thread and it therefore runs
> > a thread-specific run queue which has no events in it. It requires an
> > event to be sent to it so it will stop but none ever is because we
> > only ever send events to the main thread's event queue.
> 
> What I meant was the the NSApplication::run in ns_select_1 in Stefan's
> case seems to call something complaining about being called in the wrong
> thread. I guess it would also complain if there were an event in the
> queue. I don't know.

Oh yeah, that's a good point. The queue in NSApp will be the main
queue rather than the thread's queue.

> This whole code gets on my nerves :-).

Tell me about it.
-- 
Alan Third

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Stefan Kangas]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> I have something in my notes which might or might not help:
>
>   * Breakpoint for errors displayed by macOS on stderr
>
>   Set a breakpoint on CGPostError to find sources of errors.
>
>   #+begin_src sh
>   To find out from where macOS prints stuff to stderr, like
>   023-07-20 13:41:17.073449+0200 emacs[53072:1205906] [default]
>       CGSWindowShmemCreateWithPort failed on port 0
>   b _os_log_error_impl
>   b _os_log_impl
>   b _os_log_debug_impl
>   b _os_log_fault_impl
>   #+end_src
>
> If on of these breakpoints is hit when the error is displayed, we could
> perhaps see why ns_select_1 calls EmacsApp.run. In my understanding, it
> shouldn't because Cocoa event handling should only be done in the main
> thread.
>
> If the breakpoints are not hit, we could instrument ns_select_1 to call
> an empty function do_break on which we could set a breakpoint.
>
>   if (![NSThread isMainThread])
>     do_break ();
>   [NSApp run];
>
> Or something like that. The question would be what in the code above the
> [NSApp run] is wrong so that we land here. Probably some if condition.
>
> CC to Alan Third.

Thanks, I didn't have time to look into the above yet, but I noticed
that the bug wasn't there on emacs-29 and bisected the issue:

4ac4cec652ffaca4333d8f297b8a6c0e5bd79c68 is the first bad commit
commit 4ac4cec652ffaca4333d8f297b8a6c0e5bd79c68
Author: Gerd Möllmann <gerd@gnu.org>
Date:   Sat Mar 9 15:06:29 2024 +0100

    Prevent freezes on macOS (bug#69561)

    * src/nsterm.m (ns_select_1): Store pending input_events. Always call
    [NSApp run].

 src/nsterm.m | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Stefan Kangas <stefankangas@gmail.com> writes:

> Thanks, I didn't have time to look into the above yet, but I noticed
> that the bug wasn't there on emacs-29 and bisected the issue:
>
> 4ac4cec652ffaca4333d8f297b8a6c0e5bd79c68 is the first bad commit
> commit 4ac4cec652ffaca4333d8f297b8a6c0e5bd79c68
> Author: Gerd Möllmann <gerd@gnu.org>
> Date:   Sat Mar 9 15:06:29 2024 +0100
>
>     Prevent freezes on macOS (bug#69561)

Thanks, there he is, the culprit :-).

I don't know what to do now.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>> Thanks, I didn't have time to look into the above yet, but I noticed
>> that the bug wasn't there on emacs-29 and bisected the issue:
>>
>> 4ac4cec652ffaca4333d8f297b8a6c0e5bd79c68 is the first bad commit
>> commit 4ac4cec652ffaca4333d8f297b8a6c0e5bd79c68
>> Author: Gerd Möllmann <gerd@gnu.org>
>> Date:   Sat Mar 9 15:06:29 2024 +0100
>>
>>     Prevent freezes on macOS (bug#69561)
>
> Thanks, there he is, the culprit :-).
>
> I don't know what to do now.

Maybe this one, after what Alan explained. Could you try it?

1 file changed, 2 insertions(+), 2 deletions(-)
src/nsterm.m | 4 ++--

modified   src/nsterm.m
@@ -4875,8 +4875,8 @@ Function modeled after x_draw_glyph_string_box ().
 
   if (![NSThread isMainThread]
       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
-    thread_select (pselect, nfds, readfds, writefds,
-		   exceptfds, timeout, sigmask);
+    return thread_select (pselect, nfds, readfds, writefds,
+			  exceptfds, timeout, sigmask);
   else
     {
       struct timespec t = {0, 0};

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Stefan Kangas]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Maybe this one, after what Alan explained. Could you try it?
>
> 1 file changed, 2 insertions(+), 2 deletions(-)
> src/nsterm.m | 4 ++--
>
> modified   src/nsterm.m
> @@ -4875,8 +4875,8 @@ Function modeled after x_draw_glyph_string_box ().
>
>    if (![NSThread isMainThread]
>        || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
> -    thread_select (pselect, nfds, readfds, writefds,
> -		   exceptfds, timeout, sigmask);
> +    return thread_select (pselect, nfds, readfds, writefds,
> +			  exceptfds, timeout, sigmask);
>    else
>      {
>        struct timespec t = {0, 0};

That patch fixes it for me, indeed.  Thanks!

Eli, since this is a regression, do you agree that this should go to
emacs-30?

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> Date: Thu, 2 Jan 2025 11:03:50 +0000
> From: Alan Third <alan@idiocy.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, stefankangas@gmail.com,
> 	75275@debbugs.gnu.org
> 
> > > I don't know.  Is there anything in the following code that can be
> > > relevant to a non-main thread?  Note that non-main threads can
> > > legitimately call wait_reading_process_output, which calls ns_select.
> > > For example, what happens if a non-main Lisp thread starts a
> > > sub-process? we do expect to be able to read the output from that
> > > sub-process.
> 
> My take on how this works was that in a non-main thread ns_select
> should just act like pselect, hence it used to literally just call
> pselect and return.

In general, this is not right: calls to ns_select are expected to call
thread_select, in all threads, so that other threads could get a
chance on grabbing the global lock while this (non-main) thread waits
for sub-process output.

Maybe this cannot currently work in the NS build, but then it means
any non-main thread will never relinquish the global lock until its
function returns or signals an error.

> FWIW, I still think the NS code in its current form is unsuitable for
> multi-threaded use and must be rewritten.

Then maybe we should stop supporting a with-threads build in NS?

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Thu, 2 Jan 2025 11:03:50 +0000
>> From: Alan Third <alan@idiocy.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, stefankangas@gmail.com,
>> 	75275@debbugs.gnu.org
>> 
>> > > I don't know.  Is there anything in the following code that can be
>> > > relevant to a non-main thread?  Note that non-main threads can
>> > > legitimately call wait_reading_process_output, which calls ns_select.
>> > > For example, what happens if a non-main Lisp thread starts a
>> > > sub-process? we do expect to be able to read the output from that
>> > > sub-process.
>> 
>> My take on how this works was that in a non-main thread ns_select
>> should just act like pselect, hence it used to literally just call
>> pselect and return.
>
> In general, this is not right: calls to ns_select are expected to call
> thread_select, in all threads, so that other threads could get a
> chance on grabbing the global lock while this (non-main) thread waits
> for sub-process output.

I think Alan meant thread_select(pselect, ...) because that's what is
already done in ns_select_1.

But... when I look at fd_handler, that function calls pselect directly.

          result = pselect (nfds, &readfds, wfds, NULL, tmo, NULL);


Can that be right?

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

On Thu, Jan 02, 2025 at 04:37:19PM +0100, Gerd Möllmann wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Date: Thu, 2 Jan 2025 11:03:50 +0000
> >> From: Alan Third <alan@idiocy.org>
> >> Cc: Eli Zaretskii <eliz@gnu.org>, stefankangas@gmail.com,
> >> 	75275@debbugs.gnu.org
> >> 
> >> > > I don't know.  Is there anything in the following code that can be
> >> > > relevant to a non-main thread?  Note that non-main threads can
> >> > > legitimately call wait_reading_process_output, which calls ns_select.
> >> > > For example, what happens if a non-main Lisp thread starts a
> >> > > sub-process? we do expect to be able to read the output from that
> >> > > sub-process.
> >> 
> >> My take on how this works was that in a non-main thread ns_select
> >> should just act like pselect, hence it used to literally just call
> >> pselect and return.
> >
> > In general, this is not right: calls to ns_select are expected to call
> > thread_select, in all threads, so that other threads could get a
> > chance on grabbing the global lock while this (non-main) thread waits
> > for sub-process output.
> 
> I think Alan meant thread_select(pselect, ...) because that's what is
> already done in ns_select_1.

Yes, sorry. What Gerd said.

> But... when I look at fd_handler, that function calls pselect directly.
> 
>           result = pselect (nfds, &readfds, wfds, NULL, tmo, NULL);
> 
> 
> Can that be right?

I think that's OK as we've already called thread_select further up.
I've a vague memory of putting this in as a work-around.

We call thread_select with a zero timeout to give other threads a
chance to grab the lock, then carry on as normal, running the run loop
and fd_handler which calls pselect.

To be fair, though, I think when we set this up I didn't understand
the code as well as I do now, so perhaps it was the wrong thing to do.
-- 
Alan Third

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

[-- Attachment #1: Type: text/plain, Size: 502 bytes --]

On Thu, Jan 02, 2025 at 09:19:36AM -0600, Stefan Kangas wrote:
> That patch fixes it for me, indeed.  Thanks!

Hi Stefan, any chance you could check this version?

I have a suspicion that Gerd's will break C-g again. I'm not sure this
will be suitable for Emacs 30 though as it's a slightly larger change.

The C-g thing is hard to test as I think it relied on Gerd's machine
randomly hanging and him hammering C-g and it doing nothing. I'm not
sure how to replicate that original hang.
-- 
Alan Third

[-- Attachment #2: 0001-Fix-C-g-handling-on-NS-bug-75275.patch --]
[-- Type: text/plain, Size: 3781 bytes --]

From 528dcb934485a5a1331b4cf851216a3750df3210 Mon Sep 17 00:00:00 2001
From: Alan Third <alan@idiocy.org>
Date: Thu, 2 Jan 2025 15:56:20 +0000
Subject: [PATCH] Fix C-g handling on NS (bug#75275)

* src/nsterm.m (ns_select_1): Remove function.
(ns_select): Reinstate by copying the contents of ns_select_1 back
in. Add check for C-g at top. Stop fall-through to NS run loop when not
required.
([EmacsView showFontPanel]): Just call [NSApp run] directly, calling
ns_select for this makes ns_select more complex.
---
 src/nsterm.m | 44 ++++++++++++++++++--------------------------
 1 file changed, 18 insertions(+), 26 deletions(-)

diff --git a/src/nsterm.m b/src/nsterm.m
index a4398e79211..652eee1f988 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -4826,9 +4826,9 @@ Function modeled after x_draw_glyph_string_box ().
 
 
 static int
-ns_select_1 (int nfds, fd_set *readfds, fd_set *writefds,
-	     fd_set *exceptfds, struct timespec *timeout,
-	     sigset_t *sigmask, BOOL run_loop_only)
+ns_select (int nfds, fd_set *readfds, fd_set *writefds,
+           fd_set *exceptfds, struct timespec *timeout,
+           sigset_t *sigmask)
 /* --------------------------------------------------------------------------
      Replacement for select, checking for events
    -------------------------------------------------------------------------- */
@@ -4837,6 +4837,7 @@ Function modeled after x_draw_glyph_string_box ().
   int t, k, nr = 0;
   struct input_event event;
   char c;
+  NSEvent *ev;
 
   NSTRACE_WHEN (NSTRACE_GROUP_EVENTS, "ns_select");
 
@@ -4844,15 +4845,16 @@ Function modeled after x_draw_glyph_string_box ().
   check_native_fs ();
 #endif
 
-  /* If there are input events pending, store them so that Emacs can
-     recognize C-g.  (And we must make sure [NSApp run] is called in
-     this function, so that C-g has a chance to land in
-     hold_event_q.)  */
-  if (hold_event_q.nr > 0)
+  /* Check if there are any C-g events on the queue.  */
+  while ([NSThread isMainThread]
+         && (ev = [NSApp nextEventMatchingMask:NSEventMaskKeyDown
+                                    untilDate:nil
+                                       inMode:NSDefaultRunLoopMode
+                                       dequeue:NO]))
     {
-      for (int i = 0; i < hold_event_q.nr; ++i)
-        kbd_buffer_store_event_hold (&hold_event_q.q[i], NULL);
-      hold_event_q.nr = 0;
+      if ([ev modifierFlags] == NSEventModifierFlagControl
+          && [[ev charactersIgnoringModifiers] isEqualToString:@"g"])
+        Vquit_flag = Qt;
     }
 
   eassert (nfds <= FD_SETSIZE);
@@ -4863,13 +4865,10 @@ Function modeled after x_draw_glyph_string_box ().
     }
 
   /* emacs -nw doesn't have an NSApp, so we're done.  */
-  if (NSApp == nil)
-    return thread_select (pselect, nfds, readfds, writefds, exceptfds,
-			  timeout, sigmask);
-
-  if (![NSThread isMainThread]
+  if (NSApp == nil
+      || ![NSThread isMainThread]
       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
-    thread_select (pselect, nfds, readfds, writefds,
+    return thread_select (pselect, nfds, readfds, writefds,
 		   exceptfds, timeout, sigmask);
   else
     {
@@ -4992,14 +4991,6 @@ Function modeled after x_draw_glyph_string_box ().
   return result;
 }
 
-int
-ns_select (int nfds, fd_set *readfds, fd_set *writefds,
-	   fd_set *exceptfds, struct timespec *timeout,
-	   sigset_t *sigmask)
-{
-  return ns_select_1 (nfds, readfds, writefds, exceptfds,
-		      timeout, sigmask, NO);
-}
 
 #ifdef HAVE_PTHREAD
 void
@@ -6790,7 +6781,8 @@ - (Lisp_Object) showFontPanel
 	 && [[fm fontPanel: YES] isVisible]
 #endif
 	 )
-    ns_select_1 (0, NULL, NULL, NULL, &timeout, NULL, YES);
+
+  [NSApp run];
   unblock_input ();
 
   if (font_panel_result)
-- 
2.41.0

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Alan Third <alan@idiocy.org> writes:

>> But... when I look at fd_handler, that function calls pselect directly.
>> 
>>           result = pselect (nfds, &readfds, wfds, NULL, tmo, NULL);
>> 
>> 
>> Can that be right?
>
> I think that's OK as we've already called thread_select further up.
> I've a vague memory of putting this in as a work-around.
>
> We call thread_select with a zero timeout to give other threads a
> chance to grab the lock, then carry on as normal, running the run loop
> and fd_handler which calls pselect.
>
> To be fair, though, I think when we set this up I didn't understand
> the code as well as I do now, so perhaps it was the wrong thing to do.

OK, thanks!

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 2 Jan 2025 09:19:36 -0600
> Cc: 75275@debbugs.gnu.org, Alan Third <alan@idiocy.org>, Eli Zaretskii <eliz@gnu.org>
> 
> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> 
> > Maybe this one, after what Alan explained. Could you try it?
> >
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> > src/nsterm.m | 4 ++--
> >
> > modified   src/nsterm.m
> > @@ -4875,8 +4875,8 @@ Function modeled after x_draw_glyph_string_box ().
> >
> >    if (![NSThread isMainThread]
> >        || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
> > -    thread_select (pselect, nfds, readfds, writefds,
> > -		   exceptfds, timeout, sigmask);
> > +    return thread_select (pselect, nfds, readfds, writefds,
> > +			  exceptfds, timeout, sigmask);
> >    else
> >      {
> >        struct timespec t = {0, 0};
> 
> That patch fixes it for me, indeed.  Thanks!
> 
> Eli, since this is a regression, do you agree that this should go to
> emacs-30?

Yes, thanks.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

On Thu, Jan 02, 2025 at 04:06:25PM +0000, Alan Third wrote:
> On Thu, Jan 02, 2025 at 09:19:36AM -0600, Stefan Kangas wrote:
> > That patch fixes it for me, indeed.  Thanks!
> 
> Hi Stefan, any chance you could check this version?

Actually, don't bother. It hangs after some random amount of time.
-- 
Alan Third

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> Date: Thu, 2 Jan 2025 16:06:25 +0000
> From: Alan Third <alan@idiocy.org>
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> 	75275@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
> 
> On Thu, Jan 02, 2025 at 09:19:36AM -0600, Stefan Kangas wrote:
> > That patch fixes it for me, indeed.  Thanks!
> 
> Hi Stefan, any chance you could check this version?
> 
> I have a suspicion that Gerd's will break C-g again.

Did C-g work before Gerd's commit back in March?  If not, this is not
a regression.  If it did work, can you explain how the change Gerd
proposed now will break C-g?

> I'm not sure this will be suitable for Emacs 30 though as it's a
> slightly larger change.

It isn't, indeed.  We could install it on master, but not on emacs-30.

Thanks.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Thu, 2 Jan 2025 16:06:25 +0000
>> From: Alan Third <alan@idiocy.org>
>> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>> 	75275@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
>> 
>> On Thu, Jan 02, 2025 at 09:19:36AM -0600, Stefan Kangas wrote:
>> > That patch fixes it for me, indeed.  Thanks!
>> 
>> Hi Stefan, any chance you could check this version?
>> 
>> I have a suspicion that Gerd's will break C-g again.
>
> Did C-g work before Gerd's commit back in March?  If not, this is not
> a regression.  If it did work, can you explain how the change Gerd
> proposed now will break C-g?

It's different. I made the change because Emacs freezed and C-g did
nothing. With my change, Emacs freezed and I could get out with C-g, at
least sometimes.

Note that this didn't address the freezes themselves. The reason for the
freezes is still unknown to me. As is the reason for the freezes without
beach ball.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Alan Third <alan@idiocy.org>,  stefankangas@gmail.com,
>   75275@debbugs.gnu.org
> Date: Thu, 02 Jan 2025 18:09:46 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Date: Thu, 2 Jan 2025 16:06:25 +0000
> >> From: Alan Third <alan@idiocy.org>
> >> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> >> 	75275@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
> >> 
> >> On Thu, Jan 02, 2025 at 09:19:36AM -0600, Stefan Kangas wrote:
> >> > That patch fixes it for me, indeed.  Thanks!
> >> 
> >> Hi Stefan, any chance you could check this version?
> >> 
> >> I have a suspicion that Gerd's will break C-g again.
> >
> > Did C-g work before Gerd's commit back in March?  If not, this is not
> > a regression.  If it did work, can you explain how the change Gerd
> > proposed now will break C-g?
> 
> It's different. I made the change because Emacs freezed and C-g did
> nothing. With my change, Emacs freezed and I could get out with C-g, at
> least sometimes.

Your change included 2 hunks, and the suggestion is to revert only
one of them.  My question is whether this is likely to reintroduce
some regression.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> Your change included 2 hunks, and the suggestion is to revert only
> one of them.  My question is whether this is likely to reintroduce
> some regression.

Not that I know of. It restores things to what they were before,
prevents calling NSApplication::run from being called for non main
threads, and the first hunk should still work as before. But note that
Alan thinks differently.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

On Thu, Jan 02, 2025 at 07:22:21PM +0200, Eli Zaretskii wrote:
> > From: Gerd Möllmann <gerd.moellmann@gmail.com>
> > Cc: Alan Third <alan@idiocy.org>,  stefankangas@gmail.com,
> >   75275@debbugs.gnu.org
> > Date: Thu, 02 Jan 2025 18:09:46 +0100
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > >> Date: Thu, 2 Jan 2025 16:06:25 +0000
> > >> From: Alan Third <alan@idiocy.org>
> > >> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> > >> 	75275@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
> > >> 
> > >> On Thu, Jan 02, 2025 at 09:19:36AM -0600, Stefan Kangas wrote:
> > >> > That patch fixes it for me, indeed.  Thanks!
> > >> 
> > >> Hi Stefan, any chance you could check this version?
> > >> 
> > >> I have a suspicion that Gerd's will break C-g again.
> > >
> > > Did C-g work before Gerd's commit back in March?  If not, this is not
> > > a regression.  If it did work, can you explain how the change Gerd
> > > proposed now will break C-g?
> > 
> > It's different. I made the change because Emacs freezed and C-g did
> > nothing. With my change, Emacs freezed and I could get out with C-g, at
> > least sometimes.
> 
> Your change included 2 hunks, and the suggestion is to revert only
> one of them.  My question is whether this is likely to reintroduce
> some regression.

Perhaps I'm wrong. Gerd, can you remember, did your change for C-g
require ns_select to run the NS runloop, or did it just require the
clearing out of the input queue? I'm just concerned because this
change will reintroduce situations where it won't run the runloop.

The more I think about it the more I think I'm wrong...
-- 
Alan Third

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

On Thu, Jan 02, 2025 at 06:25:46PM +0100, Gerd Möllmann wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Your change included 2 hunks, and the suggestion is to revert only
> > one of them.  My question is whether this is likely to reintroduce
> > some regression.
> 
> Not that I know of. It restores things to what they were before,
> prevents calling NSApplication::run from being called for non main
> threads, and the first hunk should still work as before. But note that
> Alan thinks differently.

Sorry, if Gerd's sure then I expect I'm misremembering how it worked.

Please ignore me.
-- 
Alan Third

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Alan Third <alan@idiocy.org> writes:

> On Thu, Jan 02, 2025 at 07:22:21PM +0200, Eli Zaretskii wrote:
>> > From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> > Cc: Alan Third <alan@idiocy.org>,  stefankangas@gmail.com,
>> >   75275@debbugs.gnu.org
>> > Date: Thu, 02 Jan 2025 18:09:46 +0100
>> > 
>> > Eli Zaretskii <eliz@gnu.org> writes:
>> > 
>> > >> Date: Thu, 2 Jan 2025 16:06:25 +0000
>> > >> From: Alan Third <alan@idiocy.org>
>> > >> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>> > >> 	75275@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
>> > >> 
>> > >> On Thu, Jan 02, 2025 at 09:19:36AM -0600, Stefan Kangas wrote:
>> > >> > That patch fixes it for me, indeed.  Thanks!
>> > >> 
>> > >> Hi Stefan, any chance you could check this version?
>> > >> 
>> > >> I have a suspicion that Gerd's will break C-g again.
>> > >
>> > > Did C-g work before Gerd's commit back in March?  If not, this is not
>> > > a regression.  If it did work, can you explain how the change Gerd
>> > > proposed now will break C-g?
>> > 
>> > It's different. I made the change because Emacs freezed and C-g did
>> > nothing. With my change, Emacs freezed and I could get out with C-g, at
>> > least sometimes.
>> 
>> Your change included 2 hunks, and the suggestion is to revert only
>> one of them.  My question is whether this is likely to reintroduce
>> some regression.
>
> Perhaps I'm wrong. Gerd, can you remember, did your change for C-g
> require ns_select to run the NS runloop, or did it just require the
> clearing out of the input queue? I'm just concerned because this
> change will reintroduce situations where it won't run the runloop.
>
> The more I think about it the more I think I'm wrong...

AFAIR my theory went like:

- [NSApp run] + key event handler put C-g in the hold queue

- ns_select_1 calls "run" (it did before my change).

- I couldn't find how input events from the hold queue come
  to Emacs in the whole process, so I added that

- The "call run in all threads" was then a mistake

Seemed to work, to a degree.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Alan Third <alan@idiocy.org> writes:

> On Thu, Jan 02, 2025 at 06:25:46PM +0100, Gerd Möllmann wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Your change included 2 hunks, and the suggestion is to revert only
>> > one of them.  My question is whether this is likely to reintroduce
>> > some regression.
>> 
>> Not that I know of. It restores things to what they were before,
>> prevents calling NSApplication::run from being called for non main
>> threads, and the first hunk should still work as before. But note that
>> Alan thinks differently.
>
> Sorry, if Gerd's sure then I expect I'm misremembering how it worked.
>
> Please ignore me.

Rather not :-). Please see my other reply.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Alan Third <alan@idiocy.org> writes:
>
>> On Thu, Jan 02, 2025 at 07:22:21PM +0200, Eli Zaretskii wrote:
>>> > From: Gerd Möllmann <gerd.moellmann@gmail.com>
>>> > Cc: Alan Third <alan@idiocy.org>,  stefankangas@gmail.com,
>>> >   75275@debbugs.gnu.org
>>> > Date: Thu, 02 Jan 2025 18:09:46 +0100
>>> > 
>>> > Eli Zaretskii <eliz@gnu.org> writes:
>>> > 
>>> > >> Date: Thu, 2 Jan 2025 16:06:25 +0000
>>> > >> From: Alan Third <alan@idiocy.org>
>>> > >> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>>> > >> 	75275@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
>>> > >> 
>>> > >> On Thu, Jan 02, 2025 at 09:19:36AM -0600, Stefan Kangas wrote:
>>> > >> > That patch fixes it for me, indeed.  Thanks!
>>> > >> 
>>> > >> Hi Stefan, any chance you could check this version?
>>> > >> 
>>> > >> I have a suspicion that Gerd's will break C-g again.
>>> > >
>>> > > Did C-g work before Gerd's commit back in March?  If not, this is not
>>> > > a regression.  If it did work, can you explain how the change Gerd
>>> > > proposed now will break C-g?
>>> > 
>>> > It's different. I made the change because Emacs freezed and C-g did
>>> > nothing. With my change, Emacs freezed and I could get out with C-g, at
>>> > least sometimes.
>>> 
>>> Your change included 2 hunks, and the suggestion is to revert only
>>> one of them.  My question is whether this is likely to reintroduce
>>> some regression.
>>
>> Perhaps I'm wrong. Gerd, can you remember, did your change for C-g
>> require ns_select to run the NS runloop, or did it just require the
>> clearing out of the input queue? I'm just concerned because this
>> change will reintroduce situations where it won't run the runloop.
>>
>> The more I think about it the more I think I'm wrong...
>
> AFAIR my theory went like:
>
> - [NSApp run] + key event handler put C-g in the hold queue
>
> - ns_select_1 calls "run" (it did before my change).
>
> - I couldn't find how input events from the hold queue come
>   to Emacs in the whole process, so I added that
>
> - The "call run in all threads" was then a mistake
>
> Seemed to work, to a degree.

Maybe I should add that that is kind of a loop. It can be that the first
call to ns_select_1 has no C-g in the hold queue, NSApp.run leads to one
being put in the hold queue. A second ns_select_1 then finds C-g and
gives it Emacs and so on.

Why that whole thing hangs, is another question.

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Alan Third]

On Thu, Jan 02, 2025 at 06:52:11PM +0100, Gerd Möllmann wrote:
> >
> > AFAIR my theory went like:
> >
> > - [NSApp run] + key event handler put C-g in the hold queue
> >
> > - ns_select_1 calls "run" (it did before my change).
> >
> > - I couldn't find how input events from the hold queue come
> >   to Emacs in the whole process, so I added that
> >
> > - The "call run in all threads" was then a mistake
> >
> > Seemed to work, to a degree.
> 
> Maybe I should add that that is kind of a loop. It can be that the first
> call to ns_select_1 has no C-g in the hold queue, NSApp.run leads to one
> being put in the hold queue. A second ns_select_1 then finds C-g and
> gives it Emacs and so on.
> 
> Why that whole thing hangs, is another question.

I suspect it's because we removed the code in bug 65843.

We removed that because there was a crash on start using a specific
desktop file. Something to do with a certain amount of iconified
frames or something. It never made much sense to me, but if the
comment that went along with the code was right then sometimes the app
defined event is never delivered and therefore the event loop doesn't
stop.

I can't believe that's a genuine bug in Apple's code, because surely
they'd have fixed it by now, so probably it's something in our code,
but who knows what.

You could try reverting that change and see if it solves your hangs.
If so then we'll have to find another solution, like perhaps just
preventing ns_send_appdefined from doing anything while we're creating
frames.
-- 
Alan Third

bug#75275: 30.0.92; `make-thread` bug on macOS 15.2

[Gerd Möllmann]

Alan Third <alan@idiocy.org> writes:

> On Thu, Jan 02, 2025 at 06:52:11PM +0100, Gerd Möllmann wrote:
>> >
>> > AFAIR my theory went like:
>> >
>> > - [NSApp run] + key event handler put C-g in the hold queue
>> >
>> > - ns_select_1 calls "run" (it did before my change).
>> >
>> > - I couldn't find how input events from the hold queue come
>> >   to Emacs in the whole process, so I added that
>> >
>> > - The "call run in all threads" was then a mistake
>> >
>> > Seemed to work, to a degree.
>> 
>> Maybe I should add that that is kind of a loop. It can be that the first
>> call to ns_select_1 has no C-g in the hold queue, NSApp.run leads to one
>> being put in the hold queue. A second ns_select_1 then finds C-g and
>> gives it Emacs and so on.
>> 
>> Why that whole thing hangs, is another question.
>
> I suspect it's because we removed the code in bug 65843.
>
> We removed that because there was a crash on start using a specific
> desktop file. Something to do with a certain amount of iconified
> frames or something. It never made much sense to me, but if the
> comment that went along with the code was right then sometimes the app
> defined event is never delivered and therefore the event loop doesn't
> stop.

That sounds a bit like the second category of freezes I've seen in the
past, the ones without beach ball, where Emacs apparently handles
Cocoa events, but no input events are transferred to keyboard.c.

I think the other category, the freezes with beach ball are older than
bug#65834. I've had them just from the start when using Emacs again.

> I can't believe that's a genuine bug in Apple's code, because surely
> they'd have fixed it by now, so probably it's something in our code,
> but who knows what.

I don't believe it's an Apple problem either, for the same reason.

> You could try reverting that change and see if it solves your hangs.
> If so then we'll have to find another solution, like perhaps just
> preventing ns_send_appdefined from doing anything while we're creating
> frames.

I'll pass. I've never found a reproducer for either category of freeze,
and now I've had enough, at least for some time :-).