bug#13840: Fix for Emacs 24.3rc3 nextstep crash (menubar + ange-ftp + ns_select = emacs_abort)

["Silas S. Brown" <ssb22@cam.ac.uk>]

Hi, I do believe I've fixed it.  (but probably broke a
dozen GNU rules in the process ..  sorry ..  maybe a
real Emacs developer can do it properly?)  Patch below.

I downloaded emacs-24.3-rc3.tar.xz, did ./configure
--with-ns and then changed -O2 to -O0 in Makefile and
*/Makefile before running make install.  Then I ran
nextstep/Emacs.app/Contents/MacOS/Emacs inside gdb with
the -q flag (so it doesn't read my configuration), and
visited /ftp:192.168.0.3:/home/ssb22/wiki/WelcomePage (a
text file on my Linux box that opens in Fundamental mode).
Then if I click on the menu bar (Help, Buffers, etc) it
crashes immediately, reproducible every time.
An example backtrace is at the end of this email.

As far as I can tell, some code to populate the Buffers
menu is resulting in a call to verify_visited_file_modtime
on the FTP file.  This is calling the FTP process, which
results in a call to ns_select (in src/nsterm.m) during
the collection of FTP's output.  ns_select then calls
emacs_abort because ++apploopnr != 1.

The static variable 'apploopnr' is apparently designed
to make sure the [NSApp run] statements in
ns_read_socket and ns_select do not re-enter under any
circumstances.  I'm not familiar enough with Mac APIs to
understand exactly what "NSApp run" does, but I guess
the problem *might* be to do with the fact that we're
assuming we can't re-enter the API under any
circumstances but actually we're already in the API due
to handling a menubar click?  (just guessing.)

So I tried commenting out the 2 calls to emacs_abort in
ns_read_socket and ns_select.  Then when I click on the
menubar, it doesn't crash.  But the menus are all empty,
unless you first go into a non-FTP buffer and click the
menubar from there.

I then tried having the emacs_abort calls commented out
but DON'T run NSApp under these circumstances, i.e. I did:

  if (++apploopnr != 1) {} // do nothing
  else
  [NSApp run];

and this time it seemed the menus were less likely to be
empty, but there was still a certain probability that a
menu would be empty, unless you first click the menubar
from a non-FTP buffer.

Finally, I tried going back earlier in the ns_select
function, to the part that says:

  if (NSApp == nil
      || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
    return pselect (nfds, readfds, writefds, exceptfds, timeout, sigmask);

and I added " || apploopnr != 0" after NSApp == nil. 
(In other words, if apploopnr is wrong then just delegate
to POSIX pselect and ignore the timeout, it's better than
crashing.)  Then all the menus started working perfectly
and with no crashes so far.

So, my fix is the following patch (I just did "diff -u",
sorry if I should have done something else) -

---------- cut here --------------------------------
--- emacs-24.3/src/nsterm.m~	2013-01-04 09:38:13.000000000 +0000
+++ emacs-24.3/src/nsterm.m	2013-03-15 11:29:10.000000000 +0000
@@ -3461,7 +3461,7 @@
       if (writefds && FD_ISSET(k, writefds)) ++nr;
     }
 
-  if (NSApp == nil
+  if (NSApp == nil || apploopnr != 0
       || (timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0))
     return pselect (nfds, readfds, writefds, exceptfds, timeout, sigmask);
 
---------- cut here --------------------------------

This is sort-of "cargo cult programming" because I don't
100% understand what I'm doing.  But I hope it will at
least give an Emacs developer an idea.
I've added a couple of emails from the top of nsterm.m
to the CC list; sorry if that was the wrong thing to do.
(How am I supposed to say "hey I've got a patch now"?)
I really hope this issue can be fixed in time for 24.3.

Thanks.     Silas

(below is an example traceback of the unpatched code)

Program received signal SIGABRT, Aborted.
0x00007fff8986382a in __kill ()
(gdb) #0  0x00007fff8986382a in __kill ()
#1  0x0000000100126b85 in terminate_due_to_signal ()
#2  0x00000001001598a7 in emacs_abort ()
#3  0x00000001002d6600 in ns_term_shutdown ()
#4  0x00000001001298fc in shut_down_emacs ()
#5  0x0000000100126b31 in terminate_due_to_signal ()
#6  0x00000001001598a7 in emacs_abort ()
#7  0x00000001002d30c7 in ns_select ()
#8  0x000000010027c733 in wait_reading_process_output ()
#9  0x000000010027adc9 in Faccept_process_output ()
#10 0x000000010020380c in Ffuncall ()
#11 0x000000010026b2fb in exec_byte_code ()
#12 0x000000010026a176 in Fbyte_code ()
#13 0x000000010020185d in eval_sub ()
#14 0x00000001001feef2 in internal_lisp_condition_case ()
#15 0x000000010026c8f7 in exec_byte_code ()
#16 0x0000000100204636 in funcall_lambda ()
#17 0x0000000100203b46 in Ffuncall ()
#18 0x000000010026b2fb in exec_byte_code ()
#19 0x0000000100204636 in funcall_lambda ()
#20 0x0000000100203b46 in Ffuncall ()
#21 0x000000010026b2fb in exec_byte_code ()
#22 0x0000000100204636 in funcall_lambda ()
#23 0x0000000100203b46 in Ffuncall ()
#24 0x000000010026b2fb in exec_byte_code ()
#25 0x0000000100204636 in funcall_lambda ()
#26 0x0000000100203b46 in Ffuncall ()
#27 0x000000010026b2fb in exec_byte_code ()
#28 0x0000000100204636 in funcall_lambda ()
#29 0x0000000100203b46 in Ffuncall ()
#30 0x0000000100201f92 in Fapply ()
#31 0x000000010020352a in Ffuncall ()
#32 0x000000010026b2fb in exec_byte_code ()
#33 0x000000010026a176 in Fbyte_code ()
#34 0x000000010020185d in eval_sub ()
#35 0x00000001001feef2 in internal_lisp_condition_case ()
#36 0x000000010026c8f7 in exec_byte_code ()
#37 0x0000000100204636 in funcall_lambda ()
#38 0x0000000100203b46 in Ffuncall ()
#39 0x0000000100201f92 in Fapply ()
#40 0x000000010020352a in Ffuncall ()
#41 0x000000010026b2fb in exec_byte_code ()
#42 0x0000000100204636 in funcall_lambda ()
#43 0x0000000100203b46 in Ffuncall ()
#44 0x0000000100201f92 in Fapply ()
#45 0x000000010020166b in eval_sub ()
#46 0x00000001001fe90a in internal_catch ()
#47 0x000000010026c792 in exec_byte_code ()
#48 0x000000010026a176 in Fbyte_code ()
#49 0x000000010020185d in eval_sub ()
#50 0x00000001001fe90a in internal_catch ()
#51 0x000000010026c792 in exec_byte_code ()
#52 0x0000000100204636 in funcall_lambda ()
#53 0x0000000100203b46 in Ffuncall ()
#54 0x000000010020166b in eval_sub ()
#55 0x00000001001feef2 in internal_lisp_condition_case ()
#56 0x000000010026c8f7 in exec_byte_code ()
#57 0x0000000100204636 in funcall_lambda ()
#58 0x0000000100203b46 in Ffuncall ()
#59 0x0000000100202e1d in call2 ()
#60 0x000000010018d6aa in Fverify_visited_file_modtime ()
#61 0x00000001002017eb in eval_sub ()
#62 0x0000000100201707 in eval_sub ()
#63 0x00000001001fca58 in For ()
#64 0x00000001002013fe in eval_sub ()
#65 0x00000001001fcae8 in Fand ()
#66 0x00000001002013fe in eval_sub ()
#67 0x00000001001fca58 in For ()
#68 0x00000001002013fe in eval_sub ()
#69 0x0000000100201009 in Feval ()
#70 0x000000010013cbe5 in eval_dyn ()
#71 0x00000001001ff215 in internal_condition_case_1 ()
#72 0x000000010013cc8b in menu_item_eval_property ()
#73 0x000000010013d5cf in parse_menu_item ()
#74 0x000000010008b90b in single_menu_item ()
#75 0x000000010014b869 in map_keymap_item ()
#76 0x000000010014ba95 in map_keymap_internal ()
#77 0x000000010014bed5 in map_keymap_canonical ()
#78 0x000000010008b824 in single_keymap_panes ()
#79 0x000000010008c1e5 in parse_single_submenu ()
#80 0x00000001002f097f in ns_update_menubar ()
#81 0x00000001002f1c04 in -[EmacsMenu menuNeedsUpdate:] ()
#82 0x00007fff86a77c09 in -[NSMenu _populateFromDelegateWithEventRef:] ()
#83 0x00007fff8696c38c in -[NSMenu _populateWithEventRef:] ()
#84 0x00007fff86a76e33 in -[NSCarbonMenuImpl _carbonPopulateEvent:handlerCallRef:] ()
#85 0x00007fff86a769bb in NSSLMMenuEventHandler ()
#86 0x00007fff8f333234 in DispatchEventToHandlers ()
#87 0x00007fff8f332840 in SendEventToEventTargetInternal ()
#88 0x00007fff8f33267a in SendEventToEventTargetWithOptions ()
#89 0x00007fff8f364d29 in SendMenuPopulate ()
#90 0x00007fff8f4b1195 in SendMenuOpening ()
#91 0x00007fff8f4b46d9 in DrawTheMenu ()
#92 0x00007fff8f4b4be3 in MenuChanged ()
#93 0x00007fff8f384a2f in TrackMenuCommon ()
#94 0x00007fff8f37ff13 in MenuSelectCore ()
#95 0x00007fff8f37f592 in _HandleMenuSelection2 ()
#96 0x00007fff869771da in _NSHandleCarbonMenuEvent ()
#97 0x00007fff8690ccc9 in _DPSNextEvent ()
#98 0x00007fff8690c07d in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#99 0x00007fff869089b9 in -[NSApplication run] ()
#100 0x00000001002d30eb in ns_select ()
#101 0x000000010027c733 in wait_reading_process_output ()
#102 0x0000000100011a3f in sit_for ()
#103 0x00000001001304e4 in read_char ()
#104 0x00000001001417d5 in read_key_sequence ()
#105 0x000000010012cd7c in command_loop_1 ()
#106 0x00000001001ff07a in internal_condition_case ()
#107 0x000000010012c34f in command_loop_2 ()
#108 0x00000001001fe90a in internal_catch ()
#109 0x000000010012c2d2 in command_loop ()
#110 0x000000010012b6d7 in recursive_edit_1 ()
#111 0x000000010012b92a in Frecursive_edit ()
#112 0x0000000100128d95 in main ()
(gdb) #0  0x00007fff8986382a in __kill ()
#1  0x0000000100126b85 in terminate_due_to_signal ()
#2  0x00000001001598a7 in emacs_abort ()
#3  0x00000001002d6600 in ns_term_shutdown ()
#4  0x00000001001298fc in shut_down_emacs ()
#5  0x0000000100126b31 in terminate_due_to_signal ()
#6  0x00000001001598a7 in emacs_abort ()
#7  0x00000001002d30c7 in ns_select ()
#8  0x000000010027c733 in wait_reading_process_output ()
#9  0x000000010027adc9 in Faccept_process_output ()
#10 0x000000010020380c in Ffuncall ()
#11 0x000000010026b2fb in exec_byte_code ()
#12 0x000000010026a176 in Fbyte_code ()
#13 0x000000010020185d in eval_sub ()
#14 0x00000001001feef2 in internal_lisp_condition_case ()
#15 0x000000010026c8f7 in exec_byte_code ()
#16 0x0000000100204636 in funcall_lambda ()
#17 0x0000000100203b46 in Ffuncall ()
#18 0x000000010026b2fb in exec_byte_code ()
#19 0x0000000100204636 in funcall_lambda ()
#20 0x0000000100203b46 in Ffuncall ()
#21 0x000000010026b2fb in exec_byte_code ()
#22 0x0000000100204636 in funcall_lambda ()
#23 0x0000000100203b46 in Ffuncall ()
#24 0x000000010026b2fb in exec_byte_code ()
#25 0x0000000100204636 in funcall_lambda ()
#26 0x0000000100203b46 in Ffuncall ()
#27 0x000000010026b2fb in exec_byte_code ()
#28 0x0000000100204636 in funcall_lambda ()
#29 0x0000000100203b46 in Ffuncall ()
#30 0x0000000100201f92 in Fapply ()
#31 0x000000010020352a in Ffuncall ()
#32 0x000000010026b2fb in exec_byte_code ()
#33 0x000000010026a176 in Fbyte_code ()
#34 0x000000010020185d in eval_sub ()
#35 0x00000001001feef2 in internal_lisp_condition_case ()
#36 0x000000010026c8f7 in exec_byte_code ()
#37 0x0000000100204636 in funcall_lambda ()
#38 0x0000000100203b46 in Ffuncall ()
#39 0x0000000100201f92 in Fapply ()
#40 0x000000010020352a in Ffuncall ()
#41 0x000000010026b2fb in exec_byte_code ()
#42 0x0000000100204636 in funcall_lambda ()
#43 0x0000000100203b46 in Ffuncall ()
#44 0x0000000100201f92 in Fapply ()
#45 0x000000010020166b in eval_sub ()
#46 0x00000001001fe90a in internal_catch ()
#47 0x000000010026c792 in exec_byte_code ()
#48 0x000000010026a176 in Fbyte_code ()
#49 0x000000010020185d in eval_sub ()
#50 0x00000001001fe90a in internal_catch ()
#51 0x000000010026c792 in exec_byte_code ()
#52 0x0000000100204636 in funcall_lambda ()
#53 0x0000000100203b46 in Ffuncall ()
#54 0x000000010020166b in eval_sub ()
#55 0x00000001001feef2 in internal_lisp_condition_case ()
#56 0x000000010026c8f7 in exec_byte_code ()
#57 0x0000000100204636 in funcall_lambda ()
#58 0x0000000100203b46 in Ffuncall ()
#59 0x0000000100202e1d in call2 ()
#60 0x000000010018d6aa in Fverify_visited_file_modtime ()
#61 0x00000001002017eb in eval_sub ()
#62 0x0000000100201707 in eval_sub ()
#63 0x00000001001fca58 in For ()
#64 0x00000001002013fe in eval_sub ()
#65 0x00000001001fcae8 in Fand ()
#66 0x00000001002013fe in eval_sub ()
#67 0x00000001001fca58 in For ()
#68 0x00000001002013fe in eval_sub ()
#69 0x0000000100201009 in Feval ()
#70 0x000000010013cbe5 in eval_dyn ()
#71 0x00000001001ff215 in internal_condition_case_1 ()
#72 0x000000010013cc8b in menu_item_eval_property ()
#73 0x000000010013d5cf in parse_menu_item ()
#74 0x000000010008b90b in single_menu_item ()
#75 0x000000010014b869 in map_keymap_item ()
#76 0x000000010014ba95 in map_keymap_internal ()
#77 0x000000010014bed5 in map_keymap_canonical ()
#78 0x000000010008b824 in single_keymap_panes ()
#79 0x000000010008c1e5 in parse_single_submenu ()
#80 0x00000001002f097f in ns_update_menubar ()
#81 0x00000001002f1c04 in -[EmacsMenu menuNeedsUpdate:] ()
#82 0x00007fff86a77c09 in -[NSMenu _populateFromDelegateWithEventRef:] ()
#83 0x00007fff8696c38c in -[NSMenu _populateWithEventRef:] ()
#84 0x00007fff86a76e33 in -[NSCarbonMenuImpl _carbonPopulateEvent:handlerCallRef:] ()
#85 0x00007fff86a769bb in NSSLMMenuEventHandler ()
#86 0x00007fff8f333234 in DispatchEventToHandlers ()
#87 0x00007fff8f332840 in SendEventToEventTargetInternal ()
#88 0x00007fff8f33267a in SendEventToEventTargetWithOptions ()
#89 0x00007fff8f364d29 in SendMenuPopulate ()
#90 0x00007fff8f4b1195 in SendMenuOpening ()
#91 0x00007fff8f4b46d9 in DrawTheMenu ()
#92 0x00007fff8f4b4be3 in MenuChanged ()
#93 0x00007fff8f384a2f in TrackMenuCommon ()
#94 0x00007fff8f37ff13 in MenuSelectCore ()
#95 0x00007fff8f37f592 in _HandleMenuSelection2 ()
#96 0x00007fff869771da in _NSHandleCarbonMenuEvent ()
#97 0x00007fff8690ccc9 in _DPSNextEvent ()
#98 0x00007fff8690c07d in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#99 0x00007fff869089b9 in -[NSApplication run] ()
#100 0x00000001002d30eb in ns_select ()
#101 0x000000010027c733 in wait_reading_process_output ()
#102 0x0000000100011a3f in sit_for ()
#103 0x00000001001304e4 in read_char ()
#104 0x00000001001417d5 in read_key_sequence ()
#105 0x000000010012cd7c in command_loop_1 ()
#106 0x00000001001ff07a in internal_condition_case ()
#107 0x000000010012c34f in command_loop_2 ()
#108 0x00000001001fe90a in internal_catch ()
#109 0x000000010012c2d2 in command_loop ()
#110 0x000000010012b6d7 in recursive_edit_1 ()
#111 0x000000010012b92a in Frecursive_edit ()
#112 0x0000000100128d95 in main ()
(gdb) 

-- 
Silas S Brown http://people.ds.cam.ac.uk/ssb22