bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Andrew John De Angelis]

- emacs -Q
- Eval: (turn-on-flyspell)
- write a word that's not present in the dictionary
   (I wrote "incorrectlll")
- place cursor at word
- M-x flyspell-correct-word-before-point
- press down arrow (<down>) to navigate to the 'Save word' option
- emacs crashes
   (in the terminal, this is the message:
   Fatal error 11: Segmentation fault
   zsh: abort      emacs -Q
- (note: the crash only happens if I try to navigate to the option
   using keys: when I use a mouse/trackpad, there is no apparent issue

I've tried to follow the instructions to use the gdb debugger but I
wasn't able to get it to run: I don't have a src directory, as I built
emacs using homebrew. If there's anything else I can do to provide
useful information, let me know.


In GNU Emacs 29.0.50 (build 1, aarch64-apple-darwin21.6.0, NS
appkit-2113.60 Version 12.6 (Build 21G115)) of 2022-10-03 built on
andrews-mbp.lan
Windowing system distributor 'Apple', version 10.3.2113
System Description:  macOS 12.6

Configured using:
'configure --disable-dependency-tracking --disable-silent-rules
--enable-locallisppath=/opt/homebrew/share/emacs/site-lisp
--infodir=/opt/homebrew/Cellar/emacs-plus@29/29.0.50/share/info/emacs
--prefix=/opt/homebrew/Cellar/emacs-plus@29/29.0.50 --with-xml2
--with-gnutls --with-native-compilation --without-compress-install
--without-dbus --with-imagemagick --with-modules --with-rsvg
--with-xwidgets --with-ns --disable-ns-self-contained 'CFLAGS=-Os -w
-pipe -mmacosx-version-min=12
-isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk
-DFD_SETSIZE=10000 -DDARWIN_UNLIMITED_SELECT'
'CPPFLAGS=-I/opt/homebrew/opt/zlib/include
-I/opt/homebrew/opt/jpeg/include -I/opt/homebrew/opt/icu4c/include
-I/opt/homebrew/opt/openssl@1.1/include
-I/opt/homebrew/opt/readline/include -isystem/opt/homebrew/include
-F/opt/homebrew/Frameworks
-isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk'
'LDFLAGS=-L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/jpeg/lib
-L/opt/homebrew/opt/icu4c/lib -L/opt/homebrew/opt/openssl@1.1/lib
-L/opt/homebrew/opt/readline/lib -L/opt/homebrew/lib
-F/opt/homebrew/Frameworks -Wl,-headerpad_max_install_names
-isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk''

Configured features:
ACL GIF GLIB GMP GNUTLS IMAGEMAGICK JPEG JSON LCMS2 LIBXML2 MODULES
NATIVE_COMP NOTIFY KQUEUE NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP XIM XWIDGETS ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
comp comp-cstr warnings icons subr-x rx cl-seq cl-macs gv cl-extra
help-mode bytecomp byte-compile cconv cl-lib sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils rmc iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs faces cus-face macroexp files window text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads xwidget-internal kqueue cocoa
ns lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 75976 9307)
(symbols 48 7044 0)
(strings 32 18851 2756)
(string-bytes 1 602506)
(vectors 16 16382)
(vector-slots 8 329785 12666)
(floats 8 27 30)
(intervals 56 293 0)
(buffers 1000 11))

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Lars Ingebrigtsen]

Andrew John De Angelis <ajd2195@columbia.edu> writes:

> - emacs -Q
> - Eval: (turn-on-flyspell)

Or `M-x flyspell-mode'.

> - write a word that's not present in the dictionary
>    (I wrote "incorrectlll")
> - place cursor at word
> - M-x flyspell-correct-word-before-point
> - press down arrow (<down>) to navigate to the 'Save word' option
> - emacs crashes
>    (in the terminal, this is the message:
>    Fatal error 11: Segmentation fault
>    zsh: abort      emacs -Q

I can reproduce this crash on Macos (but not on Ubuntu) with the current
"master".

However, I'm not well versed in debugging stuff like this on Macos, so
if somebody else could have a look, that'd be great.

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Eli Zaretskii]

> Cc: 58288@debbugs.gnu.org
> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Tue, 04 Oct 2022 13:19:08 +0200
> 
> Andrew John De Angelis <ajd2195@columbia.edu> writes:
> 
> > - emacs -Q
> > - Eval: (turn-on-flyspell)
> 
> Or `M-x flyspell-mode'.
> 
> > - write a word that's not present in the dictionary
> >    (I wrote "incorrectlll")
> > - place cursor at word
> > - M-x flyspell-correct-word-before-point
> > - press down arrow (<down>) to navigate to the 'Save word' option
> > - emacs crashes
> >    (in the terminal, this is the message:
> >    Fatal error 11: Segmentation fault
> >    zsh: abort      emacs -Q
> 
> I can reproduce this crash on Macos (but not on Ubuntu) with the current
> "master".
> 
> However, I'm not well versed in debugging stuff like this on Macos, so
> if somebody else could have a look, that'd be great.

If you or someone else could show a meaningful backtrace, maybe the
reason will become evident.

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Robert Pluim]

>>>>> On Tue, 04 Oct 2022 15:32:00 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> I can reproduce this crash on Macos (but not on Ubuntu) with the current
    >> "master".
    >> 
    >> However, I'm not well versed in debugging stuff like this on Macos, so
    >> if somebody else could have a look, that'd be great.

    Eli> If you or someone else could show a meaningful backtrace, maybe the
    Eli> reason will become evident.

(this is not current master. I doubt that changes things, but I can
recompile if needed)

Process 48138 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x3)
    frame #0: 0x00000001000743b0 emacs`find_and_return_menu_selection [inlined] AREF(array=<unavailable>, idx=0) at lisp.h:1941:10 [opt]
   1938	AREF (Lisp_Object array, ptrdiff_t idx)
   1939	{
   1940	  eassert (0 <= idx && idx < gc_asize (array));
-> 1941	  return XVECTOR (array)->contents[idx];
   1942	}
   1943
   1944	INLINE Lisp_Object *
Target 0: (emacs) stopped.
warning: emacs was compiled with optimization - stepping may behave oddly; variables may not be available.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x3)
  * frame #0: 0x00000001000743b0 emacs`find_and_return_menu_selection [inlined] AREF(array=<unavailable>, idx=0) at lisp.h:1941:10 [opt]
    frame #1: 0x00000001000743b0 emacs`find_and_return_menu_selection(f=0x00000001040cde30, keymaps=false, client_data=0x0000000103071c80) at menu.c:988:17 [opt]
    frame #2: 0x0000000100249fce emacs`-[EmacsMenu runMenuAt:forFrame:keymaps:](self=0x000060000175d840, _cmd=<unavailable>, p=(x = 86, y = 464), f=0x00000001040cde30, keymaps=false) at nsmenu.m:767:9 [opt]

Robert
-- 

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  58288@debbugs.gnu.org,
>   ajd2195@columbia.edu
> Date: Tue, 04 Oct 2022 15:16:43 +0200
> 
> >>>>> On Tue, 04 Oct 2022 15:32:00 +0300, Eli Zaretskii <eliz@gnu.org> said:
> 
>     >> I can reproduce this crash on Macos (but not on Ubuntu) with the current
>     >> "master".
>     >> 
>     >> However, I'm not well versed in debugging stuff like this on Macos, so
>     >> if somebody else could have a look, that'd be great.
> 
>     Eli> If you or someone else could show a meaningful backtrace, maybe the
>     Eli> reason will become evident.
> 
> (this is not current master. I doubt that changes things, but I can
> recompile if needed)
> 
> Process 48138 stopped
> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x3)
>     frame #0: 0x00000001000743b0 emacs`find_and_return_menu_selection [inlined] AREF(array=<unavailable>, idx=0) at lisp.h:1941:10 [opt]
>    1938	AREF (Lisp_Object array, ptrdiff_t idx)
>    1939	{
>    1940	  eassert (0 <= idx && idx < gc_asize (array));
> -> 1941	  return XVECTOR (array)->contents[idx];
>    1942	}
>    1943
>    1944	INLINE Lisp_Object *
> Target 0: (emacs) stopped.
> warning: emacs was compiled with optimization - stepping may behave oddly; variables may not be available.
> (lldb) bt
> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x3)
>   * frame #0: 0x00000001000743b0 emacs`find_and_return_menu_selection [inlined] AREF(array=<unavailable>, idx=0) at lisp.h:1941:10 [opt]
>     frame #1: 0x00000001000743b0 emacs`find_and_return_menu_selection(f=0x00000001040cde30, keymaps=false, client_data=0x0000000103071c80) at menu.c:988:17 [opt]
>     frame #2: 0x0000000100249fce emacs`-[EmacsMenu runMenuAt:forFrame:keymaps:](self=0x000060000175d840, _cmd=<unavailable>, p=(x = 86, y = 464), f=0x00000001040cde30, keymaps=false) at nsmenu.m:767:9 [opt]

This is in NS-specific code for popup menus.  Sounds like no one is
setting up the menu_items array in that case?

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Robert Pluim]

>>>>> On Tue, 04 Oct 2022 16:47:39 +0300, Eli Zaretskii <eliz@gnu.org> said:

    Eli> This is in NS-specific code for popup menus.  Sounds like no one is
    Eli> setting up the menu_items array in that case?

Itʼs being set up, but menu_items is ending up as Qnil for some reason
(I suspect something is going wrong with save_menu_items).

Of course debugging this makes the crash go away <sigh>

Robert
-- 

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: larsi@gnus.org,  58288@debbugs.gnu.org,  ajd2195@columbia.edu
> Date: Tue, 04 Oct 2022 17:14:57 +0200
> 
> >>>>> On Tue, 04 Oct 2022 16:47:39 +0300, Eli Zaretskii <eliz@gnu.org> said:
> 
>     Eli> This is in NS-specific code for popup menus.  Sounds like no one is
>     Eli> setting up the menu_items array in that case?
> 
> Itʼs being set up, but menu_items is ending up as Qnil for some reason
> (I suspect something is going wrong with save_menu_items).

GC, perhaps?

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Robert Pluim]

>>>>> On Tue, 04 Oct 2022 19:30:00 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> Itʼs being set up, but menu_items is ending up as Qnil for some reason
    >> (I suspect something is going wrong with save_menu_items).

    Eli> GC, perhaps?

No, not GC.

frame #1: 0x00000001000754e0 emacs`find_and_return_menu_selection(f=0x00006210000c1530, keymaps=false, client_data=0x00006210006854e8) at menu.c:989:11
   986
   987 	  while (i < menu_items_used)
   988 	    {
-> 989 	      if (NILP (AREF (menu_items, i)))
   990 	        {
   991 	          subprefix_stack[submenu_depth++] = prefix;
   992 	          prefix = entry;
(lldb) p menu_items_used
(int) $36 = 59
(lldb) p menu_items
(Lisp_Object) $37 = NULL

So the code is assuming that menu_items_used > 0 means that menu_items
is valid, but we have this in menu.c:

    void
    save_menu_items (void)
    {
      Lisp_Object saved = list4 (menu_items_inuse ? menu_items : Qnil,
                                 make_fixnum (menu_items_used),
                                 make_fixnum (menu_items_n_panes),
                                 make_fixnum (menu_items_submenu_depth));
      record_unwind_protect (restore_menu_items, saved);
      menu_items_inuse = false;
      menu_items = Qnil;
    }

This fixes it, but it seems a bit dodgy to me, why has no other
platform ever run into this?

diff --git a/src/menu.c b/src/menu.c
index eeb0c9a7e5..a368da373b 100644
--- a/src/menu.c
+++ b/src/menu.c
@@ -147,7 +147,7 @@ restore_menu_items (Lisp_Object saved)
 void
 save_menu_items (void)
 {
-  Lisp_Object saved = list4 (menu_items_inuse ? menu_items : Qnil,
+  Lisp_Object saved = list4 (menu_items,
 			     make_fixnum (menu_items_used),
 			     make_fixnum (menu_items_n_panes),
 			     make_fixnum (menu_items_submenu_depth));


Robert
-- 

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: 58288@debbugs.gnu.org,  larsi@gnus.org,  ajd2195@columbia.edu
> Date: Wed, 05 Oct 2022 12:01:03 +0200
> 
> So the code is assuming that menu_items_used > 0 means that menu_items
> is valid, but we have this in menu.c:
> 
>     void
>     save_menu_items (void)
>     {
>       Lisp_Object saved = list4 (menu_items_inuse ? menu_items : Qnil,
>                                  make_fixnum (menu_items_used),
>                                  make_fixnum (menu_items_n_panes),
>                                  make_fixnum (menu_items_submenu_depth));
>       record_unwind_protect (restore_menu_items, saved);
>       menu_items_inuse = false;
>       menu_items = Qnil;
>     }
> 
> This fixes it, but it seems a bit dodgy to me, why has no other
> platform ever run into this?

How come a menu is in use, but menu_items_inuse is zero?

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Robert Pluim]

>>>>> On Wed, 05 Oct 2022 13:06:52 +0300, Eli Zaretskii <eliz@gnu.org> said:
    >> void
    >> save_menu_items (void)
    >> {
    >> Lisp_Object saved = list4 (menu_items_inuse ? menu_items : Qnil,
    >> make_fixnum (menu_items_used),
    >> make_fixnum (menu_items_n_panes),
    >> make_fixnum (menu_items_submenu_depth));
    >> record_unwind_protect (restore_menu_items, saved);
    >> menu_items_inuse = false;
    >> menu_items = Qnil;
    >> }
    >> 
    >> This fixes it, but it seems a bit dodgy to me, why has no other
    >> platform ever run into this?

    Eli> How come a menu is in use, but menu_items_inuse is zero?

`menu_items_inuse' is not zero, itʼs false. `menu_items_used' is
non-zero. I guess something called `unuse_menu_items', and the
assumption is then that `menu_items' will not be accessed (but then
why save away `menu_items_used' etc?

menu.c says:

    /* Whether the global vars defined here are already in use.
       Used to detect cases where we try to re-enter this non-reentrant code.  */
    bool menu_items_inuse;

so should we be adding "if (menu_items_inuse)" guards everywhere?

Robert
-- 

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Wed, 05 Oct 2022 13:06:52 +0300, Eli Zaretskii <eliz@gnu.org> said:
>     >> void
>     >> save_menu_items (void)
>     >> {
>     >> Lisp_Object saved = list4 (menu_items_inuse ? menu_items : Qnil,
>     >> make_fixnum (menu_items_used),
>     >> make_fixnum (menu_items_n_panes),
>     >> make_fixnum (menu_items_submenu_depth));
>     >> record_unwind_protect (restore_menu_items, saved);
>     >> menu_items_inuse = false;
>     >> menu_items = Qnil;
>     >> }
>     >> 
>     >> This fixes it, but it seems a bit dodgy to me, why has no other
>     >> platform ever run into this?
>
>     Eli> How come a menu is in use, but menu_items_inuse is zero?
>
> `menu_items_inuse' is not zero, itʼs false. `menu_items_used' is
> non-zero. I guess something called `unuse_menu_items', and the
> assumption is then that `menu_items' will not be accessed (but then
> why save away `menu_items_used' etc?
>
> menu.c says:
>
>     /* Whether the global vars defined here are already in use.
>        Used to detect cases where we try to re-enter this non-reentrant code.  */
>     bool menu_items_inuse;
>
> so should we be adding "if (menu_items_inuse)" guards everywhere?
>
> Robert

No matter how much I try I cannot reproduce this (or bug#58296) on
GNUstep, and I don't have access to Mac OS anymore.

But after the holidays are over I will be able to test stuff on a Mac
and will look into this.

Thanks.

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Robert Pluim]

>>>>> On Wed, 05 Oct 2022 18:46:48 +0800, Po Lu via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said:

    Po Lu> No matter how much I try I cannot reproduce this (or bug#58296) on
    Po Lu> GNUstep, and I don't have access to Mac OS anymore.

    Po Lu> But after the holidays are over I will be able to test stuff on a Mac
    Po Lu> and will look into this.

I have both a working lldb and gdb setup, I can test patches or debug if needed.

Robert
-- 

bug#58288: 29.0.50; (flyspell-correct-word-before-point) followed by <down> crashes emacs

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Po Lu <luangruo@yahoo.com> writes:

> But after the holidays are over I will be able to test stuff on a Mac
> and will look into this.

Here, I get some backtrace involving a crash in some Apple function
(HIToolBox`MenuElement::ResolveStore) when navigating with the arrow
keys, not related to find_and_return_menu_selection (which however is
only used on NS.)

I will try the recipe in Stefan's bug report.