bug#58113: 29.0.50; [noverlay] Segmentation fault while building on macOS

bug#58113: 29.0.50; [noverlay] Segmentation fault while building on macOS

[Gerd Möllmann]

With HEAD = 409327ff68f9ccdc8099f6a2ba2fee76abaaab70 on the noverlay
branch:

git clean -xdf
./autogen.sh
./configure
make

/bin/sh: line 1: 76907 Segmentation fault: 11  '../src/emacs' -batch --no-site-file --no-site-lisp --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" -f batch-byte-compile cedet/srecode/cpp.el
make[3]: *** [cedet/srecode/cpp.elc] Error 139

With debugger (I'll try to reprduce with -O0 later):

~/emacs/noverlay/ > cd lisp
[feature/noverlay] gerd@Mini 2022-09-27 13:31 
~/emacs/noverlay/lisp/ > lldb ../src/emacs
(lldb) target create "../src/emacs"
Current executable set to '/Users/gerd/emacs/noverlay/src/emacs' (arm64).
(lldb) run  -batch --no-site-file --no-site-lisp --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" -f batch-byte-compile cedet/srecode/cpp.el
Process 77018 launched: '/Users/gerd/emacs/noverlay/src/emacs' (arm64)
Process 77018 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xfffffffffffffffb)
    frame #0: 0x0000000100148a8c emacs`mark_buffer [inlined] set_vector_marked(v=0xfffffffffffffffb) at alloc.c:3975:5 [opt]
   3972	      pdumper_set_marked (v);
   3973	    }
   3974	  else
-> 3975	    XMARK_VECTOR (v);
   3976	}
   3977	
   3978	static bool

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xfffffffffffffffb)
  * frame #0: 0x0000000100148a8c emacs`mark_buffer [inlined] set_vector_marked(v=0xfffffffffffffffb) at alloc.c:3975:5 [opt]
    frame #1: 0x0000000100148a7c emacs`mark_buffer [inlined] set_vectorlike_marked(header=0xfffffffffffffffb) at alloc.c:3987:3 [opt]
    frame #2: 0x0000000100148a7c emacs`mark_buffer [inlined] mark_overlay(ov=0xfffffffffffffffb) at alloc.c:6508:3 [opt]
    frame #3: 0x0000000100148a7c emacs`mark_buffer(buffer=0x00000001038c9a00) at alloc.c:6536:5 [opt]
    frame #4: 0x0000000100145f3c emacs`process_mark_stack(base_sp=<unavailable>) at alloc.c:6858:3 [opt]
    frame #5: 0x0000000100148d74 emacs`mark_localized_symbol [inlined] mark_object(obj=0x00000001038c9a05) at alloc.c:7040:3 [opt]
    frame #6: 0x0000000100148d2c emacs`mark_localized_symbol(ptr=<unavailable>) at alloc.c:6577:3 [opt]
    frame #7: 0x00000001001461c0 emacs`process_mark_stack(base_sp=<unavailable>) at alloc.c:6969:3 [opt]
    frame #8: 0x0000000100144c30 emacs`garbage_collect [inlined] mark_object(obj=0x0000000000007890) at alloc.c:7040:3 [opt]
    frame #9: 0x0000000100144c14 emacs`garbage_collect [inlined] mark_object_root_visitor(root_ptr=<unavailable>, type=GC_ROOT_BUFFER_LOCAL_DEFAULT, data=0x0000000000000000) at alloc.c:5987:3 [opt]
    frame #10: 0x0000000100144c14 emacs`garbage_collect [inlined] visit_vectorlike_root(visitor=gc_root_visitor @ 0x0000600000ba0030, ptr=<unavailable>, type=GC_ROOT_BUFFER_LOCAL_DEFAULT) at alloc.c:5939:5 [opt]
    frame #11: 0x0000000100144bd8 emacs`garbage_collect [inlined] visit_buffer_root(visitor=<unavailable>, buffer=<unavailable>, type=GC_ROOT_BUFFER_LOCAL_DEFAULT) at alloc.c:5953:3 [opt]
    frame #12: 0x0000000100144bd8 emacs`garbage_collect [inlined] visit_static_gc_roots(visitor=gc_root_visitor @ 0x0000600000ba0020) at alloc.c:5965:3 [opt]
    frame #13: 0x0000000100144bd8 emacs`garbage_collect at alloc.c:6189:3 [opt]
    frame #14: 0x0000000100145a2c emacs`Fgarbage_collect at alloc.c:6341:3 [opt]
    frame #15: 0x00000001001b7670 emacs`exec_byte_code(fun=<unavailable>, args_template=<unavailable>, nargs=<unavailable>, args=<unavailable>) at bytecode.c:809:14 [opt]
    frame #16: 0x0000000100171000 emacs`funcall_lambda [inlined] fetch_and_exec_byte_code(fun=<unavailable>, args_template=<unavailable>, nargs=<unavailable>, args=<unavailable>) at eval.c:3064:10 [opt] [artificial]
    frame #17: 0x000000010016f654 emacs`apply_lambda(fun=0x0000000103d0110d, args=<unavailable>, count=(bytes = 6528)) at eval.c:3086:9 [opt]
    frame #18: 0x000000010016a818 emacs`eval_sub(form=<unavailable>) at lisp.h:0:10 [opt]
    frame #19: 0x000000010016ae40 emacs`Fprogn(body=0x00000001039a0873) at eval.c:436:13 [opt]
    frame #20: 0x000000010015faa0 emacs`Fsave_excursion(args=<unavailable>) at editfns.c:846:9 [opt]
    frame #21: 0x000000010016aa70 emacs`eval_sub(form=<unavailable>) at eval.c:2433:8 [opt]
    frame #22: 0x000000010016c818 emacs`Flet at eval.c:436:13 [opt]
    frame #23: 0x000000010016c7fc emacs`Flet(args=<unavailable>) at eval.c:1023:9 [opt]
    frame #24: 0x000000010016aa70 emacs`eval_sub(form=<unavailable>) at eval.c:2433:8 [opt]
    frame #25: 0x000000010016adf4 emacs`Fif at eval.c:436:13 [opt]
    frame #26: 0x000000010016add8 emacs`Fif(args=<unavailable>) at eval.c:392:10 [opt]
    frame #27: 0x000000010016aa70 emacs`eval_sub(form=<unavailable>) at eval.c:2433:8 [opt]
    frame #28: 0x0000000100170f88 emacs`funcall_lambda at eval.c:436:13 [opt]
    frame #29: 0x0000000100170f6c
    emacs`funcall_lambda(fun=0x00000001039a06b3, nargs=2,
    arg_vector=0x000000016f

(lldb) command script import ../etc/emacs_lldb.py 
Emacs debugging support has been installed.
(lldb) xbacktrace
(unsigned char *) data = 0x000000010027d1df "Automatic GC"
(unsigned char *) data = 0x0000000100278cc6 "garbage-collect"
(unsigned char *) data = 0x0000000103c56690 "semantic-fetch-tags"
(unsigned char *) data = 0x000000010027a568 "save-excursion"
(unsigned char *) data = 0x000000010027b174 "let"
(unsigned char *) data = 0x000000010027b0a4 "if"
(unsigned char *) data = 0x0000000103be1af0 "srecode-map-validate-file-for-mode"
(unsigned char *) data = 0x000000010027b174 "let"
(unsigned char *) data = 0x000000010027b0ac "progn"
(unsigned char *) data = 0x000000010027b0a4 "if"
(unsigned char *) data = 0x000000010027b174 "let"
(unsigned char *) data = 0x000000010027b17d "while"
(unsigned char *) data = 0x000000010027b174 "let"
(unsigned char *) data = 0x000000010027b0ac "progn"
(unsigned char *) data = 0x000000010027b0a4 "if"
(unsigned char *) data = 0x000000010027b174 "let"
(unsigned char *) data = 0x000000010027b17d "while"
(unsigned char *) data = 0x000000010027b174 "let"
(unsigned char *) data = 0x000000010027b174 "let"
(unsigned char *) data = 0x0000000103be15c8 "srecode-map-update-map"
(unsigned char *) data = 0x0000000103be1ca8 "srecode-map-load-path-set"
(unsigned char *) data = 0x0000000104b932b9 "custom-initialize-reset"
(unsigned char *) data = 0x0000000104ba7bca "custom-declare-variable"
(unsigned char *) data = 0x0000000100280751 "eval-buffer"
(unsigned char *) data = 0x0000000104b7d1cb "load-with-code-conversion"
(unsigned char *) data = 0x000000010027bb4d "require"
(unsigned char *) data = 0x0000000100280751 "eval-buffer"
(unsigned char *) data = 0x0000000104b7d1cb "load-with-code-conversion"
(unsigned char *) data = 0x000000010027bb4d "require"
(unsigned char *) data = 0x0000000100280751 "eval-buffer"
(unsigned char *) data = 0x0000000104b7d1cb "load-with-code-conversion"
(unsigned char *) data = 0x000000010027bb4d "require"
(unsigned char *) data = 0x0000000100280751 "eval-buffer"
(unsigned char *) data = 0x0000000104b7d1cb "load-with-code-conversion"
(unsigned char *) data = 0x000000010027bb4d "require"
(unsigned char *) data = 0x000000010027b205 "apply"
(unsigned char *) data = 0x00000001038d85f0 "byte-compile-file-form-require"
(unsigned char *) data = 0x00000001038d83c8 "byte-compile-file-form"
PVEC_COMPILED
(unsigned char *) data = 0x000000010389b118 "byte-compile-recurse-toplevel"
(unsigned char *) data = 0x00000001038d8018 "byte-compile-toplevel-file-form"
PVEC_COMPILED
(unsigned char *) data = 0x00000001038d7d00 "byte-compile-from-buffer"
(unsigned char *) data = 0x0000000104bab922 "byte-compile-file"
(unsigned char *) data = 0x00000001038cc6e8 "batch-byte-compile-file"
(unsigned char *) data = 0x0000000104c1b1e9 "batch-byte-compile"
(unsigned char *) data = 0x0000000104b7defc "command-line-1"
(unsigned char *) data = 0x0000000104b7fa0a "command-line"
(unsigned char *) data = 0x0000000104b80cef "normal-top-level"

(lldb) p v
(Lisp_Vector *) $457 = 0xfffffffffffffffb

frame #2: 0x0000000100148a7c emacs`mark_buffer [inlined] mark_overlay(ov=0xfffffffffffffffb) at alloc.c:6508:3 [opt]
   6505	static void
   6506	mark_overlay (struct Lisp_Overlay *ov)
   6507	{
-> 6508	  set_vectorlike_marked (&ov->header);
   6509	  mark_object (ov->plist);
   6510	}
   6511	
(lldb) p ov
(Lisp_Overlay *) $459 = 0xfffffffffffffffb
(lldb) up
frame #3: 0x0000000100148a7c emacs`mark_buffer(buffer=0x00000001038c9a00) at alloc.c:6536:5 [opt]
   6533	  struct interval_node *node;
   6534	  buffer_overlay_iter_start (buffer, PTRDIFF_MIN, PTRDIFF_MAX, ITREE_ASCENDING);
   6535	  while ((node = buffer_overlay_iter_next (buffer)))
-> 6536	    mark_overlay (XOVERLAY (node->data));
   6537	  buffer_overlay_iter_finish (buffer);
   6538	
   6539	  /* If this is an indirect buffer, mark its base buffer.  */

(lldb) p buffer->name_
(Lisp_Object) $595 = 0x0000000102f76fd4 (struct Lisp_String *) $597 = 0x0000000102f76fd0
(lldb) p *$597
(struct Lisp_String) $598 = {
  u = {
    s = {
      size = -9223372036854775790
      size_byte = -1
      intervals = NULL
      data = 0x0000000103be21b8 " *srecode-map-tmp*"

(lldb) p buffer->overlays
(interval_tree *) $599 = 0x0000600002c07d80

(lldb) p *buffer->overlays
(interval_tree) $601 = {
  root = 0x0000600002c07d88
  null = {
    parent = 0x0000600002c07d88
    left = 0x0000600002c07d88
    right = 0x0000600002c07d88
    begin = -9223372036854775808
    end = -9223372036854775808
    limit = -9223372036854775808
    offset = 0
    otick = 1
    data = NULL
    color = ITREE_BLACK
    visited = true
    rear_advance = false
    front_advance = false
  }
  otick = 1
  size = 26
  iter = 0x0000600000c41620
  iter_running = true

At this point I'm stuck, without really knowing what this is supposed
to do, and how it works.

bug#58113: 29.0.50; [noverlay] Segmentation fault while building on macOS

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> /bin/sh: line 1: 76907 Segmentation fault: 11  '../src/emacs' -batch --no-site-file --no-site-lisp --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" -f batch-byte-compile cedet/srecode/cpp.el
> make[3]: *** [cedet/srecode/cpp.elc] Error 139

Seems to be reproducible with -g -O0.

But, as I said, I don't currently know what to do next.  Maybe I enable
checking.

bug#58113: 29.0.50; [noverlay] Segmentation fault while building on macOS

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

With --enable-checking:

itree.c:389: Emacs fatal error: assertion failed: tree->size == 0 || (tree->size > 0 && tree->root != &tree->null)
/bin/sh: line 1:  2779 Abort trap: 6           '../src/emacs' -batch --no-site-file --no-site-lisp --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" -f batch-byte-compile cedet/srecode/cpp.el
make[3]: *** [cedet/srecode/cpp.elc] Error 134
make[3]: *** Waiting for unfinished jobs....

frame #5: 0x00000001003a2814 emacs`interval_tree_remove(tree=0x0000600002c03f00, node=0x0000600002112b70) at itree.c:389:3
   386 	  node->right = node->left = node->parent = NULL;
   387 	  --tree->size;
   388 	
-> 389 	  eassert (tree->size == 0 || (tree->size > 0 && tree->root != &tree->null));
   390 	
   391 	  return node;
   392 	}
(lldb) p tree->size
(intmax_t) $0 = 26
(lldb) p &tree->null
(interval_node *) $1 = 0x0000600002c03f08
(lldb) p tree->root
(interval_node *) $2 = 0x0000600002c03f08

(lldb) xbacktrace
(unsigned char *) data = 0x000000010048e3cf "delete-overlay"
(unsigned char *) data = 0x0000000103d5e158 "semantic-edits-flush-changes"
(unsigned char *) data = 0x000000010049aa58 "run-hooks"
(unsigned char *) data = 0x0000000103c573a0 "semantic-clear-toplevel-cache"
(unsigned char *) data = 0x0000000103c59048 "semantic-new-buffer-fcn"
(unsigned char *) data = 0x000000010049aea7 "let"
(unsigned char *) data = 0x000000010049add7 "if"
(unsigned char *) data = 0x0000000100498142 "save-excursion"
(unsigned char *) data = 0x000000010049aea7 "let"
(unsigned char *) data = 0x000000010049add7 "if"
(unsigned char *) data = 0x0000000103b6b8a8 "srecode-map-validate-file-for-mode"
(unsigned char *) data = 0x000000010049aea7 "let"
(unsigned char *) data = 0x000000010049addf "progn"
(unsigned char *) data = 0x000000010049add7 "if"
(unsigned char *) data = 0x000000010049aea7 "let"
(unsigned char *) data = 0x000000010049aeb0 "while"
(unsigned char *) data = 0x000000010049aea7 "let"
(unsigned char *) data = 0x000000010049addf "progn"
(unsigned char *) data = 0x000000010049add7 "if"
(unsigned char *) data = 0x000000010049aea7 "let"
(unsigned char *) data = 0x000000010049aeb0 "while"
(unsigned char *) data = 0x000000010049aea7 "let"
(unsigned char *) data = 0x000000010049aea7 "let"
(unsigned char *) data = 0x0000000103b6b380 "srecode-map-update-map"
(unsigned char *) data = 0x0000000103b6ba60 "srecode-map-load-path-set"
(unsigned char *) data = 0x0000000104b932b9 "custom-initialize-reset"
(unsigned char *) data = 0x0000000104ba7bca "custom-declare-variable"
(unsigned char *) data = 0x00000001004a69be "eval-buffer"
(unsigned char *) data = 0x0000000104b7d1cb "load-with-code-conversion"
(unsigned char *) data = 0x000000010049ca11 "require"
(unsigned char *) data = 0x00000001004a69be "eval-buffer"
(unsigned char *) data = 0x0000000104b7d1cb "load-with-code-conversion"
(unsigned char *) data = 0x000000010049ca11 "require"
(unsigned char *) data = 0x00000001004a69be "eval-buffer"
(unsigned char *) data = 0x0000000104b7d1cb "load-with-code-conversion"
(unsigned char *) data = 0x000000010049ca11 "require"
(unsigned char *) data = 0x00000001004a69be "eval-buffer"
(unsigned char *) data = 0x0000000104b7d1cb "load-with-code-conversion"
(unsigned char *) data = 0x000000010049ca11 "require"
(unsigned char *) data = 0x000000010049af38 "apply"
(unsigned char *) data = 0x00000001038e3ff0 "byte-compile-file-form-require"
(unsigned char *) data = 0x00000001038e3dc8 "byte-compile-file-form"
PVEC_COMPILED
(unsigned char *) data = 0x000000010389e6d8 "byte-compile-recurse-toplevel"
(unsigned char *) data = 0x00000001038e3a18 "byte-compile-toplevel-file-form"
PVEC_COMPILED
(unsigned char *) data = 0x00000001038e3700 "byte-compile-from-buffer"
(unsigned char *) data = 0x0000000104bab922 "byte-compile-file"
(unsigned char *) data = 0x00000001038dc2e8 "batch-byte-compile-file"
(unsigned char *) data = 0x0000000104c1b209 "batch-byte-compile"

bug#58113: 29.0.50; [noverlay] Segmentation fault while building on macOS

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Date: Tue, 27 Sep 2022 14:43:40 +0200
> 
> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> 
> With --enable-checking:
> 
> itree.c:389: Emacs fatal error: assertion failed: tree->size == 0 || (tree->size > 0 && tree->root != &tree->null)
> /bin/sh: line 1:  2779 Abort trap: 6           '../src/emacs' -batch --no-site-file --no-site-lisp --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" -f batch-byte-compile cedet/srecode/cpp.el
> make[3]: *** [cedet/srecode/cpp.elc] Error 134
> make[3]: *** Waiting for unfinished jobs....
> 
> frame #5: 0x00000001003a2814 emacs`interval_tree_remove(tree=0x0000600002c03f00, node=0x0000600002112b70) at itree.c:389:3
>    386 	  node->right = node->left = node->parent = NULL;
>    387 	  --tree->size;
>    388 	
> -> 389 	  eassert (tree->size == 0 || (tree->size > 0 && tree->root != &tree->null));
>    390 	
>    391 	  return node;
>    392 	}
> (lldb) p tree->size
> (intmax_t) $0 = 26
> (lldb) p &tree->null
> (interval_node *) $1 = 0x0000600002c03f08
> (lldb) p tree->root
> (interval_node *) $2 = 0x0000600002c03f08

What does tree->null represent?  Does it represent an empty tree?  If
so, I guess tree->size is not maintained correctly?

bug#58113: 29.0.50; [noverlay] Segmentation fault while building on macOS

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> (lldb) p tree->size
>> (intmax_t) $0 = 26
>> (lldb) p &tree->null
>> (interval_node *) $1 = 0x0000600002c03f08
>> (lldb) p tree->root
>> (interval_node *) $2 = 0x0000600002c03f08
>
> What does tree->null represent?  Does it represent an empty tree?  If
> so, I guess tree->size is not maintained correctly?

I surmise it means the tree is empty.

The tree->null thing reminds me of the MEM_NIL I used in the red-black
tree in alloc.c.  It is sort of a "trick" to make a red-black tree
implementation more "elegant".  Although--one such node per tree looks a
bit odd to me.  But what do I know.

I'll see if I can spot something obvious that goes wrong with the size.
Otherwiese, it would be good if someone who can build an Emacs from the
branch could write some more tests.

[ A more general question is why the existing red-black tree hasn't been
hoisted from alloc.c?  Or even, why not use a ready-made interval tree
like the one in the Linux kernel, which is also an augmented rb-tree?

That would be much more confidence-inducing than this. ]

bug#58113: 29.0.50; [noverlay] Segmentation fault while building on macOS

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> I'll see if I can spot something obvious that goes wrong with the
> size.

I don't see something obviously going wrong with the size field.

bug#58113: 29.0.50; [noverlay] Segmentation fault while building on macOS

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> I don't see something obviously going wrong with the size field.

Found it.  The voices told me to check my own commit a second time, and
there it was.

Closing.