bug#60237: 30.0.50; tree sitter core dumps when I edebug view a node

[Yuan Fu <casouri@gmail.com>]

Mickey Petersen <mickey@masteringemacs.org> writes:

> Yuan Fu <casouri@gmail.com> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: Mickey Petersen <mickey@masteringemacs.org>
>>>> Date: Wed, 21 Dec 2022 12:24:34 +0000
>>>
>>> Yuan, can you look into this?  The crash is in tree-sitter, so maybe
>>> it isn't our bug, but I'd like to be sure.  And even if it is a
>>> tree-sitter bug, maybe we can work around it to prevent Emacs from
>>> crashing?
>>
>> Absolutely.
>>
>>>> Happens in emacs -Q (after loading some simple elisp code that uses treesit.el) and consistently and repeatedly.
>>>>
>>>>
>>>> Here's the elisp. When I edebug it I can step and view all the
>>>> variables and expressions I like. The `combobulate-' functions are
>>>> widely used in the library and pose no issues anywhere else and do
>>>> nothing more than fetch nodes via tree sitter. It is only this bit of
>>>> code that blows up, and then only when invoked inside a python
>>>> string.
>>
>> It would be nice if you can make a reproduce recipe. Judging from the
>> backtrace, you can probably trigger it by printing the node with print
>> or princ.  And does it trigger on all python strings? Or some specific
>> string in some specific python source?
>>
>
> This issue seems entirely related to `M-x treesit-explore-mode` (and
> possibly the inspect variant also) though it is hard to reproduce
> reliably. I get either crashes or hangs, depending on whether I have
> edebug on or not.
>
> Thrown errors seem to be the common denominator?

I’m stumbled on a reliably way to trigger a crash, of possibly the same cause as
this one, by enabling the profiler and M-x garbage-collect in a
tree-sitter mode on Mac. I tried to reproduce this on Linux but with no
success.

I was also able to trigger infinite loop by the same recipe on time, but I
didn’t run that session under lldb. Anyway, we can focus on the crash
first.

Below’s the backtrace. Eli, could you see anything from this? I have the
lldb session live so let me know if you want to see anything.
Unfortunately I can’t get gdb to work on Mac.

I suspect there is some stupid mistake that I made concerning gcing
tree-sitter objects. Could you see anything suspicious from the
following description:

A Lisp_TS_Parser contains a TSParser and a TSTree, which are freed when
the Lisp_TS_Parser is collected. A Lisp_TS_Node references the parser
from which it is created, so that a Lisp_TS_Parser is only collected
when no live node references it, because the Lisp_TS_Node references the
TSTree stored in the Lisp_TS_Parser.


* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x0000000100250f3d emacs`ASIZE(array=0x00000001a1889245) at lisp.h:1768:3
   1765	ASIZE (Lisp_Object array)
   1766	{
   1767	  ptrdiff_t size = XVECTOR (array)->header.size;
-> 1768	  eassume (0 <= size);
   1769	  return size;
   1770	}
   1771
Target 0: (emacs) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x0000000100250f3d emacs`ASIZE(array=0x00000001a1889245) at lisp.h:1768:3
    frame #1: 0x0000000100250e5e emacs`get_backtrace(array=0x00000001a1889245) at eval.c:4193:28
    frame #2: 0x00000001003001ce emacs`record_backtrace(log=0x00000001a1887d68, count=64) at profiler.c:162:3
    frame #3: 0x000000010030016d emacs`malloc_probe(size=64) at profiler.c:509:3
    frame #4: 0x0000000100204e6d emacs`xmalloc(size=64) at alloc.c:760:3
    frame #5: 0x0000000100e6c0c9 libtree-sitter.0.dylib`ts_subtree_release + 158
    frame #6: 0x0000000100e6f004 libtree-sitter.0.dylib`ts_tree_delete + 44
    frame #7: 0x0000000100307379 emacs`treesit_delete_parser(lisp_parser=0x00000001a2c0f0e0) at treesit.c:1182:3
    frame #8: 0x0000000100212c1b emacs`cleanup_vector(vector=0x00000001a2c0f0e0) at alloc.c:3179:5
    frame #9: 0x00000001002124c9 emacs`sweep_vectors at alloc.c:3254:5
    frame #10: 0x000000010020c777 emacs`gc_sweep at alloc.c:7430:3
    frame #11: 0x000000010020bb67 emacs`garbage_collect at alloc.c:6262:3
    frame #12: 0x000000010020b706 emacs`maybe_garbage_collect at alloc.c:6107:5
    frame #13: 0x00000001002b4bea emacs`maybe_gc at lisp.h:5591:5
    frame #14: 0x00000001002afcaa emacs`exec_byte_code(fun=0x0000000107557b85, args_template=256, nargs=1, args=0x000000010809e338) at bytecode.c:782:6
    frame #15: 0x0000000100251e77 emacs`fetch_and_exec_byte_code(fun=0x00000001a1396be5, args_template=514, nargs=2, args=0x00007ff7bfefc628) at eval.c:3081:10
    frame #16: 0x000000010024e7c1 emacs`funcall_lambda(fun=0x00000001a1396be5, nargs=2, arg_vector=0x00007ff7bfefc628) at eval.c:3153:9
    frame #17: 0x000000010024e0a7 emacs`funcall_general(fun=0x00000001a1396be5, numargs=2, args=0x00007ff7bfefc628) at eval.c:2945:12
    frame #18: 0x00000001002494b4 emacs`Ffuncall(nargs=3, args=0x00007ff7bfefc620) at eval.c:2995:21
    frame #19: 0x000000010024d61c emacs`Fapply(nargs=2, args=0x00007ff7bfefc738) at eval.c:2666:24
    frame #20: 0x0000000100245752 emacs`apply1(fn=0x00000000a0a4d740, arg=0x00000001a38c59b3) at eval.c:2882:43
    frame #21: 0x00000001002cb671 emacs`read_process_output_call(fun_and_args=0x00000001a38c59c3) at process.c:6070:10
    frame #22: 0x000000010024a493 emacs`internal_condition_case_1(bfun=(emacs`read_process_output_call at process.c:6069), arg=0x00000001a38c59c3, handlers=0x0000000000000090, hfun=(emacs`read_process_output_error_handler at process.c:6075)) at eval.c:1498:25
    frame #23: 0x00000001002cb5c3 emacs`read_and_dispose_of_process_output(p=0x00000001a1448440, chars="Content-Length: 1184\r\n\r\n{\"jsonrpc\":\"2.0\",\"method\":\"textDocument/publishDiagnostics\",\"params\":{\"uri\":\"file:///Users/yuan/t/js/test.ts\",\"diagnostics\":[{\"range\":{\"start\":{\"line\":4,\"character\":11},\"end\":{\"line\":4,\"character\":14}},\"message\":\"Property 'get' does not exist on type 'Document'.\",\"severity\":1,\"code\":2339,\"source\":\"typescript\",\"tags\":[]},{\"range\":{\"start\":{\"line\":0,\"character\":14},\"end\":{\"line\":0,\"character\":15}},\"message\":\"Parameter 'a' implicitly has an 'any' type, but a better type may be inferred from usage.\",\"severity\":4,\"code\":7044,\"source\":\"typescript\",\"tags\":[]},{\"range\":{\"start\":{\"line\":0,\"character\":14},\"end\":{\"line\":0,\"character\":15}},\"message\":\"'a' is declared but its value is never read.\",\"severity\":4,\"code\":6133,\"source\":\"typescript\",\"tags\":[1]},{\"range\":{\"start\":{\"line\":0,\"character\":17},\"end\":{\"line\":0,\"character\":18}},\"message\":\"Parameter 'b' implicitly has an 'any' type, but a better type may be inferred from usage.\",\"severity\":4,\"code\":7044,\"source\":\"typescript\",\"tags\":[]},{\"range\":{\""..., nbytes=1208, coding=0x00000001577aed10) at process.c:6294:5
    frame #24: 0x00000001002c46df emacs`read_process_output(proc=0x00000001a1448445, channel=26) at process.c:6204:3
    frame #25: 0x00000001002c3097 emacs`wait_reading_process_output(time_limit=5, nsecs=0, read_kbd=-1, do_display=true, wait_for_cell=0x0000000000000000, wait_proc=0x0000000000000000, just_wait_proc=0) at process.c:5888:16
    frame #26: 0x000000010000c881 emacs`sit_for(timeout=0x0000000000000016, reading=true, display_option=1) at dispnew.c:6256:7
    frame #27: 0x0000000100166ffd emacs`read_char(commandflag=1, map=0x0000000193762fb3, prev_event=0x0000000000000000, used_mouse_menu=0x00007ff7bfefeb1f, end_time=0x0000000000000000) at keyboard.c:2872:11
    frame #28: 0x0000000100162e18 emacs`read_key_sequence(keybuf=0x00007ff7bfefee40, prompt=0x0000000000000000, dont_downcase_last=false, can_return_switch_frame=true, fix_current_buffer=true, prevent_redisplay=false) at keyboard.c:10074:12
    frame #29: 0x00000001001611c0 emacs`command_loop_1 at keyboard.c:1375:15
    frame #30: 0x000000010024a3c8 emacs`internal_condition_case(bfun=(emacs`command_loop_1 at keyboard.c:1269), handlers=0x0000000000000090, hfun=(emacs`cmd_error at keyboard.c:927)) at eval.c:1474:25
    frame #31: 0x0000000100160c63 emacs`command_loop_2(handlers=0x0000000000000090) at keyboard.c:1124:11
    frame #32: 0x0000000100249b53 emacs`internal_catch(tag=0x000000000000f3f0, func=(emacs`command_loop_2 at keyboard.c:1120), arg=0x0000000000000090) at eval.c:1197:25
    frame #33: 0x000000010015ffda emacs`command_loop at keyboard.c:1102:2
    frame #34: 0x000000010015fddf emacs`recursive_edit_1 at keyboard.c:711:9
    frame #35: 0x0000000100160352 emacs`Frecursive_edit at keyboard.c:794:3
    frame #36: 0x000000010015d177 emacs`main(argc=1, argv=0x00007ff7bfeff6a8) at emacs.c:2529:3
    frame #37: 0x00007ff81a314310 dyld`start + 2432
(lldb) up 4
frame #4: 0x0000000100204e6d emacs`xmalloc(size=64) at alloc.c:760:3
   757
   758 	  if (!val)
   759 	    memory_full (size);
-> 760 	  MALLOC_PROBE (size);
   761 	  return val;
   762 	}
   763
(lldb) list
   764 	/* Like the above, but zeroes out the memory just allocated.  */
   765
   766 	void *
   767 	xzalloc (size_t size)
   768 	{
   769 	  void *val;
   770
(lldb) up
frame #5: 0x0000000100e6c0c9 libtree-sitter.0.dylib`ts_subtree_release + 158
libtree-sitter.0.dylib`ts_subtree_release:
->  0x100e6c0c9 <+158>: movq   -0x30(%rbp), %rdi
    0x100e6c0cd <+162>: movq   %rax, 0x10(%rdi)
    0x100e6c0d1 <+166>: movl   %r15d, 0x1c(%rdi)
    0x100e6c0d5 <+170>: movl   0x18(%rdi), %eax
(lldb) down
frame #4: 0x0000000100204e6d emacs`xmalloc(size=64) at alloc.c:760:3
   757
   758 	  if (!val)
   759 	    memory_full (size);
-> 760 	  MALLOC_PROBE (size);
   761 	  return val;
   762 	}
   763
(lldb) down
frame #3: 0x000000010030016d emacs`malloc_probe(size=64) at profiler.c:509:3
   506 	malloc_probe (size_t size)
   507 	{
   508 	  eassert (HASH_TABLE_P (memory_log));
-> 509 	  record_backtrace (XHASH_TABLE (memory_log), min (size, MOST_POSITIVE_FIXNUM));
   510 	}
   511
   512 	DEFUN ("function-equal", Ffunction_equal, Sfunction_equal, 2, 2, 0,
(lldb) down
frame #2: 0x00000001003001ce emacs`record_backtrace(log=0x00000001a1887d68, count=64) at profiler.c:162:3
   159 	  /* Get a "working memory" vector.  */
   160 	  Lisp_Object backtrace = HASH_VALUE (log, index);
   161 	  eassert (BASE_EQ (Qunbound, HASH_KEY (log, index)));
-> 162 	  get_backtrace (backtrace);
   163
   164 	  { /* We basically do a `gethash+puthash' here, except that we have to be
   165 	       careful to avoid memory allocation since we're in a signal
(lldb) down
frame #1: 0x0000000100250e5e emacs`get_backtrace(array=0x00000001a1889245) at eval.c:4193:28
   4190	get_backtrace (Lisp_Object array)
   4191	{
   4192	  union specbinding *pdl = backtrace_next (backtrace_top ());
-> 4193	  ptrdiff_t i = 0, asize = ASIZE (array);
   4194
   4195	  /* Copy the backtrace contents into working memory.  */
   4196	  for (; i < asize; i++)
(lldb) down
frame #0: 0x0000000100250f3d emacs`ASIZE(array=0x00000001a1889245) at lisp.h:1768:3
   1765	ASIZE (Lisp_Object array)
   1766	{
   1767	  ptrdiff_t size = XVECTOR (array)->header.size;
-> 1768	  eassume (0 <= size);
   1769	  return size;
   1770	}
   1771
(lldb) p size
(ptrdiff_t) $0 = -9223372036854775792
(lldb) p XVECTOR(array)
(Lisp_Vector *) $1 = 0x00000001a1889240
(lldb) p XVECTOR(array)->header
(vectorlike_header) $2 = (size = -9223372036854775792)
(lldb) p *array
error: expression failed to parse:
error: <user expression 3>:1:1: incomplete type 'Lisp_X' where a complete type is required
*array
^
(lldb)