bug#41321: 27.0.91; Emacs aborts due to invalid pseudovector objects

[Eli Zaretskii <eliz@gnu.org>]

> Date: Fri, 22 May 2020 10:22:56 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 41321@debbugs.gnu.org
> 
> > > I'm already running with such a breakpoint, let's how it will catch
> > > something.                                        ^^^
> > 
> > Should have been "hope".  Sorry.
> 
> It happened again, and now insert-file-contents wasn't involved, so I
> guess it's off the hook.  The command which triggered the problem was
> self-insert-command, as shown in the backtrace below.  The problem
> seems to be with handling overlays when buffer text changes.

One more segfault very similar to the last one I reported: it happened
when calling report_overlay_modification due to text being inserted
into a buffer.

The backtrace and the debugging session are below.  Noteworthy
observations:

. The buffer's overlay chain and the buffer's marker chain are both
  intact and valid.

. The two markers, start_marker and end_marker, which are created by
  PRESERVE_START_END before calling before-change-functions, are NOT
  in the buffer's marker chain after run-hook-with-args returns.  This
  most probably means GC was invoked while run-hook-with-args ran and
  decided to GC those 2 markers, which then unchains them via
  unchain_dead_markers.

. last_marked[] doesn't seem to mention start_marker or end_marker, at
  least not in its last 470 slots:

    (gdb) find /g1 &last_marked[0], last_marked[last_marked_index-1], 0xa00000001ffac2c8
    Pattern not found.

  This seems to be a supporting evidence that those two markers were
  GC'ed.

. start_marker and end_marker encode pointers which are 8-byte
  aligned, not 16-byte aligned.  The values of the pointers are
  0x1ffac2a8 and 0x1ffac2c8, as can be seen from the debug session.

. There's nothing wrong with rvoe_arg.location; in the previous
  sessions we forgot to dereference it (it's a pointer to a Lisp
  object).  Here's how it looks when shown correctly:

    (gdb) p rvoe_arg.location
    $14 = (Lisp_Object *) 0x15c9298 <globals+120>
    (gdb) p *rvoe_arg.location
    $15 = XIL(0xc00000001646b9b0)
    (gdb) xtype
    Lisp_Cons
    (gdb) xcar
    $16 = 0x30
    (gdb) xsymbol
    $17 = (struct Lisp_Symbol *) 0x15ca210 <lispsym+48>
    "t"
    (gdb) p *rvoe_arg.location
    $18 = XIL(0xc00000001646b9b0)
    (gdb) xcdr
    $19 = 0xc00000001646b9d0
    (gdb) xtype
    Lisp_Cons
    (gdb) xcar
    $20 = 0xd5c0
    (gdb) xtype
    Lisp_Symbol
    (gdb) xsymbol
    $21 = (struct Lisp_Symbol *) 0x15d77a0 <lispsym+54720>
    "syntax-ppss-flush-cache"
    (gdb) p *rvoe_arg.location
    $22 = XIL(0xc00000001646b9b0)
    (gdb) xcdr
    $23 = 0xc00000001646b9d0
    (gdb) xcdr
    $24 = 0x0
    [...]
    (gdb) pp *rvoe_arg.location
    (t syntax-ppss-flush-cache)

. There's nothing wrong with GDB's xtype command: it fails when a Lisp
  object encodes a pointer to invalid memory:

    (gdb) p start_marker
    $25 = XIL(0xa00000001ffac2a8)
    (gdb) xtype
    Lisp_Vectorlike
    Cannot access memory at address 0x1ffac2a8
    (gdb) p/x start_marker
    $26 = 0xa00000001ffac2a8
    (gdb) xgettype $26
    (gdb) p $type
    $27 = Lisp_Vectorlike
    (gdb) xvectype $26
    Cannot access memory at address 0x1ffac2a8
    (gdb) p/x ((struct Lisp_Vector *) $26)->header.size
    warning: value truncated
    Cannot access memory at address 0x1ffac2a8
    (gdb) p/x ((struct Lisp_Vector *) $26)->header
    warning: value truncated
    Cannot access memory at address 0x1ffac2a8
    (gdb) p/x ((struct Lisp_Vector *) $26)
    warning: value truncated
    $35 = 0x1ffac2a8
    (gdb) p/x end_marker
    $38 = 0xa00000001ffac2c8
    (gdb) xtype
    Lisp_Vectorlike
    Cannot access memory at address 0x1ffac2a8
    (gdb) p/x ((struct Lisp_Vector *)0x1ffac2c8)->header
    Cannot access memory at address 0x1ffac2c8

. Provisional conclusion: the two temporary markers created by
  signal_before_change were on the stack (see my other message with
  code disassembly), and were GC'ed as side effect or running
  syntax-ppss-flush-cache via before-change-functions.  So we should
  see whether fixing the LISP_ALIGNMENT vs GCALIGNMENT discrepancy
  fixes this problem.

Here's the backtrace and the full debug session after the crash, with
some omissions:

Thread 1 received signal SIGSEGV, Segmentation fault.
PSEUDOVECTORP (code=<optimized out>, a=<optimized out>) at lisp.h:1720
1720          return PSEUDOVECTOR_TYPEP (XUNTAG (a, Lisp_Vectorlike,
(gdb) bt
#0  PSEUDOVECTORP (code=<optimized out>, a=<optimized out>) at lisp.h:1720
#1  MARKERP (x=<optimized out>) at lisp.h:2618
#2  CHECK_MARKER (x=XIL(0xa00000001ffac2c8)) at marker.c:133
#3  0x010f073c in Fmarker_position (marker=XIL(0xa00000001ffac2c8))
    at marker.c:452
#4  0x010edd34 in signal_before_change (preserve_ptr=0x0, end_int=276884,
    start_int=276884) at insdel.c:2179
#5  prepare_to_modify_buffer_1 (start=start@entry=276884,
    end=end@entry=276884, preserve_ptr=preserve_ptr@entry=0x0)
    at insdel.c:2007
#6  0x010ee27d in prepare_to_modify_buffer (start=276884, end=276884,
    preserve_ptr=preserve_ptr@entry=0x0) at insdel.c:2018
#7  0x010ee54d in insert_1_both (
    string=0x1e3c9c08 " 2823D 26-May  gdb-patches@sourceware.or [244] Re: [PATCH, testsuite] Fix some duplicate test names\n\r...",
    nchars=100, nbytes=100, inherit=false, prepare=true, before_markers=false)
    at insdel.c:896
#8  0x010ee5c5 in insert_1_both (string=<optimized out>,
    nchars=<optimized out>, nchars@entry=100, nbytes=<optimized out>,
    nbytes@entry=100, inherit=inherit@entry=false,
    prepare=prepare@entry=true, before_markers=before_markers@entry=false)
    at insdel.c:947
#9  0x01174188 in Fprinc (object=XIL(0x800000001e05f278),
    printcharfun=<optimized out>) at print.c:734
#10 0x0114fc5c in funcall_subr (subr=<optimized out>,
    numargs=<optimized out>, numargs@entry=2, args=<optimized out>,
    args@entry=0x82d9b8) at eval.c:2869
#11 0x0114daed in Ffuncall (nargs=3, args=args@entry=0x82d9b0) at eval.c:2794
#12 0x0118ebe7 in exec_byte_code (bytestr=<optimized out>,
    vector=<optimized out>, maxdepth=<optimized out>,
    args_template=<optimized out>, nargs=<optimized out>, nargs@entry=4,
    args=<optimized out>, args@entry=0x82dde8) at bytecode.c:633
#13 0x0115134f in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=4,
    arg_vector=arg_vector@entry=0x82dde8) at eval.c:2989
#14 0x0114da43 in Ffuncall (nargs=5, args=args@entry=0x82dde0) at eval.c:2808
#15 0x0118ebe7 in exec_byte_code (bytestr=<optimized out>,
    vector=<optimized out>, maxdepth=<optimized out>,
    args_template=<optimized out>, nargs=<optimized out>, nargs@entry=3,
    args=<optimized out>, args@entry=0x82e1b0) at bytecode.c:633
#16 0x0115134f in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=3,
    arg_vector=arg_vector@entry=0x82e1b0) at eval.c:2989
#17 0x0114da43 in Ffuncall (nargs=4, args=args@entry=0x82e1a8) at eval.c:2808
#18 0x0118ebe7 in exec_byte_code (bytestr=<optimized out>,
    vector=<optimized out>, maxdepth=<optimized out>,
    args_template=<optimized out>, nargs=<optimized out>, nargs@entry=0,
    args=<optimized out>, args@entry=0x82e570) at bytecode.c:633
#19 0x0115134f in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=0,
    arg_vector=arg_vector@entry=0x82e570) at eval.c:2989
#20 0x0114da43 in Ffuncall (nargs=nargs@entry=1, args=args@entry=0x82e568)
    at eval.c:2808
#21 0x0114de2d in Fapply (nargs=2, args=0x82e568) at eval.c:2377
#22 0x0114daed in Ffuncall (nargs=3, args=args@entry=0x82e560) at eval.c:2794
#23 0x0118ebe7 in exec_byte_code (bytestr=<optimized out>,
    vector=<optimized out>, maxdepth=<optimized out>,
    args_template=<optimized out>, nargs=<optimized out>, nargs@entry=0,
    args=<optimized out>, args@entry=0x82e8c0) at bytecode.c:633
#24 0x0115134f in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=0,
    arg_vector=arg_vector@entry=0x82e8c0) at eval.c:2989
#25 0x0114da43 in Ffuncall (nargs=1, args=args@entry=0x82e8b8) at eval.c:2808
#26 0x0118ebe7 in exec_byte_code (bytestr=<optimized out>,
    vector=<optimized out>, maxdepth=<optimized out>,
    args_template=<optimized out>, nargs=<optimized out>, nargs@entry=3,
    args=<optimized out>, args@entry=0x82ed30) at bytecode.c:633
#27 0x0115134f in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=3,
    arg_vector=arg_vector@entry=0x82ed30) at eval.c:2989
#28 0x0114da43 in Ffuncall (nargs=4, args=args@entry=0x82ed28) at eval.c:2808
#29 0x0118ebe7 in exec_byte_code (bytestr=<optimized out>,
    vector=<optimized out>, maxdepth=<optimized out>,
    args_template=<optimized out>, nargs=<optimized out>, nargs@entry=1,
    args=<optimized out>, args@entry=0x82f298) at bytecode.c:633
#30 0x0115134f in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=1,
    arg_vector=arg_vector@entry=0x82f298) at eval.c:2989
#31 0x0114da43 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x82f290)
    at eval.c:2808
#32 0x0114906d in Ffuncall_interactively (nargs=2, args=0x82f290)
    at callint.c:254
#33 0x0114daed in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x82f288)
    at eval.c:2794
#34 0x0114df22 in Fapply (nargs=nargs@entry=3, args=args@entry=0x82f288)
    at eval.c:2381
#35 0x0114afbb in Fcall_interactively (function=XIL(0x5f2c790),
    record_flag=<optimized out>, keys=XIL(0xa00000000759f578))
    at callint.c:342
#36 0x0114fc89 in funcall_subr (subr=<optimized out>,
    numargs=<optimized out>, numargs@entry=3, args=<optimized out>,
    args@entry=0x82f430) at eval.c:2872
#37 0x0114daed in Ffuncall (nargs=4, args=args@entry=0x82f428) at eval.c:2794
#38 0x0118ebe7 in exec_byte_code (bytestr=<optimized out>,
    vector=<optimized out>, maxdepth=<optimized out>,
    args_template=<optimized out>, nargs=<optimized out>, nargs@entry=1,
    args=<optimized out>, args@entry=0x82f7b8) at bytecode.c:633
#39 0x0115134f in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=1,
    arg_vector=arg_vector@entry=0x82f7b8) at eval.c:2989
#40 0x0114da43 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x82f7b0)
    at eval.c:2808
#41 0x0114dc1c in call1 (fn=XIL(0x3f30), arg1=XIL(0x5f2c790)) at eval.c:2654
#42 0x010d0efe in command_loop_1 () at keyboard.c:1463
#43 0x0114ca0f in internal_condition_case (
    bfun=bfun@entry=0x10d0a0e <command_loop_1>, handlers=XIL(0x90),
    hfun=hfun@entry=0x10c5049 <cmd_error>) at eval.c:1355
#44 0x010bdbda in command_loop_2 (ignore=XIL(0)) at keyboard.c:1091
#45 0x0114c996 in internal_catch (tag=XIL(0xdfb0),
    func=func@entry=0x10bdbb3 <command_loop_2>, arg=XIL(0)) at eval.c:1116
#46 0x010bdb5d in command_loop () at keyboard.c:1070
#47 0x010c4bf3 in recursive_edit_1 () at keyboard.c:714
#48 0x010c4f0c in Frecursive_edit () at keyboard.c:786
#49 0x0124a594 in main (argc=<optimized out>, argv=<optimized out>)
    at emacs.c:2054

Lisp Backtrace:
"princ" (0x82d9b8)
"rmail-new-summary-1" (0x82dde8)
"rmail-new-summary" (0x82e1b0)
"rmail-summary" (0x82e570)
"apply" (0x82e568)
"rmail-update-summary" (0x82e8c0)
"rmail-get-new-mail-1" (0x82ed30)
"rmail-get-new-mail" (0x82f298)
"funcall-interactively" (0x82f290)
"call-interactively" (0x82f430)
"command-execute" (0x82f7b8)
(gdb) fr 4
#4  0x010f073c in Fmarker_position (marker=XIL(0xa00000001ffac2c8))
    at marker.c:452
452       CHECK_MARKER (marker);
(gdb) up
#5  0x010edd34 in signal_before_change (preserve_ptr=0x0, end_int=276884,
    start_int=276884) at insdel.c:2179
2179          report_overlay_modification (FETCH_START, FETCH_END, 0,
(gdb) p current_buffer->overlays_before
$1 = (struct Lisp_Overlay *) 0x75ac520
(gdb) p *$
$2 = {
  header = {
    size = 1140854787
  },
  start = XIL(0xa0000000075ac4e0),
  end = XIL(0xa0000000075ac500),
  plist = XIL(0xc0000000077f2340),
  next = 0x0
}
(gdb) p/x $1->header.size
$3 = 0x44001003
(gdb) p current_buffer->name_
$4 = XIL(0x8000000007364540)
(gdb) xtype
Lisp_String
(gdb) xstring
$5 = (struct Lisp_String *) 0x7364540
"INBOX-summary"
(gdb) p current_buffer->overlays_before->start
$6 = XIL(0xa0000000075ac4e0)
(gdb) p *$
$7 = 1124081664
(gdb) p current_buffer->overlays_before->start
$8 = XIL(0xa0000000075ac4e0)
(gdb) xtype
Lisp_Vectorlike
PVEC_MARKER
(gdb) xmarker
$9 = (struct Lisp_Marker *) 0x75ac4e0
(gdb) p *$
$10 = {
  header = {
    size = 1124081664
  },
  buffer = 0x7519948,
  need_adjustment = 0,
  insertion_type = 0,
  next = 0x0,
  charpos = 1,
  bytepos = 1
}
(gdb) p current_buffer->overlays_before->next
$11 = (struct Lisp_Overlay *) 0x0
(gdb) p current_buffer->overlays_after
$12 = (struct Lisp_Overlay *) 0x0
(gdb) p rvoe_arg
$13 = {
  location = 0x15c9298 <globals+120>,
  errorp = false
}
(gdb) p rvoe_arg.location
$14 = (Lisp_Object *) 0x15c9298 <globals+120>
(gdb) p *rvoe_arg.location
$15 = XIL(0xc00000001646b9b0)
(gdb) xtype
Lisp_Cons
(gdb) xcar
$16 = 0x30
(gdb) xsymbol
$17 = (struct Lisp_Symbol *) 0x15ca210 <lispsym+48>
"t"
(gdb) p *rvoe_arg.location
$18 = XIL(0xc00000001646b9b0)
(gdb) xcdr
$19 = 0xc00000001646b9d0
(gdb) xtype
Lisp_Cons
(gdb) xcar
$20 = 0xd5c0
(gdb) xtype
Lisp_Symbol
(gdb) xsymbol
$21 = (struct Lisp_Symbol *) 0x15d77a0 <lispsym+54720>
"syntax-ppss-flush-cache"
(gdb) p *rvoe_arg.location
$22 = XIL(0xc00000001646b9b0)
(gdb) xcdr
$23 = 0xc00000001646b9d0
(gdb) xcdr
$24 = 0x0
(gdb) p start_marker
$25 = XIL(0xa00000001ffac2a8)
(gdb) xtype
Lisp_Vectorlike
Cannot access memory at address 0x1ffac2a8
(gdb) p/x start_marker
$26 = 0xa00000001ffac2a8
(gdb) xgettype $26
(gdb) p $type
$27 = Lisp_Vectorlike
(gdb) xvectype $26
Cannot access memory at address 0x1ffac2a8
(gdb) p/x ((struct Lisp_Vector *) $26)->header.size
warning: value truncated
Cannot access memory at address 0x1ffac2a8
(gdb) p/x ((struct Lisp_Vector *) $26)->header
warning: value truncated
Cannot access memory at address 0x1ffac2a8
(gdb) p/x ((struct Lisp_Vector *) $26)
warning: value truncated
$35 = 0x1ffac2a8
(gdb) p/x $26
$36 = 0xa00000001ffac2a8
(gdb) p/x ((struct Lisp_Vector *)0x1ffac2a8
A syntax error in expression, near `'.
(gdb) p/x ((struct Lisp_Vector *)0x1ffac2a8)
$37 = 0x1ffac2a8
(gdb) p/x *((struct Lisp_Vector *)0x1ffac2a8)
Cannot access memory at address 0x1ffac2a8
(gdb) p/x end_marker
$38 = 0xa00000001ffac2c8
(gdb) xtype
Lisp_Vectorlike
Cannot access memory at address 0x1ffac2a8
(gdb) p/x ((struct Lisp_Vector *)0x1ffac2c8)->header
Cannot access memory at address 0x1ffac2c8
(gdb) p Vfirst_change_hook
$39 = XIL(0)
(gdb) p current_buffer->text->markers
$40 = (struct Lisp_Marker *) 0x76353a0
(gdb) p *$
$41 = {
  header = {
    size = 1124081664
  },
  buffer = 0x7519948,
  need_adjustment = 0,
  insertion_type = 0,
  next = 0x76353e0,
  charpos = 1,
  bytepos = 1
}
(gdb) p current_buffer->text->markers->next
$42 = (struct Lisp_Marker *) 0x76353e0
(gdb) p *$
$43 = {
  header = {
    size = 1124081664
  },
  buffer = 0x7519948,
  need_adjustment = 0,
  insertion_type = 0,
  next = 0x7635420,
  charpos = 1,
  bytepos = 1
}
(gdb) p current_buffer->text->markers->next->next
$44 = (struct Lisp_Marker *) 0x7635420
(gdb) p *$
$45 = {
  header = {
    size = 1124081664
  },
  buffer = 0x7519948,
  need_adjustment = 0,
  insertion_type = 0,
  next = 0x16b6a5d0,
  charpos = 1,
  bytepos = 1
}
(gdb) p current_buffer->text->markers->next->next->next
$46 = (struct Lisp_Marker *) 0x16b6a5d0
(gdb) p *$
$47 = {
  header = {
    size = 1124081664
  },
  buffer = 0x7519948,
  need_adjustment = 0,
  insertion_type = 0,
  next = 0x16b6a5b0,
  charpos = 1,
  bytepos = 1
}
(gdb) p/x start_marker
$98 = 0xa00000001ffac2c8
(gdb) pp *rvoe_arg.location
(t syntax-ppss-flush-cache)
(gdb) p last_mar
last_marked        last_marked_index
(gdb) p last_marked_index
$99 = 498
(gdb) p last_marked[497]
$100 = XIL(0x439c370)
(gdb) xtype
Lisp_Vectorlike
Cannot access memory at address 0x1ffac2a8
(gdb) find /g1 &last_marked[0], last_marked[last_marked_index-1], 0xa00000001ffac2a8
Pattern not found.