bug#68690: Segmentation fault building with native-comp

[Eli Zaretskii <eliz@gnu.org>]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: jm@pub.pink,  68690@debbugs.gnu.org
> Date: Thu, 25 Jan 2024 21:43:01 -0500
> 
> >> Hmm... I can't reproduce it here (even with native-comp and
> >> `--with-wide-int`).
> > This build is without native-comp, but it's a 32-bit build.  Did you
> > try that?  I think that's the key to unlock this (see below).
> 
> I tried 32bit with and without native-comp, and with and without wide-int,
> but can't reproduce it here.  Maybe it only manifest itself under w32?

It could be, but I cannot imagine why the way we dump charsets has
anything to do with w32.

> In your message I see it crashes compiling `mwheel.el`; is that the
> first place where it crashes?

"First place" in what sense?

What happens in the build is that temacs is built and dumped to
produce bootstrap-emacs, then bootstrap-emacs starts compiling Lisp
files and crashes on the first one it tries to compile, with the
backtrace I posted.

> Does it crash on most other files as well?

It seems to crash with any Lisp file I try.  It also crashes in this
command:

  '../src/emacs.exe' -batch --no-site-file --no-site-lisp \
	      -l ./emacs-lisp/loaddefs-gen.elc \
      -f loaddefs-generate--emacs-batch . ./calc ./calendar ./cedet ./cedet/ede ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./image ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./use-package ./vc

> In interactive use?

If I invoke bootstrap-emacs interactively, it crashes during startup,
in window-system-initialization.  The backtrace for that is below, and
it again seems to show a bogus value of charset attributes.

> >> The above stack frame suggests it might be related
> >> to commit 33b8d5b6c5a (and hence unrelated to the original bug#68690
> >> which was a bug in `DOHASH`).
> >> Any chance you can investigate what is this `0x92348b000000000`?
> > It's obviously a bogus value, since Lisp objects in this build should
> > have their high 32 bits zero except for the type tag in the MSBs.
> 
> Indeed.
> 
> >> It should be a charset's attributes and the "idx=1" is because
> >> we're using `CHARSET_ATTR_NAME` to extract the name.
> > It sounds like we are not dumping the charset attributes correctly,
> > and that also corrupts all the fields of a struct charset following
> > the attributes.  Here's this charset in temacs:
> [...]
> >   (gdb) p cs->attributes
> >   $3 = XIL(0xa000000009023d88)
> [...]
> > And here's the same charset in emacs, after we restore from dump:
> [...]
> >     attributes = XIL(0x92848b000000000),
> 
> Yup, sure looks like the bytes got shifted by 4 bytes for some reason.
> 
> > I tried to figure out what is wrong with how we dump this new field,
> > but got lost in the proverbial twisty little passages of pdumper.c,
> > all alike.
> 
> 🙁
> 
> > For example, I cannot understand why some fields which are
> > Lisp objects are dumped with dump_field_lv while others with
> > dump_field_lv_or_rawptr, and what is the significance of WEIGHT_NORMAL
> > vs WEIGHT_STRONG.  Hopefully, the above gives enough information for
> > you to figure this out.
> 
> I'm just as lost as you are in pdumper.c, sadly.

Well, can I provide more info for you to try to figure out what's
wrong?  E.g., why and how did you decide to use dump_field_lv for
dumping the charset's attributes (which is a Lisp vector)?
Alternatively, any ideas for how should I proceed to debug this
myself?

See, I cannot afford to have the master branch be broken for me for
prolonged periods of time, as that prevents me from doing my job of
installing patches by others and testing various issues.  We must fix
this build, or face the need to revert the charset-related changes for
now.

Here's the backtrace during startup when bootstrap-emacs is invoked
with -Q:

  lisp.h:1784: Emacs fatal error: assertion failed: VECTORLIKEP (a)

  Thread 1 hit Breakpoint 1, terminate_due_to_signal (sig=22,
      backtrace_limit=2147483647) at emacs.c:442
  442       signal (sig, SIG_DFL);
  (gdb) bt
  #0  terminate_due_to_signal (sig=22, backtrace_limit=2147483647) at emacs.c:442
  #1  0x00802435 in die (msg=0xe6c80d <b_fwd+233> "VECTORLIKEP (a)",
      file=0xe6c740 <b_fwd+28> "lisp.h", line=1784) at alloc.c:8062
  #2  0x006b6a78 in XVECTOR (a=XIL(0x926755800000000)) at lisp.h:1784
  #3  0x006b6b02 in gc_asize (array=XIL(0x926755800000000)) at lisp.h:1800
  #4  0x006b6bee in AREF (array=XIL(0x926755800000000), idx=7) at lisp.h:1971
  #5  0x006ba13f in map_charset_chars (c_function=0x9a7fa1 <set_fontset_font>,
      function=XIL(0), arg=XIL(0xa000000009140a08), charset=0x91d912c, from=0,
      to=33) at charset.c:775
  #6  0x009a9cf2 in Fset_fontset_font (fontset=XIL(0x800000000912a370),
      characters=XIL(0xa3e0), font_spec=XIL(0xa000000009122ed0),
      frame=XIL(0xa0000000090d9de8), add=XIL(0x2a90)) at fontset.c:1668
  #7  0x009aa8f2 in Fnew_fontset (name=XIL(0x800000000912a370),
      fontlist=XIL(0xc000000005ed6570)) at fontset.c:1786
  #8  0x0084a583 in funcall_subr (subr=0xe5a5c0 <Snew_fontset>, numargs=2,
      args=0x9cd7250) at eval.c:3092
  #9  0x008bd5eb in exec_byte_code (fun=XIL(0xa000000009576978),
      args_template=0, nargs=0, args=0x9cd7208) at bytecode.c:815
  #10 0x0084ab2e in fetch_and_exec_byte_code (fun=XIL(0xa000000009279ff0),
      args_template=256, nargs=0, args=0x9cd7148) at eval.c:3135
  #11 0x0084b08d in funcall_lambda (fun=XIL(0xa000000009279ff0), nargs=0,
      arg_vector=0x9cd7148) at eval.c:3207
  #12 0x00849eb1 in funcall_general (fun=XIL(0xa000000009279ff0), numargs=0,
      args=0x9cd7148) at eval.c:2972
  #13 0x0084a236 in Ffuncall (nargs=1, args=0x9cd7140) at eval.c:3022
  #14 0x00848c4a in Fapply (nargs=2, args=0x9cd7140) at eval.c:2646
  #15 0x0084a9df in funcall_subr (subr=0xe51f40 <Sapply>, numargs=2,
      args=0x9cd7140) at eval.c:3113
  #16 0x008bd5eb in exec_byte_code (fun=XIL(0xa0000000095b4058),
      args_template=770, nargs=3, args=0x9cd7390) at bytecode.c:815
  #17 0x0084ab2e in fetch_and_exec_byte_code (fun=XIL(0xa00000000945dc90),
      args_template=0, nargs=0, args=0x5e5f5a0) at eval.c:3135
  #18 0x0084b08d in funcall_lambda (fun=XIL(0xa00000000945dc90), nargs=0,
      arg_vector=0x5e5f5a0) at eval.c:3207
  #19 0x0084acdd in apply_lambda (fun=XIL(0xa00000000945dc90), args=XIL(0),
      count=128) at eval.c:3157
  #20 0x0084861a in eval_sub (form=XIL(0xc0000000098d6248)) at eval.c:2572
  #21 0x008475ed in Feval (form=XIL(0xc0000000098d6248), lexical=XIL(0x30))
      at eval.c:2389
  #22 0x00740220 in top_level_2 () at keyboard.c:1173
  #23 0x00844564 in internal_condition_case (bfun=0x740197 <top_level_2>,
      handlers=XIL(0x90), hfun=0x73f6f6 <cmd_error>) at eval.c:1537
  #24 0x007402b3 in top_level_1 (ignore=XIL(0)) at keyboard.c:1185
  #25 0x008431d2 in internal_catch (tag=XIL(0x10a40),
      func=0x74023f <top_level_1>, arg=XIL(0)) at eval.c:1217
  #26 0x0074008a in command_loop () at keyboard.c:1134
  #27 0x0073f156 in recursive_edit_1 () at keyboard.c:744
  #28 0x0073f3f4 in Frecursive_edit () at keyboard.c:827
  #29 0x0073a400 in main (argc=2, argv=0x1f2570) at emacs.c:2624

  Lisp Backtrace:
  "new-fontset" (0x9cd7250)
  "setup-default-fontset" (0x9cd7208)
  "create-default-fontset" (0x9cd71c0)
  0x9279ff0 PVEC_COMPILED
  "apply" (0x9cd7140)
  "window-system-initialization" (0x9cd70b8)
  "command-line" (0x9cd7048)
  "normal-top-level" (0x5e5f5a0)
  (gdb) fr 5
  #5  0x006ba13f in map_charset_chars (c_function=0x9a7fa1 <set_fontset_font>,
      function=XIL(0), arg=XIL(0xa000000009140a08), charset=0x91d912c, from=0,
      to=33) at charset.c:775
  775           for (parents = CHARSET_SUPERSET (charset); CONSP (parents);
  (gdb) p charset
  $1 = (struct charset *) 0x91d912c
  (gdb) p *$
  $2 = {
    id = 0,
    attributes = XIL(0x926755800000000),
    dimension = -1610612736,
    code_space = {1, 33, 126, 94, 94, 0, 0, 1, 94, 0, 0, 1, 94, 0, 0},
    code_space_mask = 0x1 <error: Cannot access memory at address 0x1>,
    code_linear_p = 0,
    iso_chars_96 = 0,
    ascii_compatible_p = 0,
    supplementary_p = 0,
    compact_codes_p = 0,
    unified_p = 0,
    iso_final = 25,
    iso_revision = 49,
    emacs_mule_id = -1,
    method = 167,
    min_code = 0,
    max_code = 33,
    char_index_offset = 126,
    min_char = 0,
    max_char = 3713,
    invalid_code = 3806,
    fast_map = "\000\000\000\000\001\000\000 ", '\000' <repeats 181 times>,
    code_offset = 0
  }
  (gdb)