bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

I was hit by a segfault while scrolling through a C source file, in
this case dispnew.c.  The sequence of commands was this:

 emacs -Q
 C-h sit-for RET
 Click on the link to subr.el
 In subr.el go to where sit-for calls sleep-for and type C-h f RET
 Click on "C source code" to display dispnew.c
 Scroll down with C-n or C-v

The backtrace appears below, with some data I collected.  The argument
'args' to Flss is obviously bogus, but I don't understand how it came
into existence.  Maybe related to 0x30, which stands for the symbol t?
The first call-stack frame above that I can examine, frame #4, calls
c-beginning-of-statement-1 with 4 nil args and the last argument of t.
The levels below that are impenetrable for me: is there a way of
digging into this
F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0
thing?

Any suggestions for how to debug this further or what data to collect
that will give you an idea for the root cause(s)?

P.S. Note the stopped backtrace: this is something I see for the last
couple of days on the native-comp branch, not sure if it's related.  I
will report that separately.

P.P.S. I tried to start another instance of Emacs from the branch, and
it immediately displayed this:

  Re-entering top level after C stack overflow

Which probably means something unhealthy happens when you start Emacs
while another instance is under a debugger with the same *.eln files
loaded.

Here's the backtrace and some related variables from the crash site:

  Thread 1 received signal SIGSEGV, Segmentation fault.
  0x01236788 in arithcompare_driver (nargs=2, args=0x28, comparison=ARITH_LESS)
      at data.c:2673
  2673        if (NILP (arithcompare (args[i - 1], args[i], comparison)))
  (gdb) bt
  #0  0x01236788 in arithcompare_driver (nargs=2, args=0x28,
      comparison=ARITH_LESS) at data.c:2673
  #1  0x01236860 in Flss (nargs=2, args=0x28) at data.c:2691
  #2  0x61a92285 in F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0 ()
     from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-engine-ccfcb170-1b345b21.eln
  #3  0x01261898 in funcall_lambda (fun=XIL(0xa00000000796aed8), nargs=5,
      arg_vector=0x827a78) at eval.c:3292
  #4  0x012601ed in Ffuncall (nargs=6, args=0x827a70) at eval.c:3013
  #5  0x61b00dbf in F632d6a7573742d61667465722d66756e632d6172676c6973742d70_c_just_after_func_arglist_p_0 ()
     from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-engine-ccfcb170-1b345b21.eln
  #6  0x01261898 in funcall_lambda (fun=XIL(0xa000000007973cb8), nargs=0,
      arg_vector=0x827c50) at eval.c:3292
  #7  0x012601ed in Ffuncall (nargs=1, args=0x827c48) at eval.c:3013
  #8  0x61aee041 in F632d6261636b2d6f7665722d6d656d6265722d696e697469616c697a657273_c_back_over_member_initializers_0 ()
     from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-engine-ccfcb170-1b345b21.eln
  #9  0x01261898 in funcall_lambda (fun=XIL(0xa0000000079739f8), nargs=1,
      arg_vector=0x827e28) at eval.c:3292
  #10 0x012601ed in Ffuncall (nargs=2, args=0x827e20) at eval.c:3013
  #11 0x0a525b36 in ?? ()
  #12 0x01261898 in funcall_lambda (fun=XIL(0xa0000000079b97c0), nargs=1,
      arg_vector=0x8280c0) at eval.c:3292
  #13 0x012601ed in Ffuncall (nargs=2, args=0x8280b8) at eval.c:3013
  #14 0x0686af93 in ?? ()
  #15 0x012de838 in helper_save_restriction () at comp.c:4575
  #16 0x0122e9aa in wrong_type_argument (predicate=XIL(0x892404890c245c89),
      value=XIL(0x8244c89e45d8be0)) at data.c:143
  Backtrace stopped: previous frame inner to this frame (corrupt stack?)

  Lisp Backtrace:
  "c-beginning-of-statement-1" (0x827a78)
  "c-just-after-func-arglist-p" (0x827c50)
  "c-back-over-member-initializers" (0x827e28)
  "c-font-lock-cut-off-declarators" (0x8280c0)
  "font-lock-fontify-keywords-region" (0x828418)
  "font-lock-default-fontify-region" (0x828728)
  "c-font-lock-fontify-region" (0x8288d8)
  "font-lock-fontify-region" (0x828ac8)
  0x78fb7e8 PVEC_COMPILED
  "jit-lock--run-functions" (0x829460)
  "jit-lock-fontify-now" (0x829720)
  "jit-lock-function" (0x829948)
  "redisplay_internal (C function)" (0x0)
  (gdb) fr 3
  #3  0x01261898 in funcall_lambda (fun=XIL(0xa00000000796aed8), nargs=5,
      arg_vector=0x827a78) at eval.c:3292
  3292          val = XSUBR (fun)->function.a0 ();
  (gdb) p nargs
  $1 = 5
  (gdb) p args[0]
  No symbol "args" in current context.
  (gdb) p arg_vector
  $2 = (Lisp_Object *) 0x827a78
  (gdb) p arg_vector [0]
  $3 = XIL(0)
  (gdb) p arg_vector [1]
  $4 = XIL(0)
  (gdb) p arg_vector[0]
  $5 = XIL(0)
  (gdb) p arg_vector[1]
  $6 = XIL(0)
  (gdb) p arg_vector[2]
  $7 = XIL(0)
  (gdb) p arg_vector[3]
  $8 = XIL(0)
  (gdb) p arg_vector[4]
  $9 = XIL(0x30)
  (gdb) xtype
  Lisp_Symbol
  (gdb) xsymbol
  $10 = (struct Lisp_Symbol *) 0x186a390 <lispsym+48>
  "t"
  (gdb) up
  #4  0x012601ed in Ffuncall (nargs=6, args=0x827a70) at eval.c:3013
  3013        val = funcall_lambda (fun, numargs, args + 1);
  (gdb) p args[0]
  $11 = XIL(0x60800a8)
  (gdb) xtype
  Lisp_Symbol
  (gdb) xsymbol
  $12 = (struct Lisp_Symbol *) 0x78ea408
  "c-beginning-of-statement-1"
  (gdb) p args[1]
  $13 = XIL(0)
  (gdb) p args[2]
  $14 = XIL(0)
  (gdb) p args[3]
  $15 = XIL(0)
  (gdb) p args[4]
  $16 = XIL(0)
  (gdb) p args[5]
  $17 = XIL(0x30)
  (gdb) down
  #3  0x01261898 in funcall_lambda (fun=XIL(0xa00000000796aed8), nargs=5,
      arg_vector=0x827a78) at eval.c:3292
  3292          val = XSUBR (fun)->function.a0 ();
  (gdb) p fun
  $18 = XIL(0xa00000000796aed8)
  (gdb) xtype
  Lisp_Vectorlike
  PVEC_SUBR
  (gdb) xsubr
  $19 = (struct Lisp_Subr *) 0x796aed8
  {
    header = {
      size = 1342205952
    },
    function = {
      a0 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      a1 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      a2 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      a3 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      a4 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      a5 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      a6 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      a7 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      a8 = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      aUNEVALLED = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>,
      aMANY = 0x61a8d020 <F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0>
    },
    min_args = 0,
    max_args = 5,
    symbol_name = 0x796eac0 "c-beginning-of-statement-1",
    {
      intspec = 0x0,
      native_intspec = XIL(0)
    },
    doc = 91,
    native_comp_u = {XIL(0xa0000000078884c0)},
    native_c_name = {
      0x796eaf8 "F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0"},
    lambda_list = {XIL(0xc0000000079155b0)},
    type = {XIL(0)}
  }
  (gdb) p 0x28
  $20 = 40
  (gdb) xtype
  Lisp_Symbol
  (gdb) xsymbol
  $21 = (struct Lisp_Symbol *) 0x186a388 <lispsym+40>
  Cannot access memory at address 0x1a4
  (gdb)


In GNU Emacs 28.0.50 (build 1080, i686-pc-mingw32)
 of 2021-03-11 built on HOME-C4E4A596F7
Repository revision: 8497af6892fcf9b08a1c120e897c9f5c21ea64fa
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
System Description: Microsoft Windows XP Service Pack 3 (v5.1.0.2600)

Configured using:
 'configure -C --prefix=/d/usr --with-wide-int --with-modules
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-4 -g3''

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NOTIFY
W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS XPM
ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads w32notify
w32 lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 56717 12106)
 (symbols 48 7804 1)
 (strings 16 21565 2060)
 (string-bytes 1 626902)
 (vectors 16 13077)
 (vector-slots 8 172292 12096)
 (floats 8 23 61)
 (intervals 40 263 114)
 (buffers 888 10))

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> Date: Thu, 11 Mar 2021 13:27:52 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: Andrea Corallo <akrl@sdf.org>
> 
> P.P.S. I tried to start another instance of Emacs from the branch, and
> it immediately displayed this:
> 
>   Re-entering top level after C stack overflow
> 
> Which probably means something unhealthy happens when you start Emacs
> while another instance is under a debugger with the same *.eln files
> loaded.

This part with stack overflow is not reproducible, unfortunately:
subsequent attempts to do the same start Emacs normally.  Too bad.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> Date: Thu, 11 Mar 2021 13:27:52 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: Andrea Corallo <akrl@sdf.org>
> 
>   #15 0x012de838 in helper_save_restriction () at comp.c:4575
>   #16 0x0122e9aa in wrong_type_argument (predicate=XIL(0x892404890c245c89),
>       value=XIL(0x8244c89e45d8be0)) at data.c:143
>   Backtrace stopped: previous frame inner to this frame (corrupt stack?)
> 

Btw, it's clear that the arguments to wrong_type_argument are
garbled.  Perhaps some code somewhere clobbers the C stack?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> I was hit by a segfault while scrolling through a C source file, in
> this case dispnew.c.  The sequence of commands was this:
>
>  emacs -Q
>  C-h sit-for RET
>  Click on the link to subr.el
>  In subr.el go to where sit-for calls sleep-for and type C-h f RET
>  Click on "C source code" to display dispnew.c
>  Scroll down with C-n or C-v

I can't reproduce here :/

> The backtrace appears below, with some data I collected.  The argument
> 'args' to Flss is obviously bogus, but I don't understand how it came
> into existence.  Maybe related to 0x30, which stands for the symbol t?
> The first call-stack frame above that I can examine, frame #4, calls
> c-beginning-of-statement-1 with 4 nil args and the last argument of t.
> The levels below that are impenetrable for me: is there a way of
> digging into this
> F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0
> thing?
>
> Any suggestions for how to debug this further or what data to collect
> that will give you an idea for the root cause(s)?

Assuming is a miscompilation it's gonna be tricky to reduce it without a
reproducible testcase.

But if is a miscompilation is should be reproducible so either is not a
miscompilation or either the initial conditions are different.

> P.S. Note the stopped backtrace: this is something I see for the last
> couple of days on the native-comp branch, not sure if it's related.  I
> will report that separately.
>
> P.P.S. I tried to start another instance of Emacs from the branch, and
> it immediately displayed this:
>
>   Re-entering top level after C stack overflow
>
> Which probably means something unhealthy happens when you start Emacs
> while another instance is under a debugger with the same *.eln files
> loaded.

I often used more than one Emacs session from the same binary so at
least on GNU/Linux this does not appear to be a problem.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 06:46:50 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I was hit by a segfault while scrolling through a C source file, in
> > this case dispnew.c.  The sequence of commands was this:
> >
> >  emacs -Q
> >  C-h sit-for RET
> >  Click on the link to subr.el
> >  In subr.el go to where sit-for calls sleep-for and type C-h f RET
> >  Click on "C source code" to display dispnew.c
> >  Scroll down with C-n or C-v
> 
> I can't reproduce here :/

Did you try the 32-bit build --with-wide-int?  It could be specific to
that configuration.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 12 Mar 2021 06:46:50 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I was hit by a segfault while scrolling through a C source file, in
>> > this case dispnew.c.  The sequence of commands was this:
>> >
>> >  emacs -Q
>> >  C-h sit-for RET
>> >  Click on the link to subr.el
>> >  In subr.el go to where sit-for calls sleep-for and type C-h f RET
>> >  Click on "C source code" to display dispnew.c
>> >  Scroll down with C-n or C-v
>> 
>> I can't reproduce here :/
>
> Did you try the 32-bit build --with-wide-int?  It could be specific to
> that configuration.

Good point, it tried on 32-bit before and now 32-bit --with-wide-int but
still could not reproduce.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 12:04:34 +0000
> 
> >> >  emacs -Q
> >> >  C-h sit-for RET
> >> >  Click on the link to subr.el
> >> >  In subr.el go to where sit-for calls sleep-for and type C-h f RET
> >> >  Click on "C source code" to display dispnew.c
> >> >  Scroll down with C-n or C-v
> >> 
> >> I can't reproduce here :/
> >
> > Did you try the 32-bit build --with-wide-int?  It could be specific to
> > that configuration.
> 
> Good point, it tried on 32-bit before and now 32-bit --with-wide-int but
> still could not reproduce.

Is there any data I can collect to help diagnose the issue?  Anything
at all?  Like maybe disassembly of this F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0()
function or some part of it?

IOW, if the problem is miscompilation to native code, what facilities
do we have to report the details if the simple recipe doesn't
reproduce the problem?  We will have this kind of problems in the near
future, so having a good way of reporting the details might help
eliminate bugs faster.

Thanks.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 12 Mar 2021 12:04:34 +0000
>> 
>> >> >  emacs -Q
>> >> >  C-h sit-for RET
>> >> >  Click on the link to subr.el
>> >> >  In subr.el go to where sit-for calls sleep-for and type C-h f RET
>> >> >  Click on "C source code" to display dispnew.c
>> >> >  Scroll down with C-n or C-v
>> >> 
>> >> I can't reproduce here :/
>> >
>> > Did you try the 32-bit build --with-wide-int?  It could be specific to
>> > that configuration.
>> 
>> Good point, it tried on 32-bit before and now 32-bit --with-wide-int but
>> still could not reproduce.
>
> Is there any data I can collect to help diagnose the issue?  Anything
> at all?  Like maybe disassembly of this F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0()
> function or some part of it?
>
> IOW, if the problem is miscompilation to native code, what facilities
> do we have to report the details if the simple recipe doesn't
> reproduce the problem?  We will have this kind of problems in the near
> future, so having a good way of reporting the details might help
> eliminate bugs faster.

Generally speaking the first step is to identify the function that is
responsible for the bug, this is often on the top of the back-trace but
not necessarily.  In the unfortunate case I typically proceed by
bisection.

When the function is identified I typically construct a single function
reproducer, for this I typically need the input parameters and I try to
substitute all other values coming from the environment with something I
can control.  This step involve understanding which part of the
environment are captured by the function (say: point, current buffer
content etc etc...).

At that point I reduce the function searching for the minimal piece of
code that behaves differently when native compiled.

At this point will typically start the "smart" part of the
investigation.

Here the problem is that being not reproducible we are stuck in the
first steps, reproducibility is tipically a pre for this kind of
analysis.  But again if it's a miscompilation it *must* be reproducible
because code is not morphing so probably we are not reproducing it
precisely?

BTW cc-engine.el is dynamic scope, this means we do not perform any
optimization in comp.el and we perform a bare 1:1 translation, so at
this stage I'd be rather skeptical this is miscompiled.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 15:27:30 +0000
> 
> Generally speaking the first step is to identify the function that is
> responsible for the bug, this is often on the top of the back-trace but
> not necessarily.  In the unfortunate case I typically proceed by
> bisection.

In my case the top of the stack looks like this:

  #0  0x01236788 in arithcompare_driver (nargs=2, args=0x28,
      comparison=ARITH_LESS) at data.c:2673
  #1  0x01236860 in Flss (nargs=2, args=0x28) at data.c:2691
  #2  0x0a872285 in ?? ()
  #3  0x01261898 in funcall_lambda (fun=XIL(0xa00000000a0bf230), nargs=5,
      arg_vector=0x826a08) at eval.c:3292
  #4  0x012601ed in Ffuncall (nargs=6, args=0x826a00) at eval.c:3013
  #5  0x0a8e0dbf in ?? ()
  #6  0x012601ed in Ffuncall (nargs=1, args=0x826bd8) at eval.c:3013
  #7  0x0a8ce041 in ?? ()
  #8  0x01261898 in funcall_lambda (fun=XIL(0xa0000000069f2a50), nargs=1,
      arg_vector=0x826db8) at eval.c:3292
  #9  0x012601ed in Ffuncall (nargs=2, args=0x826db0) at eval.c:3013
  #10 0x70895b36 in F632d666f6e742d6c6f636b2d6375742d6f66662d6465636c617261746f7273_c_font_lock_cut_off_declarators_0 ()
     from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-fonts-d7d8a7f5-b7c359cd.eln
  #11 0x01261898 in funcall_lambda (fun=XIL(0xa0000000079249a0), nargs=1,
      arg_vector=0x827050) at eval.c:3292
  #12 0x012601ed in Ffuncall (nargs=2, args=0x827048) at eval.c:3013

And the corresponding Lisp backtrace:

  "c-beginning-of-statement-1" (0x826a08)
  "c-just-after-func-arglist-p" (0x826be0)
  "c-back-over-member-initializers" (0x826db8)
  "c-font-lock-cut-off-declarators" (0x827050)
  "font-lock-fontify-keywords-region" (0x8273a8)
  "font-lock-default-fontify-region" (0x8276b8)

(Don't ask me why "<", i.e. Flss, doesn't appear in the Lisp
backtrace: something strange happens with backtraces here, as I will
describe in another message.  I think the "??" things in the backtrace
are related.)

How do I go about finding the function that's responsible for the
problem given the above?  The problem is 100% reproducible for me.

> Here the problem is that being not reproducible we are stuck in the
> first steps, reproducibility is tipically a pre for this kind of
> analysis.  But again if it's a miscompilation it *must* be reproducible
> because code is not morphing so probably we are not reproducing it
> precisely?

Here's another reproducer:

  emacs -Q
  C-x C-f src/dispnew.c
  C-s sleep-for

I usually get a SIGSEGV before I even type the whole of "sleep-for".

Do you have all of the cc-*.el files natively-compiled?  I do.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 12 Mar 2021 15:27:30 +0000
>> 
>> Generally speaking the first step is to identify the function that is
>> responsible for the bug, this is often on the top of the back-trace but
>> not necessarily.  In the unfortunate case I typically proceed by
>> bisection.
>
> In my case the top of the stack looks like this:
>
>   #0  0x01236788 in arithcompare_driver (nargs=2, args=0x28,
>       comparison=ARITH_LESS) at data.c:2673
>   #1  0x01236860 in Flss (nargs=2, args=0x28) at data.c:2691
>   #2  0x0a872285 in ?? ()
>   #3  0x01261898 in funcall_lambda (fun=XIL(0xa00000000a0bf230), nargs=5,
>       arg_vector=0x826a08) at eval.c:3292
>   #4  0x012601ed in Ffuncall (nargs=6, args=0x826a00) at eval.c:3013
>   #5  0x0a8e0dbf in ?? ()
>   #6  0x012601ed in Ffuncall (nargs=1, args=0x826bd8) at eval.c:3013
>   #7  0x0a8ce041 in ?? ()
>   #8  0x01261898 in funcall_lambda (fun=XIL(0xa0000000069f2a50), nargs=1,
>       arg_vector=0x826db8) at eval.c:3292
>   #9  0x012601ed in Ffuncall (nargs=2, args=0x826db0) at eval.c:3013
>   #10 0x70895b36 in F632d666f6e742d6c6f636b2d6375742d6f66662d6465636c617261746f7273_c_font_lock_cut_off_declarators_0 ()
>      from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-fonts-d7d8a7f5-b7c359cd.eln
>   #11 0x01261898 in funcall_lambda (fun=XIL(0xa0000000079249a0), nargs=1,
>       arg_vector=0x827050) at eval.c:3292
>   #12 0x012601ed in Ffuncall (nargs=2, args=0x827048) at eval.c:3013
>
> And the corresponding Lisp backtrace:
>
>   "c-beginning-of-statement-1" (0x826a08)
>   "c-just-after-func-arglist-p" (0x826be0)
>   "c-back-over-member-initializers" (0x826db8)
>   "c-font-lock-cut-off-declarators" (0x827050)
>   "font-lock-fontify-keywords-region" (0x8273a8)
>   "font-lock-default-fontify-region" (0x8276b8)
>
> (Don't ask me why "<", i.e. Flss, doesn't appear in the Lisp
> backtrace: something strange happens with backtraces here, as I will
> describe in another message.  I think the "??" things in the backtrace
> are related.)
>
> How do I go about finding the function that's responsible for the
> problem given the above?  The problem is 100% reproducible for me.

One easy option is to evaluate say `c-beginning-of-statement-1' (as
first defendant) and see if afterwards it still crashes.  Same one can
load entire files to exclude entirely their content from the equation.

>> Here the problem is that being not reproducible we are stuck in the
>> first steps, reproducibility is tipically a pre for this kind of
>> analysis.  But again if it's a miscompilation it *must* be reproducible
>> because code is not morphing so probably we are not reproducing it
>> precisely?
>
> Here's another reproducer:
>
>   emacs -Q
>   C-x C-f src/dispnew.c
>   C-s sleep-for
>
> I usually get a SIGSEGV before I even type the whole of "sleep-for".

Can't reproduce this either :(  That's odd.

> Do you have all of the cc-*.el files natively-compiled?  I do.

Looks so.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 16:08:33 +0000
> 
> > In my case the top of the stack looks like this:
> >
> >   #0  0x01236788 in arithcompare_driver (nargs=2, args=0x28,
> >       comparison=ARITH_LESS) at data.c:2673
> >   #1  0x01236860 in Flss (nargs=2, args=0x28) at data.c:2691
> >   #2  0x0a872285 in ?? ()
> >   #3  0x01261898 in funcall_lambda (fun=XIL(0xa00000000a0bf230), nargs=5,
> >       arg_vector=0x826a08) at eval.c:3292
> >   #4  0x012601ed in Ffuncall (nargs=6, args=0x826a00) at eval.c:3013
> >   #5  0x0a8e0dbf in ?? ()
> >   #6  0x012601ed in Ffuncall (nargs=1, args=0x826bd8) at eval.c:3013
> >   #7  0x0a8ce041 in ?? ()
> >   #8  0x01261898 in funcall_lambda (fun=XIL(0xa0000000069f2a50), nargs=1,
> >       arg_vector=0x826db8) at eval.c:3292

Btw, what are those "??" there instead of function names?  Do you see
the same on your system?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 16:08:33 +0000
> 
> > And the corresponding Lisp backtrace:
> >
> >   "c-beginning-of-statement-1" (0x826a08)
> >   "c-just-after-func-arglist-p" (0x826be0)
> >   "c-back-over-member-initializers" (0x826db8)
> >   "c-font-lock-cut-off-declarators" (0x827050)
> >   "font-lock-fontify-keywords-region" (0x8273a8)
> >   "font-lock-default-fontify-region" (0x8276b8)
> >
> > (Don't ask me why "<", i.e. Flss, doesn't appear in the Lisp
> > backtrace: something strange happens with backtraces here, as I will
> > describe in another message.  I think the "??" things in the backtrace
> > are related.)
> >
> > How do I go about finding the function that's responsible for the
> > problem given the above?  The problem is 100% reproducible for me.
> 
> One easy option is to evaluate say `c-beginning-of-statement-1' (as
> first defendant) and see if afterwards it still crashes.  Same one can
> load entire files to exclude entirely their content from the equation.

Just evaluating c-beginning-of-statement-1 doesn't help.  But if I
load cc-engine.el, then the crash goes away.

(Btw, if I load cc-engine.elc, it says it loads the .eln file
instead?  is that intentional?)

> > Do you have all of the cc-*.el files natively-compiled?  I do.
> 
> Looks so.

Here's the full list of *.eln files I have in the eln-cache, maybe you
can spot some differences wrt what you have:

  ansi-color-75eac800-c3f61e7e.eln
  apropos-7c1ecbdf-9d649fe6.eln
  auth-source-49df7eef-368f27dd.eln
  bug-reference-79735dad-1dd033e2.eln
  bytecomp-12882072-5aabcc96.eln
  byte-opt-9c5f25f5-125a1c43.eln
  cc-align-bb265728-02034f8e.eln
  cc-bytecomp-4817e810-d16f606e.eln
  cc-cmds-9eab15be-9c2d9c5e.eln
  cc-defs-029877ca-e2bf1927.eln
  cc-e.n
  cc-engine-ccfcb170-1b345b21.eln
  cc-fonts-d7d8a7f5-b7c359cd.eln
  cc-guess-fa39ca73-10a5145e.eln
  cc-langs-de47cd28-5b5f4a5fhuBjCz.eln.tmp
  cc-menus-d370fa63-f0f22492.eln
  cc-mode-515a98a5-c7761ade.eln
  cconv-3b1f1f98-1f92e386.eln
  cc-styles-e4992978-134c4151.eln
  cc-vars-6cc3f0fc-b327aefc.eln
  cl-extra-d7051cba-ff815ded.eln
  cl-lib-8b938900-fbcee53d.eln
  cl-macs-7ae82f81-71a226f6.eln
  cl-seq-f464c4e2-95d0eb23.eln
  comint-faef15ad-9b304e32.eln
  comp-7672a6ed-9eeeb410.eln
  comp-cstr-ef162ef7-50619cfa.eln
  compile-91e1c2a0-004c9cb4.eln
  cus-edit-3cd01345-0185ea92.eln
  derived-381801a7-192e647c.eln
  descr-text-4ed9ee33-0bda188d.eln
  dframe-2a07085b-11be22d3.eln
  diff-mode-ac82991e-01e3cc24.eln
  dired-6a3ae2bc-f9b00bd0.eln
  dired-aux-1ff8c91a-5e8d68e3.eln
  easy-mmode-5bf0223e-cf7b61bf.eln
  eieio-0db8d1d4-4e501056.eln
  eieio-core-63c05904-38f7202b.eln
  eieio-opt-b31de7ef-12184a12.eln
  emacsbug-2b184f71-c7cc3f7a.eln
  enriched-ae445047-ffd49ae6.eln
  epa-bdd8ea1c-9c6ccc43.eln
  epg-config-78240760-62de08d4.eln
  epg-de089247-fae49f41.eln
  ezimage-53d8406d-4cbd31ae.eln
  find-func-64af57a4-ba832011.eln
  fortran-ad9735b1-90f9ed86.eln
  gmm-utils-3a9b481d-cecf0e84.eln
  gnus-util-b645aa9c-39a0db66.eln
  gv-e0cf7478-6f822f02.eln
  help-fns-d233c6e8-5e828b84.eln
  help-mode-d4dbae3d-3727161b.eln
  ietf-drums-23d9ecfe-579c3b1e.eln
  imenu-a6693d03-62e7d385.eln
  info-ce12c0ca-df930c16.eln
  info-look-27e24920-050cbc72.eln
  json-a90a1eab-9034d030.eln
  lao-util-4c214197-8053260c.eln
  mailabbrev-287b6d20-a5fad4b7.eln
  mail-extr-e04131ae-12208c1c.eln
  mailheader-2dff755b-9d40572b.eln
  mail-utils-ae525238-708bf20f.eln
  man-9b8001be-e7203284.eln
  map-5d7b616a-baad437f.eln
  message-4a315c5a-7ec20cdd.eln
  misearch-3d1286b0-e86a9158.eln
  mm-bodies-5a92b2dc-7e472c08.eln
  mm-decode-4a4c16d8-6d118abe.eln
  mm-encode-4f0ce800-8f6b2a84.eln
  mml-34028078-23c8ee7c.eln
  mml-sec-827872f6-5d3f8ae5.eln
  mm-util-3a4439f0-4318e260.eln
  password-cache-187e4eec-b2e0084a.eln
  pp-5d47c1cc-e2c622e6.eln
  puny-eb5a151f-06b0d248.eln
  radix-tree-669a468d-e66242bf.eln
  rfc2045-713c994b-94aee2d3.eln
  rfc2047-b84e264d-42c8bbe1.eln
  rfc2231-e2cbed75-f8579d90.eln
  rfc822-a91daeff-329e7b69.eln
  ring-bff0b981-59f7c140.eln
  rmail-f41733e3-6e1970d3.eln
  rmc-9c382b5d-38a8af9e.eln
  ruler-mode-3c3fd53f-72bfc9a1.eln
  rx-627d8c83-551f997f.eln
  sendmail-bcf4df3f-704bb7ae.eln
  seq-c314b42f-92293e32.eln
  shadow-87c8aad9-c7cf33e5.eln
  shortdoc-c079c0cb-d9ab4584.eln
  sort-14dd51e7-a0f100a3.eln
  speedbar-2a9b6d1b-3e59deba.eln
  subr--trampoline-782d646973706c61792d6261636b696e672d73746f7265_x_display_backing_store_0.eln
  subr--trampoline-782d646973706c61792d636f6c6f722d63656c6c73_x_display_color_cells_0.eln
  subr--trampoline-782d646973706c61792d6d6d2d686569676874_x_display_mm_height_0.eln
  subr--trampoline-782d646973706c61792d6d6d2d7769647468_x_display_mm_width_0.eln
  subr--trampoline-782d646973706c61792d706978656c2d686569676874_x_display_pixel_height_0.eln
  subr--trampoline-782d646973706c61792d706978656c2d7769647468_x_display_pixel_width_0.eln
  subr--trampoline-782d646973706c61792d706c616e6573_x_display_planes_0.eln
  subr--trampoline-782d646973706c61792d73637265656e73_x_display_screens_0.eln
  subr--trampoline-782d646973706c61792d76697375616c2d636c617373_x_display_visual_class_0.eln
  subr--trampoline-782d6c6973742d666f6e7473_x_list_fonts_0.eln
  subr--trampoline-782d7365727665722d6d61782d726571756573742d73697a65_x_server_max_request_size_0.eln
  subr--trampoline-782d7365727665722d76656e646f72_x_server_vendor_0.eln
  subr--trampoline-782d7365727665722d76657273696f6e_x_server_version_0.eln
  subr-x-02dfef32-17faeb1d.eln
  tabify-b74f3a50-59f526ca.eln
  text-property-search-db1383f6-3658aa80.eln
  thai-util-75470e18-576e35e0.eln
  thai-word-5d4a80fb-6bc70d61.eln
  thingatpt-6fc8a4ab-f84cd1e6.eln
  time-date-40951a48-0eafe94e.eln
  vc-d8599e21-047fc768.eln
  vc-dispatcher-d82eb341-d439fc51.eln
  vc-git-5d6756dc-43af02c2.eln
  view-faefc6b2-24d3a53e.eln
  warnings-28e75f4d-c98d4e00.eln
  wid-edit-5b92861a-77dee7ff.eln

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Fri, Mar 12, 2021 at 12:52 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Andrea Corallo <akrl@sdf.org>
> > Cc: 47067@debbugs.gnu.org
> > Date: Fri, 12 Mar 2021 12:04:34 +0000
> >
> > >> >  emacs -Q
> > >> >  C-h sit-for RET
> > >> >  Click on the link to subr.el
> > >> >  In subr.el go to where sit-for calls sleep-for and type C-h f RET
> > >> >  Click on "C source code" to display dispnew.c
> > >> >  Scroll down with C-n or C-v
> > >>
> > >> I can't reproduce here :/
> > >
> > > Did you try the 32-bit build --with-wide-int?  It could be specific to
> > > that configuration.
> >
> > Good point, it tried on 32-bit before and now 32-bit --with-wide-int but
> > still could not reproduce.
>
> Is there any data I can collect to help diagnose the issue?  Anything
> at all?  Like maybe disassembly of this F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0()
> function or some part of it?

I think disassembling that function couldn't hurt, and it might help,
particularly the insns around the call site (but, of course, Andrea's
the expert). Passing 0x28 where the argument pointer should be is very
wrong; my suspicion is that the frame base pointer is NULL and there
are five arguments, leaving us with 0x28 pointing to what's allegedly
the base of the "proper" stack, but I don't think that's even how it's
supposed to work in the dynamic-scope case...

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 12 Mar 2021 16:08:33 +0000
>> 
>> > And the corresponding Lisp backtrace:
>> >
>> >   "c-beginning-of-statement-1" (0x826a08)
>> >   "c-just-after-func-arglist-p" (0x826be0)
>> >   "c-back-over-member-initializers" (0x826db8)
>> >   "c-font-lock-cut-off-declarators" (0x827050)
>> >   "font-lock-fontify-keywords-region" (0x8273a8)
>> >   "font-lock-default-fontify-region" (0x8276b8)
>> >
>> > (Don't ask me why "<", i.e. Flss, doesn't appear in the Lisp
>> > backtrace: something strange happens with backtraces here, as I will
>> > describe in another message.  I think the "??" things in the backtrace
>> > are related.)
>> >
>> > How do I go about finding the function that's responsible for the
>> > problem given the above?  The problem is 100% reproducible for me.
>> 
>> One easy option is to evaluate say `c-beginning-of-statement-1' (as
>> first defendant) and see if afterwards it still crashes.  Same one can
>> load entire files to exclude entirely their content from the equation.
>
> Just evaluating c-beginning-of-statement-1 doesn't help.  But if I
> load cc-engine.el, then the crash goes away.

Okay, then probably is one of the other four c-* functions we see in the
backtrace.

> (Btw, if I load cc-engine.elc, it says it loads the .eln file
> instead?  is that intentional?)

Yes, .eln load is "transparent" and triggered automatically while
loading a .elc file when the corresponding .eln is found in the
`comp-eln-load-path'.

To force the .elc to be loaded one has to bind `load-no-native' to
non-nil.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 12 Mar 2021 16:08:33 +0000
>> 
>> > In my case the top of the stack looks like this:
>> >
>> >   #0  0x01236788 in arithcompare_driver (nargs=2, args=0x28,
>> >       comparison=ARITH_LESS) at data.c:2673
>> >   #1  0x01236860 in Flss (nargs=2, args=0x28) at data.c:2691
>> >   #2  0x0a872285 in ?? ()
>> >   #3  0x01261898 in funcall_lambda (fun=XIL(0xa00000000a0bf230), nargs=5,
>> >       arg_vector=0x826a08) at eval.c:3292
>> >   #4  0x012601ed in Ffuncall (nargs=6, args=0x826a00) at eval.c:3013
>> >   #5  0x0a8e0dbf in ?? ()
>> >   #6  0x012601ed in Ffuncall (nargs=1, args=0x826bd8) at eval.c:3013
>> >   #7  0x0a8ce041 in ?? ()
>> >   #8  0x01261898 in funcall_lambda (fun=XIL(0xa0000000069f2a50), nargs=1,
>> >       arg_vector=0x826db8) at eval.c:3292
>
> Btw, what are those "??" there instead of function names?  Do you see
> the same on your system?

Yes I think too they are in place of function names.  You could verify
if the address is mapped by an .eln file.

I do not see those in my back-traces so it might a bug of the Windows
toolchain?

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 19:04:07 +0000
> 
> > Just evaluating c-beginning-of-statement-1 doesn't help.  But if I
> > load cc-engine.el, then the crash goes away.
> 
> Okay, then probably is one of the other four c-* functions we see in the
> backtrace.

Yes, but how to determine which one?

> > (Btw, if I load cc-engine.elc, it says it loads the .eln file
> > instead?  is that intentional?)
> 
> Yes, .eln load is "transparent" and triggered automatically while
> loading a .elc file when the corresponding .eln is found in the
> `comp-eln-load-path'.
> 
> To force the .elc to be loaded one has to bind `load-no-native' to
> non-nil.

I think if load-file is invoked interactively, and the user actually
types "foo.elc", we need to bind load-no-native non-nil
automatically.  Otherwise users would be surprised, as it goes against
the logic of what we do when the user types "foo.el".

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 19:30:31 +0000
> 
> > Btw, what are those "??" there instead of function names?  Do you see
> > the same on your system?
> 
> Yes I think too they are in place of function names.  You could verify
> if the address is mapped by an .eln file.

Doesn't look like that.

> I do not see those in my back-traces so it might a bug of the Windows
> toolchain?

Please show the full backtrace (including the Lisp backtrace) you get
as result of the following steps:

  gdb ./emacs -Q
  (gdb) break Fredraw_display
  (gdb) r -Q

  C-x C-f dispnew.c
  M-x redraw-display

  (gdb) break Fskip_chars_backward
  (gdb) c

  C-s sleep

  (gdb) bt

I'd like to compare your backtrace with what I get here.

Btw, is your build configured --enable-checking='yes,glyphs' ?  If
not, could you please reconfigure the 32-bit build with wide ints, and
see if you can reproduce the crashes then?  In any case, please show
the backtrace from that configuration, to avoid gratuitous
differences.

Thanks.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Fri, 12 Mar 2021 18:42:17 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> I think disassembling that function couldn't hurt, and it might help,
> particularly the insns around the call site (but, of course, Andrea's
> the expert).

They are large functions.  I will post the disassembly if someone
wants to look at it.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 12 Mar 2021 19:04:07 +0000
>> 
>> > Just evaluating c-beginning-of-statement-1 doesn't help.  But if I
>> > load cc-engine.el, then the crash goes away.
>> 
>> Okay, then probably is one of the other four c-* functions we see in the
>> backtrace.
>
> Yes, but how to determine which one?

Given they are 4 one could go evaluating these one by one, if they were
more bisection would have been the best strategy.  Yeah that's not the
most fun...

>
>> > (Btw, if I load cc-engine.elc, it says it loads the .eln file
>> > instead?  is that intentional?)
>> 
>> Yes, .eln load is "transparent" and triggered automatically while
>> loading a .elc file when the corresponding .eln is found in the
>> `comp-eln-load-path'.
>> 
>> To force the .elc to be loaded one has to bind `load-no-native' to
>> non-nil.
>
> I think if load-file is invoked interactively, and the user actually
> types "foo.elc", we need to bind load-no-native non-nil
> automatically.  Otherwise users would be surprised, as it goes against
> the logic of what we do when the user types "foo.el".

We certanly can do this if this is what we want.  This breaks a little
the idea to have the system as much transparent as possible, I went this
way cause this was my understanding of what we wanted but I've no strong
feeling with that.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 12 Mar 2021 19:30:31 +0000
>> 
>> > Btw, what are those "??" there instead of function names?  Do you see
>> > the same on your system?
>> 
>> Yes I think too they are in place of function names.  You could verify
>> if the address is mapped by an .eln file.
>
> Doesn't look like that.
>
>> I do not see those in my back-traces so it might a bug of the Windows
>> toolchain?
>
> Please show the full backtrace (including the Lisp backtrace) you get
> as result of the following steps:
>
>   gdb ./emacs -Q
>   (gdb) break Fredraw_display
>   (gdb) r -Q
>
>   C-x C-f dispnew.c
>   M-x redraw-display
>
>   (gdb) break Fskip_chars_backward
>   (gdb) c
>
>   C-s sleep
>
>   (gdb) bt
>
> I'd like to compare your backtrace with what I get here.

Mmmh, my Emacs on the 32bit system I prepared when running interactively
under gdb is unusable because all keys except the basic letters are
mixed-up.  I never experienced this, is this common?  How can I solve
it?

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 20:21:26 +0000
> 
> > I'd like to compare your backtrace with what I get here.
> 
> Mmmh, my Emacs on the 32bit system I prepared when running interactively
> under gdb is unusable because all keys except the basic letters are
> mixed-up.  I never experienced this, is this common?  How can I solve
> it?

Rebuild Emacs not under GDB.  I think the breakpoints and other stuff
you set up in GDB get dumped into the pdmp file, and thus render the
dumped Emacs not very usable.  I always rebuild Emacs when I let it
dump itself (which happens rarely, because I rarely have to debug
temacs).

If the above doesn't help, it could be that some of the *.eln files
are damaged for similar reasons, so maybe remove them and let Emacs
recompile them as well.

If none of the above helps, please describe how did you "prepare Emacs
on the 32bit system when running interactively under GDB", maybe I
don't understand what exactly you did there.

Meanwhile, could you please post the backtrace I asked for from the
64-bit build?  Maybe it will tell me enough to get some ideas.

Thanks.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Fri, Mar 12, 2021 at 7:50 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Fri, 12 Mar 2021 18:42:17 +0000
> > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> >
> > I think disassembling that function couldn't hurt, and it might help,
> > particularly the insns around the call site (but, of course, Andrea's
> > the expert).
>
> They are large functions.  I will post the disassembly if someone
> wants to look at it.

I'm volunteering :-) I've looked at the architecture-independent
earlier stages of the compile process, and they look fine here...

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 12 Mar 2021 20:10:27 +0000
> 
> >> > Just evaluating c-beginning-of-statement-1 doesn't help.  But if I
> >> > load cc-engine.el, then the crash goes away.
> >> 
> >> Okay, then probably is one of the other four c-* functions we see in the
> >> backtrace.
> >
> > Yes, but how to determine which one?
> 
> Given they are 4 one could go evaluating these one by one, if they were
> more bisection would have been the best strategy.  Yeah that's not the
> most fun...

I was wrong: it _is_ c-beginning-of-statement-1, after all.  It's just
that the procedure I followed to evaluate is was wrong.  AFAIU now, it
should be: first load-library cc-engine (which loads the .eln file),
then evaluate the function's definition in the .el file.

So now, given that this (humongous) function is the suspect, how do I
proceed with the next steps, which you described as follows:

  When the function is identified I typically construct a single function
  reproducer, for this I typically need the input parameters and I try to
  substitute all other values coming from the environment with something I
  can control.  This step involves understanding which part of the
  environment are captured by the function (say: point, current buffer
  content etc etc...).

  At that point I reduce the function searching for the minimal piece of
  code that behaves differently when native compiled.

  At this point will typically start the "smart" part of the
  investigation.

> >> To force the .elc to be loaded one has to bind `load-no-native' to
> >> non-nil.
> >
> > I think if load-file is invoked interactively, and the user actually
> > types "foo.elc", we need to bind load-no-native non-nil
> > automatically.  Otherwise users would be surprised, as it goes against
> > the logic of what we do when the user types "foo.el".
> 
> We certanly can do this if this is what we want.  This breaks a little
> the idea to have the system as much transparent as possible, I went this
> way cause this was my understanding of what we wanted but I've no strong
> feeling with that.

I don't think this will break the transparent operation, because
loading a package non-interactively (as in when the corresponding
feature is 'require'd by some code) will still load the .eln file.
Only the following 2 use cases will be affected:

  M-x load-file RET /path/to/FOO.elc RET
  M-x load-library RET FOO.elc RET

IOW, when the user loads the file/library interactively, and
explicitly uses the .elc extension, we load the file the user
specified, not the corresponding .eln file.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

[-- Attachment #1: Type: text/plain, Size: 373 bytes --]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 08:18:43 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> > They are large functions.  I will post the disassembly if someone
> > wants to look at it.
> 
> I'm volunteering :-)

Thanks.  I attach disassembly of 2 functions somehow involved in this,
with the corresponding broken backtraces.


[-- Attachment #2: gdb.txt.xz --]
[-- Type: application/octet-stream, Size: 9432 bytes --]

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sat, Mar 13, 2021 at 8:47 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Sat, 13 Mar 2021 08:18:43 +0000
> > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> >
> > > They are large functions.  I will post the disassembly if someone
> > > wants to look at it.
> >
> > I'm volunteering :-)
>
> Thanks.  I attach disassembly of 2 functions somehow involved in this,
> with the corresponding broken backtraces.

It's c-beginning-of-statement-1 that I think is the immediate caller.
Can you disassemble the function around 0x09c32285 (or, in another
dump, whatever calls Flss), particularly the 256 bytes or so before
that EIP?

Thanks!
Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 08:53:04 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> It's c-beginning-of-statement-1 that I think is the immediate caller.

It's nowhere in the C backtrace, only its caller
c-font-lock-cut-off-declarators is.  And I'm not sure we can trust the
C backtrace anyway, given its abnormalities that I cannot explain yet.
In particular, if I put a breakpoint in wrong_type_argument, which
seems to appear in the backtrace, that breakpoint never breaks during
the recipe.

> Can you disassemble the function around 0x09c32285 (or, in another
> dump, whatever calls Flss), particularly the 256 bytes or so before
> that EIP?

Below.  The updated address for the caller of Flss is 0x09d82285,
which is not in any function whose name is known to GDB.

(gdb) bt
#0  0x01236964 in arithcompare_driver (nargs=2, args=0x28, 
    comparison=ARITH_LESS) at data.c:2673
#1  0x01236a3c in Flss (nargs=2, args=0x28) at data.c:2691
#2  0x09d82285 in ?? ()
#3  0x01261a74 in funcall_lambda (fun=XIL(0xa000000000fcb7f8), nargs=5, 
    arg_vector=0x826a08) at eval.c:3292
#4  0x012603c9 in Ffuncall (nargs=6, args=0x826a00) at eval.c:3013
#5  0x09df0dbf in ?? ()
#6  0x012603c9 in Ffuncall (nargs=1, args=0x826bd8) at eval.c:3013
#7  0x09dde041 in ?? ()
#8  0x01261a74 in funcall_lambda (fun=XIL(0xa000000006a5d2c8), nargs=1, 
    arg_vector=0x826db8) at eval.c:3292
#9  0x012603c9 in Ffuncall (nargs=2, args=0x826db0) at eval.c:3013
#10 0x70895b36 in F632d666f6e742d6c6f636b2d6375742d6f66662d6465636c617261746f7273_c_font_lock_cut_off_declarators_0 ()
   from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-fonts-d7d8a7f5-b7c359cd.eln
#11 0x01261a74 in funcall_lambda (fun=XIL(0xa0000000077a8918), nargs=1, 
    arg_vector=0x827050) at eval.c:3292
#12 0x012603c9 in Ffuncall (nargs=2, args=0x827048) at eval.c:3013
#13 0x068daf93 in ?? ()
#14 0x012dea14 in helper_save_restriction () at comp.c:4575
#15 0x0122eb86 in wrong_type_argument (predicate=XIL(0x892404890c245c89), 
    value=XIL(0x8244c89e45d8be0)) at data.c:143
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Lisp Backtrace:
"c-beginning-of-statement-1" (0x826a08)
"c-just-after-func-arglist-p" (0x826be0)
"c-back-over-member-initializers" (0x826db8)
"c-font-lock-cut-off-declarators" (0x827050)
"font-lock-fontify-keywords-region" (0x8273a8)
"font-lock-default-fontify-region" (0x8276b8)
"c-font-lock-fontify-region" (0x827868)
"font-lock-fontify-region" (0x827a58)
0x6ac13c8 PVEC_COMPILED
"jit-lock--run-functions" (0x8283f0)
"jit-lock-fontify-now" (0x8286b0)
"jit-lock-function" (0x8288d8)
"redisplay_internal (C function)" (0x0)
"sit-for" (0x82e040)
"isearch-lazy-highlight-new-loop" (0x82e320)
"isearch-update" (0x82e620)
"isearch-search-and-update" (0x82e830)
"isearch-process-search-string" (0x82ea58)
"isearch-process-search-char" (0x82ec90)
"isearch-printing-char" (0x82f030)
"funcall-interactively" (0x82f028)
"command-execute" (0x82f598)
(gdb) disassemble 0x9d82100,+0x200
Dump of assembler code from 0x9d82100 to 0x9d82300:
   0x09d82100:	mov    %edx,-0xe4(%ebp)
   0x09d82106:	mov    0x9e87564,%edx
   0x09d8210c:	mov    %eax,-0xe0(%ebp)
   0x09d82112:	mov    -0x100(%ebp),%eax
   0x09d82118:	mov    %edx,-0xdc(%ebp)
   0x09d8211e:	mov    %eax,0x4(%esp)
   0x09d82122:	call   *0xd84(%ebx)
   0x09d82128:	mov    %eax,-0xf0(%ebp)
   0x09d8212e:	mov    %edx,-0xec(%ebp)
   0x09d82134:	mov    %eax,(%esp)
   0x09d82137:	mov    %edx,0x4(%esp)
   0x09d8213b:	call   *0xad8(%ebx)
   0x09d82141:	mov    %eax,-0xf0(%ebp)
   0x09d82147:	mov    %edx,-0xec(%ebp)
   0x09d8214d:	call   *0xab0(%ebx)
   0x09d82153:	jmp    0x9d80934
   0x09d82158:	lea    0x0(%esi,%eiz,1),%esi
   0x09d8215f:	nop
   0x09d82160:	mov    0x9e87b50,%eax
   0x09d82165:	mov    0x9e87b54,%edx
   0x09d8216b:	mov    %eax,-0xf0(%ebp)
   0x09d82171:	mov    0x9e87aa0,%eax
   0x09d82176:	mov    %edx,-0xec(%ebp)
   0x09d8217c:	mov    0x9e87aa4,%edx
   0x09d82182:	mov    %eax,(%esp)
   0x09d82185:	mov    %edx,0x4(%esp)
   0x09d82189:	call   *0x1358(%ebx)
   0x09d8218f:	movl   $0x4,(%esp)
   0x09d82196:	mov    %eax,-0xe8(%ebp)
   0x09d8219c:	mov    0x9e875f0,%eax
   0x09d821a1:	mov    %edx,-0xe4(%ebp)
   0x09d821a7:	mov    0x9e875f4,%edx
   0x09d821ad:	mov    %eax,-0xe0(%ebp)
   0x09d821b3:	mov    %eax,-0xd8(%ebp)
   0x09d821b9:	mov    -0x100(%ebp),%eax
   0x09d821bf:	mov    %edx,-0xdc(%ebp)
   0x09d821c5:	mov    %edx,-0xd4(%ebp)
   0x09d821cb:	mov    %eax,0x4(%esp)
   0x09d821cf:	call   *0xd84(%ebx)
   0x09d821d5:	mov    0x9e8a750,%ecx
   0x09d821db:	mov    %eax,-0xf0(%ebp)
   0x09d821e1:	mov    (%ecx),%ecx
   0x09d821e3:	mov    %edx,-0xec(%ebp)
   0x09d821e9:	mov    0x54(%ecx),%esi
   0x09d821ec:	mov    0x20(%esi),%esi
   0x09d821ef:	mov    %esi,0x54(%ecx)
   0x09d821f2:	jmp    0x9d808b2
   0x09d821f7:	lea    0x0(%esi,%eiz,1),%esi
   0x09d821fe:	xchg   %ax,%ax
   0x09d82200:	mov    0x9e87950,%eax
   0x09d82205:	mov    0x9e87954,%edx
   0x09d8220b:	mov    %eax,(%esp)
   0x09d8220e:	mov    %edx,0x4(%esp)
   0x09d82212:	call   *0x1358(%ebx)
   0x09d82218:	mov    %edx,-0xec(%ebp)
   0x09d8221e:	or     %eax,%edx
   0x09d82220:	mov    %eax,-0xf0(%ebp)
   0x09d82226:	je     0x9d82308
   0x09d8222c:	mov    0x9e87690,%eax
   0x09d82231:	mov    0x9e87694,%edx
   0x09d82237:	mov    %eax,(%esp)
   0x09d8223a:	mov    %edx,0x4(%esp)
   0x09d8223e:	call   *0x1358(%ebx)
   0x09d82244:	mov    %eax,-0xf0(%ebp)
   0x09d8224a:	mov    0x9e875a8,%eax
   0x09d8224f:	mov    %edx,-0xec(%ebp)
   0x09d82255:	mov    0x9e875ac,%edx
   0x09d8225b:	mov    %eax,(%esp)
   0x09d8225e:	mov    %edx,0x4(%esp)
   0x09d82262:	call   *0x1358(%ebx)
   0x09d82268:	mov    %edi,0x4(%esp)
   0x09d8226c:	mov    %eax,-0xe8(%ebp)
   0x09d82272:	mov    %edx,-0xe4(%ebp)
   0x09d82278:	movl   $0x2,(%esp)
   0x09d8227f:	call   *0x1318(%ebx)
   0x09d82285:	mov    %edx,-0xec(%ebp)  <<<<<<<<<<<<<<<<<<<<
   0x09d8228b:	or     %eax,%edx
   0x09d8228d:	mov    %eax,-0xf0(%ebp)
   0x09d82293:	jne    0x9d82308
   0x09d82295:	mov    0x9e87690,%eax
   0x09d8229a:	mov    0x9e87694,%edx
   0x09d822a0:	mov    %eax,(%esp)
   0x09d822a3:	mov    %edx,0x4(%esp)
   0x09d822a7:	call   *0x1358(%ebx)
   0x09d822ad:	mov    %eax,-0xf0(%ebp)
   0x09d822b3:	mov    0x9e87988,%eax
   0x09d822b8:	mov    %edx,-0xec(%ebp)
   0x09d822be:	mov    0x9e8798c,%edx
   0x09d822c4:	mov    %eax,(%esp)
   0x09d822c7:	mov    %edx,0x4(%esp)
   0x09d822cb:	call   *0x1358(%ebx)
   0x09d822d1:	mov    %edi,0x4(%esp)
   0x09d822d5:	mov    %eax,-0xe8(%ebp)
   0x09d822db:	mov    %edx,-0xe4(%ebp)
   0x09d822e1:	movl   $0x2,(%esp)
   0x09d822e8:	call   *0x130c(%ebx)
   0x09d822ee:	mov    %edx,-0xec(%ebp)
   0x09d822f4:	or     %eax,%edx
   0x09d822f6:	mov    %eax,-0xf0(%ebp)
   0x09d822fc:	je     0x9d7ee24
End of assembler dump.
(gdb) fr 2
#2  0x09d82285 in ?? ()
(gdb) info registers
eax            0x30                48
ecx            0x6a54fe8           111497192
edx            0x8                 8
ebx            0x187b8c0           25671872
esp            0x826650            0x826650
ebp            0x826778            0x826778
esi            0x4002d2b0          1073926832
edi            0x28                40
eip            0x9d82285           0x9d82285
eflags         0x10216             [ PF AF IF RF ]
cs             0x1b                27
ss             0x23                35
ds             0x23                35
es             0x23                35
fs             0x3b                59
gs             0x0                 0
(gdb) info symbol (0x187b8c0+0x1318)
freloc + 4888 in section .bss of D:\gnu\git\emacs\native-comp\src\emacs.exe

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> Date: Sat, 13 Mar 2021 13:10:29 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: akrl@sdf.org, 47067@debbugs.gnu.org
> 
> (gdb) info symbol (0x187b8c0+0x1318)
> freloc + 4888 in section .bss of D:\gnu\git\emacs\native-comp\src\emacs.exe

One more relevant piece of info:

  (gdb) p freloc.link_table[4888/4]
  $1 = (void *) 0x1236a1c <Flss>

Where is the source code that calls through freloc.link_table?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sat, Mar 13, 2021 at 11:10 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Sat, 13 Mar 2021 08:53:04 +0000
> > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> >
> > It's c-beginning-of-statement-1 that I think is the immediate caller.
>
> It's nowhere in the C backtrace, only its caller

But it was in one of the previous backtraces?

> c-font-lock-cut-off-declarators is.  And I'm not sure we can trust the
> C backtrace anyway, given its abnormalities that I cannot explain yet.

Good point.

> > Can you disassemble the function around 0x09c32285 (or, in another
> > dump, whatever calls Flss), particularly the 256 bytes or so before
> > that EIP?
>
> Below.  The updated address for the caller of Flss is 0x09d82285,
> which is not in any function whose name is known to GDB.

That's normal, assuming you didn't compile with comp-debug > 0.

> (gdb) disassemble 0x9d82100,+0x200
> Dump of assembler code from 0x9d82100 to 0x9d82300:
>    0x09d82100:  mov    %edx,-0xe4(%ebp)
>    0x09d82106:  mov    0x9e87564,%edx
>    0x09d8210c:  mov    %eax,-0xe0(%ebp)
>    0x09d82112:  mov    -0x100(%ebp),%eax
>    0x09d82118:  mov    %edx,-0xdc(%ebp)
>    0x09d8211e:  mov    %eax,0x4(%esp)
>    0x09d82122:  call   *0xd84(%ebx)
>    0x09d82128:  mov    %eax,-0xf0(%ebp)
>    0x09d8212e:  mov    %edx,-0xec(%ebp)
>    0x09d82134:  mov    %eax,(%esp)
>    0x09d82137:  mov    %edx,0x4(%esp)
>    0x09d8213b:  call   *0xad8(%ebx)
>    0x09d82141:  mov    %eax,-0xf0(%ebp)
>    0x09d82147:  mov    %edx,-0xec(%ebp)
>    0x09d8214d:  call   *0xab0(%ebx)
>    0x09d82153:  jmp    0x9d80934
>    0x09d82158:  lea    0x0(%esi,%eiz,1),%esi
>    0x09d8215f:  nop
>    0x09d82160:  mov    0x9e87b50,%eax
>    0x09d82165:  mov    0x9e87b54,%edx
>    0x09d8216b:  mov    %eax,-0xf0(%ebp)
>    0x09d82171:  mov    0x9e87aa0,%eax
>    0x09d82176:  mov    %edx,-0xec(%ebp)
>    0x09d8217c:  mov    0x9e87aa4,%edx
>    0x09d82182:  mov    %eax,(%esp)
>    0x09d82185:  mov    %edx,0x4(%esp)
>    0x09d82189:  call   *0x1358(%ebx)
>    0x09d8218f:  movl   $0x4,(%esp)
>    0x09d82196:  mov    %eax,-0xe8(%ebp)
>    0x09d8219c:  mov    0x9e875f0,%eax
>    0x09d821a1:  mov    %edx,-0xe4(%ebp)
>    0x09d821a7:  mov    0x9e875f4,%edx
>    0x09d821ad:  mov    %eax,-0xe0(%ebp)
>    0x09d821b3:  mov    %eax,-0xd8(%ebp)
>    0x09d821b9:  mov    -0x100(%ebp),%eax
>    0x09d821bf:  mov    %edx,-0xdc(%ebp)
>    0x09d821c5:  mov    %edx,-0xd4(%ebp)
>    0x09d821cb:  mov    %eax,0x4(%esp)
>    0x09d821cf:  call   *0xd84(%ebx)
>    0x09d821d5:  mov    0x9e8a750,%ecx
>    0x09d821db:  mov    %eax,-0xf0(%ebp)
>    0x09d821e1:  mov    (%ecx),%ecx
>    0x09d821e3:  mov    %edx,-0xec(%ebp)
>    0x09d821e9:  mov    0x54(%ecx),%esi
>    0x09d821ec:  mov    0x20(%esi),%esi
>    0x09d821ef:  mov    %esi,0x54(%ecx)
>    0x09d821f2:  jmp    0x9d808b2
>    0x09d821f7:  lea    0x0(%esi,%eiz,1),%esi
>    0x09d821fe:  xchg   %ax,%ax
>    0x09d82200:  mov    0x9e87950,%eax
>    0x09d82205:  mov    0x9e87954,%edx
>    0x09d8220b:  mov    %eax,(%esp)
>    0x09d8220e:  mov    %edx,0x4(%esp)
>    0x09d82212:  call   *0x1358(%ebx)
>    0x09d82218:  mov    %edx,-0xec(%ebp)
>    0x09d8221e:  or     %eax,%edx
>    0x09d82220:  mov    %eax,-0xf0(%ebp)
>    0x09d82226:  je     0x9d82308
>    0x09d8222c:  mov    0x9e87690,%eax
>    0x09d82231:  mov    0x9e87694,%edx
>    0x09d82237:  mov    %eax,(%esp)
>    0x09d8223a:  mov    %edx,0x4(%esp)
>    0x09d8223e:  call   *0x1358(%ebx)
>    0x09d82244:  mov    %eax,-0xf0(%ebp)
>    0x09d8224a:  mov    0x9e875a8,%eax
>    0x09d8224f:  mov    %edx,-0xec(%ebp)
>    0x09d82255:  mov    0x9e875ac,%edx
>    0x09d8225b:  mov    %eax,(%esp)
>    0x09d8225e:  mov    %edx,0x4(%esp)
>    0x09d82262:  call   *0x1358(%ebx)
>    0x09d82268:  mov    %edi,0x4(%esp)
>    0x09d8226c:  mov    %eax,-0xe8(%ebp)
>    0x09d82272:  mov    %edx,-0xe4(%ebp)
>    0x09d82278:  movl   $0x2,(%esp)
>    0x09d8227f:  call   *0x1318(%ebx)
>    0x09d82285:  mov    %edx,-0xec(%ebp)  <<<<<<<<<<<<<<<<<<<<

So EDI is bunk at this point. Can you go back a bit further to where
it's initialized?

> (gdb) info registers
> eax            0x30                48
> ecx            0x6a54fe8           111497192
> edx            0x8                 8
> ebx            0x187b8c0           25671872
> esp            0x826650            0x826650
> ebp            0x826778            0x826778
> esi            0x4002d2b0          1073926832
> edi            0x28                40

That value should be between 0x826650 and 0x826778.

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 11:24:28 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> On Sat, Mar 13, 2021 at 11:10 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > > From: Pip Cet <pipcet@gmail.com>
> > > Date: Sat, 13 Mar 2021 08:53:04 +0000
> > > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> > >
> > > It's c-beginning-of-statement-1 that I think is the immediate caller.
> >
> > It's nowhere in the C backtrace, only its caller
> 
> But it was in one of the previous backtraces?

Too many moons ago.  The ABI was bumped since then, and so did the
*.eln files.

> > > Can you disassemble the function around 0x09c32285 (or, in another
> > > dump, whatever calls Flss), particularly the 256 bytes or so before
> > > that EIP?
> >
> > Below.  The updated address for the caller of Flss is 0x09d82285,
> > which is not in any function whose name is known to GDB.
> 
> That's normal, assuming you didn't compile with comp-debug > 0.

If that's normal, why did Andrea say that he didn't see such "??" in
his backtraces?  And if that's because Andrea builds with comp-debug
above the default zero, then I think either the doc string or the set
of values (or both) need to be updated:

  (defcustom comp-debug 0
    "Debug level for native compilation, a number between 0 and 3.
  This is intended for debugging the compiler itself.
    0 no debugging output.
      This is the recommended value unless you are debugging the compiler itself.
    1 emit debug symbols and dump pseudo C code.
    2 dump gcc passes and libgccjit log file.
    3 dump libgccjit reproducers."

The first line is somewhat deceptive, as it doesn't say portions of
the backtrace will be anonymous; the backtrace is useful even if not
"debugging the compiler itself".  And the description of level 1 says
it will also "dump pseudo C code", which I don't think is needed for
most of the debugging.  Maybe we need an intermediate level that just
leaves the symbols without dumping the pseudo code?

> >    0x09d82268:  mov    %edi,0x4(%esp)
> >    0x09d8226c:  mov    %eax,-0xe8(%ebp)
> >    0x09d82272:  mov    %edx,-0xe4(%ebp)
> >    0x09d82278:  movl   $0x2,(%esp)
> >    0x09d8227f:  call   *0x1318(%ebx)
> >    0x09d82285:  mov    %edx,-0xec(%ebp)  <<<<<<<<<<<<<<<<<<<<
> 
> So EDI is bunk at this point. Can you go back a bit further to where
> it's initialized?

Sorry, I don't understand: I gave you the disassembly of 512 bytes
before, isn't that enough to see where EDI is assigned the value?  Or
what do you mean by "go back"?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sat, Mar 13, 2021 at 3:27 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > > It's nowhere in the C backtrace, only its caller
> >
> > But it was in one of the previous backtraces?
>
> Too many moons ago.  The ABI was bumped since then, and so did the
> *.eln files.

The code you pasted matches c-beginning-of-statement-1, and so does
the Lisp backtrace, so I would suggest we go with it...

> > > > Can you disassemble the function around 0x09c32285 (or, in another
> > > > dump, whatever calls Flss), particularly the 256 bytes or so before
> > > > that EIP?
> > >
> > > Below.  The updated address for the caller of Flss is 0x09d82285,
> > > which is not in any function whose name is known to GDB.
> >
> > That's normal, assuming you didn't compile with comp-debug > 0.
>
> If that's normal, why did Andrea say that he didn't see such "??" in
> his backtraces?  And if that's because Andrea builds with comp-debug
> above the default zero, then I think either the doc string or the set
> of values (or both) need to be updated:
>
>   (defcustom comp-debug 0
>     "Debug level for native compilation, a number between 0 and 3.
>   This is intended for debugging the compiler itself.
>     0 no debugging output.
>       This is the recommended value unless you are debugging the compiler itself.
>     1 emit debug symbols and dump pseudo C code.
>     2 dump gcc passes and libgccjit log file.
>     3 dump libgccjit reproducers."
>
> The first line is somewhat deceptive, as it doesn't say portions of
> the backtrace will be anonymous; the backtrace is useful even if not
> "debugging the compiler itself".  And the description of level 1 says
> it will also "dump pseudo C code", which I don't think is needed for
> most of the debugging.  Maybe we need an intermediate level that just
> leaves the symbols without dumping the pseudo code?

I think that's an excellent idea.

> > >    0x09d82268:  mov    %edi,0x4(%esp)
> > >    0x09d8226c:  mov    %eax,-0xe8(%ebp)
> > >    0x09d82272:  mov    %edx,-0xe4(%ebp)
> > >    0x09d82278:  movl   $0x2,(%esp)
> > >    0x09d8227f:  call   *0x1318(%ebx)
> > >    0x09d82285:  mov    %edx,-0xec(%ebp)  <<<<<<<<<<<<<<<<<<<<
> >
> > So EDI is bunk at this point. Can you go back a bit further to where
> > it's initialized?
>
> Sorry, I don't understand: I gave you the disassembly of 512 bytes
> before, isn't that enough to see where EDI is assigned the value?  Or
> what do you mean by "go back"?

It's not enough, no. we're looking for an insn of the form mov XXX,
%edi or lea XXX, %edi, or anything like that.

I'm suspicious because EDI is a register variable that is clobbered
somehow right after a setjmp returned. Which setjmp implementation are
you using? Is it possible that you're on Windows, but unlike other
Windows setjmps, it's unsafe to call your setjmp through a function
pointer?

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 15:45:52 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> On Sat, Mar 13, 2021 at 3:27 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > > > It's nowhere in the C backtrace, only its caller
> > >
> > > But it was in one of the previous backtraces?
> >
> > Too many moons ago.  The ABI was bumped since then, and so did the
> > *.eln files.
> 
> The code you pasted matches c-beginning-of-statement-1, and so does
> the Lisp backtrace, so I would suggest we go with it...

I don't know how to go with it: the backtrace is truncated (for
reasons I don't yet understand) before it gets to it.

> > > So EDI is bunk at this point. Can you go back a bit further to where
> > > it's initialized?
> >
> > Sorry, I don't understand: I gave you the disassembly of 512 bytes
> > before, isn't that enough to see where EDI is assigned the value?  Or
> > what do you mean by "go back"?
> 
> It's not enough, no. we're looking for an insn of the form mov XXX,
> %edi or lea XXX, %edi, or anything like that.

I went back 4KB, and the only two instructions that write into EDI are
the following:

   0x09e3159d:	mov    -0x100(%ebp),%edi
   0x09e31c71:	mov    0x9f37b9c,%edi

> I'm suspicious because EDI is a register variable that is clobbered
> somehow right after a setjmp returned. Which setjmp implementation are
> you using?

Not sure how to answer that.  AFAIK, it's a setjmp from the MS runtime.

> Is it possible that you're on Windows, but unlike other Windows
> setjmps, it's unsafe to call your setjmp through a function pointer?

How do I tell?

And why I never had any problems with setjmp elsewhere in Emacs,
although we use it all the time in keyboard.c and elsewhere?

Here's an interesting factoid: while most addresses in the backtraces
I see with this recipe are identical from run to run, the 'fun'
arguments of funcall_lambda's aren't.  Compare the backtrace I sent 3
messages ago with this one:

  #0  0x01236964 in arithcompare_driver (nargs=2, args=0x28,
      comparison=ARITH_LESS) at data.c:2673
  #1  0x01236a3c in Flss (nargs=2, args=0x28) at data.c:2691
  #2  0x09e32285 in ?? ()
  #3  0x01261a74 in funcall_lambda (fun=XIL(0xa000000007650188), nargs=5,
      arg_vector=0x826a08) at eval.c:3292
  #4  0x012603c9 in Ffuncall (nargs=6, args=0x826a00) at eval.c:3013
  #5  0x09ea0dbf in ?? ()
  #6  0x012603c9 in Ffuncall (nargs=1, args=0x826bd8) at eval.c:3013
  #7  0x09e8e041 in ?? ()
  #8  0x01261a74 in funcall_lambda (fun=XIL(0xa00000000778d5b8), nargs=1,
      arg_vector=0x826db8) at eval.c:3292
  #9  0x012603c9 in Ffuncall (nargs=2, args=0x826db0) at eval.c:3013
  #10 0x70895b36 in F632d666f6e742d6c6f636b2d6375742d6f66662d6465636c617261746f7273_c_font_lock_cut_off_declarators_0 ()
     from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-fonts-d7d8a7f5-b7c359cd.eln
  #11 0x01261a74 in funcall_lambda (fun=XIL(0xa000000007785f78), nargs=1,
      arg_vector=0x827050) at eval.c:3292
  #12 0x012603c9 in Ffuncall (nargs=2, args=0x827048) at eval.c:3013
  #13 0x068daf93 in ?? ()
  #14 0x012dea14 in helper_save_restriction () at comp.c:4575
  #15 0x0122eb86 in wrong_type_argument (predicate=XIL(0x892404890c245c89),
      value=XIL(0x8244c89e45d8be0)) at data.c:143
  Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Note how arguments to Funcall's are the same, whereas arguments to
funcall_lambda's aren't.  Even the garbage in the 2 arguments to
wrong_type_argument are identical.

Sounds like something is uninitialized somewhere?  Hmm...

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

[-- Attachment #1: Type: text/plain, Size: 4656 bytes --]

On Sat, Mar 13, 2021 at 4:21 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Sat, 13 Mar 2021 15:45:52 +0000
> > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> >
> > On Sat, Mar 13, 2021 at 3:27 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > > > > It's nowhere in the C backtrace, only its caller
> > > >
> > > > But it was in one of the previous backtraces?
> > >
> > > Too many moons ago.  The ABI was bumped since then, and so did the
> > > *.eln files.
> >
> > The code you pasted matches c-beginning-of-statement-1, and so does
> > the Lisp backtrace, so I would suggest we go with it...
>
> I don't know how to go with it: the backtrace is truncated (for
> reasons I don't yet understand) before it gets to it.

I'm pretty sure the first ?? is c-beginning-of-stmt-1. It's a function
with five arguments, at least.

> > > > So EDI is bunk at this point. Can you go back a bit further to where
> > > > it's initialized?
> > >
> > > Sorry, I don't understand: I gave you the disassembly of 512 bytes
> > > before, isn't that enough to see where EDI is assigned the value?  Or
> > > what do you mean by "go back"?
> >
> > It's not enough, no. we're looking for an insn of the form mov XXX,
> > %edi or lea XXX, %edi, or anything like that.
>
> I went back 4KB, and the only two instructions that write into EDI are

It's a long function, that might not have been enough.

> the following:
>
>    0x09e3159d:  mov    -0x100(%ebp),%edi
>    0x09e31c71:  mov    0x9f37b9c,%edi
>
> > I'm suspicious because EDI is a register variable that is clobbered
> > somehow right after a setjmp returned. Which setjmp implementation are
> > you using?
>
> Not sure how to answer that.  AFAIK, it's a setjmp from the MS runtime.

So not some mingw wrapper for it? I just checked the only "mingw"-like
sources I could find, and they don't appear to use the frame pointer
argument properly...

> > Is it possible that you're on Windows, but unlike other Windows
> > setjmps, it's unsafe to call your setjmp through a function pointer?
>
> How do I tell?

Well, you could just apply this untested patch, fix any obvious
compile errors I might not have spotted, and try to reproduce it. I'm
not currently on a Windows (or x86) machine, so it's a bit hard for me
to test...

> And why I never had any problems with setjmp elsewhere in Emacs,
> although we use it all the time in keyboard.c and elsewhere?

It's only natively-compiled code that attempts to call setjmp through
a function pointer. This was fixed in the POSIX case, but I didn't
touch the Windows code because I assumed that they used the extra
argument in their non-standard API to do this right...

> Here's an interesting factoid: while most addresses in the backtraces
> I see with this recipe are identical from run to run, the 'fun'
> arguments of funcall_lambda's aren't.  Compare the backtrace I sent 3
> messages ago with this one:
>
>   #0  0x01236964 in arithcompare_driver (nargs=2, args=0x28,
>       comparison=ARITH_LESS) at data.c:2673
>   #1  0x01236a3c in Flss (nargs=2, args=0x28) at data.c:2691
>   #2  0x09e32285 in ?? ()
>   #3  0x01261a74 in funcall_lambda (fun=XIL(0xa000000007650188), nargs=5,
>       arg_vector=0x826a08) at eval.c:3292
>   #4  0x012603c9 in Ffuncall (nargs=6, args=0x826a00) at eval.c:3013
>   #5  0x09ea0dbf in ?? ()
>   #6  0x012603c9 in Ffuncall (nargs=1, args=0x826bd8) at eval.c:3013
>   #7  0x09e8e041 in ?? ()
>   #8  0x01261a74 in funcall_lambda (fun=XIL(0xa00000000778d5b8), nargs=1,
>       arg_vector=0x826db8) at eval.c:3292
>   #9  0x012603c9 in Ffuncall (nargs=2, args=0x826db0) at eval.c:3013
>   #10 0x70895b36 in F632d666f6e742d6c6f636b2d6375742d6f66662d6465636c617261746f7273_c_font_lock_cut_off_declarators_0 ()
>      from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-fonts-d7d8a7f5-b7c359cd.eln
>   #11 0x01261a74 in funcall_lambda (fun=XIL(0xa000000007785f78), nargs=1,
>       arg_vector=0x827050) at eval.c:3292
>   #12 0x012603c9 in Ffuncall (nargs=2, args=0x827048) at eval.c:3013
>   #13 0x068daf93 in ?? ()
>   #14 0x012dea14 in helper_save_restriction () at comp.c:4575
>   #15 0x0122eb86 in wrong_type_argument (predicate=XIL(0x892404890c245c89),
>       value=XIL(0x8244c89e45d8be0)) at data.c:143
>   Backtrace stopped: previous frame inner to this frame (corrupt stack?)
>
> Note how arguments to Funcall's are the same, whereas arguments to
> funcall_lambda's aren't.  Even the garbage in the 2 arguments to
> wrong_type_argument are identical.

Which non-stack addresses are invariant in that backtrace?

> Sounds like something is uninitialized somewhere?  Hmm...

Maybe...

Pip

[-- Attachment #2: bug46824.diff --]
[-- Type: text/x-diff, Size: 1056 bytes --]

diff --git a/src/comp.c b/src/comp.c
index b24fcb5bf5a99..ce3ad22b30576 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -1946,6 +1946,11 @@ emit_setjmp (gcc_jit_rvalue *buf)
   return gcc_jit_context_new_call (comp.ctxt, NULL, f, 1, args);
 #else
   /* _setjmp (buf, __builtin_frame_address (0)) */
+  gcc_jit_param *params[] =
+  {
+    gcc_jit_context_new_param (comp.ctxt, NULL, comp.void_ptr_type, "buf"),
+    gcc_jit_context_new_param (comp.ctxt, NULL, comp.unsigned_type, "ret"),
+  };
   gcc_jit_rvalue *args[2];
 
   args[0] =
@@ -1959,8 +1964,14 @@ emit_setjmp (gcc_jit_rvalue *buf)
 					    "__builtin_frame_address"),
       1, args);
   args[0] = buf;
-  return emit_call (intern_c_string (STR (SETJMP_NAME)), comp.int_type, 2, args,
-                    false);
+  gcc_jit_function *f =
+    gcc_jit_context_new_function (comp.ctxt, NULL,
+				  GCC_JIT_FUNCTION_IMPORTED,
+				  comp.int_type, STR (SETJMP_NAME),
+				  ARRAYELTS (params), params,
+				  false);
+
+  return gcc_jit_context_new_call (comp.ctxt, NULL, f, 2, args);
 #endif
 }
 

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 16:32:50 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> > > > > So EDI is bunk at this point. Can you go back a bit further to where
> > > > > it's initialized?
> > > >
> > > > Sorry, I don't understand: I gave you the disassembly of 512 bytes
> > > > before, isn't that enough to see where EDI is assigned the value?  Or
> > > > what do you mean by "go back"?
> > >
> > > It's not enough, no. we're looking for an insn of the form mov XXX,
> > > %edi or lea XXX, %edi, or anything like that.
> >
> > I went back 4KB, and the only two instructions that write into EDI are
> 
> It's a long function, that might not have been enough.

But since I found those two, everything before that is irrelevant,
right?

> > > I'm suspicious because EDI is a register variable that is clobbered
> > > somehow right after a setjmp returned. Which setjmp implementation are
> > > you using?
> >
> > Not sure how to answer that.  AFAIK, it's a setjmp from the MS runtime.
> 
> So not some mingw wrapper for it?

No, not that I could see.

> I just checked the only "mingw"-like sources I could find, and they
> don't appear to use the frame pointer argument properly...

Why is this suddenly relevant when native compilation is involved?

> > > Is it possible that you're on Windows, but unlike other Windows
> > > setjmps, it's unsafe to call your setjmp through a function pointer?
> >
> > How do I tell?
> 
> Well, you could just apply this untested patch, fix any obvious
> compile errors I might not have spotted, and try to reproduce it. I'm
> not currently on a Windows (or x86) machine, so it's a bit hard for me
> to test...

I'd like this investigation to be less of a blind search, sorry.  can
you tell what to check or look at to see if this is relevant?

And how is setjmp related to the code which causes segfault?  I see no
call to setjmp in the disassembly.

> > Note how arguments to Funcall's are the same, whereas arguments to
> > funcall_lambda's aren't.  Even the garbage in the 2 arguments to
> > wrong_type_argument are identical.
> 
> Which non-stack addresses are invariant in that backtrace?

Not sure how stack-based vs non-stack based is important here.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 16:32:50 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> > And why I never had any problems with setjmp elsewhere in Emacs,
> > although we use it all the time in keyboard.c and elsewhere?
> 
> It's only natively-compiled code that attempts to call setjmp through
> a function pointer. This was fixed in the POSIX case, but I didn't
> touch the Windows code because I assumed that they used the extra
> argument in their non-standard API to do this right...

And if you remove that fix, you get the same or similar crashes in the
recipe I provided?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sat, Mar 13, 2021 at 4:53 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Sat, 13 Mar 2021 16:32:50 +0000
> > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> >
> > > > > > So EDI is bunk at this point. Can you go back a bit further to where
> > > > > > it's initialized?
> > > > >
> > > > > Sorry, I don't understand: I gave you the disassembly of 512 bytes
> > > > > before, isn't that enough to see where EDI is assigned the value?  Or
> > > > > what do you mean by "go back"?
> > > >
> > > > It's not enough, no. we're looking for an insn of the form mov XXX,
> > > > %edi or lea XXX, %edi, or anything like that.
> > >
> > > I went back 4KB, and the only two instructions that write into EDI are
> >
> > It's a long function, that might not have been enough.
>
> But since I found those two, everything before that is irrelevant,
> right?

Assuming all code paths hit these insns, yes.

> > I just checked the only "mingw"-like sources I could find, and they
> > don't appear to use the frame pointer argument properly...
>
> Why is this suddenly relevant when native compilation is involved?

Native-compiled code assumes, on Windows, that you can call setjmp
through a function pointer. If the implementation of setjmp is such
that you can't, we shouldn't do that. Portable code will never call
setjmp through a function pointer.

Since mingw (at least the version I could find) declares setjmp with
the "returns_twice" attribute, I'm assuming their implementation is
not such that you can call it through a function pointer.

> > > > Is it possible that you're on Windows, but unlike other Windows
> > > > setjmps, it's unsafe to call your setjmp through a function pointer?
> > >
> > > How do I tell?
> >
> > Well, you could just apply this untested patch, fix any obvious
> > compile errors I might not have spotted, and try to reproduce it. I'm
> > not currently on a Windows (or x86) machine, so it's a bit hard for me
> > to test...
>
> I'd like this investigation to be less of a blind search, sorry.  can
> you tell what to check or look at to see if this is relevant?
>
> And how is setjmp related to the code which causes segfault?  I see no
> call to setjmp in the disassembly.

That might be because it's called through a function pointer?

If you look at the Lisp code, the thing it does just before the code
dies is to exit from a long-ish (catch ...) form. Which involves
setjmp, unless the byte compiler is smart enough to optimize it away.

> > > Note how arguments to Funcall's are the same, whereas arguments to
> > > funcall_lambda's aren't.  Even the garbage in the 2 arguments to
> > > wrong_type_argument are identical.
> >
> > Which non-stack addresses are invariant in that backtrace?
>
> Not sure how stack-based vs non-stack based is important here.

If non-stack addresses vary between runs and stack addresses don't, I
don't see any evidence we're looking at corruption here.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 17:10:08 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> > > It's a long function, that might not have been enough.
> >
> > But since I found those two, everything before that is irrelevant,
> > right?
> 
> Assuming all code paths hit these insns, yes.

Then tell me how far back to go.

Or, better yet, perhaps there's a way of displaying that code in C?

> Since mingw (at least the version I could find) declares setjmp with
> the "returns_twice" attribute, I'm assuming their implementation is
> not such that you can call it through a function pointer.

I think you are looking at a 64-bit MinGW64, but I'm out of my depth
here anyway.  If you want to pursue this further, the implementation I
use is in MSVCRT.DLL.

> > > > Note how arguments to Funcall's are the same, whereas arguments to
> > > > funcall_lambda's aren't.  Even the garbage in the 2 arguments to
> > > > wrong_type_argument are identical.
> > >
> > > Which non-stack addresses are invariant in that backtrace?
> >
> > Not sure how stack-based vs non-stack based is important here.
> 
> If non-stack addresses vary between runs and stack addresses don't, I
> don't see any evidence we're looking at corruption here.

Why would non-stack base addresses change?  There's no ASLR here.

Anyway, if that doesn't help, just forget it.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sat, Mar 13, 2021 at 5:26 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Sat, 13 Mar 2021 17:10:08 +0000
> > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> >
> > > > It's a long function, that might not have been enough.
> > >
> > > But since I found those two, everything before that is irrelevant,
> > > right?
> >
> > Assuming all code paths hit these insns, yes.
>
> Then tell me how far back to go.
>
> Or, better yet, perhaps there's a way of displaying that code in C?

Sure! Put the defun in a file called breakage.el. Then open emacs -Q
in GDB and evaluate

(require 'comp)
(let ((comp-debug 3) (comp-verbose 3))
  (load (native-compile "breakage.el")))

Then set a GDB breakpoint on
F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0,
continue and evaluate

(c-beginning-of-stmt-1)

In GDB, you should now see the "source" code interspersed with the
disassembly (if you have that enabled).

I'd suggest to try disass/rs next and post the entire output, but you
know your way around gdb much better than I do :-)

The code you're looking for looks like this:

7961      /* calling subr: < */
7962      frame[(int)0] = freloc->R3c__0 ((long long)2, (&frame[(int)0]));
   0x00007fffec660f41 <+3409>:    4c 89 f6    mov    %r14,%rsi
   0x00007fffec660f44 <+3412>:    bf 02 00 00 00    mov    $0x2,%edi

(but different, of course, because of the different architecture).

> > Since mingw (at least the version I could find) declares setjmp with
> > the "returns_twice" attribute, I'm assuming their implementation is
> > not such that you can call it through a function pointer.
>
> I think you are looking at a 64-bit MinGW64, but I'm out of my depth
> here anyway.  If you want to pursue this further, the implementation I
> use is in MSVCRT.DLL.

Thanks!

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 18:55:37 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> Sure! Put the defun in a file called breakage.el. Then open emacs -Q
> in GDB and evaluate
> 
> (require 'comp)
> (let ((comp-debug 3) (comp-verbose 3))
>   (load (native-compile "breakage.el")))
> 
> Then set a GDB breakpoint on
> F632d626567696e6e696e672d6f662d73746174656d656e742d31_c_beginning_of_statement_1_0,
> continue and evaluate
> 
> (c-beginning-of-stmt-1)
> 
> In GDB, you should now see the "source" code interspersed with the
> disassembly (if you have that enabled).
> 
> I'd suggest to try disass/rs next and post the entire output, but you
> know your way around gdb much better than I do :-)
> 
> The code you're looking for looks like this:
> 
> 7961      /* calling subr: < */
> 7962      frame[(int)0] = freloc->R3c__0 ((long long)2, (&frame[(int)0]));
>    0x00007fffec660f41 <+3409>:    4c 89 f6    mov    %r14,%rsi
>    0x00007fffec660f44 <+3412>:    bf 02 00 00 00    mov    $0x2,%edi

Thanks, will try that some day.  This week's cup is full with
frustration already.

Btw, could there be a stack alignment issue?  When GDB cannot report a
full backtrace, I see:

  Thread 1 hit Breakpoint 4, Fskip_chars_backward (
      string=XIL(0x800000000696bf28), lim=XIL(0)) at syntax.c:1610
  1610      return skip_chars (0, string, lim, 1);
  (gdb) bt
  #0  Fskip_chars_backward (string=XIL(0x800000000696bf28), lim=XIL(0))
      at syntax.c:1610
  #1  0x09d8d5b2 in ?? ()
  #2  0x012608fa in funcall_subr (subr=0x765a848, numargs=0, args=0x828d20)
      at eval.c:3093
  #3  0x0126035f in Ffuncall (nargs=1, args=0x828d18) at eval.c:3016
  #4  0x7088e5ff in F632d666f6e742d6c6f636b2d636f6d706c65782d6465636c2d70726570617265_c_font_lock_complex_decl_prepare_0 ()
     from d:\usr\eli\.emacs.d\eln-cache\28.0.50-7d88f6c1\cc-fonts-d7d8a7f5-b7c359cd.eln
  #5  0x01261aa5 in funcall_lambda (fun=XIL(0xa000000007789a90), nargs=1,
      arg_vector=0x8290a0) at eval.c:3303
  #6  0x012603fa in Ffuncall (nargs=2, args=0x829098) at eval.c:3023
  #7  0x068daf93 in ?? ()
  #8  0x012dea48 in helper_save_restriction () at comp.c:4575
  #9  0x0122eb86 in wrong_type_argument (predicate=XIL(0x892404890c245c89),
      value=XIL(0x8244c89e45d8be0)) at data.c:143
  Backtrace stopped: previous frame inner to this frame (corrupt stack?)
  (gdb) fr 9
  #9  0x0122eb86 in wrong_type_argument (predicate=XIL(0x892404890c245c89),
      value=XIL(0x8244c89e45d8be0)) at data.c:143
  143       xsignal2 (Qwrong_type_argument, predicate, value);
  (gdb) p/x $ebp
  $7 = 0x122eaf2
  (gdb) down
  #8  0x012dea48 in helper_save_restriction () at comp.c:4575
  4575    }
  (gdb) p/x $ebp
  $8 = 0x122eaf2

The EBP values are not 8-byte aligned.  They aren't even 4-byte
aligned, which I don't understand how can happen on 32-bit x86
architecture.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

[...]

> Btw, is your build configured --enable-checking='yes,glyphs' ?  If
> not, could you please reconfigure the 32-bit build with wide ints, and
> see if you can reproduce the crashes then?  In any case, please show
> the backtrace from that configuration, to avoid gratuitous
> differences.

Rebuilding now, will follow-up tomorrow.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 12 Mar 2021 20:21:26 +0000
>>
>> > I'd like to compare your backtrace with what I get here.
>>
>> Mmmh, my Emacs on the 32bit system I prepared when running interactively
>> under gdb is unusable because all keys except the basic letters are
>> mixed-up.  I never experienced this, is this common?  How can I solve
>> it?
>
> Rebuild Emacs not under GDB.  I think the breakpoints and other stuff
> you set up in GDB get dumped into the pdmp file, and thus render the
> dumped Emacs not very usable.  I always rebuild Emacs when I let it
> dump itself (which happens rarely, because I rarely have to debug
> temacs).
>
> If the above doesn't help, it could be that some of the *.eln files
> are damaged for similar reasons, so maybe remove them and let Emacs
> recompile them as well.
>
> If none of the above helps, please describe how did you "prepare Emacs
> on the 32bit system when running interactively under GDB", maybe I
> don't understand what exactly you did there.

I've done nothing special, I've just a regular wide-int 32bit build on
the virtual machine I've prepared, running interactivelly Emacs underg
gdb is unusable for the reason I've described (outside gdb it works
perfectly).  I think it has nothing to do with native compilation.

> Meanwhile, could you please post the backtrace I asked for from the
> 64-bit build?  Maybe it will tell me enough to get some ideas.

Here it is:

#0  Fskip_chars_backward (string=0xd7f594, lim=0x0) at syntax.c:1610
#1  0x00007ffff2ce170d in F632d6261636b776172642d737773_c_backward_sws_0 () from /home/andcor03/.emacs.d/eln-cache/28.0.50-85dc790d/cc-engine-ccfcb170-1b345b21.eln
#2  0x000000000052445b in Ffuncall (nargs=1, args=0x7fffffff74e0) at eval.c:3009
#3  0x00007ffff2dfe124 in F632d666c2d6465636c2d656e64_c_fl_decl_end_0 () from /home/andcor03/.emacs.d/eln-cache/28.0.50-85dc790d/cc-mode-515a98a5-c7761ade.eln
#4  0x00000000005274f0 in funcall_lambda (fun=0xdc4cc5, nargs=1, arg_vector=0x7fffffff7618) at lisp.h:731
#5  0x00000000005242fd in Ffuncall (nargs=2, args=0x7fffffff7610) at eval.c:3025
#6  0x00007ffff2dfec94 in F632d636f6e746578742d657870616e642d666c2d726567696f6e_c_context_expand_fl_region_0 ()
   from /home/andcor03/.emacs.d/eln-cache/28.0.50-85dc790d/cc-mode-515a98a5-c7761ade.eln
#7  0x00000000005274f0 in funcall_lambda (fun=0xdc4d65, nargs=2, arg_vector=0x7fffffff7738) at lisp.h:731
#8  0x00000000005242fd in Ffuncall (nargs=3, args=0x7fffffff7730) at eval.c:3025
#9  0x00007ffff2dfee34 in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_62 () from /home/andcor03/.emacs.d/eln-cache/28.0.50-85dc790d/cc-mode-515a98a5-c7761ade.eln
#10 0x00000000005274f0 in funcall_lambda (fun=0xcbd8c5, nargs=1, arg_vector=0x7fffffff7848) at lisp.h:731
#11 0x00000000005242fd in Ffuncall (nargs=2, args=0x7fffffff7840) at eval.c:3025
#12 0x000000000052450a in call1 (fn=<optimized out>, arg1=<optimized out>) at eval.c:2869
#13 0x000000000052e6e4 in mapcar1 (leni=1, vals=vals@entry=0x0, fn=fn@entry=0xcbd8c5, seq=seq@entry=0xe111d3) at lisp.h:731
#14 0x0000000000530655 in Fmapc (function=0xcbd8c5, sequence=0xe111d3) at fns.c:2867
#15 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffff7930) at eval.c:3009
#16 0x00007ffff2dfef77 in F632d6265666f72652d636f6e746578742d666c2d657870616e642d726567696f6e_c_before_context_fl_expand_region_0 ()
   from /home/andcor03/.emacs.d/eln-cache/28.0.50-85dc790d/cc-mode-515a98a5-c7761ade.eln
#17 0x00000000005274f0 in funcall_lambda (fun=0xdc4db5, nargs=2, arg_vector=0x7fffffff7ae8) at lisp.h:731
#18 0x00000000005242fd in Ffuncall (nargs=3, args=0x7fffffff7ae0) at eval.c:3025
#19 0x00007ffff2dff40f in F632d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_c_font_lock_fontify_region_0 ()
   from /home/andcor03/.emacs.d/eln-cache/28.0.50-85dc790d/cc-mode-515a98a5-c7761ade.eln
#20 0x00000000005274f0 in funcall_lambda (fun=0xdc4e05, nargs=3, arg_vector=0x7fffffff7c68) at lisp.h:731
#21 0x00000000005242fd in Ffuncall (nargs=4, args=0x7fffffff7c60) at eval.c:3025
#22 0x00007ffff40c06b5 in F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/font-lock-895216f6-4cc69ac6.eln
#23 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffff7d20) at eval.c:3009
#24 0x000000000055da00 in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized out>, nargs=<optimized out>,
    args=<optimized out>) at bytecode.c:632
#25 0x00000000005242fd in Ffuncall (nargs=2, args=0x7fffffff80b0) at eval.c:3025
#26 0x00000000005244bc in run_hook_wrapped_funcall (nargs=<optimized out>, args=0x7fffffff80b0) at eval.c:2746
#27 0x0000000000523b3d in run_hook_with_args (nargs=2, args=0x7fffffff80b0, funcall=0x5244a0 <run_hook_wrapped_funcall>) at eval.c:2827
#28 0x00007ffff4318bc5 in F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/jit-lock-8a988e43-73eb7028.eln
#29 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffff8200) at eval.c:3009
#30 0x00007ffff4319521 in F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/jit-lock-8a988e43-73eb7028.eln-----------------------------------------------------------------------------
#31 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffff8350) at eval.c:3009
#32 0x00007ffff431894d in F6a69742d6c6f636b2d66756e6374696f6e_jit_lock_function_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/jit-lock-8a988e43-73eb7028.eln
#33 0x000000000052445b in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x7fffffff8440) at eval.c:3009
#34 0x00000000005236c9 in internal_condition_case_n (bfun=0x524190 <Ffuncall>, nargs=2, args=0x7fffffff8440, handlers=<optimized out>, hfun=0x4330e0 <safe_eval_handler>)
    at eval.c:1612
#35 0x0000000000427a50 in safe__call (inhibit_quit=inhibit_quit@entry=false, nargs=nargs@entry=2, func=<optimized out>, ap=ap@entry=0x7fffffff84b8) at lisp.h:1002
#36 0x0000000000432305 in safe_call (nargs=nargs@entry=2, func=<optimized out>) at xdisp.c:2996
#37 0x000000000043244f in safe_call1 (arg=0x1f10a, fn=<optimized out>) at xdisp.c:3007
#38 handle_fontified_prop (it=0x7fffffff8700) at xdisp.c:4313
#39 handle_fontified_prop (it=0x7fffffff8700) at xdisp.c:4250
#40 0x0000000000434bfd in handle_stop (it=0x7fffffff8700) at xdisp.c:3841
#41 0x000000000044f2d2 in redisplay_window (window=<optimized out>, just_this_one_p=<optimized out>) at xdisp.c:18960
#42 0x0000000000452f2b in redisplay_window_0 (window=window@entry=0xb6228d) at xdisp.c:16581
#43 0x00000000005234e4 in internal_condition_case_1 (bfun=0x452f00 <redisplay_window_0>, arg=0xb6228d, handlers=<optimized out>, hfun=0x4276f0 <redisplay_window_error>)
    at eval.c:1472
#44 0x0000000000429791 in redisplay_windows (window=0xb6228d) at xdisp.c:16561
#45 0x0000000000448e65 in redisplay_internal () at xdisp.c:16029
#46 0x000000000044abd0 in redisplay_preserve_echo_area (from_where=<optimized out>) at xdisp.c:16378
#47 0x0000000000418fdf in Fredisplay (force=<optimized out>) at dispnew.c:6157
#48 Fredisplay (force=0x0) at dispnew.c:6135
#49 0x00007ffff44facc2 in F7369742d666f72_sit_for_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/subr-13adf6a6-4a99f497.eln
#50 0x000000000052445b in Ffuncall (nargs=2, args=0x7fffffffd5b0) at eval.c:3009
#51 0x00007ffff408705d in F697365617263682d6c617a792d686967686c696768742d6e65772d6c6f6f70_isearch_lazy_highlight_new_loop_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#52 0x000000000052445b in Ffuncall (nargs=1, args=0x7fffffffd750) at eval.c:3009
#53 0x00007ffff40772fd in F697365617263682d757064617465_isearch_update_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#54 0x000000000052445b in Ffuncall (nargs=1, args=0x7fffffffd920) at eval.c:3009
#55 0x00007ffff4080ec4 in F697365617263682d7365617263682d616e642d757064617465_isearch_search_and_update_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#56 0x000000000052445b in Ffuncall (nargs=1, args=0x7fffffffd9d8) at eval.c:3009
#57 0x00007ffff4082ead in F697365617263682d70726f636573732d7365617263682d737472696e67_isearch_process_search_string_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#58 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffffdac0) at eval.c:3009
#59 0x00007ffff4082cf9 in F697365617263682d70726f636573732d7365617263682d63686172_isearch_process_search_char_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#60 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffffdbc0) at eval.c:3009
#61 0x00007ffff4082bb5 in F697365617263682d7072696e74696e672d63686172_isearch_printing_char_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#62 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffffdcf8) at eval.c:3009
#63 0x0000000000520c31 in Ffuncall_interactively (nargs=3, args=0x7fffffffdcf8) at callint.c:260
#64 0x000000000052445b in Ffuncall (nargs=4, args=0x7fffffffdcf0) at eval.c:3009
#65 0x0000000000526210 in Fapply (nargs=3, args=0x7fffffffde40) at eval.c:2639
#66 0x00000000005222ae in Fcall_interactively (function=0x7ffff3f38050, record_flag=0x0, keys=0x1040f35) at callint.c:353
#67 0x00007ffff43b7a51 in F636f6d6d616e642d65786563757465_command_execute_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/simple-fab5b0cf-9fc01ab7.eln
#68 0x000000000052445b in Ffuncall (nargs=2, args=0x7fffffffdfc0) at eval.c:3009
#69 0x000000000052450a in call1 (fn=<optimized out>, arg1=<optimized out>) at eval.c:2869
#70 0x00000000004b9846 in command_loop_1 () at lisp.h:1002
#42 0x0000000000452f2b in redisplay_window_0 (window=window@entry=0xb6228d) at xdisp.c:16581
#43 0x00000000005234e4 in internal_condition_case_1 (bfun=0x452f00 <redisplay_window_0>, arg=0xb6228d, handlers=<optimized out>, hfun=0x4276f0 <redisplay_window_error>)
    at eval.c:1472
#44 0x0000000000429791 in redisplay_windows (window=0xb6228d) at xdisp.c:16561
#45 0x0000000000448e65 in redisplay_internal () at xdisp.c:16029
#46 0x000000000044abd0 in redisplay_preserve_echo_area (from_where=<optimized out>) at xdisp.c:16378
#47 0x0000000000418fdf in Fredisplay (force=<optimized out>) at dispnew.c:6157
#48 Fredisplay (force=0x0) at dispnew.c:6135
#49 0x00007ffff44facc2 in F7369742d666f72_sit_for_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/subr-13adf6a6-4a99f497.eln
#50 0x000000000052445b in Ffuncall (nargs=2, args=0x7fffffffd5b0) at eval.c:3009
#51 0x00007ffff408705d in F697365617263682d6c617a792d686967686c696768742d6e65772d6c6f6f70_isearch_lazy_highlight_new_loop_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#52 0x000000000052445b in Ffuncall (nargs=1, args=0x7fffffffd750) at eval.c:3009
#53 0x00007ffff40772fd in F697365617263682d757064617465_isearch_update_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#54 0x000000000052445b in Ffuncall (nargs=1, args=0x7fffffffd920) at eval.c:3009
#55 0x00007ffff4080ec4 in F697365617263682d7365617263682d616e642d757064617465_isearch_search_and_update_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#56 0x000000000052445b in Ffuncall (nargs=1, args=0x7fffffffd9d8) at eval.c:3009
#57 0x00007ffff4082ead in F697365617263682d70726f636573732d7365617263682d737472696e67_isearch_process_search_string_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#58 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffffdac0) at eval.c:3009
#59 0x00007ffff4082cf9 in F697365617263682d70726f636573732d7365617263682d63686172_isearch_process_search_char_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#60 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffffdbc0) at eval.c:3009
#61 0x00007ffff4082bb5 in F697365617263682d7072696e74696e672d63686172_isearch_printing_char_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/isearch-693bdd75-d165ea70.eln
#62 0x000000000052445b in Ffuncall (nargs=3, args=0x7fffffffdcf8) at eval.c:3009
#63 0x0000000000520c31 in Ffuncall_interactively (nargs=3, args=0x7fffffffdcf8) at callint.c:260
#64 0x000000000052445b in Ffuncall (nargs=4, args=0x7fffffffdcf0) at eval.c:3009
#65 0x0000000000526210 in Fapply (nargs=3, args=0x7fffffffde40) at eval.c:2639
#66 0x00000000005222ae in Fcall_interactively (function=0x7ffff3f38050, record_flag=0x0, keys=0x1040f35) at callint.c:353
#67 0x00007ffff43b7a51 in F636f6d6d616e642d65786563757465_command_execute_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-85dc790d/simple-fab5b0cf-9fc01ab7.eln
#68 0x000000000052445b in Ffuncall (nargs=2, args=0x7fffffffdfc0) at eval.c:3009
#69 0x000000000052450a in call1 (fn=<optimized out>, arg1=<optimized out>) at eval.c:2869
#70 0x00000000004b9846 in command_loop_1 () at lisp.h:1002
--Type <RET> for more, q to quit, c to continue without paging--
#71 0x0000000000523457 in internal_condition_case (bfun=0x4b9490 <command_loop_1>, handlers=<optimized out>, hfun=0x4b0770 <cmd_error>) at eval.c:1448
#72 0x00000000004ab5ec in command_loop_2 (ignore=ignore@entry=0x0) at lisp.h:1002
#73 0x0000000000525913 in internal_catch (tag=0xca80, func=0x4ab5d0 <command_loop_2>, arg=0x0) at eval.c:1198
#74 0x00000000004ab594 in command_loop () at lisp.h:1002
#75 0x00000000004b0391 in recursive_edit_1 () at keyboard.c:720
#76 0x00000000004b06b3 in Frecursive_edit () at keyboard.c:789
#77 0x0000000000418036 in main (argc=<optimized out>, argv=<optimized out>) at emacs.c:2095

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pip Cet <pipcet@gmail.com>
>> Date: Sat, 13 Mar 2021 11:24:28 +0000
>> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
>> 
>> On Sat, Mar 13, 2021 at 11:10 AM Eli Zaretskii <eliz@gnu.org> wrote:
>> > > From: Pip Cet <pipcet@gmail.com>
>> > > Date: Sat, 13 Mar 2021 08:53:04 +0000
>> > > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
>> > >
>> > > It's c-beginning-of-statement-1 that I think is the immediate caller.
>> >
>> > It's nowhere in the C backtrace, only its caller
>> 
>> But it was in one of the previous backtraces?
>
> Too many moons ago.  The ABI was bumped since then, and so did the
> *.eln files.
>
>> > > Can you disassemble the function around 0x09c32285 (or, in another
>> > > dump, whatever calls Flss), particularly the 256 bytes or so before
>> > > that EIP?
>> >
>> > Below.  The updated address for the caller of Flss is 0x09d82285,
>> > which is not in any function whose name is known to GDB.
>> 
>> That's normal, assuming you didn't compile with comp-debug > 0.
>
> If that's normal, why did Andrea say that he didn't see such "??" in
> his backtraces?  And if that's because Andrea builds with comp-debug
> above the default zero, then I think either the doc string or the set
> of values (or both) need to be updated:

On my x86_64-pc-linux-gnu system I've always seen the all the function
names (no ??)  even with no debug symbols, see my other backtrace
(compiled with the default comp-debug 0).

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Sat, 13 Mar 2021 19:51:59 +0000
> 
> I've done nothing special, I've just a regular wide-int 32bit build on
> the virtual machine I've prepared, running interactivelly Emacs underg
> gdb is unusable for the reason I've described (outside gdb it works
> perfectly).  I think it has nothing to do with native compilation.

Maybe your GDB has problems with debugging 32-bit executables?

> > Meanwhile, could you please post the backtrace I asked for from the
> > 64-bit build?  Maybe it will tell me enough to get some ideas.
> 
> Here it is:

Very different.  I don't understand why all those functions you see
appear as ?? in my case.  Maybe it indeed has something to do with ELF
vs PE-COFF binaries.  I'm just guessing.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Sat, 13 Mar 2021 19:51:59 +0000
>> 
>> I've done nothing special, I've just a regular wide-int 32bit build on
>> the virtual machine I've prepared, running interactivelly Emacs underg
>> gdb is unusable for the reason I've described (outside gdb it works
>> perfectly).  I think it has nothing to do with native compilation.
>
> Maybe your GDB has problems with debugging 32-bit executables?

Everything is possible... BTW this is gdb (Debian 8.2.1-2+b3) 8.2.1.

Tomorrow I'll try with a recent gdb from trunk to see if it's better.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pip Cet <pipcet@gmail.com>
>> Date: Sat, 13 Mar 2021 16:32:50 +0000
>> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
>> 
>> > > > > So EDI is bunk at this point. Can you go back a bit further to where
>> > > > > it's initialized?
>> > > >
>> > > > Sorry, I don't understand: I gave you the disassembly of 512 bytes
>> > > > before, isn't that enough to see where EDI is assigned the value?  Or
>> > > > what do you mean by "go back"?
>> > >
>> > > It's not enough, no. we're looking for an insn of the form mov XXX,
>> > > %edi or lea XXX, %edi, or anything like that.
>> >
>> > I went back 4KB, and the only two instructions that write into EDI are
>> 
>> It's a long function, that might not have been enough.
>
> But since I found those two, everything before that is irrelevant,
> right?
>
>> > > I'm suspicious because EDI is a register variable that is clobbered
>> > > somehow right after a setjmp returned. Which setjmp implementation are
>> > > you using?
>> >
>> > Not sure how to answer that.  AFAIK, it's a setjmp from the MS runtime.
>> 
>> So not some mingw wrapper for it?
>
> No, not that I could see.
>
>> I just checked the only "mingw"-like sources I could find, and they
>> don't appear to use the frame pointer argument properly...
>
> Why is this suddenly relevant when native compilation is involved?
>
>> > > Is it possible that you're on Windows, but unlike other Windows
>> > > setjmps, it's unsafe to call your setjmp through a function pointer?
>> >
>> > How do I tell?
>> 
>> Well, you could just apply this untested patch, fix any obvious
>> compile errors I might not have spotted, and try to reproduce it. I'm
>> not currently on a Windows (or x86) machine, so it's a bit hard for me
>> to test...
>
> I'd like this investigation to be less of a blind search, sorry.  can
> you tell what to check or look at to see if this is relevant?

One confirmation that the issue is the one suggested by Pip would be
running the test we added for this with like:

$ ./src/emacs -batch -l test/src/comp-tests.el --eval '(ert-run-tests-batch-and-exit "46824-1")'

If the test-case fails it would be a clear marker, if it doesn't the
issue might still the be the suggested one but the different architecture
might play a role here making the test-case ineffective.

Thanks

  Andtea

PS Eli, even better would be to run all tests in comp-tests.el as a
quick sanity check to verify that all is okay.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sat, Mar 13, 2021 at 8:53 PM Andrea Corallo <akrl@sdf.org> wrote:
> One confirmation that the issue is the one suggested by Pip would be
> running the test we added for this with like:

Andrea, are you aware of any reason not to call setjmp directly on Windows?

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Pip Cet <pipcet@gmail.com> writes:

> On Sat, Mar 13, 2021 at 8:53 PM Andrea Corallo <akrl@sdf.org> wrote:
>> One confirmation that the issue is the one suggested by Pip would be
>> running the test we added for this with like:
>
> Andrea, are you aware of any reason not to call setjmp directly on Windows?

No I'm not, as I don't see a reason to do it probably we should just go
for the direct call as we do for other systems.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Pip Cet <pipcet@gmail.com> writes:
>
>> On Sat, Mar 13, 2021 at 8:53 PM Andrea Corallo <akrl@sdf.org> wrote:
>>> One confirmation that the issue is the one suggested by Pip would be
>>> running the test we added for this with like:
>>
>> Andrea, are you aware of any reason not to call setjmp directly on Windows?
>
> No I'm not, as I don't see a reason to do it probably we should just go
> for the direct call as we do for other systems.

(as your suggestion) ;)

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Pip Cet <pipcet@gmail.com>, 47067@debbugs.gnu.org
> Date: Sat, 13 Mar 2021 20:53:00 +0000
> 
> > I'd like this investigation to be less of a blind search, sorry.  can
> > you tell what to check or look at to see if this is relevant?
> 
> One confirmation that the issue is the one suggested by Pip would be
> running the test we added for this with like:
> 
> $ ./src/emacs -batch -l test/src/comp-tests.el --eval '(ert-run-tests-batch-and-exit "46824-1")'
> 
> If the test-case fails it would be a clear marker, if it doesn't the
> issue might still the be the suggested one but the different architecture
> might play a role here making the test-case ineffective.

This test succeeds.

> PS Eli, even better would be to run all tests in comp-tests.el as a
> quick sanity check to verify that all is okay.

This is on my list of things to do, but with such basic instability I
thought it was premature.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Date: Sat, 13 Mar 2021 21:07:55 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, 47067@debbugs.gnu.org
> 
> Pip Cet <pipcet@gmail.com> writes:
> 
> > On Sat, Mar 13, 2021 at 8:53 PM Andrea Corallo <akrl@sdf.org> wrote:
> >> One confirmation that the issue is the one suggested by Pip would be
> >> running the test we added for this with like:
> >
> > Andrea, are you aware of any reason not to call setjmp directly on Windows?
> 
> No I'm not, as I don't see a reason to do it probably we should just go
> for the direct call as we do for other systems.

If someone can send a patch, I could try it.  TIA.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Date: Sat, 13 Mar 2021 21:07:55 +0000
>> Cc: Eli Zaretskii <eliz@gnu.org>, 47067@debbugs.gnu.org
>> 
>> Pip Cet <pipcet@gmail.com> writes:
>> 
>> > On Sat, Mar 13, 2021 at 8:53 PM Andrea Corallo <akrl@sdf.org> wrote:
>> >> One confirmation that the issue is the one suggested by Pip would be
>> >> running the test we added for this with like:
>> >
>> > Andrea, are you aware of any reason not to call setjmp directly on Windows?
>> 
>> No I'm not, as I don't see a reason to do it probably we should just go
>> for the direct call as we do for other systems.
>
> If someone can send a patch, I could try it.  TIA.

Should be what Pip suggested here
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#98>

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Pip Cet <pipcet@gmail.com>, 47067@debbugs.gnu.org
>> Date: Sat, 13 Mar 2021 20:53:00 +0000
>> 
>> > I'd like this investigation to be less of a blind search, sorry.  can
>> > you tell what to check or look at to see if this is relevant?
>> 
>> One confirmation that the issue is the one suggested by Pip would be
>> running the test we added for this with like:
>> 
>> $ ./src/emacs -batch -l test/src/comp-tests.el --eval '(ert-run-tests-batch-and-exit "46824-1")'
>> 
>> If the test-case fails it would be a clear marker, if it doesn't the
>> issue might still the be the suggested one but the different architecture
>> might play a role here making the test-case ineffective.
>
> This test succeeds.
>
>> PS Eli, even better would be to run all tests in comp-tests.el as a
>> quick sanity check to verify that all is okay.
>
> This is on my list of things to do, but with such basic instability I
> thought it was premature.

Running tests in comp-tests.el should take a couple of minutes and in
case we catch a fail there we'd already have a reduced reproducer to
look at, so yeah I suggest to have a run when you can as I think might
have a good return for the investment.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: 47067@debbugs.gnu.org
>>> Date: Sat, 13 Mar 2021 19:51:59 +0000
>>>
>>> I've done nothing special, I've just a regular wide-int 32bit build on
>>> the virtual machine I've prepared, running interactivelly Emacs underg
>>> gdb is unusable for the reason I've described (outside gdb it works
>>> perfectly).  I think it has nothing to do with native compilation.
>>
>> Maybe your GDB has problems with debugging 32-bit executables?
>
> Everything is possible... BTW this is gdb (Debian 8.2.1-2+b3) 8.2.1.
>
> Tomorrow I'll try with a recent gdb from trunk to see if it's better.

Okay with a trunk gdb I'm able to debug...

Here is my backtrace for the suggested reproducer on my Emacs configured
with --with-wide-int --enable-checking=yes,glyphs and compiled with the
default `comp-debug' 0.

=============
#0  Fskip_chars_backward (string=-9223372036710370816, lim=0) at syntax.c:1610
#1  0xb3f3224b in F632d6261636b776172642d737773_c_backward_sws_0 () from /home/andcor03/.emacs.d/eln-cache/28.0.50-d8064d9c/cc-engine-ccfcb170-1b345b21.eln
#2  0x081c68fa in Ffuncall (nargs=1, args=0xbfff9c78) at eval.c:3009
#3  0xb3dfc844 in F632d666f6e742d6c6f636b2d636f6d706c65782d6465636c2d70726570617265_c_font_lock_complex_decl_prepare_0 ()
   from /home/andcor03/.emacs.d/eln-cache/28.0.50-d8064d9c/cc-fonts-d7d8a7f5-b7c359cd.eln
#4  0x081c9f7a in funcall_lambda (fun=<optimized out>, nargs=1, arg_vector=0xbfff9f50) at eval.c:3292
#5  0x081c6824 in Ffuncall (nargs=2, args=0xbfff9f48) at eval.c:3025
#6  0xb4a4d023 in F666f6e742d6c6f636b2d666f6e746966792d6b6579776f7264732d726567696f6e_font_lock_fontify_keywords_region_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/font-lock-895216f6-4cc69ac6.eln
#7  0x081c8c4f in funcall_subr (subr=<optimized out>, numargs=<optimized out>, args=<optimized out>) at eval.c:3092
#8  0x081c68fa in Ffuncall (nargs=4, args=0xbfffa1e0) at eval.c:3009
#9  0xb4a49542 in F666f6e742d6c6f636b2d64656661756c742d666f6e746966792d726567696f6e_font_lock_default_fontify_region_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/font-lock-895216f6-4cc69ac6.eln
#10 0x081c8c4f in funcall_subr (subr=<optimized out>, numargs=<optimized out>, args=<optimized out>) at eval.c:3092
#11 0x081c68fa in Ffuncall (nargs=4, args=0xbfffa430) at eval.c:3009 */
#12 0xb40f1b8c in F632d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_c_font_lock_fontify_region_0 ()
   from /home/andcor03/.emacs.d/eln-cache/28.0.50-d8064d9c/cc-mode-515a98a5-c7761ade.eln
#13 0x081c9f7a in funcall_lambda (fun=<optimized out>, nargs=3, arg_vector=0xbfffa558) at eval.c:3292
#14 0x081c6824 in Ffuncall (nargs=4, args=0xbfffa550) at eval.c:3025
#15 0xb4a4807a in F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/font-lock-895216f6-4cc69ac6.eln
#16 0x081c8c4f in funcall_subr (subr=<optimized out>, numargs=<optimized out>, args=<optimized out>) at eval.c:3092
#17 0x081c68fa in Ffuncall (nargs=3, args=0xbfffa690) at eval.c:3009
#18 0x08212ffd in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized out>, nargs=<optimized out>,
    args=<optimized out>) at bytecode.c:632dows.  */
#19 0x081c8323 in fetch_and_exec_byte_code (fun=<optimized out>, syms_left=4611686018427388161, nargs=1, args=0xbfffaae0) at eval.c:3133
#20 0x081c9f30 in funcall_lambda (fun=<optimized out>, nargs=1, arg_vector=0xbfffaae0) at eval.c:3214
#21 0x081c6824 in Ffuncall (nargs=2, args=0xbfffaad8) at eval.c:3025
#22 0x081c699a in run_hook_wrapped_funcall (nargs=2, args=0xbfffaad8) at eval.c:2746
#23 0x081c5c8c in run_hook_with_args (nargs=2, args=0xbfffaad8, funcall=0x81c6970 <run_hook_wrapped_funcall>) at eval.c:2827
#24 0x081c5fa5 in Frun_hook_wrapped (nargs=2, args=0xbfffaad8) at eval.c:2761
#25 0xb4e83640 in F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/jit-lock-8a988e43-ae750d1b.eln
#26 0x081c8c2e in funcall_subr (subr=0xb54731e8, numargs=2, args=0xbfffacb0) at eval.c:3086
#27 0x081c68fa in Ffuncall (nargs=3, args=0xbfffaca8) at eval.c:3009
#28 0xb4e8440b in F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/jit-lock-8a988e43-ae750d1b.eln
#29 0x081c8c2e in funcall_subr (subr=0xb548a2d4, numargs=2, args=0xbfffae90) at eval.c:3086----------------------------------------------------------------------------------
#30 0x081c68fa in Ffuncall (nargs=3, args=0xbfffae88) at eval.c:3009
#31 0xb4e83082 in F6a69742d6c6f636b2d66756e6374696f6e_jit_lock_function_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/jit-lock-8a988e43-ae750d1b.eln
#32 0x081c68fa in Ffuncall (nargs=2, args=0xbfffaf60) at eval.c:3009
#33 0x081c55c7 in internal_condition_case_n (bfun=0x81c6610 <Ffuncall>, nargs=2, args=0xbfffaf60, handlers=40, hfun=0x8093ec0 <safe_eval_handler>) at eval.c:1612
#34 0x080847f0 in safe__call (inhibit_quit=inhibit_quit@entry=false, nargs=nargs@entry=2, func=<optimized out>, ap=<optimized out>) at xdisp.c:2981
#35 0x08092abe in safe_call (nargs=2, func=2902088064) at xdisp.c:2996
#36 0x08092c9b in safe_call1 (arg=4611686018427572803, fn=2902088064) at xdisp.c:3007
#37 handle_fontified_prop (it=<optimized out>) at xdisp.c:4313
#38 0x08096e6e in handle_stop (it=0xbfffb178) at xdisp.c:3841
#39 0x080b4700 in redisplay_window (window=<optimized out>, just_this_one_p=<optimized out>) at xdisp.c:18960
#40 0x080b8723 in redisplay_window_0 (window=-6917529027498444184) at xdisp.c:16581
#41 0x081c544f in internal_condition_case_1 (bfun=0x80b86e0 <redisplay_window_0>, arg=-6917529027498444184, handlers=-4611686015382928776,
    hfun=0x8081820 <redisplay_window_error>) at eval.c:1472
#42 0x08082086 in redisplay_windows (window=<optimized out>) at xdisp.c:16561
#43 0x080addfe in redisplay_internal () at xdisp.c:16029
#44 0x080b054e in redisplay_preserve_echo_area (from_where=2) at xdisp.c:16378
#45 0x08070366 in Fredisplay (force=0) at dispnew.c:6157
#46 0xb50fcd59 in F7369742d666f72_sit_for_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/subr-13adf6a6-656b9117.eln
#47 0x081c8c4f in funcall_subr (subr=<optimized out>, numargs=<optimized out>, args=<optimized out>) at eval.c:3092
#48 0x081c68fa in Ffuncall (nargs=2, args=0xbfffe5a8) at eval.c:3009
#49 0xb4be1cab in F697365617263682d6c617a792d686967686c696768742d6e65772d6c6f6f70_isearch_lazy_highlight_new_loop_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/isearch-693bdd75-2b63c98a.eln
#50 0x081c8c2e in funcall_subr (subr=0xb564b4fc, numargs=0, args=0xbfffe7c0) at eval.c:3086
#51 0x081c68fa in Ffuncall (nargs=1, args=0xbfffe7b8) at eval.c:3009
#52 0xb4bcad02 in F697365617263682d757064617465_isearch_update_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/isearch-693bdd75-2b63c98a.eln
#53 0x081c68fa in Ffuncall (nargs=1, args=0xbfffe978) at eval.c:3009
#54 0xb4bd943e in F697365617263682d7365617263682d616e642d757064617465_isearch_search_and_update_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/isearch-693bdd75-2b63c98a.eln
#55 0x081c68fa in Ffuncall (nargs=1, args=0xbfffea38) at eval.c:3009
#56 0xb4bdc01c in F697365617263682d70726f636573732d7365617263682d737472696e67_isearch_process_search_string_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/isearch-693bdd75-2b63c98a.eln
#57 0x081c8c2e in funcall_subr (subr=0xb55cc904, numargs=2, args=0xbfffebb8) at eval.c:3086
#58 0x081c68fa in Ffuncall (nargs=3, args=0xbfffebb0) at eval.c:3009
#59 0xb4bdbe01 in F697365617263682d70726f636573732d7365617263682d63686172_isearch_process_search_char_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/isearch-693bdd75-2b63c98a.eln
#60 0x081c8c2e in funcall_subr (subr=0xb55db120, numargs=2, args=0xbfffed30) at eval.c:3086
#61 0x081c68fa in Ffuncall (nargs=3, args=0xbfffed28) at eval.c:3009
#62 0xb4bdbb93 in F697365617263682d7072696e74696e672d63686172_isearch_printing_char_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/isearch-693bdd75-2b63c98a.eln
#63 0x081c8c2e in funcall_subr (subr=0xb5525e5c, numargs=2, args=0xbfffeec0) at eval.c:3086
#64 0x081c68fa in Ffuncall (nargs=3, args=0xbfffeeb8) at eval.c:3009
#65 0x081c1e7e in Ffuncall_interactively (nargs=3, args=0xbfffeeb8) at callint.c:260
#66 0x081c68fa in Ffuncall (nargs=4, args=0xbfffeeb0) at eval.c:3009
#67 0x081c9400 in Fapply (nargs=<optimized out>, args=<optimized out>) at eval.c:2639
#68 0x081c3ae9 in Fcall_interactively (function=2900825492, record_flag=<optimized out>, keys=<optimized out>) at callint.c:353
#69 0xb4f50b3b in F636f6d6d616e642d65786563757465_command_execute_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-d8064d9c/simple-fab5b0cf-fa0c27ba.eln
#70 0x081c8c4f in funcall_subr (subr=<optimized out>, numargs=<optimized out>, args=<optimized out>) at eval.c:3092
#71 0x081c68fa in Ffuncall (nargs=2, args=0xbffff1d0) at eval.c:3009
#72 0x081c6a2f in call1 (fn=11520, arg1=2900825492) at eval.c:2869
#73 0x0813a4ed in command_loop_1 () at keyboard.c:1466
#74 0x081c53a4 in internal_condition_case (bfun=0x8139f70 <command_loop_1>, handlers=120, hfun=0x812e310 <cmd_error>) at eval.c:1448
#75 0x0812677b in command_loop_2 (ignore=0) at keyboard.c:1094
#76 0x081c89ae in internal_catch (tag=42000, func=0x8126760 <command_loop_2>, arg=0) at eval.c:1198
#77 0x081270ac in command_loop () at keyboard.c:1073
#78 0x0812de09 in recursive_edit_1 () at keyboard.c:720
#79 0x0812e196 in Frecursive_edit () at keyboard.c:789
#80 0x0806ceb1 in main (argc=<optimized out>, argv=<optimized out>) at emacs.c:2095
=============

Emacs does not crash if then I continue and scroll through down through
in the file.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
> Date: Sun, 14 Mar 2021 06:53:01 +0000
> 
> > If someone can send a patch, I could try it.  TIA.
> 
> Should be what Pip suggested here
> <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#98>

Shouldn't that bump the ABI number as well?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
>> Date: Sun, 14 Mar 2021 06:53:01 +0000
>> 
>> > If someone can send a patch, I could try it.  TIA.
>> 
>> Should be what Pip suggested here
>> <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#98>
>
> Shouldn't that bump the ABI number as well?

Right good point I think it should.

Anyway thinking about: this should be good for a quick test maybe
removing (or renaming) the discussed .eln.  But if it proves this to be
the bug you'll have to rebuild from scratch to have a reliable system
anyway , as the ABI number will not cover us for the native code that
was already dumped with the image.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
> Date: Sun, 14 Mar 2021 07:31:14 +0000
> 
> >> <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#98>
> >
> > Shouldn't that bump the ABI number as well?
> 
> Right good point I think it should.
> 
> Anyway thinking about: this should be good for a quick test maybe
> removing (or renaming) the discussed .eln.  But if it proves this to be
> the bug you'll have to rebuild from scratch to have a reliable system
> anyway , as the ABI number will not cover us for the native code that
> was already dumped with the image.

Sorry, I don't understand: what is the procedure for testing this
patch?

I thought I should do this, in the order specified:

  . apply the patch
  . increase the ABI version
  . rebuild Emacs
  . natively-compile all the preloaded Lisp files, then redump Emacs
  . visit a C file and let Emacs native-compile all the cc-*.el files
  . restart Emacs and try reproducing the crash

Are there any shortcuts that don't run the risk of invalidating the
test results?  Since we are changing something so basic in how Lisp is
native-compiled, and since non-local exits are so common on our
sources, can I really do less than the above and be sure the solution
works reliably?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, 47067@debbugs.gnu.org
> Date: Sun, 14 Mar 2021 07:19:17 +0000
> 
> Okay with a trunk gdb I'm able to debug...

Yes, 8.2 is quite an old version.

> Here is my backtrace for the suggested reproducer on my Emacs configured
> with --with-wide-int --enable-checking=yes,glyphs and compiled with the
> default `comp-debug' 0.

Thanks.  However, your build is optimized, whereas mine isn't.  This
has a significant effect on the backtrace (static functions are
inlined and don't appear in the backtrace).  Please reconfigure with
CFLAGS='-O0 -g3' (and the rest of the switches as you used them for
this build), and show the backtrace after rebuilding.

As I established earlier, in

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#80
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#83

the code which GDB fails to identify with any symbols calls Emacs
primitives via freloc.link_table.  Where is the C source for this
code?  If there is no C source, then how is that code generated?

> Emacs does not crash if then I continue and scroll through down through
> in the file.

And if you type "C-s sleep"?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
>> Date: Sun, 14 Mar 2021 07:31:14 +0000
>> 
>> >> <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#98>
>> >
>> > Shouldn't that bump the ABI number as well?
>> 
>> Right good point I think it should.
>> 
>> Anyway thinking about: this should be good for a quick test maybe
>> removing (or renaming) the discussed .eln.  But if it proves this to be
>> the bug you'll have to rebuild from scratch to have a reliable system
>> anyway , as the ABI number will not cover us for the native code that
>> was already dumped with the image.
>
> Sorry, I don't understand: what is the procedure for testing this
> patch?
>
> I thought I should do this, in the order specified:
>
>   . apply the patch
>   . increase the ABI version
>   . rebuild Emacs
>   . natively-compile all the preloaded Lisp files, then redump Emacs
>   . visit a C file and let Emacs native-compile all the cc-*.el files
>   . restart Emacs and try reproducing the crash

Yeah this would be 100% the proper way...  Given the function
potentially miscompiled has been isolated I was suggesting just to have
a quick confirmation doing:

  - apply the patch
  - "make" so we have the new compiler working
  - rename the .eln containing the maybe miscompiled function so that
    is not loaded
  - start Emacs and load cc-mode so the file gets native compiled
  - when this is done (the maybe miscompiled function will be native)
    try to reproduce on dispnew.c

If this gets further with no crash should be prove that now the function
is working properly.  Indeed as you suggest to have a relialble system
and probably to do a more certain test the full procedure you mentioned
will in case be necessary.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sun, Mar 14, 2021 at 7:31 AM Andrea Corallo <akrl@sdf.org> wrote:
> >> Should be what Pip suggested here
> >> <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#98>
> >
> > Shouldn't that bump the ABI number as well?
>
> Right good point I think it should.

The patch should be fine to test without bumping the ABI number, but
if we want to include it, we should remove the SETJMP export and bump
the ABI number.

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sun, Mar 14, 2021 at 7:48 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Andrea Corallo <akrl@sdf.org>
> > Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
> > Date: Sun, 14 Mar 2021 07:31:14 +0000
> >
> > >> <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#98>
> > >
> > > Shouldn't that bump the ABI number as well?
> >
> > Right good point I think it should.
> >
> > Anyway thinking about: this should be good for a quick test maybe
> > removing (or renaming) the discussed .eln.  But if it proves this to be
> > the bug you'll have to rebuild from scratch to have a reliable system
> > anyway , as the ABI number will not cover us for the native code that
> > was already dumped with the image.
>
> Sorry, I don't understand: what is the procedure for testing this
> patch?

- apply the patch
- rebuild Emacs
- remove the problematic file (but not necessarily any others)
- rerun your recipe

Bumping the ABI number should not be required to test the patch, and
rebuilding everything is, indeed, unnecessary and potentially spoils
test results.

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, 47067@debbugs.gnu.org
>> Date: Sun, 14 Mar 2021 07:19:17 +0000
>> 
>> Okay with a trunk gdb I'm able to debug...
>
> Yes, 8.2 is quite an old version.
>
>> Here is my backtrace for the suggested reproducer on my Emacs configured
>> with --with-wide-int --enable-checking=yes,glyphs and compiled with the
>> default `comp-debug' 0.
>
> Thanks.  However, your build is optimized, whereas mine isn't.  This
> has a significant effect on the backtrace (static functions are
> inlined and don't appear in the backtrace).  Please reconfigure with
> CFLAGS='-O0 -g3' (and the rest of the switches as you used them for
> this build), and show the backtrace after rebuilding.

Okay will rebuild and report this evening (have to leave the PC now).

> As I established earlier, in
>
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#80
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47067#83
>
> the code which GDB fails to identify with any symbols calls Emacs
> primitives via freloc.link_table.  Where is the C source for this
> code?  If there is no C source, then how is that code generated?

There's no C source for that as this is generated by us from Lisp (Emacs
is acting as a GCC frontend).

We can anyway have libgccjit generate some pseudo C code to ease
debugging (see 'gcc_jit_context_dump_to_file' in [1] if you like).  We
trigger this compiling with `comp-debug' >= 1, in this case gdb should
let you step into the .eln following the dumped pseudo C code.

>> Emacs does not crash if then I continue and scroll through down through
>> in the file.
>
> And if you type "C-s sleep"?

Doesn't crash.

Thanks

  Andrea

[1] <https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html>

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
> Date: Sun, 14 Mar 2021 06:57:36 +0000
> 
> >> PS Eli, even better would be to run all tests in comp-tests.el as a
> >> quick sanity check to verify that all is okay.
> >
> > This is on my list of things to do, but with such basic instability I
> > thought it was premature.
> 
> Running tests in comp-tests.el should take a couple of minutes and in
> case we catch a fail there we'd already have a reduced reproducer to
> look at, so yeah I suggest to have a run when you can as I think might
> have a good return for the investment.

The results of running "make src/comp-tests" are below.  As you see,
stage2 compilation crashed.  (Is 153 sec for stage1 a reasonable time,
btw?)  The backtrace is not useful: it shows only the stages since the
SIGSEGV delivery.

Any other tests you'd like me to try?

  $ make src/comp-tests
  make[1]: Entering directory `/d/gnu/git/emacs/native-comp/test'
    ELC+ELN      src/comp-tests.elc

  In toplevel form:
  src/comp-tests.el:120:14: Warning: reference to free variable
      `comp-tests-var1'
  src/comp-tests.el:676:15: Warning: reference to free variable
      `comp-test-up-val'

  In end of data:
  src/comp-tests.el:1438:1: Warning: the following functions are not known to be
      defined: comp-tests-varref-f, comp-tests-list-f, comp-tests-list2-f,
      comp-tests-car-f, comp-tests-cdr-f, comp-tests-car-safe-f,
      comp-tests-cdr-safe-f, comp-tests-cons-car-f, comp-tests-cons-cdr-f,
      comp-tests-varset0-f, comp-tests-varset1-f, comp-tests-length-f,
      comp-tests-aref-aset-f, comp-tests-symbol-value-f, comp-tests-concat-f,
      comp-tests-ffuncall-callee-optional-f, comp-tests-ffuncall-callee-rest-f,
      comp-tests-ffuncall-callee-more8-f,
      comp-tests-ffuncall-callee-more8-rest-f, comp-tests-ffuncall-native-f,
      comp-tests-ffuncall-native-rest-f, comp-tests-ffuncall-apply-many-f,
      comp-tests-ffuncall-lambda-f, comp-tests-jump-table-1-f,
      comp-tests-jump-table-2-f, comp-tests-conditionals-1-f,
      comp-tests-conditionals-2-f, comp-tests-fixnum-1-minus-f,
      comp-tests-fixnum-1-plus-f, comp-tests-fixnum-minus-f,
      comp-tests-hint-fixnum-f, comp-tests-hint-cons-f, comp-tests-eqlsign-f,
      comp-tests-gtr-f, comp-tests-lss-f, comp-tests-les-f, comp-tests-geq-f,
      comp-tests-setcar-f, comp-tests-setcdr-f, comp-bubble-sort-f,
      comp-tests-consp-f, comp-tests-setcar2-f, comp-tests-integerp-f,
      comp-tests-numberp-f, comp-tests-discardn-f, comp-tests-insertn-f,
      comp-tests-condition-case-0-f, comp-tests-condition-case-1-f,
      comp-tests-catch-f, comp-tests-throw-f, comp-tests-buff0-f,
      comp-tests-lambda-return-f, comp-tests-fib-f, comp-tests-string-trim-f,
      comp-tests-trampoline-removal-f, comp-tests-signal-f,
      comp-tests-func-call-removal-f, comp-tests-doc-f,
      comp-test-interactive-form0-f, comp-test-interactive-form1-f,
      comp-test-interactive-form2-f, comp-tests-free-fun-f,
      comp-tests/freefun-f, comp-test-40187-1-f, comp-test-40187-2-f,
      comp-test-speed--1-f, comp-test-42360-f, comp-test-44968-f,
      comp-test-45342-f, comp-test-assume-double-neg-f,
      comp-test-assume-in-loop-1-f, comp-test-45376-1-f, comp-test-45376-2-f,
      comp-test-primitive-advice-f, comp-test-primitive-redefine-f,
      comp-test-defsubst-f, comp-test-and-3-f, comp-test-copy-insn-f,
      comp-test-cond-rw-1-2-f, comp-test-not-cons-f, comp-test-45576-f,
      comp-test-45635-f, comp-test-45603--file-local-name, comp-test-46670-2-f,
      comp-test-46824-1-f, comp-test-consp, comp-test-listp, comp-test-stringp,
      comp-test-symbolp, comp-test-integerp, comp-test-numberp, comp-test-add1,
      comp-test-sub1, comp-test-negate, comp-test-not, comp-test-bobp,
      comp-test-eobp, comp-test-point-min, comp-test-point, comp-test-point-max,
      comp-test-car, comp-test-cdr, comp-test-car-safe, comp-test-cdr-safe,
      comp-test-eq, comp-test-if, comp-test-and, comp-test-or,
      comp-test-save-excursion, comp-test-current-buffer, comp-test->,
      comp-test-catch, comp-test-memq, comp-test-listN, comp-test-concatN,
      comp-test-opt-rest, comp-test-opt, comp-test-unwind-protect,
      comp-tests-ffuncall-callee-dyn-f, comp-tests-ffuncall-callee-opt-dyn-f,
      comp-tests-ffuncall-callee-rest-dyn-f,
      comp-tests-ffuncall-callee-opt-rest-dyn-f, comp-tests-cl-macro-exp-f,
      comp-tests-cl-uninterned-arg-parse-f, comp-tests-tco-f,
      comp-tests-fw-prop-1-f, comp-tests-pure-caller-f,
      comp-tests-pure-fibn-entry-f
    GEN      src/comp-tests.log
  Compiling tests...

  In comp-tests-discardn-f:
  comp-test-funcs.el:192:31: Warning: Unused lexical argument `x'

  In comp-test-40187-2-f:
  comp-test-funcs.el:286:8: Warning: function `comp-test-40187-2-f' defined
      multiple times in this file

  In comp-test-42360-f:
  comp-test-funcs.el:299:26: Warning: Unused lexical variable `str-width'
  comp-test-funcs.el:318:25: Warning: Unused lexical variable `ellipsis-width'

  In comp-test-46824-1-f:
  comp-test-funcs.el:488:11: Warning: value returned from (format "%S"
      (error-message-string err)) is unused
  comp-test-funcs.el:493:18: Warning: Unused lexical variable `recipe'

  In comp-test-silly-frame2:
  comp-test-funcs.el:628:32: Warning: Unused lexical argument `token'
  comp-test-funcs.el:630:10: Warning: reference to free variable `c'

  In comp-test-big-interactive:
  comp-test-funcs.el:665:12: Warning: byte-compile-file called with 2 arguments,
      but accepts only 1
  Loading d:/usr/MSYS/nonexistent/.emacs.d/eln-cache/28.0.50-7d88f6c1/comp-test-funcs-5c1d19b4-2b9f725f.eln (native compiled elisp)...
  Loading d:/usr/MSYS/nonexistent/.emacs.d/eln-cache/28.0.50-7d88f6c1/comp-test-funcs-dyn-a078e8f7-55bd648f.eln (native compiled elisp)...
  Running 165 tests (2021-03-14 10:08:59+0200, selector `(not (tag :unstable))')
     passed    1/165  comp-tests-45576-1 (0.000000 sec)

  In comp-test-45603--call-marked:
  comp-test-45603.el:10:38: Warning: Unused lexical argument `action'
  comp-test-45603.el:11:30: Warning: Unused lexical variable `marked-candidates'
  comp-test-45603.el:13:12: Warning: `mapcar' called for effect; use `mapc' or
      `dolist' instead
  comp-test-45603.el:19:12: Warning: Unused lexical variable `multi-action'
  comp-test-45603.el:20:59: Warning: Empty let* body

  In end of data:
  comp-test-45603.el:29:1: Warning: the function
      `comp-test-45603--get-multi-action' is not known to be defined.
  Loading d:/usr/MSYS/nonexistent/.emacs.d/eln-cache/28.0.50-7d88f6c1/comp-test-45603-25594de9-dc7d2de5.eln (native compiled elisp)...
     passed    2/165  comp-tests-45603-1 (2.031250 sec)
     passed    3/165  comp-tests-45635-1 (0.000000 sec)
     passed    4/165  comp-tests-46670-1 (0.000000 sec)
     passed    5/165  comp-tests-46824-1 (0.015625 sec)
     passed    6/165  comp-tests-> (0.000000 sec)
     passed    7/165  comp-tests-add1 (0.000000 sec)
     passed    8/165  comp-tests-and (0.000000 sec)
     passed    9/165  comp-tests-and-3 (0.000000 sec)
     passed   10/165  comp-tests-apply (0.000000 sec)
     passed   11/165  comp-tests-aref-aset (0.000000 sec)
     passed   12/165  comp-tests-arith-comp (0.000000 sec)
     passed   13/165  comp-tests-assume-double-neg (0.000000 sec)
     passed   14/165  comp-tests-assume-in-loop-1 (0.000000 sec)
     passed   15/165  comp-tests-bobp-and-eobp (0.000000 sec)
  Loading d:/gnu/git/emacs/native-comp/test/src/../../lisp/emacs-lisp/comp.elc...
  Compiling stage1...
  libgccjit-0.dll: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]
  Done in 153 secs
  Loading d:/gnu/git/emacs/native-comp/native-lisp/28.0.50-7d88f6c1/stage1-Wif12F-15c85b93-ed123e94.eln (native compiled elisp)...
  Compiling stage2...
  libgccjit-0.dll: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]

  Exception 0xc0000005 at this address:
  074bbd71

  Backtrace:
  0135ae16
  0135ae95
  0115c633
  0119b8c1
  0119b89b
  0119b8f6
  010010cd
  7c864eb5
  7c843e7e

  This application has requested the Runtime to terminate it in an unusual way.
  Please contact the application's support team for more information.
  Makefile:177: recipe for target `src/comp-tests.log' failed
  make[1]: *** [src/comp-tests.log] Error 3
  make[1]: Leaving directory `/d/gnu/git/emacs/native-comp/test'
  Makefile:243: recipe for target `src/comp-tests' failed
  make: *** [src/comp-tests] Error 2

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sun, 14 Mar 2021 08:06:30 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> > Sorry, I don't understand: what is the procedure for testing this
> > patch?
> 
> - apply the patch
> - rebuild Emacs
> - remove the problematic file (but not necessarily any others)
> - rerun your recipe

Thanks.  After patching and rebuilding Emacs, compilation of
cc-engine.el fails with these error messages:

  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *) 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_unary_op: NULL rvalue 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_block_end_with_conditional: NULL boolval 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *) 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_unary_op: NULL rvalue 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_block_end_with_conditional: NULL boolval 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *) 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_unary_op: NULL rvalue 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_block_end_with_conditional: NULL boolval 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *) 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_unary_op: NULL rvalue 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_block_end_with_conditional: NULL boolval 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *) 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_unary_op: NULL rvalue 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_block_end_with_conditional: NULL boolval 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *) 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_unary_op: NULL rvalue 
  Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_block_end_with_conditional: NULL boolval 
  Warning (comp): d:/gnu/git/emacs/native-comp/lisp/progmodes/cc-engine.el: Error: Internal native compiler error failing to compile function 

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

On Sun, Mar 14, 2021 at 11:39 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Sun, 14 Mar 2021 08:06:30 +0000
> > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> >
> > > Sorry, I don't understand: what is the procedure for testing this
> > > patch?
> >
> > - apply the patch
> > - rebuild Emacs
> > - remove the problematic file (but not necessarily any others)
> > - rerun your recipe
>
> Thanks.  After patching and rebuilding Emacs, compilation of
> cc-engine.el fails with these error messages:
>
>   Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *)

Sorry, thinko. Better patch attached.

Pip

[-- Attachment #2: bug46824-002.diff --]
[-- Type: text/x-patch, Size: 1058 bytes --]

diff --git a/src/comp.c b/src/comp.c
index b24fcb5bf5a99..7df8c6a93e1c6 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -1946,6 +1946,11 @@ emit_setjmp (gcc_jit_rvalue *buf)
   return gcc_jit_context_new_call (comp.ctxt, NULL, f, 1, args);
 #else
   /* _setjmp (buf, __builtin_frame_address (0)) */
+  gcc_jit_param *params[] =
+  {
+    gcc_jit_context_new_param (comp.ctxt, NULL, comp.void_ptr_type, "buf"),
+    gcc_jit_context_new_param (comp.ctxt, NULL, comp.void_ptr_type, "frame"),
+  };
   gcc_jit_rvalue *args[2];
 
   args[0] =
@@ -1959,8 +1964,14 @@ emit_setjmp (gcc_jit_rvalue *buf)
 					    "__builtin_frame_address"),
       1, args);
   args[0] = buf;
-  return emit_call (intern_c_string (STR (SETJMP_NAME)), comp.int_type, 2, args,
-                    false);
+  gcc_jit_function *f =
+    gcc_jit_context_new_function (comp.ctxt, NULL,
+				  GCC_JIT_FUNCTION_IMPORTED,
+				  comp.int_type, STR (SETJMP_NAME),
+				  ARRAYELTS (params), params,
+				  false);
+
+  return gcc_jit_context_new_call (comp.ctxt, NULL, f, 2, args);
 #endif
 }
 

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sun, 14 Mar 2021 11:44:14 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> >   Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *)
> 
> Sorry, thinko. Better patch attached.

Thanks, the crash is gone, it seems.  (The problems with the truncated
backtrace are still there, though.)

So I think this should be pushed, with the corresponding bump of the
ABI version.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

On Sun, Mar 14, 2021 at 12:07 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Sun, 14 Mar 2021 11:44:14 +0000
> > Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> >
> > >   Warning (comp): D:\usr\bin\libgccjit-0.dll: error: gcc_jit_context_new_call: mismatching types for argument 2 of function "_setjmp": assignment to param ret (type: unsigned int) from __builtin_frame_address ((unsigned int)0) (type: void *)
> >
> > Sorry, thinko. Better patch attached.
>
> Thanks, the crash is gone, it seems.  (The problems with the truncated
> backtrace are still there, though.)
>
> So I think this should be pushed, with the corresponding bump of the
> ABI version.

I'll prepare a version which actually changes the ABI and bumps the ABI version.

Pip

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Pip Cet]

[-- Attachment #1: Type: text/plain, Size: 545 bytes --]

On Sun, Mar 14, 2021 at 12:09 PM Pip Cet <pipcet@gmail.com> wrote:
> On Sun, Mar 14, 2021 at 12:07 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > So I think this should be pushed, with the corresponding bump of the
> > ABI version.
>
> I'll prepare a version which actually changes the ABI and bumps the ABI version.

Still don't have a Windows machine (and I gave up on setting up a Wine
environment for Emacs testing), so can someone check this compiles on
WINDOWSNT and avoids the crash?

Any nitpicks would, as always, be appreciated, too!

Pip

[-- Attachment #2: 0001-Don-t-call-setjmp-through-a-function-pointer-on-Wind.patch --]
[-- Type: text/x-patch, Size: 2587 bytes --]

From 980929876c9fa6db046eef021854be20a8d49875 Mon Sep 17 00:00:00 2001
From: Pip Cet <pipcet@gmail.com>
Date: Sun, 14 Mar 2021 12:13:40 +0000
Subject: [PATCH] Don't call setjmp through a function pointer on Windows
 (bug#47067)

* src/comp.c (ABI_VERSION): Bump.
(emit_setjmp): Call setjmp directly.
(declare_runtime_imported_funcs): Remove setjmp.
(helper_link_table): Remove entry for setjmp.
---
 src/comp.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/src/comp.c b/src/comp.c
index 2ed893cbe0404..7d10f1f2b6926 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -429,7 +429,7 @@ load_gccjit_if_necessary (bool mandatory)
 
 \f
 /* Increase this number to force a new Vcomp_abi_hash to be generated.  */
-#define ABI_VERSION "3"
+#define ABI_VERSION "4"
 
 /* Length of the hashes used for eln file naming.  */
 #define HASH_LENGTH 8
@@ -654,9 +654,6 @@ #define NUM_CAST_TYPES 15
     helper_PSEUDOVECTOR_TYPEP_XUNTAG,
     pure_write_error,
     push_handler,
-#ifdef WINDOWSNT
-    SETJMP_NAME,
-#endif
     record_unwind_protect_excursion,
     helper_unbind_n,
     helper_save_restriction,
@@ -1972,6 +1969,11 @@ emit_setjmp (gcc_jit_rvalue *buf)
   return gcc_jit_context_new_call (comp.ctxt, NULL, f, 1, args);
 #else
   /* _setjmp (buf, __builtin_frame_address (0)) */
+  gcc_jit_param *params[] =
+  {
+    gcc_jit_context_new_param (comp.ctxt, NULL, comp.void_ptr_type, "buf"),
+    gcc_jit_context_new_param (comp.ctxt, NULL, comp.void_ptr_type, "frame"),
+  };
   gcc_jit_rvalue *args[2];
 
   args[0] =
@@ -1985,8 +1987,14 @@ emit_setjmp (gcc_jit_rvalue *buf)
 					    "__builtin_frame_address"),
       1, args);
   args[0] = buf;
-  return emit_call (intern_c_string (STR (SETJMP_NAME)), comp.int_type, 2, args,
-                    false);
+  gcc_jit_function *f =
+    gcc_jit_context_new_function (comp.ctxt, NULL,
+				  GCC_JIT_FUNCTION_IMPORTED,
+				  comp.int_type, STR (SETJMP_NAME),
+				  ARRAYELTS (params), params,
+				  false);
+
+  return gcc_jit_context_new_call (comp.ctxt, NULL, f, 2, args);
 #endif
 }
 
@@ -2701,12 +2709,6 @@ #define ADD_IMPORTED(f_name, ret_type, nargs, args)			       \
   args[1] = comp.int_type;
   ADD_IMPORTED (push_handler, comp.handler_ptr_type, 2, args);
 
-#ifdef WINDOWSNT
-  args[0] = gcc_jit_type_get_pointer (gcc_jit_struct_as_type (comp.jmp_buf_s));
-  args[1] = comp.void_ptr_type;
-  ADD_IMPORTED (SETJMP_NAME, comp.int_type, 2, args);
-#endif
-
   ADD_IMPORTED (record_unwind_protect_excursion, comp.void_type, 0, NULL);
 
   args[0] = comp.lisp_obj_type;
-- 
2.30.1

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sat, 13 Mar 2021 15:45:52 +0000
> Cc: 47067@debbugs.gnu.org, Andrea Corallo <akrl@sdf.org>
> 
> >   (defcustom comp-debug 0
> >     "Debug level for native compilation, a number between 0 and 3.
> >   This is intended for debugging the compiler itself.
> >     0 no debugging output.
> >       This is the recommended value unless you are debugging the compiler itself.
> >     1 emit debug symbols and dump pseudo C code.
> >     2 dump gcc passes and libgccjit log file.
> >     3 dump libgccjit reproducers."
> >
> > The first line is somewhat deceptive, as it doesn't say portions of
> > the backtrace will be anonymous; the backtrace is useful even if not
> > "debugging the compiler itself".  And the description of level 1 says
> > it will also "dump pseudo C code", which I don't think is needed for
> > most of the debugging.  Maybe we need an intermediate level that just
> > leaves the symbols without dumping the pseudo code?
> 
> I think that's an excellent idea.

Andrea, is it possible to have debug level 1 produce only the debug
info within the generated code, and leave the pseudo-code dump for
higher levels?  If you don't have time to implement this yourself, can
you instruct me what changes need to be made?  I'd like to try
recompiling everything with debug level 1 and see if that helps with
the backtraces.

Thanks.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Sun, 14 Mar 2021 12:17:35 +0000
> Cc: Andrea Corallo <akrl@sdf.org>, 47067@debbugs.gnu.org
> 
> Still don't have a Windows machine (and I gave up on setting up a Wine
> environment for Emacs testing), so can someone check this compiles on
> WINDOWSNT and avoids the crash?

Compiles and avoids the crash, thanks.

This also has a positive effect on backtraces: fewer "??" thingies,
and the backtrace is truncated further upward and without the error
message from GDB.

So I think you should install this.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> Date: Sun, 14 Mar 2021 15:31:43 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: akrl@sdf.org, 47067@debbugs.gnu.org
> 
> > Still don't have a Windows machine (and I gave up on setting up a Wine
> > environment for Emacs testing), so can someone check this compiles on
> > WINDOWSNT and avoids the crash?
> 
> Compiles and avoids the crash, thanks.
> 
> This also has a positive effect on backtraces: fewer "??" thingies,
> and the backtrace is truncated further upward and without the error
> message from GDB.

stage2 of comp-tests.el still crashes.  But this time I have a
backtrace (let me know if you need me to demangle the libgccjit part
of the backtrace):

  Thread 1 received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 8020.0x16c4]
  0x074abd71 in libgccjit-0!_Z23ipcp_transform_functionP11cgraph_node ()
   from d:\usr\bin\libgccjit-0.dll
  (gdb) bt
  #0  0x074abd71 in libgccjit-0!_Z23ipcp_transform_functionP11cgraph_node ()
   from d:\usr\bin\libgccjit-0.dll
  #1  0x0759b7c5 in libgccjit-0!_Z26execute_all_ipa_transformsb ()
   from d:\usr\bin\libgccjit-0.dll
  #2  0x072d76ea in libgccjit-0!_ZN11cgraph_node6expandEv ()
   from d:\usr\bin\libgccjit-0.dll
  #3  0x072d86c9 in libgccjit-0!_ZN12symbol_table15output_weakrefsEv ()
   from d:\usr\bin\libgccjit-0.dll
  #4  0x072da6f1 in libgccjit-0!_ZN12symbol_table25finalize_compilation_unitEv
    () from d:\usr\bin\libgccjit-0.dll
  #5  0x0765c551 in libgccjit-0!_ZN5timer3popE12timevar_id_t ()
   from d:\usr\bin\libgccjit-0.dll
  #6  0x07fc9e4c in libgccjit-0!_ZN6toplev4mainEiPPc ()
   from d:\usr\bin\libgccjit-0.dll
  #7  0x072478ca in libgccjit-0!_ZN3gcc3jit8playback7context7compileEv ()
   from d:\usr\bin\libgccjit-0.dll
  #8  0x0723b9b8 in libgccjit-0!_ZN3gcc3jit9recording7context15compile_to_fileE19gcc_jit_output_kindPKc () from d:\usr\bin\libgccjit-0.dll
  #9  0x0722e29e in libgccjit-0!gcc_jit_context_compile_to_file ()
   from d:\usr\bin\libgccjit-0.dll
  #10 0x012de6fe in Fcomp__compile_ctxt_to_file (
    filename=XIL(0x800000000968c910)) at comp.c:4502
  #11 0x01260916 in funcall_subr (subr=0x17340c0 <Scomp__compile_ctxt_to_file>,
    numargs=1, args=0x82b3f8) at eval.c:3095
  #12 0x0126035f in Ffuncall (nargs=2, args=0x82b3f0) at eval.c:3016
  #13 0x0ee85513 in F636f6d702d636f6d70696c652d637478742d746f2d66696c65_comp_compile_ctxt_to_file_0 ()
   from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
  #14 0x01260916 in funcall_subr (subr=0xc5c08d0, numargs=1, args=0x82b608)
    at eval.c:3095
  #15 0x0126035f in Ffuncall (nargs=2, args=0x82b600) at eval.c:3016
  #16 0x0ee856e6 in F636f6d702d66696e616c31_comp_final1_0 ()
   from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
  #17 0x012608fa in funcall_subr (subr=0xc5c9018, numargs=0, args=0x82b930)
    at eval.c:3093
  #18 0x0126035f in Ffuncall (nargs=1, args=0x82b928) at eval.c:3016
  #19 0x0ee86601 in F636f6d702d66696e616c_comp_final_0 ()
   from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
  #20 0x01260916 in funcall_subr (subr=0xc5c9dc8, numargs=1, args=0x82bbc8)
    at eval.c:3095
  #21 0x0126035f in Ffuncall (nargs=2, args=0x82bbc0) at eval.c:3016
  #22 0x0ee8ad5a in F636f6d702d2d6e61746976652d636f6d70696c65_comp__native_compile_0 ()
   from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
  #23 0x0126098d in funcall_subr (subr=0xc533ff0, numargs=3, args=0x82bdc8)
    at eval.c:3100
  #24 0x0126035f in Ffuncall (nargs=4, args=0x82bdc0) at eval.c:3016
  #25 0x0ee8c050 in F6e61746976652d636f6d70696c65_native_compile_0 ()
   from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
  #26 0x0125de95 in eval_sub (form=XIL(0xc000000006c9c180)) at eval.c:2487
  #27 0x01258978 in Flet (args=XIL(0xc000000006d68130)) at eval.c:1027
  #28 0x0125da4f in eval_sub (form=XIL(0xc000000006d680f0)) at eval.c:2437
  #29 0x01256014 in Fprogn (body=XIL(0)) at eval.c:471
  #30 0x01258672 in FletX (args=XIL(0xc000000006d68090)) at eval.c:989
  #31 0x0125da4f in eval_sub (form=XIL(0xc000000006d68050)) at eval.c:2437
  #32 0x01256014 in Fprogn (body=XIL(0)) at eval.c:471
  #33 0x01258672 in FletX (args=XIL(0xc000000006d67fa0)) at eval.c:989
  #34 0x0125da4f in eval_sub (form=XIL(0xc000000006d67f90)) at eval.c:2437
  #35 0x01256014 in Fprogn (body=XIL(0)) at eval.c:471
  #36 0x01261a30 in funcall_lambda (fun=XIL(0xc000000006f79830), nargs=0,
    arg_vector=0x82c828) at eval.c:3297
  #37 0x01260598 in Ffuncall (nargs=1, args=0x82c820) at eval.c:3036
  #38 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006ce8d20),
    vector=XIL(0xa000000006790318), maxdepth=make_fixnum(10),
    args_template=make_fixnum(257), nargs=1, args=0x82ced8) at bytecode.c:632
  #39 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa000000006790420),
    syms_left=make_fixnum(257), nargs=1, args=0x82ced0) at eval.c:3144
  #40 0x0126144c in funcall_lambda (fun=XIL(0xa000000006790420), nargs=1,
    arg_vector=0x82ced0) at eval.c:3225
  #41 0x012603fa in Ffuncall (nargs=2, args=0x82cec8) at eval.c:3023
  #42 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006ce8dc0),
    vector=XIL(0xa000000006790610), maxdepth=make_fixnum(11),
    args_template=make_fixnum(257), nargs=1, args=0x82d530) at bytecode.c:632
  #43 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa0000000067906f0),
    syms_left=make_fixnum(257), nargs=1, args=0x82d528) at eval.c:3144
  #44 0x0126144c in funcall_lambda (fun=XIL(0xa0000000067906f0), nargs=1,
    arg_vector=0x82d528) at eval.c:3225
  #45 0x012603fa in Ffuncall (nargs=2, args=0x82d520) at eval.c:3023
  #46 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006cec680),
    vector=XIL(0xa000000006886b08), maxdepth=make_fixnum(11),
    args_template=make_fixnum(771), nargs=3, args=0x82dba8) at bytecode.c:632
  #47 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa000000006886ba8),
    syms_left=make_fixnum(771), nargs=3, args=0x82db90) at eval.c:3144
  #48 0x0126144c in funcall_lambda (fun=XIL(0xa000000006886ba8), nargs=3,
    arg_vector=0x82db90) at eval.c:3225
  #49 0x012603fa in Ffuncall (nargs=4, args=0x82db88) at eval.c:3023
  #50 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006cec6e0),
    vector=XIL(0xa000000006886c90), maxdepth=make_fixnum(13),
    args_template=make_fixnum(770), nargs=3, args=0x82e1e8) at bytecode.c:632
  #51 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa000000006886d68),
    syms_left=make_fixnum(770), nargs=3, args=0x82e1d0) at eval.c:3144
  #52 0x0126144c in funcall_lambda (fun=XIL(0xa000000006886d68), nargs=3,
    arg_vector=0x82e1d0) at eval.c:3225
  #53 0x012603fa in Ffuncall (nargs=4, args=0x82e1c8) at eval.c:3023
  #54 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006cefe10),
    vector=XIL(0xa00000000688acc0), maxdepth=make_fixnum(7),
    args_template=make_fixnum(256), nargs=1, args=0x82e7b0) at bytecode.c:632
  #55 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa00000000688acf0),
    syms_left=make_fixnum(256), nargs=1, args=0x82e7a8) at eval.c:3144
  #56 0x0126144c in funcall_lambda (fun=XIL(0xa00000000688acf0), nargs=1,
    arg_vector=0x82e7a8) at eval.c:3225
  #57 0x012603fa in Ffuncall (nargs=2, args=0x82e7a0) at eval.c:3023
  #58 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006ceffe0),
    vector=XIL(0xa00000000688ad90), maxdepth=make_fixnum(6),
    args_template=make_fixnum(256), nargs=1, args=0x82ecd8) at bytecode.c:632
  #59 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa00000000688adf8),
    syms_left=make_fixnum(256), nargs=1, args=0x82ecd0) at eval.c:3144
  #60 0x0126144c in funcall_lambda (fun=XIL(0xa00000000688adf8), nargs=1,
    arg_vector=0x82ecd0) at eval.c:3225
  #61 0x01261095 in apply_lambda (fun=XIL(0xa00000000688adf8),
    args=XIL(0xc00000000970f360), count=11) at eval.c:3169
  #62 0x0125e420 in eval_sub (form=XIL(0xc00000000970f430)) at eval.c:2531
  #63 0x0125d2cb in Feval (form=XIL(0xc00000000970f430), lexical=XIL(0x30))
    at eval.c:2313
  #64 0x06b757b6 in F636f6d6d616e642d6c696e652d31_command_line_1_0 ()
   from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\startup-bbc6ea72-9be7c541.eln
  #65 0x01260916 in funcall_subr (subr=0x5dd3bcc, numargs=1, args=0x82f3e8)
    at eval.c:3095
  #66 0x0126035f in Ffuncall (nargs=2, args=0x82f3e0) at eval.c:3016
  #67 0x06b6af88 in F636f6d6d616e642d6c696e65_command_line_0 ()
   from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\startup-bbc6ea72-9be7c541.eln
  #68 0x012608fa in funcall_subr (subr=0x5de5bec, numargs=0, args=0x82f638)
    at eval.c:3093
  #69 0x0126035f in Ffuncall (nargs=1, args=0x82f630) at eval.c:3016
  #70 0x06b65dd5 in F6e6f726d616c2d746f702d6c6576656c_normal_top_level_0 ()
   from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\startup-bbc6ea72-9be7c541.eln
  #71 0x0125de17 in eval_sub (form=XIL(0xc000000005dcd95c)) at eval.c:2481
  #72 0x0125d2cb in Feval (form=XIL(0xc000000005dcd95c), lexical=XIL(0))
    at eval.c:2313
  #73 0x01164353 in top_level_2 () at keyboard.c:1103
  #74 0x0125a2e9 in internal_condition_case (bfun=0x1164320 <top_level_2>,
    handlers=XIL(0x90), hfun=0x1163ad1 <cmd_error>) at eval.c:1448
  #75 0x011643cd in top_level_1 (ignore=XIL(0)) at keyboard.c:1111
  #76 0x012593cc in internal_catch (tag=XIL(0xee80),
    func=0x1164359 <top_level_1>, arg=XIL(0)) at eval.c:1198
  #77 0x01164225 in command_loop () at keyboard.c:1072
  #78 0x01163561 in recursive_edit_1 () at keyboard.c:720
  #79 0x011637cf in Frecursive_edit () at keyboard.c:789
  #80 0x0115ee6e in main (argc=14, argv=0xa445f8) at emacs.c:2095

  Lisp Backtrace:
  "comp--compile-ctxt-to-file" (0x82b3f8)
  "comp-compile-ctxt-to-file" (0x82b608)
  "comp-final1" (0x82b930)
  "comp-final" (0x82bbc8)
  "comp--native-compile" (0x82bdc8)
  "native-compile" (0x82be88)
  "let" (0x82c118)
  "let*" (0x82c368)
  "let*" (0x82c5b8)
  0x6f79820 Lisp type 6
  "ert--run-test-internal" (0x82ced0)
  "ert-run-test" (0x82d528)
  "ert-run-or-rerun-test" (0x82db90)
  "ert-run-tests" (0x82e1d0)
  "ert-run-tests-batch" (0x82e7a8)
  "ert-run-tests-batch-and-exit" (0x82ecd0)
  "command-line-1" (0x82f3e8)
  "command-line" (0x82f638)
  "normal-top-level" (0x82f728)
  (gdb)

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 10479 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sun, 14 Mar 2021 15:31:43 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: akrl@sdf.org, 47067@debbugs.gnu.org
>> 
>> > Still don't have a Windows machine (and I gave up on setting up a Wine
>> > environment for Emacs testing), so can someone check this compiles on
>> > WINDOWSNT and avoids the crash?
>> 
>> Compiles and avoids the crash, thanks.
>> 
>> This also has a positive effect on backtraces: fewer "??" thingies,
>> and the backtrace is truncated further upward and without the error
>> message from GDB.
>
> stage2 of comp-tests.el still crashes.  But this time I have a
> backtrace (let me know if you need me to demangle the libgccjit part
> of the backtrace):
>
>   Thread 1 received signal SIGSEGV, Segmentation fault.
>   [Switching to Thread 8020.0x16c4]
>   0x074abd71 in libgccjit-0!_Z23ipcp_transform_functionP11cgraph_node ()
>    from d:\usr\bin\libgccjit-0.dll
>   (gdb) bt
>   #0  0x074abd71 in libgccjit-0!_Z23ipcp_transform_functionP11cgraph_node ()
>    from d:\usr\bin\libgccjit-0.dll
>   #1  0x0759b7c5 in libgccjit-0!_Z26execute_all_ipa_transformsb ()
>    from d:\usr\bin\libgccjit-0.dll
>   #2  0x072d76ea in libgccjit-0!_ZN11cgraph_node6expandEv ()
>    from d:\usr\bin\libgccjit-0.dll
>   #3  0x072d86c9 in libgccjit-0!_ZN12symbol_table15output_weakrefsEv ()
>    from d:\usr\bin\libgccjit-0.dll
>   #4  0x072da6f1 in libgccjit-0!_ZN12symbol_table25finalize_compilation_unitEv
>     () from d:\usr\bin\libgccjit-0.dll
>   #5  0x0765c551 in libgccjit-0!_ZN5timer3popE12timevar_id_t ()
>    from d:\usr\bin\libgccjit-0.dll
>   #6  0x07fc9e4c in libgccjit-0!_ZN6toplev4mainEiPPc ()
>    from d:\usr\bin\libgccjit-0.dll
>   #7  0x072478ca in libgccjit-0!_ZN3gcc3jit8playback7context7compileEv ()
>    from d:\usr\bin\libgccjit-0.dll
>   #8  0x0723b9b8 in libgccjit-0!_ZN3gcc3jit9recording7context15compile_to_fileE19gcc_jit_output_kindPKc () from d:\usr\bin\libgccjit-0.dll
>   #9  0x0722e29e in libgccjit-0!gcc_jit_context_compile_to_file ()
>    from d:\usr\bin\libgccjit-0.dll
>   #10 0x012de6fe in Fcomp__compile_ctxt_to_file (
>     filename=XIL(0x800000000968c910)) at comp.c:4502
>   #11 0x01260916 in funcall_subr (subr=0x17340c0 <Scomp__compile_ctxt_to_file>,
>     numargs=1, args=0x82b3f8) at eval.c:3095
>   #12 0x0126035f in Ffuncall (nargs=2, args=0x82b3f0) at eval.c:3016
>   #13 0x0ee85513 in F636f6d702d636f6d70696c652d637478742d746f2d66696c65_comp_compile_ctxt_to_file_0 ()
>    from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
>   #14 0x01260916 in funcall_subr (subr=0xc5c08d0, numargs=1, args=0x82b608)
>     at eval.c:3095
>   #15 0x0126035f in Ffuncall (nargs=2, args=0x82b600) at eval.c:3016
>   #16 0x0ee856e6 in F636f6d702d66696e616c31_comp_final1_0 ()
>    from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
>   #17 0x012608fa in funcall_subr (subr=0xc5c9018, numargs=0, args=0x82b930)
>     at eval.c:3093
>   #18 0x0126035f in Ffuncall (nargs=1, args=0x82b928) at eval.c:3016
>   #19 0x0ee86601 in F636f6d702d66696e616c_comp_final_0 ()
>    from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
>   #20 0x01260916 in funcall_subr (subr=0xc5c9dc8, numargs=1, args=0x82bbc8)
>     at eval.c:3095
>   #21 0x0126035f in Ffuncall (nargs=2, args=0x82bbc0) at eval.c:3016
>   #22 0x0ee8ad5a in F636f6d702d2d6e61746976652d636f6d70696c65_comp__native_compile_0 ()
>    from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
>   #23 0x0126098d in funcall_subr (subr=0xc533ff0, numargs=3, args=0x82bdc8)
>     at eval.c:3100
>   #24 0x0126035f in Ffuncall (nargs=4, args=0x82bdc0) at eval.c:3016
>   #25 0x0ee8c050 in F6e61746976652d636f6d70696c65_native_compile_0 ()
>    from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\stage1-oKpQqd-5af2e625-ed123e94.eln
>   #26 0x0125de95 in eval_sub (form=XIL(0xc000000006c9c180)) at eval.c:2487
>   #27 0x01258978 in Flet (args=XIL(0xc000000006d68130)) at eval.c:1027
>   #28 0x0125da4f in eval_sub (form=XIL(0xc000000006d680f0)) at eval.c:2437
>   #29 0x01256014 in Fprogn (body=XIL(0)) at eval.c:471
>   #30 0x01258672 in FletX (args=XIL(0xc000000006d68090)) at eval.c:989
>   #31 0x0125da4f in eval_sub (form=XIL(0xc000000006d68050)) at eval.c:2437
>   #32 0x01256014 in Fprogn (body=XIL(0)) at eval.c:471
>   #33 0x01258672 in FletX (args=XIL(0xc000000006d67fa0)) at eval.c:989
>   #34 0x0125da4f in eval_sub (form=XIL(0xc000000006d67f90)) at eval.c:2437
>   #35 0x01256014 in Fprogn (body=XIL(0)) at eval.c:471
>   #36 0x01261a30 in funcall_lambda (fun=XIL(0xc000000006f79830), nargs=0,
>     arg_vector=0x82c828) at eval.c:3297
>   #37 0x01260598 in Ffuncall (nargs=1, args=0x82c820) at eval.c:3036
>   #38 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006ce8d20),
>     vector=XIL(0xa000000006790318), maxdepth=make_fixnum(10),
>     args_template=make_fixnum(257), nargs=1, args=0x82ced8) at bytecode.c:632
>   #39 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa000000006790420),
>     syms_left=make_fixnum(257), nargs=1, args=0x82ced0) at eval.c:3144
>   #40 0x0126144c in funcall_lambda (fun=XIL(0xa000000006790420), nargs=1,
>     arg_vector=0x82ced0) at eval.c:3225
>   #41 0x012603fa in Ffuncall (nargs=2, args=0x82cec8) at eval.c:3023
>   #42 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006ce8dc0),
>     vector=XIL(0xa000000006790610), maxdepth=make_fixnum(11),
>     args_template=make_fixnum(257), nargs=1, args=0x82d530) at bytecode.c:632
>   #43 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa0000000067906f0),
>     syms_left=make_fixnum(257), nargs=1, args=0x82d528) at eval.c:3144
>   #44 0x0126144c in funcall_lambda (fun=XIL(0xa0000000067906f0), nargs=1,
>     arg_vector=0x82d528) at eval.c:3225
>   #45 0x012603fa in Ffuncall (nargs=2, args=0x82d520) at eval.c:3023
>   #46 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006cec680),
>     vector=XIL(0xa000000006886b08), maxdepth=make_fixnum(11),
>     args_template=make_fixnum(771), nargs=3, args=0x82dba8) at bytecode.c:632
>   #47 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa000000006886ba8),
>     syms_left=make_fixnum(771), nargs=3, args=0x82db90) at eval.c:3144
>   #48 0x0126144c in funcall_lambda (fun=XIL(0xa000000006886ba8), nargs=3,
>     arg_vector=0x82db90) at eval.c:3225
>   #49 0x012603fa in Ffuncall (nargs=4, args=0x82db88) at eval.c:3023
>   #50 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006cec6e0),
>     vector=XIL(0xa000000006886c90), maxdepth=make_fixnum(13),
>     args_template=make_fixnum(770), nargs=3, args=0x82e1e8) at bytecode.c:632
>   #51 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa000000006886d68),
>     syms_left=make_fixnum(770), nargs=3, args=0x82e1d0) at eval.c:3144
>   #52 0x0126144c in funcall_lambda (fun=XIL(0xa000000006886d68), nargs=3,
>     arg_vector=0x82e1d0) at eval.c:3225
>   #53 0x012603fa in Ffuncall (nargs=4, args=0x82e1c8) at eval.c:3023
>   #54 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006cefe10),
>     vector=XIL(0xa00000000688acc0), maxdepth=make_fixnum(7),
>     args_template=make_fixnum(256), nargs=1, args=0x82e7b0) at bytecode.c:632
>   #55 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa00000000688acf0),
>     syms_left=make_fixnum(256), nargs=1, args=0x82e7a8) at eval.c:3144
>   #56 0x0126144c in funcall_lambda (fun=XIL(0xa00000000688acf0), nargs=1,
>     arg_vector=0x82e7a8) at eval.c:3225
>   #57 0x012603fa in Ffuncall (nargs=2, args=0x82e7a0) at eval.c:3023
>   #58 0x012cbfbc in exec_byte_code (bytestr=XIL(0x8000000006ceffe0),
>     vector=XIL(0xa00000000688ad90), maxdepth=make_fixnum(6),
>     args_template=make_fixnum(256), nargs=1, args=0x82ecd8) at bytecode.c:632
>   #59 0x01260ecf in fetch_and_exec_byte_code (fun=XIL(0xa00000000688adf8),
>     syms_left=make_fixnum(256), nargs=1, args=0x82ecd0) at eval.c:3144
>   #60 0x0126144c in funcall_lambda (fun=XIL(0xa00000000688adf8), nargs=1,
>     arg_vector=0x82ecd0) at eval.c:3225
>   #61 0x01261095 in apply_lambda (fun=XIL(0xa00000000688adf8),
>     args=XIL(0xc00000000970f360), count=11) at eval.c:3169
>   #62 0x0125e420 in eval_sub (form=XIL(0xc00000000970f430)) at eval.c:2531
>   #63 0x0125d2cb in Feval (form=XIL(0xc00000000970f430), lexical=XIL(0x30))
>     at eval.c:2313
>   #64 0x06b757b6 in F636f6d6d616e642d6c696e652d31_command_line_1_0 ()
>    from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\startup-bbc6ea72-9be7c541.eln
>   #65 0x01260916 in funcall_subr (subr=0x5dd3bcc, numargs=1, args=0x82f3e8)
>     at eval.c:3095
>   #66 0x0126035f in Ffuncall (nargs=2, args=0x82f3e0) at eval.c:3016
>   #67 0x06b6af88 in F636f6d6d616e642d6c696e65_command_line_0 ()
>    from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\startup-bbc6ea72-9be7c541.eln
>   #68 0x012608fa in funcall_subr (subr=0x5de5bec, numargs=0, args=0x82f638)
>     at eval.c:3093
>   #69 0x0126035f in Ffuncall (nargs=1, args=0x82f630) at eval.c:3016
>   #70 0x06b65dd5 in F6e6f726d616c2d746f702d6c6576656c_normal_top_level_0 ()
>    from d:\gnu\git\emacs\native-comp\native-lisp\28.0.50-88a242e7\startup-bbc6ea72-9be7c541.eln
>   #71 0x0125de17 in eval_sub (form=XIL(0xc000000005dcd95c)) at eval.c:2481
>   #72 0x0125d2cb in Feval (form=XIL(0xc000000005dcd95c), lexical=XIL(0))
>     at eval.c:2313
>   #73 0x01164353 in top_level_2 () at keyboard.c:1103
>   #74 0x0125a2e9 in internal_condition_case (bfun=0x1164320 <top_level_2>,
>     handlers=XIL(0x90), hfun=0x1163ad1 <cmd_error>) at eval.c:1448
>   #75 0x011643cd in top_level_1 (ignore=XIL(0)) at keyboard.c:1111
>   #76 0x012593cc in internal_catch (tag=XIL(0xee80),
>     func=0x1164359 <top_level_1>, arg=XIL(0)) at eval.c:1198
>   #77 0x01164225 in command_loop () at keyboard.c:1072
>   #78 0x01163561 in recursive_edit_1 () at keyboard.c:720
>   #79 0x011637cf in Frecursive_edit () at keyboard.c:789
>   #80 0x0115ee6e in main (argc=14, argv=0xa445f8) at emacs.c:2095

Hi Eli,

this really look like a libgccjit bug.  I think would be interesting if
you have time to:

1- produce a libgccjit reproducer (applying the attached patch before
running the test) and post it so we can check if we can reproduce and
investigate this on other setups.

2- run all the comp-tests.el tests excluding `comp-tests-bootstrap' just
to make sure we are fine with all the reduced testcases.

Thanks!

  Andrea


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: repro-bootstrap.patch --]
[-- Type: text/x-diff, Size: 763 bytes --]

diff --git a/test/src/comp-tests.el b/test/src/comp-tests.el
index f60e4ab049..3813728687 100644
--- a/test/src/comp-tests.el
+++ b/test/src/comp-tests.el
@@ -75,8 +75,9 @@ comp-tests-bootstrap
       (load comp1-eln nil nil t t)
       (should (subr-native-elisp-p (symbol-function 'native-compile)))
       (message "Compiling stage2...")
-      (let ((t0 (current-time))
-            (comp2-eln (native-compile comp2-src)))
+      (let* ((comp-libgccjit-reproducer t)
+             (t0 (current-time))
+             (comp2-eln (native-compile comp2-src)))
         (message "Done in %d secs" (float-time (time-since t0)))
         (message "Comparing %s %s" comp1-eln comp2-eln)
         (should (= (call-process "cmp" nil nil nil comp1-eln comp2-eln) 0))))))

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1982 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pip Cet <pipcet@gmail.com>
>> Date: Sat, 13 Mar 2021 15:45:52 +0000
>> Cc: 47067@debbugs.gnu.org, Andrea Corallo <akrl@sdf.org>
>> 
>> >   (defcustom comp-debug 0
>> >     "Debug level for native compilation, a number between 0 and 3.
>> >   This is intended for debugging the compiler itself.
>> >     0 no debugging output.
>> >       This is the recommended value unless you are debugging the compiler itself.
>> >     1 emit debug symbols and dump pseudo C code.
>> >     2 dump gcc passes and libgccjit log file.
>> >     3 dump libgccjit reproducers."
>> >
>> > The first line is somewhat deceptive, as it doesn't say portions of
>> > the backtrace will be anonymous; the backtrace is useful even if not
>> > "debugging the compiler itself".  And the description of level 1 says
>> > it will also "dump pseudo C code", which I don't think is needed for
>> > most of the debugging.  Maybe we need an intermediate level that just
>> > leaves the symbols without dumping the pseudo code?
>> 
>> I think that's an excellent idea.
>
> Andrea, is it possible to have debug level 1 produce only the debug
> info within the generated code, and leave the pseudo-code dump for
> higher levels?  If you don't have time to implement this yourself, can
> you instruct me what changes need to be made?  I'd like to try
> recompiling everything with debug level 1 and see if that helps with
> the backtraces.

Sure that's very easy (the attached should do the job), is this
something you'd like to have a local modification or a change in the
branch?

If the case is the second before I'd just like to understand why the
Windows toolchain needs debug symbols for function names and if this is
a bug or the expected behavior.

IME this was never the case on systems I've worked on and this is why I
though having debug symbols and dump pseudo C code together in the same
debug level made sense, essentially to allow stepping.

Thanks

  Andrea


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: no-cdump.patch --]
[-- Type: text/x-diff, Size: 382 bytes --]

diff --git a/src/comp.c b/src/comp.c
index a79ee4ad87..f295106576 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -4484,7 +4484,7 @@ DEFUN ("comp--compile-ctxt-to-file", Fcomp__compile_ctxt_to_file,
 
   add_driver_options ();
 
-  if (comp.debug)
+  if (comp.debug > 1)
       gcc_jit_context_dump_to_file (comp.ctxt,
 				    format_string ("%s.c", SSDATA (ebase_name)),
 				    1);

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: Eli Zaretskii <eliz@gnu.org>, 47067@debbugs.gnu.org
>>> Date: Sun, 14 Mar 2021 07:19:17 +0000
>>>
>>> Okay with a trunk gdb I'm able to debug...
>>
>> Yes, 8.2 is quite an old version.
>>
>>> Here is my backtrace for the suggested reproducer on my Emacs configured
>>> with --with-wide-int --enable-checking=yes,glyphs and compiled with the
>>> default `comp-debug' 0.
>>
>> Thanks.  However, your build is optimized, whereas mine isn't.  This
>> has a significant effect on the backtrace (static functions are
>> inlined and don't appear in the backtrace).  Please reconfigure with
>> CFLAGS='-O0 -g3' (and the rest of the switches as you used them for
>> this build), and show the backtrace after rebuilding.
>
> Okay will rebuild and report this evening (have to leave the PC now).

Not sure this is still relevant but anyway here it is adding the
suggested CFLAGS='-O0 -g3':

========
#0  Fskip_chars_backward (string=-9223372036709943776, lim=0) at syntax.c:1610
#1  0xb3f3224b in F632d6261636b776172642d737773_c_backward_sws_0 () from /home/andcor03/.emacs.d/eln-cache/28.0.50-f36e1a77/cc-engine-ccfcb170-1b345b21.eln
#2  0x0825114a in funcall_subr (subr=0x8af5b40, numargs=0, args=0xbfff94e0) at eval.c:3082
#3  0x08250c5b in Ffuncall (nargs=1, args=0xbfff94d8) at eval.c:3009
#4  0xb40f03f7 in F632d666c2d6465636c2d656e64_c_fl_decl_end_0 () from /home/andcor03/.emacs.d/eln-cache/28.0.50-f36e1a77/cc-mode-515a98a5-c7761ade.eln
#5  0x082520a2 in funcall_lambda (fun=-6917529027497101736, nargs=1, arg_vector=0x0) at eval.c:3292
#6  0x08250cbd in Ffuncall (nargs=2, args=0xbfff9680) at eval.c:3013
#7  0xb40f1353 in F632d636f6e746578742d657870616e642d666c2d726567696f6e_c_context_expand_fl_region_0 ()
   from /home/andcor03/.emacs.d/eln-cache/28.0.50-f36e1a77/cc-mode-515a98a5-c7761ade.eln
#8  0x082520a2 in funcall_lambda (fun=-6917529027497101608, nargs=2, arg_vector=0x0) at eval.c:3292
#9  0x08250cbd in Ffuncall (nargs=3, args=0xbfff9818) at eval.c:3013
#10 0xb40f1534 in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_62 () from /home/andcor03/.emacs.d/eln-cache/28.0.50-f36e1a77/cc-mode-515a98a5-c7761ade.eln
#11 0x082520a2 in funcall_lambda (fun=-6917529027496760720, nargs=1, arg_vector=0x0) at eval.c:3292
#12 0x08250cbd in Ffuncall (nargs=2, args=0xbfff9990) at eval.c:3013
#13 0x0824ffa5 in call1 (fn=-6917529027496760720, arg1=2366592) at eval.c:2869
#14 0x082648c1 in mapcar1 (leni=1, vals=0x0, fn=-6917529027496760720, seq=-4611686018282341936) at fns.c:2793
#15 0x08264fa2 in Fmapc (function=-6917529027496760720, sequence=-4611686018282341936) at fns.c:2866
#16 0x0825118b in funcall_subr (subr=0x87042e0 <Smapc>, numargs=2, args=0xbfff9c20) at eval.c:3086
#17 0x08250c5b in Ffuncall (nargs=3, args=0xbfff9c18) at eval.c:3009
#18 0xb40f1709 in F632d6265666f72652d636f6e746578742d666c2d657870616e642d726567696f6e_c_before_context_fl_expand_region_0 ()
   from /home/andcor03/.emacs.d/eln-cache/28.0.50-f36e1a77/cc-mode-515a98a5-c7761ade.eln
#19 0x082520a2 in funcall_lambda (fun=-6917529027497101544, nargs=2, arg_vector=0x0) at eval.c:3292
#20 0x08250cbd in Ffuncall (nargs=3, args=0xbfff9e38) at eval.c:3013
#21 0xb40f1a2f in F632d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_c_font_lock_fontify_region_0 ()
   from /home/andcor03/.emacs.d/eln-cache/28.0.50-f36e1a77/cc-mode-515a98a5-c7761ade.eln
#22 0x082520a2 in funcall_lambda (fun=-6917529027497101480, nargs=3, arg_vector=0x0) at eval.c:3292
#23 0x08250cbd in Ffuncall (nargs=4, args=0xbfffa010) at eval.c:3013
#24 0xb4a4807a in F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/font-lock-895216f6-4cc69ac6.eln
#25 0x082511c8 in funcall_subr (subr=0xb53b1f14, numargs=2, args=0xbfffa1c8) at eval.c:3089
#26 0x08250c5b in Ffuncall (nargs=3, args=0xbfffa1c0) at eval.c:3009
#27 0x082b4be2 in exec_byte_code (bytestr=-9223372033810787668, vector=-6917529027493170984, maxdepth=4611686018427387914, args_template=4611686018427388161, nargs=1,
    args=0xbfffa8d8) at bytecode.c:632
#28 0x082515c8 in fetch_and_exec_byte_code (fun=-6917529027493170912, syms_left=4611686018427388161, nargs=1, args=0xbfffa8d0) at eval.c:3133
#29 0x08251ade in funcall_lambda (fun=-6917529027493170912, nargs=1, arg_vector=0xbfffa8d0) at eval.c:3214
#30 0x08250cbd in Ffuncall (nargs=2, args=0xbfffa8c8) at eval.c:3013
#31 0x0824f8eb in run_hook_wrapped_funcall (nargs=2, args=0xbfffa8c8) at eval.c:2746
#32 0x0824fd80 in run_hook_with_args (nargs=2, args=0xbfffa8c8, funcall=0x824f8a7 <run_hook_wrapped_funcall>) at eval.c:2827-------------------------------------------------
#33 0x0824f936 in Frun_hook_wrapped (nargs=2, args=0xbfffa8c8) at eval.c:2761
#34 0xb4e83640 in F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/jit-lock-8a988e43-ae750d1b.eln
#35 0x0825118b in funcall_subr (subr=0xb54731e8, numargs=2, args=0xbfffab10) at eval.c:3086
#36 0x08250c5b in Ffuncall (nargs=3, args=0xbfffab08) at eval.c:3009
#37 0xb4e8440b in F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/jit-lock-8a988e43-ae750d1b.eln
#38 0x0825118b in funcall_subr (subr=0xb548a2d4, numargs=2, args=0xbfffad60) at eval.c:3086
#39 0x08250c5b in Ffuncall (nargs=3, args=0xbfffad58) at eval.c:3009
#40 0xb4e83082 in F6a69742d6c6f636b2d66756e6374696f6e_jit_lock_function_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/jit-lock-8a988e43-ae750d1b.eln
#41 0x08251164 in funcall_subr (subr=0xb565a248, numargs=1, args=0xbfffaf48) at eval.c:3084
#42 0x08250c5b in Ffuncall (nargs=2, args=0xbfffaf40) at eval.c:3009
#43 0x0824b6dc in internal_condition_case_n (bfun=0x8250a79 <Ffuncall>, nargs=2, args=0xbfffaf40, handlers=40, hfun=0x8077099 <safe_eval_handler>) at eval.c:1612
#44 0x080772a2 in safe__call (inhibit_quit=false, nargs=2, func=2901362656, ap=0xbfffb004 "") at xdisp.c:2981
#45 0x08077301 in safe_call (nargs=2, func=2901362656) at xdisp.c:2996
#46 0x08077346 in safe_call1 (fn=2901362656, arg=4611686018427419714) at xdisp.c:3007
#47 0x0807aa72 in handle_fontified_prop (it=0xbfffbf90) at xdisp.c:4313
#48 0x08079566 in handle_stop (it=0xbfffbf90) at xdisp.c:3841
#49 0x08082d45 in reseat (it=0xbfffbf90, pos=..., force_p=true) at xdisp.c:7087
#50 0x080789c7 in init_iterator (it=0xbfffbf90, w=0x88b8a68, charpos=31810, bytepos=31810, row=0x0, base_face_id=DEFAULT_FACE_ID) at xdisp.c:3442
#51 0x080a004b in redisplay_window (window=-6917529027497719192, just_this_one_p=false) at xdisp.c:18960
#52 0x08098233 in redisplay_window_0 (window=-6917529027497719192) at xdisp.c:16581
#53 0x0824b1d7 in internal_condition_case_1 (bfun=0x80981e5 <redisplay_window_0>, arg=-6917529027497719192, handlers=-4611686015382928776,
    hfun=0x80981a1 <redisplay_window_error>) at eval.c:1472
#54 0x0809815f in redisplay_windows (window=-6917529027497719192) at xdisp.c:16561
#55 0x08096b75 in redisplay_internal () at xdisp.c:16029
#56 0x08097863 in redisplay_preserve_echo_area (from_where=2) at xdisp.c:16378
#57 0x0805e1de in Fredisplay (force=0) at dispnew.c:6157
#58 0xb50fcd59 in F7369742d666f72_sit_for_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/subr-13adf6a6-656b9117.eln
#59 0x082511c8 in funcall_subr (subr=0xb53e6b88, numargs=1, args=0xbfffdde0) at eval.c:3089
#60 0x08250c5b in Ffuncall (nargs=2, args=0xbfffddd8) at eval.c:3009
#61 0xb4be1cab in F697365617263682d6c617a792d686967686c696768742d6e65772d6c6f6f70_isearch_lazy_highlight_new_loop_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/isearch-693bdd75-2b63c98a.eln
#62 0x0825118b in funcall_subr (subr=0xb564b4fc, numargs=0, args=0xbfffe060) at eval.c:3086
#63 0x08250c5b in Ffuncall (nargs=1, args=0xbfffe058) at eval.c:3009
#64 0xb4bcad02 in F697365617263682d757064617465_isearch_update_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/isearch-693bdd75-2b63c98a.eln
#65 0x0825114a in funcall_subr (subr=0xb53d26e0, numargs=0, args=0xbfffe300) at eval.c:3082
#66 0x08250c5b in Ffuncall (nargs=1, args=0xbfffe2f8) at eval.c:3009
#67 0xb4bd943e in F697365617263682d7365617263682d616e642d757064617465_isearch_search_and_update_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/isearch-693bdd75-2b63c98a.eln
#68 0x0825114a in funcall_subr (subr=0xb55d453c, numargs=0, args=0xbfffe4a0) at eval.c:3082
#69 0x08250c5b in Ffuncall (nargs=1, args=0xbfffe498) at eval.c:3009
#70 0xb4bdc01c in F697365617263682d70726f636573732d7365617263682d737472696e67_isearch_process_search_string_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/isearch-693bdd75-2b63c98a.eln
#71 0x0825118b in funcall_subr (subr=0xb55cc904, numargs=2, args=0xbfffe688) at eval.c:3086
#72 0x08250c5b in Ffuncall (nargs=3, args=0xbfffe680) at eval.c:3009
#73 0xb4bdbe01 in F697365617263682d70726f636573732d7365617263682d63686172_isearch_process_search_char_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/isearch-693bdd75-2b63c98a.eln
#74 0x0825118b in funcall_subr (subr=0xb55db120, numargs=2, args=0xbfffe870) at eval.c:3086
#75 0x08250c5b in Ffuncall (nargs=3, args=0xbfffe868) at eval.c:3009
#76 0xb4bdbb93 in F697365617263682d7072696e74696e672d63686172_isearch_printing_char_0 ()
   from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/isearch-693bdd75-2b63c98a.eln
#77 0x0825118b in funcall_subr (subr=0xb5525e5c, numargs=2, args=0xbfffeb70) at eval.c:3086
#78 0x08250c5b in Ffuncall (nargs=3, args=0xbfffeb68) at eval.c:3009
#79 0x08241b04 in Ffuncall_interactively (nargs=3, args=0xbfffeb68) at callint.c:260
#80 0x08251090 in funcall_subr (subr=0x87021e0 <Sfuncall_interactively>, numargs=3, args=0xbfffeb68) at eval.c:3064
#81 0x08250c5b in Ffuncall (nargs=4, args=0xbfffeb60) at eval.c:3009
#82 0x0824f6fa in Fapply (nargs=3, args=0xbfffecb8) at eval.c:2639
#83 0x082421c0 in Fcall_interactively (function=2900100084, record_flag=0, keys=-6917529027494038576) at callint.c:353
#84 0xb4f50b3b in F636f6d6d616e642d65786563757465_command_execute_0 () from /home/andcor03/emacs2/src/../native-lisp/28.0.50-f36e1a77/simple-fab5b0cf-fa0c27ba.eln
#85 0x08251225 in funcall_subr (subr=0xb5376bd4, numargs=1, args=0xbffff058) at eval.c:3092
#86 0x08250c5b in Ffuncall (nargs=2, args=0xbffff050) at eval.c:3009
#87 0x0824ffa5 in call1 (fn=11520, arg1=2900100084) at eval.c:2869
#88 0x0816067a in command_loop_1 () at keyboard.c:1466
#89 0x0824b0fa in internal_condition_case (bfun=0x815faa0 <command_loop_1>, handlers=120, hfun=0x815edc2 <cmd_error>) at eval.c:1448
#90 0x0815f54e in command_loop_2 (ignore=0) at keyboard.c:1094
#91 0x0824a2fd in internal_catch (tag=42000, func=0x815f51b <command_loop_2>, arg=0) at eval.c:1198
#92 0x0815f4d2 in command_loop () at keyboard.c:1073
#93 0x0815e869 in recursive_edit_1 () at keyboard.c:720
#94 0x0815eace in Frecursive_edit () at keyboard.c:789
#95 0x0815a674 in main (argc=2, argv=0xbffff654) at emacs.c:2095
========

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Pip Cet <pipcet@gmail.com>, 47067@debbugs.gnu.org
> Date: Sun, 14 Mar 2021 19:00:26 +0000
> 
> > Andrea, is it possible to have debug level 1 produce only the debug
> > info within the generated code, and leave the pseudo-code dump for
> > higher levels?  If you don't have time to implement this yourself, can
> > you instruct me what changes need to be made?  I'd like to try
> > recompiling everything with debug level 1 and see if that helps with
> > the backtraces.
> 
> Sure that's very easy (the attached should do the job), is this
> something you'd like to have a local modification or a change in the
> branch?

I don't know yet.

> If the case is the second before I'd just like to understand why the
> Windows toolchain needs debug symbols for function names and if this is
> a bug or the expected behavior.
> 
> IME this was never the case on systems I've worked on and this is why I
> though having debug symbols and dump pseudo C code together in the same
> debug level made sense, essentially to allow stepping.

I'm not yet sure there's something special on MS-Windows in this
regard.  After Pip's last patch I see much fewer "??" in the
backtraces, and when they appear, there's something else wrong with
the backtrace as a whole, for example it ends prematurely, before it
gets all the way to 'main'.  I don't yet understand why that happens,
but it doesn't happen right away, the first backtraces I see are
completely normal.  My current observation is that the backtraces
begin to show abnormalities as soon as Isearch calls sit-for (to show
the lazy-highlight of matches).  Before that, I see no "??"
whatsoever.

Let me try the patch you sent and see what I think about its effects.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
>> Date: Sun, 14 Mar 2021 18:47:12 +0000
>>
>> this really look like a libgccjit bug.  I think would be interesting if
>> you have time to:
>>
>> 1- produce a libgccjit reproducer (applying the attached patch before
>> running the test) and post it so we can check if we can reproduce and
>> investigate this on other setups.
>>
>> 2- run all the comp-tests.el tests excluding `comp-tests-bootstrap' just
>> to make sure we are fine with all the reduced testcases.
>
> The reproducer for stage2 crash is attached.  It doesn't compile,
> because the code is really invalid C.

That's sad :(

> First, there's a string which
> provides the absolute file name of libgccjit, and it uses backslashes
> (as the original file name does), but the backslashes aren't doubled
> when the reproducer is generated, so GCC complains about unknown
> escapes.  I fixed that manually, but it still fails to compile because
> there are literal "nil"s in several places where (I think) strings
> were supposed to be.

I've the impression it's "just" failing to quote strings...  And on GCC
pre 11 we rely on strings to store immediates in the eln (so we have a
ton of).


I tried to produce a reproducer for bootstrap stage2 on my system 64bit
using the libgccjit I've (11) but forcing back the use of the strings
and the result is not compilable either.

> And maybe there are more problems, I didn't try
> to pursue this further.  If you'd like me to try to compile a
> cleaned-up reproducer, send it to me and I will try.

I've the impression it might be a *very* time consuming task unless you
have an automated way of doing it.  The outcome might be allowing for
investigating a GCC bug that most likely is not reproducible on a
contemporary GCC and that we observe only in a test for now... So I'd be
a little reluctant.

But I think would be nice to try to understand if the issue is "just"
the quoting or what's the state and report it as a GCC bug.  I'll try to
have a look (not really sure when, unless we consider this urgent).

> All the other tests in comp-tests.el pass.

That's something...

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Pip Cet <pipcet@gmail.com>, 47067@debbugs.gnu.org
>> Date: Sun, 14 Mar 2021 19:00:26 +0000
>> 
>> > Andrea, is it possible to have debug level 1 produce only the debug
>> > info within the generated code, and leave the pseudo-code dump for
>> > higher levels?  If you don't have time to implement this yourself, can
>> > you instruct me what changes need to be made?  I'd like to try
>> > recompiling everything with debug level 1 and see if that helps with
>> > the backtraces.
>> 
>> Sure that's very easy (the attached should do the job), is this
>> something you'd like to have a local modification or a change in the
>> branch?
>
> I don't know yet.
>
>> If the case is the second before I'd just like to understand why the
>> Windows toolchain needs debug symbols for function names and if this is
>> a bug or the expected behavior.
>> 
>> IME this was never the case on systems I've worked on and this is why I
>> though having debug symbols and dump pseudo C code together in the same
>> debug level made sense, essentially to allow stepping.
>
> I'm not yet sure there's something special on MS-Windows in this
> regard.  After Pip's last patch I see much fewer "??" in the
> backtraces, and when they appear, there's something else wrong with
> the backtrace as a whole, for example it ends prematurely, before it
> gets all the way to 'main'.  I don't yet understand why that happens,
> but it doesn't happen right away, the first backtraces I see are
> completely normal.  My current observation is that the backtraces
> begin to show abnormalities as soon as Isearch calls sit-for (to show
> the lazy-highlight of matches).  Before that, I see no "??"
> whatsoever.

Very dumb question, I guess you are (as opposed to me) already using a
farly recent gdb correct?

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
> Date: Sun, 14 Mar 2021 20:36:57 +0000
> 
> Very dumb question, I guess you are (as opposed to me) already using a
> farly recent gdb correct?

GDB 10.1, the latest release.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

I still see some strange phenomena with backtraces under GDB, which
tell me we sometimes clobber saved registers or something.  How else
to explain that one backtrace may be truncated after 20 frames, while
the next backtrace shows all the frames back to 'main'?

What could explain this? another snafu with setjmp?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> I still see some strange phenomena with backtraces under GDB, which
> tell me we sometimes clobber saved registers or something.  How else
> to explain that one backtrace may be truncated after 20 frames, while
> the next backtrace shows all the frames back to 'main'?
>
> What could explain this? another snafu with setjmp?

Dunno, but the only place where we emit setjmps is the one now fixed.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: pipcet@gmail.com, 47067@debbugs.gnu.org
> Date: Mon, 15 Mar 2021 20:17:18 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I still see some strange phenomena with backtraces under GDB, which
> > tell me we sometimes clobber saved registers or something.  How else
> > to explain that one backtrace may be truncated after 20 frames, while
> > the next backtrace shows all the frames back to 'main'?
> >
> > What could explain this? another snafu with setjmp?
> 
> Dunno, but the only place where we emit setjmps is the one now fixed.

Fair enough.  I asked the GDB folks to help me analyze this.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

[...]

>> >> To force the .elc to be loaded one has to bind `load-no-native' to
>> >> non-nil.
>> >
>> > I think if load-file is invoked interactively, and the user actually
>> > types "foo.elc", we need to bind load-no-native non-nil
>> > automatically.  Otherwise users would be surprised, as it goes against
>> > the logic of what we do when the user types "foo.el".
>> 
>> We certanly can do this if this is what we want.  This breaks a little
>> the idea to have the system as much transparent as possible, I went this
>> way cause this was my understanding of what we wanted but I've no strong
>> feeling with that.
>
> I don't think this will break the transparent operation, because
> loading a package non-interactively (as in when the corresponding
> feature is 'require'd by some code) will still load the .eln file.
> Only the following 2 use cases will be affected:
>
>   M-x load-file RET /path/to/FOO.elc RET
>   M-x load-library RET FOO.elc RET
>
> IOW, when the user loads the file/library interactively, and
> explicitly uses the .elc extension, we load the file the user
> specified, not the corresponding .eln file.

Hi Eli,

I've a patch that works modifying `load' to prevent native code being
loaded when a file with .elc suffix is explicitly presented as argument.

I was going to post it but re-reading your message I now wanted to ask:
do we want to act at `load' level or at `load-file' + `load-library'
level?

TIA

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 19 Mar 2021 09:42:33 +0000
> 
> I've a patch that works modifying `load' to prevent native code being
> loaded when a file with .elc suffix is explicitly presented as argument.
> 
> I was going to post it but re-reading your message I now wanted to ask:
> do we want to act at `load' level or at `load-file' + `load-library'
> level?

The former, I think.  This is to keep the original semantics of 'load'
as it existed before the native-compilation: if you say

  (load "foo.el")

Emacs loads the .el file, even if the corresponding .elc file exists.

But if doing that in 'load' causes some trouble, I think we should at
least do this on the level of 'load-file' and 'load-library'.

Thanks.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Fri, 19 Mar 2021 09:42:33 +0000
>> 
>> I've a patch that works modifying `load' to prevent native code being
>> loaded when a file with .elc suffix is explicitly presented as argument.
>> 
>> I was going to post it but re-reading your message I now wanted to ask:
>> do we want to act at `load' level or at `load-file' + `load-library'
>> level?
>
> The former, I think.  This is to keep the original semantics of 'load'
> as it existed before the native-compilation: if you say
>
>   (load "foo.el")
>
> Emacs loads the .el file, even if the corresponding .elc file exists.
>
> But if doing that in 'load' causes some trouble, I think we should at
> least do this on the level of 'load-file' and 'load-library'.

Okay b3ad62f8a3 does that in `load'.

At a quick test it does not create problems, will see.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Fri, 19 Mar 2021 14:05:06 +0000
> 
> > The former, I think.  This is to keep the original semantics of 'load'
> > as it existed before the native-compilation: if you say
> >
> >   (load "foo.el")
> >
> > Emacs loads the .el file, even if the corresponding .elc file exists.
> >
> > But if doing that in 'load' causes some trouble, I think we should at
> > least do this on the level of 'load-file' and 'load-library'.
> 
> Okay b3ad62f8a3 does that in `load'.
> 
> At a quick test it does not create problems, will see.

Thanks.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

A related question: what happens when a .el which was preloaded at
some point is no longer preloaded?

When it was preloaded, we had its .eln file under native/lisp/, but
when it ceases to be preloaded, its name is removed from $shortlisp
(in src/lisp.mk), and we no longer ELC+ELN it during the build.  So we
now have a stale .eln file in native-lisp, and at some future point in
time, when that file is loaded into Emacs, it will be
natively-compiled into ~/.emacs.d/eln-cache/.  At which point we have
2 versions of the .eln file.

Question #1: will Emacs load the up-to-date .eln file in eln-cache, or
could it by any chance load the outdated file in native-lisp?
Question #2: can we somehow remove the stale .eln file in these
situations?

Thanks.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> A related question: what happens when a .el which was preloaded at
> some point is no longer preloaded?
>
> When it was preloaded, we had its .eln file under native/lisp/, but
> when it ceases to be preloaded, its name is removed from $shortlisp
> (in src/lisp.mk), and we no longer ELC+ELN it during the build.  So we
> now have a stale .eln file in native-lisp, and at some future point in
> time, when that file is loaded into Emacs, it will be
> natively-compiled into ~/.emacs.d/eln-cache/.  At which point we have
> 2 versions of the .eln file.
>
> Question #1: will Emacs load the up-to-date .eln file in eln-cache, or
> could it by any chance load the outdated file in native-lisp?

If the .el file was not changed Emacs will load the original .eln in
'native-lisp' without having to recompile it.

If the .el was modified we will recompile it in 'eln-cache' and load it,
when we do this we clean-up the stale .eln present in
`comp-eln-load-path'.  Unfortunatelly for this case we don't do this for
the last directory in `comp-eln-load-path' (as this is tipically
read-only when Emacs is installed).

> Question #2: can we somehow remove the stale .eln file in these
> situations?

I guess we should make it work for non installed case.

be22cda7be should do this.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Sun, 21 Mar 2021 08:26:45 +0000
> 
> If the .el was modified we will recompile it in 'eln-cache' and load it,
> when we do this we clean-up the stale .eln present in
> `comp-eln-load-path'.  Unfortunatelly for this case we don't do this for
> the last directory in `comp-eln-load-path' (as this is tipically
> read-only when Emacs is installed).

The cleanup didn't happen in my case, I needed to do it manually.  Is
that because of that "last directory in comp-eln-load-path" case, or
because I run Emacs uninstalled?

> > Question #2: can we somehow remove the stale .eln file in these
> > situations?
> 
> I guess we should make it work for non installed case.
> 
> be22cda7be should do this.

Thanks.  However, does this also attempt to remove stale *.eln files
that are preloaded?  Because if they are removed, older binaries which
were dumped with those stale *.eln files will be unable to start after
the cleanup, right?  It's a bit of a mess, I guess.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> Date: Sun, 21 Mar 2021 11:52:11 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 47067@debbugs.gnu.org
> 
> > > Question #2: can we somehow remove the stale .eln file in these
> > > situations?
> > 
> > I guess we should make it work for non installed case.
> > 
> > be22cda7be should do this.
> 
> Thanks.  However, does this also attempt to remove stale *.eln files
> that are preloaded?  Because if they are removed, older binaries which
> were dumped with those stale *.eln files will be unable to start after
> the cleanup, right?  It's a bit of a mess, I guess.

So maybe we should separate the preloaded *.eln files from the rest of
them.  Say, put the *.eln files for the preloaded Lisp files into a
special subdirectory of native-lisp/XX.YY.ZZ-xxxxxx/, let's call it
"preloaded".  Then in that subdirectory we never remove stale *.eln
files, because there might be a .pdmp file that references them.  The
*.eln files in other places can be cleaned up when they become stale.

WDYT?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Sun, 21 Mar 2021 08:26:45 +0000
>> 
>> If the .el was modified we will recompile it in 'eln-cache' and load it,
>> when we do this we clean-up the stale .eln present in
>> `comp-eln-load-path'.  Unfortunatelly for this case we don't do this for
>> the last directory in `comp-eln-load-path' (as this is tipically
>> read-only when Emacs is installed).
>
> The cleanup didn't happen in my case, I needed to do it manually.  Is
> that because of that "last directory in comp-eln-load-path" case, or
> because I run Emacs uninstalled?

Being uninstalled 'native-lisp' is the last entry of
`comp-eln-load-path' so yeah should be the former case.

>> > Question #2: can we somehow remove the stale .eln file in these
>> > situations?
>> 
>> I guess we should make it work for non installed case.
>> 
>> be22cda7be should do this.
>
> Thanks.  However, does this also attempt to remove stale *.eln files
> that are preloaded?  Because if they are removed, older binaries which
> were dumped with those stale *.eln files will be unable to start after
> the cleanup, right?  It's a bit of a mess, I guess.

Right good point, d0280ce1b1 reverts be22cda7be...

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sun, 21 Mar 2021 11:52:11 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: 47067@debbugs.gnu.org
>> 
>> > > Question #2: can we somehow remove the stale .eln file in these
>> > > situations?
>> > 
>> > I guess we should make it work for non installed case.
>> > 
>> > be22cda7be should do this.
>> 
>> Thanks.  However, does this also attempt to remove stale *.eln files
>> that are preloaded?  Because if they are removed, older binaries which
>> were dumped with those stale *.eln files will be unable to start after
>> the cleanup, right?  It's a bit of a mess, I guess.
>
> So maybe we should separate the preloaded *.eln files from the rest of
> them.  Say, put the *.eln files for the preloaded Lisp files into a
> special subdirectory of native-lisp/XX.YY.ZZ-xxxxxx/, let's call it
> "preloaded".  Then in that subdirectory we never remove stale *.eln
> files, because there might be a .pdmp file that references them.  The
> *.eln files in other places can be cleaned up when they become stale.
>
> WDYT?

What should be choose as XX.YY.ZZ-xxxxxx ?

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Sun, 21 Mar 2021 14:37:45 +0000
> 
> > So maybe we should separate the preloaded *.eln files from the rest of
> > them.  Say, put the *.eln files for the preloaded Lisp files into a
> > special subdirectory of native-lisp/XX.YY.ZZ-xxxxxx/, let's call it
> > "preloaded".  Then in that subdirectory we never remove stale *.eln
> > files, because there might be a .pdmp file that references them.  The
> > *.eln files in other places can be cleaned up when they become stale.
> >
> > WDYT?
> 
> What should be choose as XX.YY.ZZ-xxxxxx ?

That's the usual VERSION+ABI-HASH way you generate that subdirectory
today, I don't propose any change in that part.  Just have a
subdirectory _under_ that, called "preloaded", where you put all the
preloaded *.eln files, and put all the other *.eln files generated
during the build as you do today.

Did I explain myself?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Sun, 21 Mar 2021 14:37:45 +0000
>> 
>> > So maybe we should separate the preloaded *.eln files from the rest of
>> > them.  Say, put the *.eln files for the preloaded Lisp files into a
>> > special subdirectory of native-lisp/XX.YY.ZZ-xxxxxx/, let's call it
>> > "preloaded".  Then in that subdirectory we never remove stale *.eln
>> > files, because there might be a .pdmp file that references them.  The
>> > *.eln files in other places can be cleaned up when they become stale.
>> >
>> > WDYT?
>> 
>> What should be choose as XX.YY.ZZ-xxxxxx ?
>
> That's the usual VERSION+ABI-HASH way you generate that subdirectory
> today, I don't propose any change in that part.  Just have a
> subdirectory _under_ that, called "preloaded", where you put all the
> preloaded *.eln files, and put all the other *.eln files generated
> during the build as you do today.
>
> Did I explain myself?

Yep sorry, it's clear now.

I was thinking, the only downside I see of this solution is that while
we can tweak the build system to signal to Emacs that a certain file
being compiled will be used for dump, this will indeed not be possible
for subsequent dumps.  Not sure if this is a real issue or not.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Sun, 21 Mar 2021 15:41:48 +0000
> 
> I was thinking, the only downside I see of this solution is that while
> we can tweak the build system to signal to Emacs that a certain file
> being compiled will be used for dump, this will indeed not be possible
> for subsequent dumps.

I don't think I follow: why would this be impossible for subsequent
dumps?

Maybe I should describe the use case this is supposed to solve.
Imagine a repository where Emacs is being built regularly, and where
old Emacs builds are kept for long periods of time, together with
their *.pdmp files.  So we will have emacs-28.0.50.1, emacs-28.0.50.2,
... emacs-28-0.50.1000, ..., and their respective *.pdmp files.  The
intent is to be able to run these old binaries for quick "bisecting"
of issues reported by users.  This is what I do on my system: I keep
one build from each of the previous months, and delete the rest.

Until native-comp, keeping the binary and the .pdmp file was enough
for being able to run that binary.  With native-comp, we also need the
*.eln files that were dumped into each of the *.pdmp files.  I'm
trying to find a reasonable solution to this problem.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Sun, 21 Mar 2021 15:41:48 +0000
>> 
>> I was thinking, the only downside I see of this solution is that while
>> we can tweak the build system to signal to Emacs that a certain file
>> being compiled will be used for dump, this will indeed not be possible
>> for subsequent dumps.
>
> I don't think I follow: why would this be impossible for subsequent
> dumps?
>
> Maybe I should describe the use case this is supposed to solve.
> Imagine a repository where Emacs is being built regularly, and where
> old Emacs builds are kept for long periods of time, together with
> their *.pdmp files.  So we will have emacs-28.0.50.1, emacs-28.0.50.2,
> ... emacs-28-0.50.1000, ..., and their respective *.pdmp files.  The
> intent is to be able to run these old binaries for quick "bisecting"
> of issues reported by users.  This is what I do on my system: I keep
> one build from each of the previous months, and delete the rest.
>
> Until native-comp, keeping the binary and the .pdmp file was enough
> for being able to run that binary.  With native-comp, we also need the
> *.eln files that were dumped into each of the *.pdmp files.  I'm
> trying to find a reasonable solution to this problem.

Thanks I think I've understood the case.

I was referring to a different case, future dumps done by the user
calling directly `dump-emacs-portable', in that case some .eln might
stay where they were placed originally (not in the preloaded subfolder).

I know for now we don't support officially `dump-emacs-portable' to be
used directly by the user, so I'm not sure that's a real issue, but I
was trying to think future proof.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Sun, 21 Mar 2021 16:41:14 +0000
> 
> I was referring to a different case, future dumps done by the user
> calling directly `dump-emacs-portable', in that case some .eln might
> stay where they were placed originally (not in the preloaded subfolder).
> 
> I know for now we don't support officially `dump-emacs-portable' to be
> used directly by the user, so I'm not sure that's a real issue, but I
> was trying to think future proof.

Ah, okay.  I think we can leave this use case for posterity, to be
figured out when re-dumping is fully supported.  How about adding this
bit to the etc/TODO item that talks about re-dumping?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Sun, 21 Mar 2021 16:41:14 +0000
>> 
>> I was referring to a different case, future dumps done by the user
>> calling directly `dump-emacs-portable', in that case some .eln might
>> stay where they were placed originally (not in the preloaded subfolder).
>> 
>> I know for now we don't support officially `dump-emacs-portable' to be
>> used directly by the user, so I'm not sure that's a real issue, but I
>> was trying to think future proof.
>
> Ah, okay.  I think we can leave this use case for posterity, to be
> figured out when re-dumping is fully supported.  How about adding this
> bit to the etc/TODO item that talks about re-dumping?

Right, will do that contextually when I prepare the patch.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo <akrl@sdf.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: 47067@debbugs.gnu.org
>>> Date: Sun, 21 Mar 2021 16:41:14 +0000
>>> 
>>> I was referring to a different case, future dumps done by the user
>>> calling directly `dump-emacs-portable', in that case some .eln might
>>> stay where they were placed originally (not in the preloaded subfolder).
>>> 
>>> I know for now we don't support officially `dump-emacs-portable' to be
>>> used directly by the user, so I'm not sure that's a real issue, but I
>>> was trying to think future proof.
>>
>> Ah, okay.  I think we can leave this use case for posterity, to be
>> figured out when re-dumping is fully supported.  How about adding this
>> bit to the etc/TODO item that talks about re-dumping?
>
> Right, will do that contextually when I prepare the patch.
>
>   Andrea

The only non trivial part here is to identify at compile time if a file
will be preloaded or not (so we can output it in the preloaded
directory).

Any obvious suggestion here before I start hacking the build system?

TIA

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Thu, 25 Mar 2021 16:51:48 +0000
> 
> >>> I was referring to a different case, future dumps done by the user
> >>> calling directly `dump-emacs-portable', in that case some .eln might
> >>> stay where they were placed originally (not in the preloaded subfolder).
> >>> 
> >>> I know for now we don't support officially `dump-emacs-portable' to be
> >>> used directly by the user, so I'm not sure that's a real issue, but I
> >>> was trying to think future proof.
> >>
> >> Ah, okay.  I think we can leave this use case for posterity, to be
> >> figured out when re-dumping is fully supported.  How about adding this
> >> bit to the etc/TODO item that talks about re-dumping?
> >
> > Right, will do that contextually when I prepare the patch.
> >
> >   Andrea
> 
> The only non trivial part here is to identify at compile time if a file
> will be preloaded or not (so we can output it in the preloaded
> directory).
> 
> Any obvious suggestion here before I start hacking the build system?

Why does it have to be the build system?  dump-emacs-portable is an
Emacs function, it doesn't use any of the build machinery.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Thu, 25 Mar 2021 16:51:48 +0000
>> 
>> >>> I was referring to a different case, future dumps done by the user
>> >>> calling directly `dump-emacs-portable', in that case some .eln might
>> >>> stay where they were placed originally (not in the preloaded subfolder).
>> >>> 
>> >>> I know for now we don't support officially `dump-emacs-portable' to be
>> >>> used directly by the user, so I'm not sure that's a real issue, but I
>> >>> was trying to think future proof.
>> >>
>> >> Ah, okay.  I think we can leave this use case for posterity, to be
>> >> figured out when re-dumping is fully supported.  How about adding this
>> >> bit to the etc/TODO item that talks about re-dumping?
>> >
>> > Right, will do that contextually when I prepare the patch.
>> >
>> >   Andrea
>> 
>> The only non trivial part here is to identify at compile time if a file
>> will be preloaded or not (so we can output it in the preloaded
>> directory).
>> 
>> Any obvious suggestion here before I start hacking the build system?
>
> Why does it have to be the build system?  dump-emacs-portable is an
> Emacs function, it doesn't use any of the build machinery.

If we want to have the eln files dumped into the "preloaded" subfolder
we need to know where to output when they are compiled, so before
`dump-emacs-portable' is called.

Maybe you have a different model in mind?

TIA

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Thu, 25 Mar 2021 20:24:12 +0000
> 
> >> The only non trivial part here is to identify at compile time if a file
> >> will be preloaded or not (so we can output it in the preloaded
> >> directory).
> >> 
> >> Any obvious suggestion here before I start hacking the build system?
> >
> > Why does it have to be the build system?  dump-emacs-portable is an
> > Emacs function, it doesn't use any of the build machinery.
> 
> If we want to have the eln files dumped into the "preloaded" subfolder
> we need to know where to output when they are compiled, so before
> `dump-emacs-portable' is called.

Ah, I see now.  The list of the files that could be preloaded is in
$shortlisp, see src/Makefile.in.  The file names are read from
src/lisp.mk.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Thu, 25 Mar 2021 20:24:12 +0000
>> 
>> >> The only non trivial part here is to identify at compile time if a file
>> >> will be preloaded or not (so we can output it in the preloaded
>> >> directory).
>> >> 
>> >> Any obvious suggestion here before I start hacking the build system?
>> >
>> > Why does it have to be the build system?  dump-emacs-portable is an
>> > Emacs function, it doesn't use any of the build machinery.
>> 
>> If we want to have the eln files dumped into the "preloaded" subfolder
>> we need to know where to output when they are compiled, so before
>> `dump-emacs-portable' is called.
>
> Ah, I see now.  The list of the files that could be preloaded is in
> $shortlisp, see src/Makefile.in.  The file names are read from
> src/lisp.mk.

Here I'm, 6f8ec14491 should introduce the behaviour we wanted, preloaded
files are now deposed in the 'preloaded' subfolder.

Also, 1ad0ecea2b reinstablish the fact that at each re-compilation we
clean-up stale eln files in all the directories in `comp-eln-load-path',
indeed this is excluding the introduced 'preloaded' subfolder.

Regards

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Sun, 04 Apr 2021 20:52:51 +0000
> 
> > Ah, I see now.  The list of the files that could be preloaded is in
> > $shortlisp, see src/Makefile.in.  The file names are read from
> > src/lisp.mk.
> 
> Here I'm, 6f8ec14491 should introduce the behaviour we wanted, preloaded
> files are now deposed in the 'preloaded' subfolder.
> 
> Also, 1ad0ecea2b reinstablish the fact that at each re-compilation we
> clean-up stale eln files in all the directories in `comp-eln-load-path',
> indeed this is excluding the introduced 'preloaded' subfolder.

Thanks.  Things seem to work quite well, AFAICS, but there's a subtle
problem (or maybe 2):

 . First, the problem with corrupted backtraces is back, although I
   verified that comp-debug is 1.  Strangely, it happens only for some
   *.eln files; others show full backtraces as expected.  The telltale
   sign of a "problematic" .eln file is that even when functions are
   shown in the backtrace with their fill names, the arguments of
   function calls aren't shown, something that shouldn't happen when
   there's debug info.

 . If I manually recompile a preloaded .el file, like this:

     emacs -batch -l comp -f batch-byte-native-compile-for-bootstrap ../lisp/isearch.el

   then the produced .eln file is written not into the preloaded
   folder, but into its parent, i.e. native-lisp/28.0.50-NNNNNN/.

And here is the possible link between these two issues: if I start
Emacs after compiling preloaded files manually as above, then Emacs
seems to load the *.eln files from native-lisp/28.0.50-NNNNNN/, not
from native-lisp/28.0.50-NNNNNN/preloaded, and the problems with the
backtraces miraculously disappear!

Any idea what is going on here?

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> Date: Mon, 05 Apr 2021 18:06:03 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 47067@debbugs.gnu.org
> 
> And here is the possible link between these two issues: if I start
> Emacs after compiling preloaded files manually as above, then Emacs
> seems to load the *.eln files from native-lisp/28.0.50-NNNNNN/, not
> from native-lisp/28.0.50-NNNNNN/preloaded, and the problems with the
> backtraces miraculously disappear!

Scratch that: the problems with backtraces don't disappear, they still
appear from time to time.

I'm now starting to think that setting comp-debug to 1 didn't really
fix the issue with backtraces, just made it less frequent.

Don't know what to do with this.  Very disappointing.  It means that
for me it will be much harder to debug the natively-compiled Emacs.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> Date: Mon, 05 Apr 2021 19:09:52 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 47067@debbugs.gnu.org
> 
> Scratch that: the problems with backtraces don't disappear, they still
> appear from time to time.
> 
> I'm now starting to think that setting comp-debug to 1 didn't really
> fix the issue with backtraces, just made it less frequent.
> 
> Don't know what to do with this.  Very disappointing.  It means that
> for me it will be much harder to debug the natively-compiled Emacs.

Hold on, I think I cracked this, see

  https://sourceware.org/pipermail/gdb-patches/2021-April/177499.html

It looks like a Windows-specific GDB bug, indeed.  I will try to code
a fix for that in GDB and see if that solves the problems with
backtraces once and for all.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Sun, 04 Apr 2021 20:52:51 +0000
>> 
>> > Ah, I see now.  The list of the files that could be preloaded is in
>> > $shortlisp, see src/Makefile.in.  The file names are read from
>> > src/lisp.mk.
>> 
>> Here I'm, 6f8ec14491 should introduce the behaviour we wanted, preloaded
>> files are now deposed in the 'preloaded' subfolder.
>> 
>> Also, 1ad0ecea2b reinstablish the fact that at each re-compilation we
>> clean-up stale eln files in all the directories in `comp-eln-load-path',
>> indeed this is excluding the introduced 'preloaded' subfolder.
>
> Thanks.  Things seem to work quite well, AFAICS, but there's a subtle
> problem (or maybe 2):
>
>  . First, the problem with corrupted backtraces is back, although I
>    verified that comp-debug is 1.  Strangely, it happens only for some
>    *.eln files; others show full backtraces as expected.  The telltale
>    sign of a "problematic" .eln file is that even when functions are
>    shown in the backtrace with their fill names, the arguments of
>    function calls aren't shown, something that shouldn't happen when
>    there's debug info.
>
>  . If I manually recompile a preloaded .el file, like this:
>
>      emacs -batch -l comp -f batch-byte-native-compile-for-bootstrap ../lisp/isearch.el
>
>    then the produced .eln file is written not into the preloaded
>    folder, but into its parent, i.e. native-lisp/28.0.50-NNNNNN/.
>
> And here is the possible link between these two issues: if I start
> Emacs after compiling preloaded files manually as above, then Emacs
> seems to load the *.eln files from native-lisp/28.0.50-NNNNNN/, not
> from native-lisp/28.0.50-NNNNNN/preloaded, and the problems with the
> backtraces miraculously disappear!
>
> Any idea what is going on here?

I think I understand the second issue (at least):

ATM which file is preloaded and which is not is suggested by the build
system to Emacs (using the LISP_PRELOADED env var) so that Emacs decides
where to depose the eln.  Unfortunately when the build system is by
passed with a manual invocation Emacs is unaware of this information and
not targeting the preloaded sub-directory.

Not sure what's the best strategy to improve this.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Mon, 05 Apr 2021 18:04:53 +0000
> 
> >  . If I manually recompile a preloaded .el file, like this:
> >
> >      emacs -batch -l comp -f batch-byte-native-compile-for-bootstrap ../lisp/isearch.el
> >
> >    then the produced .eln file is written not into the preloaded
> >    folder, but into its parent, i.e. native-lisp/28.0.50-NNNNNN/.
> >
> > And here is the possible link between these two issues: if I start
> > Emacs after compiling preloaded files manually as above, then Emacs
> > seems to load the *.eln files from native-lisp/28.0.50-NNNNNN/, not
> > from native-lisp/28.0.50-NNNNNN/preloaded, and the problems with the
> > backtraces miraculously disappear!
> >
> > Any idea what is going on here?
> 
> I think I understand the second issue (at least):
> 
> ATM which file is preloaded and which is not is suggested by the build
> system to Emacs (using the LISP_PRELOADED env var) so that Emacs decides
> where to depose the eln.  Unfortunately when the build system is by
> passed with a manual invocation Emacs is unaware of this information and
> not targeting the preloaded sub-directory.
> 
> Not sure what's the best strategy to improve this.

How about a boolean variable comp-file-preloaded-p, which could be set
to signal to comp.el that the file being compiled should be preloaded?
Then the manual command would need to set that via -eval.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 05 Apr 2021 18:06:03 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: 47067@debbugs.gnu.org
>> 
>> And here is the possible link between these two issues: if I start
>> Emacs after compiling preloaded files manually as above, then Emacs
>> seems to load the *.eln files from native-lisp/28.0.50-NNNNNN/, not
>> from native-lisp/28.0.50-NNNNNN/preloaded, and the problems with the
>> backtraces miraculously disappear!
>
> Scratch that: the problems with backtraces don't disappear, they still
> appear from time to time.
>
> I'm now starting to think that setting comp-debug to 1 didn't really
> fix the issue with backtraces, just made it less frequent.
>
> Don't know what to do with this.  Very disappointing.  It means that
> for me it will be much harder to debug the natively-compiled Emacs.

Yes that's sad, OTOH I suspect that providing a disassembled frame to
the gdb guys should be easy for them to add/fix the gdb unwinder.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Mon, 05 Apr 2021 18:04:53 +0000
>> 
>> >  . If I manually recompile a preloaded .el file, like this:
>> >
>> >      emacs -batch -l comp -f batch-byte-native-compile-for-bootstrap ../lisp/isearch.el
>> >
>> >    then the produced .eln file is written not into the preloaded
>> >    folder, but into its parent, i.e. native-lisp/28.0.50-NNNNNN/.
>> >
>> > And here is the possible link between these two issues: if I start
>> > Emacs after compiling preloaded files manually as above, then Emacs
>> > seems to load the *.eln files from native-lisp/28.0.50-NNNNNN/, not
>> > from native-lisp/28.0.50-NNNNNN/preloaded, and the problems with the
>> > backtraces miraculously disappear!
>> >
>> > Any idea what is going on here?
>> 
>> I think I understand the second issue (at least):
>> 
>> ATM which file is preloaded and which is not is suggested by the build
>> system to Emacs (using the LISP_PRELOADED env var) so that Emacs decides
>> where to depose the eln.  Unfortunately when the build system is by
>> passed with a manual invocation Emacs is unaware of this information and
>> not targeting the preloaded sub-directory.
>> 
>> Not sure what's the best strategy to improve this.
>
> How about a boolean variable comp-file-preloaded-p, which could be set
> to signal to comp.el that the file being compiled should be preloaded?
> Then the manual command would need to set that via -eval.

Sounds good should be easy to add, doing it.

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: 47067@debbugs.gnu.org
>>> Date: Mon, 05 Apr 2021 18:04:53 +0000
>>> 
>>> >  . If I manually recompile a preloaded .el file, like this:
>>> >
>>> >      emacs -batch -l comp -f batch-byte-native-compile-for-bootstrap ../lisp/isearch.el
>>> >
>>> >    then the produced .eln file is written not into the preloaded
>>> >    folder, but into its parent, i.e. native-lisp/28.0.50-NNNNNN/.
>>> >
>>> > And here is the possible link between these two issues: if I start
>>> > Emacs after compiling preloaded files manually as above, then Emacs
>>> > seems to load the *.eln files from native-lisp/28.0.50-NNNNNN/, not
>>> > from native-lisp/28.0.50-NNNNNN/preloaded, and the problems with the
>>> > backtraces miraculously disappear!
>>> >
>>> > Any idea what is going on here?
>>> 
>>> I think I understand the second issue (at least):
>>> 
>>> ATM which file is preloaded and which is not is suggested by the build
>>> system to Emacs (using the LISP_PRELOADED env var) so that Emacs decides
>>> where to depose the eln.  Unfortunately when the build system is by
>>> passed with a manual invocation Emacs is unaware of this information and
>>> not targeting the preloaded sub-directory.
>>> 
>>> Not sure what's the best strategy to improve this.
>>
>> How about a boolean variable comp-file-preloaded-p, which could be set
>> to signal to comp.el that the file being compiled should be preloaded?
>> Then the manual command would need to set that via -eval.
>
> Sounds good should be easy to add, doing it.

Right, 0a3e715e1f should do the job please have a try when you like.

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> Date: Mon, 05 Apr 2021 20:58:36 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 47067@debbugs.gnu.org
> 
> > Don't know what to do with this.  Very disappointing.  It means that
> > for me it will be much harder to debug the natively-compiled Emacs.
> 
> Hold on, I think I cracked this, see
> 
>   https://sourceware.org/pipermail/gdb-patches/2021-April/177499.html
> 
> It looks like a Windows-specific GDB bug, indeed.  I will try to code
> a fix for that in GDB and see if that solves the problems with
> backtraces once and for all.

OK, I fixed the bug in GDB, and now the backtraces are full and
complete.  So we can now close the issue with the corrupted
backtraces.  Yay!

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 05 Apr 2021 20:58:36 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: 47067@debbugs.gnu.org
>> 
>> > Don't know what to do with this.  Very disappointing.  It means that
>> > for me it will be much harder to debug the natively-compiled Emacs.
>> 
>> Hold on, I think I cracked this, see
>> 
>>   https://sourceware.org/pipermail/gdb-patches/2021-April/177499.html
>> 
>> It looks like a Windows-specific GDB bug, indeed.  I will try to code
>> a fix for that in GDB and see if that solves the problems with
>> backtraces once and for all.
>
> OK, I fixed the bug in GDB, and now the backtraces are full and
> complete.  So we can now close the issue with the corrupted
> backtraces.  Yay!

Wow great job!

Do we still want to produce the debug symbols by default on Windows?

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Tue, 06 Apr 2021 13:29:02 +0000
> 
> >>   https://sourceware.org/pipermail/gdb-patches/2021-April/177499.html
> >> 
> >> It looks like a Windows-specific GDB bug, indeed.  I will try to code
> >> a fix for that in GDB and see if that solves the problems with
> >> backtraces once and for all.
> >
> > OK, I fixed the bug in GDB, and now the backtraces are full and
> > complete.  So we can now close the issue with the corrupted
> > backtraces.  Yay!
> 
> Wow great job!
> 
> Do we still want to produce the debug symbols by default on Windows?

Probably no need, but let's leave this as it is now for the time
being, just so we don't rock the boat too much before landing on
master.  I will look into this later, and reset comp-debug to zero if
indeed it doesn't need to be higher.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Tue, 06 Apr 2021 13:29:02 +0000
>> 
>> >>   https://sourceware.org/pipermail/gdb-patches/2021-April/177499.html
>> >> 
>> >> It looks like a Windows-specific GDB bug, indeed.  I will try to code
>> >> a fix for that in GDB and see if that solves the problems with
>> >> backtraces once and for all.
>> >
>> > OK, I fixed the bug in GDB, and now the backtraces are full and
>> > complete.  So we can now close the issue with the corrupted
>> > backtraces.  Yay!
>> 
>> Wow great job!
>> 
>> Do we still want to produce the debug symbols by default on Windows?
>
> Probably no need, but let's leave this as it is now for the time
> being, just so we don't rock the boat too much before landing on
> master.  I will look into this later, and reset comp-debug to zero if
> indeed it doesn't need to be higher.

Sure no problem, thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Mon, 05 Apr 2021 19:22:19 +0000
> 
> >> How about a boolean variable comp-file-preloaded-p, which could be set
> >> to signal to comp.el that the file being compiled should be preloaded?
> >> Then the manual command would need to set that via -eval.
> >
> > Sounds good should be easy to add, doing it.
> 
> Right, 0a3e715e1f should do the job please have a try when you like.

Thanks, this works.

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47067@debbugs.gnu.org
>> Date: Mon, 05 Apr 2021 19:22:19 +0000
>> 
>> >> How about a boolean variable comp-file-preloaded-p, which could be set
>> >> to signal to comp.el that the file being compiled should be preloaded?
>> >> Then the manual command would need to set that via -eval.
>> >
>> > Sounds good should be easy to add, doing it.
>> 
>> Right, 0a3e715e1f should do the job please have a try when you like.
>
> Thanks, this works.

Nice, is there anything left we should look into for this bug?

Thanks

  Andrea

bug#47067: 28.0.50; [feature/native-comp] Crash while scrolling through dispnew.c

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47067@debbugs.gnu.org
> Date: Tue, 06 Apr 2021 16:06:53 +0000
> 
> >> Right, 0a3e715e1f should do the job please have a try when you like.
> >
> > Thanks, this works.
> 
> Nice, is there anything left we should look into for this bug?

No, closing.