bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Hi all,

I'm on Win10 with Msys2/MinGW64 and GCC 13.1 landed on my HD.  Now Emacs
doesn't build anymore when I pass '--with-native-compilation' to
configure.  In summary (tried with master 3adc1e7f37):

  $ git clone --depth=1 https://git.savannah.gnu.org/git/emacs.git emacs-build-test
  $ cd emacs-build-test
  $ ./autogen.sh
  $ ./configure --with-native-compilation
  $ make

Works with

  gcc.exe (Rev7, Built by MSYS2 project) 12.2.0

and breaks with

  gcc.exe (Rev4, Built by MSYS2 project) 13.1.0

emitting

--8<---------------cut here---------------start------------->8---
  ELC+ELN  ../lisp/font-core.elc
  ELC+ELN  ../lisp/font-lock.elc
  ELC+ELN  ../lisp/format.elc
  ELC+ELN  ../lisp/frame.elc

Error: wrong-type-argument ("../lisp/frame.el" number-or-marker-p
Backtrace:
00007ff6e577a12e
00007ff6e5648be1
00007ff6e5669601
00007ff6e57de84a
00007ff9b7977ff0
00007ff9b81b23d7
00007ff9b816149c
00007ff9b81b0f06
00007ff6e56ffc23
00007ff6e5701300
00007ff6e56dc492
00007ff6e56dd3e5
00007ff6e56d80fa
00007ff6e56d8448
00007ff6e56d8af9
00007ff6e56d9725
00007ff6e56d9979
00007ff6e56d80fa
00007ff996e9af72
00007ff6e56d80fa
00007ff996e9c1a7
00007ff6e56d80fa
00007ff996e9c3ad
00007ff6e56d80fa
00007ff6e56dc0f6
00007ff6e56dce05
00007ff6e56dc2bf
00007ff6e56dce95
00007ff6e56dc2bf
00007ff6e56de205
00007ff6e56dc2bf
00007ff6e56dccb5
00007ff6e56dc2bf
00007ff6e56de205
00007ff6e56dc2bf
00007ff6e56dc5e5
00007ff6e56dc2bf
00007ff6e56dc2bf
00007ff6e56ddded
00007ff6e56dc2bf
00007ff6e56ddded
00007ff6e56dc2bf
00007ff6e56dd3e5
00007ff6e56dd9d6
00007ff6e56dbc5d
00007ff6e56ddded
00007ff6e56dc2bf
00007ff6e56dd3e5
00007ff6e56dd9d6
00007ff6e56dbc5d
00007ff6e56de3ae
00007ff6e56dc2bf
00007ff6e56ddded
00007ff6e56dc2bf
00007ff6e56dce05
00007ff6e56dc2bf
00007ff6e56dd3e5
00007ff6e56dd9d6
00007ff6e56dbc5d
00007ff6e56de9aa
00007ff6e56d6735
00007ff6e5649fb5
...
make[3]: *** [Makefile:283: ../lisp/frame.elc] Error 3
make[2]: *** [Makefile:842: ../lisp/frame.elc] Error 2
make[2]: Leaving directory '/z/emacs-build-test/src'
make[1]: *** [Makefile:544: src] Error 2
make[1]: Leaving directory '/z/emacs-build-test'
make[1]: Entering directory '/z/emacs-build-test'
***
*** "make all" failed with exit status 2.
***
*** You could try to:
*** - run "make bootstrap", which might fix the problem
*** - run "make V=1", which displays the full commands invoked by make,
***   to further investigate the problem
***
make[1]: *** [Makefile:414: advice-on-failure] Error 2
make[1]: Leaving directory '/z/emacs-build-test'
make: *** [Makefile:370: all] Error 2
--8<---------------cut here---------------end--------------->8---

I haven't tried "make V=1" yet.

Building Emacs with

  $ git clone --depth=1 https://git.savannah.gnu.org/git/emacs.git emacs-build-test
  $ cd emacs-build-test
  $ ./autogen.sh
  $ ./configure --without-native-compilation
  $ make

is successful with GCC 13.1, OTOH.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Date: Mon, 08 May 2023 10:16:31 +0200
> 
> --8<---------------cut here---------------start------------->8---
>   ELC+ELN  ../lisp/font-core.elc
>   ELC+ELN  ../lisp/font-lock.elc
>   ELC+ELN  ../lisp/format.elc
>   ELC+ELN  ../lisp/frame.elc
> 
> Error: wrong-type-argument ("../lisp/frame.el" number-or-marker-p
> Backtrace:
> 00007ff6e577a12e
> 00007ff6e5648be1
> 00007ff6e5669601
> 00007ff6e57de84a
> 00007ff9b7977ff0

These addresses are not useful anywhere but on your system.  So either
run this command under GDB and post a human-readable source-level
backtrace, or at least convert the above list of addresses to a list
of file names, function names, and line numbers as explained in the
node "Crashing" of the Emacs user manual.

Thanks.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> These addresses are not useful anywhere but on your system.  So either
> run this command under GDB and post a human-readable source-level
> backtrace,

Thank.  I tried that, i.e.,

  $ ./autogen.sh
  $ ./configure --with-native-compilation
  $ gdb
  $ (gdb) make

But backtrace in gdb returns nothing (nothing on stack IIRC).  Do I have
to do something else?

> or at least convert the above list of addresses to a list of file
> names, function names, and line numbers as explained in the node
> "Crashing" of the Emacs user manual.

The recipe from the manual[1] doesn't work since:

  $ sed -n 's/.*\[\(.*\)]$/\1/p' lisp/emacs_backtrace.txt

prints nothing on addresses like:

  00007ff67feca12e
  00007ff67fd98be1
  00007ff67fdb9601

Am I missing something?

Another observation after reading etc/DEBUG: The build is successful if
I follow the advice there and do:

  $ ./configure --with-native-compilation  CFLAGS='-O0 -g3'

Does this make sense?

Best, Arash

Footnotes:
[1]  https://www.gnu.org/software/emacs/manual/html_node/emacs/Crashing.html

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: Andrea Corallo <akrl@sdf.org>,  63365@debbugs.gnu.org
> Date: Mon, 08 May 2023 16:36:57 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > These addresses are not useful anywhere but on your system.  So either
> > run this command under GDB and post a human-readable source-level
> > backtrace,
> 
> Thank.  I tried that, i.e.,
> 
>   $ ./autogen.sh
>   $ ./configure --with-native-compilation
>   $ gdb
>   $ (gdb) make
> 
> But backtrace in gdb returns nothing (nothing on stack IIRC).  Do I have
> to do something else?

Yes, of course.  The GDB command "make" doesn't run anything under the
debugger, it is just a conveniency for rebuilding a program without
exiting GDB.

What you need is to run Emacs under GDB while it (Emacs) executes the
command which crashes.  To find out what is that crashing command, you
should say:

  $ make V=1

(note: "make", not "make -jN"!).  Then wait for the crash.  The
(longish) command shown by Make immediately before the crash is the
command that you want; copy it into the clipboard, and then do

  $ cd  /path/to/emacs/src
  $ gdb ./emacs.exe
  ...
  (gdb) cd ../lisp
  (gdb) run <paste here the crashing command you copied>

and wait for the crash.  Then:

  (gdb) thread 1
  (gdb) bt

Note: the command you copy into the clipboard may need massaging
before you can paste it to the GDB prompt: remove any stray newlines
and backslashes that escape newlines etc.  Use some editor for that:
either (an older) Emacs or even Notepad.

> > or at least convert the above list of addresses to a list of file
> > names, function names, and line numbers as explained in the node
> > "Crashing" of the Emacs user manual.
> 
> The recipe from the manual[1] doesn't work since:
> 
>   $ sed -n 's/.*\[\(.*\)]$/\1/p' lisp/emacs_backtrace.txt
> 
> prints nothing on addresses like:
> 
>   00007ff67feca12e
>   00007ff67fd98be1
>   00007ff67fdb9601
> 
> Am I missing something?

The recipe goes like this:

  sed -n 's/.*\[\(.*\)]$/\1/p' BACKTRACE | addr2line -C -f -i -p -e BINDIR/EMACS-BINARY

(It is broken in the manual into two lines, because the line is long,
but it is a single long command that runs Sed and pipes its output
into addr2line, a program that comes with GNU Binutils.)

> Another observation after reading etc/DEBUG: The build is successful if
> I follow the advice there and do:
> 
>   $ ./configure --with-native-compilation  CFLAGS='-O0 -g3'
> 
> Does this make sense?

It probably means this is a GCC bug that rears its ugly head when you
compile with optimizations.

Can you show a full C compilation command of one of the C source
files?  Like this:

  $ cd /path/to/emacs/src
  $ make data.o -W data.c V=1

I'd like to see all of the compiler's command-line options your build
uses.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> What you need is to run Emacs under GDB while it (Emacs) executes the
> command which crashes.  To find out what is that crashing command, you
> should say:
>
>   $ make V=1

Thanks for the detailed instructions; I will play with it, not there yet.

> The recipe goes like this:
>
>   sed -n 's/.*\[\(.*\)]$/\1/p' BACKTRACE | addr2line -C -f -i -p -e BINDIR/EMACS-BINARY
>
> (It is broken in the manual into two lines, because the line is long,
> but it is a single long command that runs Sed and pipes its output
> into addr2line, a program that comes with GNU Binutils.)

Yes, I think I grasped that one.  The problem I'm facing is that my
lisp/emacs_backtrace.txt looks like this:

  Backtrace:
  00007ff61166a12e
  00007ff611538be1
  00007ff611559601
  00007ff6116ce84a
  00007ff9b7977ff0
  ...

and the sed command doesn't match on the addresses above.  It works for
the example given in the manual like this:

  Backtrace:
  emacs[0x5094e4]
  emacs[0x4ed3e6]
  emacs[0x4ed504]
  /lib64/libpthread.so.0[0x375220efe0]
  /lib64/libpthread.so.0(read+0xe)[0x375220e08e]
  emacs[0x509af6]
  emacs[0x5acc26]

So in my case, sed returns nothing which is piped into addr2line.

> Can you show a full C compilation command of one of the C source
> files?  Like this:
>
>   $ cd /path/to/emacs/src
>   $ make data.o -W data.c V=1
>
> I'd like to see all of the compiler's command-line options your build
> uses.

It looks like this; line-breaks added manually:

--8<---------------cut here---------------start------------->8---
../lib-src/make-docfile -d . -g dispnew.o frame.o scroll.o         \
xdisp.o menu.o window.o charset.o coding.o category.o ccl.o        \
character.o chartab.o bidi.o  term.o terminal.o xfaces.o           \
emacs.o keyboard.o macros.o keymap.o sysdep.o bignum.o buffer.o    \
filelock.o insdel.o marker.o minibuf.o fileio.o dired.o cmds.o     \
casetab.o casefiddle.o indent.o search.o regex-emacs.o undo.o      \
alloc.o pdumper.o data.o doc.o editfns.o callint.o eval.o          \
floatfns.o fns.o sort.o font.o print.o lread.o emacs-module.o      \
syntax.o  bytecode.o comp.o dynlib.o process.o gnutls.o callproc.o \
region-cache.o sound.o timefns.o atimer.o doprnt.o intervals.o     \
textprop.o composite.o xml.o lcms.o w32notify.o  profiler.o        \
decompress.o thread.o systhread.o sqlite.o  treesit.o itree.o      \
hbfont.o w32fns.o w32menu.o w32reg.o w32font.o w32term.o w32xfns.o \
w32select.o w32uniscribe.o w32cygwinx.o w32.o w32console.o         \
w32heap.o w32inevt.o w32proc.o w32image.o fontset.o fringe.o       \
image.o  json.o    > globals.tmp
../build-aux/move-if-change globals.tmp globals.h
echo timestamp > gl-stamp
gcc  -c -mtune=generic                                \
-DUSE_CRT_DLL=1                                       \
-I /d/arash/Dev/Emacs/emacs-build-test/nt/inc -Demacs \
-I. -I. -I../lib -I../lib  -mtune=generic             \
-isystem C:/msys64/mingw64/include/librsvg-2.0        \
-isystem C:/msys64/mingw64/include/glib-2.0           \
-isystem C:/msys64/mingw64/lib/glib-2.0/include       \
-isystem C:/msys64/mingw64/include/gdk-pixbuf-2.0     \
-isystem C:/msys64/mingw64/include/libpng16           \
-isystem C:/msys64/mingw64/include/webp               \
-DLIBDEFLATE_DLL                                      \
-isystem C:/msys64/mingw64/include/cairo              \
-isystem C:/msys64/mingw64/include/freetype2          \
-isystem C:/msys64/mingw64/include/harfbuzz           \
-isystem C:/msys64/mingw64/include/pixman-1           \
-isystem C:/msys64/mingw64/include/libxml2            \
-isystem C:/msys64/mingw64/include/webp               \
-isystem C:/msys64/mingw64/include/harfbuzz           \
-isystem C:/msys64/mingw64/include/freetype2          \
-isystem C:/msys64/mingw64/include/libpng16           \
-isystem C:/msys64/mingw64/include/glib-2.0           \
-isystem C:/msys64/mingw64/lib/glib-2.0/include       \
-MMD -MF deps/data.d -MP                              \
-isystem C:/msys64/mingw64/include/p11-kit-1          \
-fno-common                                           \
-Wall                                                 \
-Warith-conversion                                    \
-Wdate-time                                           \
-Wdisabled-optimization                               \
-Wdouble-promotion                                    \
-Wduplicated-cond                                     \
-Wextra                                               \
-Wformat-signedness                                   \
-Winit-self                                           \
-Winvalid-pch                                         \
-Wlogical-op                                          \
-Wmissing-declarations                                \
-Wmissing-include-dirs                                \
-Wmissing-prototypes                                  \
-Wnested-externs                                      \
-Wnull-dereference                                    \
-Wold-style-definition                                \
-Wopenmp-simd                                         \
-Wpacked                                              \
-Wpointer-arith                                       \
-Wstrict-prototypes                                   \
-Wsuggest-attribute=noreturn                          \
-Wsuggest-final-methods                               \
-Wsuggest-final-types                                 \
-Wtrampolines                                         \
-Wuninitialized                                       \
-Wunknown-pragmas                                     \
-Wunused-macros                                       \
-Wvariadic-macros                                     \
-Wvector-operation-performance                        \
-Wwrite-strings                                       \
-Warray-bounds=2                                      \
-Wattribute-alias=2                                   \
-Wformat=2                                            \
-Wformat-truncation=2                                 \
-Wimplicit-fallthrough=5                              \
-Wshift-overflow=2                                    \
-Wuse-after-free=3                                    \
-Wvla-larger-than=4031                                \
-Wno-missing-field-initializers                       \
-Wno-override-init                                    \
-Wno-sign-compare                                     \
-Wno-type-limits                                      \
-Wno-unused-parameter                                 \
-Wno-format-nonliteral                                \
-Wno-bidi-chars                                       \
-Wno-pointer-sign                                     \
-g3 -O2 -gdwarf-2  data.c
--8<---------------cut here---------------end--------------->8---

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: akrl@sdf.org,  63365@debbugs.gnu.org
> Date: Mon, 08 May 2023 21:34:22 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The recipe goes like this:
> >
> >   sed -n 's/.*\[\(.*\)]$/\1/p' BACKTRACE | addr2line -C -f -i -p -e BINDIR/EMACS-BINARY
> >
> > (It is broken in the manual into two lines, because the line is long,
> > but it is a single long command that runs Sed and pipes its output
> > into addr2line, a program that comes with GNU Binutils.)
> 
> Yes, I think I grasped that one.  The problem I'm facing is that my
> lisp/emacs_backtrace.txt looks like this:
> 
>   Backtrace:
>   00007ff61166a12e
>   00007ff611538be1
>   00007ff611559601
>   00007ff6116ce84a
>   00007ff9b7977ff0
>   ...
> 
> and the sed command doesn't match on the addresses above.  It works for
> the example given in the manual like this:
> 
>   Backtrace:
>   emacs[0x5094e4]
>   emacs[0x4ed3e6]
>   emacs[0x4ed504]
>   /lib64/libpthread.so.0[0x375220efe0]
>   /lib64/libpthread.so.0(read+0xe)[0x375220e08e]
>   emacs[0x509af6]
>   emacs[0x5acc26]
> 
> So in my case, sed returns nothing which is piped into addr2line.

Right, forgot about that factoid.  On Windows, the Sed part is not
required, so the command is just

  addr2line -C -f -i -p -e BINDIR/EMACS-BINARY < emacs_backtrace.txt

> > Can you show a full C compilation command of one of the C source
> > files?  Like this:
> >
> >   $ cd /path/to/emacs/src
> >   $ make data.o -W data.c V=1
> >
> > I'd like to see all of the compiler's command-line options your build
> > uses.
> 
> It looks like this; line-breaks added manually:

OK, so you compile with "-O2 -gdwarf-2 -g3", the default optimization
options, and with -mtune=generic (also the default).  Things to try,
in order to better understand the scope of the problem:

  1) try removing -mtune=generic
  2) try using -O1 instead of -O2

For 1), edit src/Makefile to remove -mtune=generic from any GCC
options there, then remove src/*.o files and say "make" to rebuild.
For 2), edit src/Makefile to change -O2 to -O1, and again remove *.o
files and rebuild.  But let's first establish the exact locus of the
crashes first, without any changes in the build options.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> Right, forgot about that factoid.  On Windows, the Sed part is not
> required, so the command is just
>
>   addr2line -C -f -i -p -e BINDIR/EMACS-BINARY < emacs_backtrace.txt

Tried this (with different setups, incl. adjusting sed to pipe the
addresses), and the result always looks like this:

  ?? ??:0
  ?? ??:0
  ?? ??:0
  ?? ??:0
  ?? ??:0
  ...

So that didn't work, for whatever reason :-(

> OK, so you compile with "-O2 -gdwarf-2 -g3", the default optimization
> options, and with -mtune=generic (also the default).  Things to try,
> in order to better understand the scope of the problem:
>
>   1) try removing -mtune=generic
>   2) try using -O1 instead of -O2
>
> For 1), edit src/Makefile to remove -mtune=generic from any GCC
> options there, then remove src/*.o files and say "make" to rebuild.
> For 2), edit src/Makefile to change -O2 to -O1, and again remove *.o
> files and rebuild.  But let's first establish the exact locus of the
> crashes first, without any changes in the build options.

Thanks, and I agree, let's keep the build options; I will try your
recipe for gdb next.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: akrl@sdf.org,  63365@debbugs.gnu.org
> Date: Tue, 09 May 2023 14:12:56 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Right, forgot about that factoid.  On Windows, the Sed part is not
> > required, so the command is just
> >
> >   addr2line -C -f -i -p -e BINDIR/EMACS-BINARY < emacs_backtrace.txt
> 
> Tried this (with different setups, incl. adjusting sed to pipe the
> addresses), and the result always looks like this:
> 
>   ?? ??:0
>   ?? ??:0
>   ?? ??:0
>   ?? ??:0
>   ?? ??:0
>   ...

??? Did you rebuild Emacs in-between?  Or did you install Emacs with
something like "make install-strip", so the Emacs binary has no
symbols?

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> ??? Did you rebuild Emacs in-between?  Or did you install Emacs with
> something like "make install-strip", so the Emacs binary has no
> symbols?

Yes, I built Emacs anew, but the recipe is always the same:

  $ ./autogen.sh
  $ ./configure --with-native-compilation
  $ make

No installation, stripping, etc. and that gives me an emacs.exe of 137MB
size.  If I strip and optimize, it is around 8.5MB.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: akrl@sdf.org,  63365@debbugs.gnu.org
> Date: Wed, 10 May 2023 14:37:55 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > ??? Did you rebuild Emacs in-between?  Or did you install Emacs with
> > something like "make install-strip", so the Emacs binary has no
> > symbols?
> 
> Yes, I built Emacs anew, but the recipe is always the same:
> 
>   $ ./autogen.sh
>   $ ./configure --with-native-compilation
>   $ make
> 
> No installation, stripping, etc. and that gives me an emacs.exe of 137MB
> size.  If I strip and optimize, it is around 8.5MB.

Did you submit to addr2line the 137MB binary or the stripped 8.5MB
binary?  It sounds like you did that with the latter?

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> Did you submit to addr2line the 137MB binary or the stripped 8.5MB
> binary?  It sounds like you did that with the latter?

No, I tried many versions of passing the absolute path of src/emacs.exe
or relative one to addr2line; it (ideally) didn't pick up anything from
my $PATH.  So I did things like:

  $ addr2line -C -f -i -p -e src/emacs.exe < lisp/emacs_backtrace.txt
  $ addr2line -C -f -i -p -e ../src/emacs.exe < emacs_backtrace.txt
  $ addr2line -C -f -i -p -e `cygpath.exe -w -a src/emacs.exe` \
      < lisp/emacs_backtrace.txt
  ...

The result was always the same.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: akrl@sdf.org,  63365@debbugs.gnu.org
> Date: Wed, 10 May 2023 16:27:11 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Did you submit to addr2line the 137MB binary or the stripped 8.5MB
> > binary?  It sounds like you did that with the latter?
> 
> No, I tried many versions of passing the absolute path of src/emacs.exe
> or relative one to addr2line; it (ideally) didn't pick up anything from
> my $PATH.  So I did things like:
> 
>   $ addr2line -C -f -i -p -e src/emacs.exe < lisp/emacs_backtrace.txt
>   $ addr2line -C -f -i -p -e ../src/emacs.exe < emacs_backtrace.txt
>   $ addr2line -C -f -i -p -e `cygpath.exe -w -a src/emacs.exe` \
>       < lisp/emacs_backtrace.txt
>   ...
> 
> The result was always the same.

Maybe it's a bug in your addr2line, then.  It works perfectly here,
FWIW.

Can you make the unstripped emacs.exe available for download
somewhere, along with the emacs_backtrace.txt produced by that same
binary?  I'd like to take a look.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> Cc: 63365@debbugs.gnu.org, akrl@sdf.org
> Date: Wed, 10 May 2023 19:54:06 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> Maybe it's a bug in your addr2line, then.  It works perfectly here,
> FWIW.
> 
> Can you make the unstripped emacs.exe available for download
> somewhere, along with the emacs_backtrace.txt produced by that same
> binary?  I'd like to take a look.

Hmm... looking at the files you sent, I'm beginning to think something
is wrong with generating backtraces on MS-Windows in the 64-bit
builds.  Maybe it has to do with ASLR, or maybe the code which
produces the backtrace is wrong in 64-bit executables.

I'd like to see the backtrace from GDB and compare the addresses there
with those in emacs_backtrace.txt.  Did you try to run the crashing
command under GDB, as I described earlier?

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> Cc: 63365@debbugs.gnu.org, akrl@sdf.org
> Date: Wed, 10 May 2023 22:08:57 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > Cc: 63365@debbugs.gnu.org, akrl@sdf.org
> > Date: Wed, 10 May 2023 19:54:06 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > 
> > Maybe it's a bug in your addr2line, then.  It works perfectly here,
> > FWIW.
> > 
> > Can you make the unstripped emacs.exe available for download
> > somewhere, along with the emacs_backtrace.txt produced by that same
> > binary?  I'd like to take a look.
> 
> Hmm... looking at the files you sent, I'm beginning to think something
> is wrong with generating backtraces on MS-Windows in the 64-bit
> builds.  Maybe it has to do with ASLR, or maybe the code which
> produces the backtrace is wrong in 64-bit executables.

Actually, I take this back: the addresses in emacs_backtrace.txt seem
reasonable, similar to addresses shown in GDB backtraces from a 64-bit
Windows build we have on the bug tracker.

So the suspicion goes back to addr2line, although I see the same
problem with my version, which I built myself and which works
flawlessly with the 32-bit MinGW build of Emacs.

> I'd like to see the backtrace from GDB and compare the addresses there
> with those in emacs_backtrace.txt.  Did you try to run the crashing
> command under GDB, as I described earlier?

And also, please try the other method described in the manual:

  The hexadecimal numbers are program addresses, which can be
  associated with source code lines using a debugging tool.  For
  example, the GDB command ‘list *0x509af6’ prints the source-code
  lines corresponding to the ‘emacs[0x509af6]’ entry.

So could you try this method with a couple of addresses shown in
emacs_backtrace.txt you have?  Does "list *0xNNNNN" yield reasonable
source locations?

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> Date: Thu, 11 May 2023 13:06:15 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 63365@debbugs.gnu.org, akrl@sdf.org
> 
> > Cc: 63365@debbugs.gnu.org, akrl@sdf.org
> > Date: Wed, 10 May 2023 22:08:57 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > 
> > > Cc: 63365@debbugs.gnu.org, akrl@sdf.org
> > > Date: Wed, 10 May 2023 19:54:06 +0300
> > > From: Eli Zaretskii <eliz@gnu.org>
> > > 
> > > Maybe it's a bug in your addr2line, then.  It works perfectly here,
> > > FWIW.
> > > 
> > > Can you make the unstripped emacs.exe available for download
> > > somewhere, along with the emacs_backtrace.txt produced by that same
> > > binary?  I'd like to take a look.
> > 
> > Hmm... looking at the files you sent, I'm beginning to think something
> > is wrong with generating backtraces on MS-Windows in the 64-bit
> > builds.  Maybe it has to do with ASLR, or maybe the code which
> > produces the backtrace is wrong in 64-bit executables.
> 
> Actually, I take this back: the addresses in emacs_backtrace.txt seem
> reasonable, similar to addresses shown in GDB backtraces from a 64-bit
> Windows build we have on the bug tracker.
> 
> So the suspicion goes back to addr2line, although I see the same
> problem with my version, which I built myself and which works
> flawlessly with the 32-bit MinGW build of Emacs.

This conclusion was incorrect.  There's nothing wrong with addr2line;
the reason for the problem _was_ ASLR.  addr2line interprets the
addresses in emacs_backtrace.txt relative to the base address of the
program image as recorded in the PE header of the executable file.  It
cannot do anything else, since it doesn't have a live program to work
with (unlike when running Emacs under GDB).  But ASLR causes programs
be loaded at base addresses that are randomized from run to run, and
thus are usually different from what is recorded in the PE header.  So
for this to work on modern versions of Windows, Emacs must correct the
addresses it writes to emacs_backtrace.txt, so as to make them match
the situation where Emacs is loaded at the fixed base address recorded
in the header.

I recently upgraded my main development machine, and, as luck would
have it, had crashes during the build, so I could study the problem
and fix it.  It should now be fixed on the master branch.  I could
only test the solution in a 32-bit MinGW build, so please try testing
it in the 64-bit build when you have a crash on your hands.

Thanks.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> I recently upgraded my main development machine, and, as luck would
> have it, had crashes during the build, so I could study the problem
> and fix it.  It should now be fixed on the master branch.  I could
> only test the solution in a 32-bit MinGW build, so please try testing
> it in the 64-bit build when you have a crash on your hands.

I can confirm that Emacs now builds on Win10 64-bit with MSYS2/MINGW64
without '-fno-optimize-sibling-calls'.  This is with:

  gcc.exe (Rev3, Built by MSYS2 project) 13.2.0

and

  CFLAGS='-O2 -g0 -pipe'

and --with-native-compilation.  Do you need another confirmation?
Otherwise I can close this bug.  Thanks for fixing this.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 26 Jan 2024 19:24:50 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I recently upgraded my main development machine, and, as luck would
> > have it, had crashes during the build, so I could study the problem
> > and fix it.  It should now be fixed on the master branch.  I could
> > only test the solution in a 32-bit MinGW build, so please try testing
> > it in the 64-bit build when you have a crash on your hands.
> 
> I can confirm that Emacs now builds on Win10 64-bit with MSYS2/MINGW64
> without '-fno-optimize-sibling-calls'.  This is with:
> 
>   gcc.exe (Rev3, Built by MSYS2 project) 13.2.0
> 
> and
> 
>   CFLAGS='-O2 -g0 -pipe'
> 
> and --with-native-compilation.  Do you need another confirmation?

That's good to know, but I also wanted to be sure that
emacs_backtrace.txt can now be processed with addr2line.  If you have
a way of causing Emacs to crash and output emacs_backtrace.txt, please
test that.  If not, this can wait until there is some crash.

> Otherwise I can close this bug.  Thanks for fixing this.

Thanks.

P.S. Btw, I recommend using -g3, not -g0, as the former will produce
more useful debug info.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> That's good to know, but I also wanted to be sure that
> emacs_backtrace.txt can now be processed with addr2line.  If you have
> a way of causing Emacs to crash and output emacs_backtrace.txt, please
> test that.  If not, this can wait until there is some crash.

Ok, I will play with this and see if I can trigger the issue again; I'll
report back.

> P.S. Btw, I recommend using -g3, not -g0, as the former will produce
> more useful debug info.

Thanks, got it.  I have an aged Windows box here and tried to optimize
the build as far as possible for daily usage, and then rebuild for
specific bug reports with other flags.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

[-- Attachment #1: Type: text/plain, Size: 418 bytes --]

Arash Esbati <arash@gnu.org> writes:

> Ok, I will play with this and see if I can trigger the issue again; I'll
> report back.

Here is the report.  Suppose the repo is inside Z:/emacs-build-test
up-to-date with de020255a5c, I did:

  $ ./autogen.sh
  $ ./configure --with-native-compilation
  $ make
  $ addr2line -C -f -i -p -e src/emacs.exe < lisp/emacs_backtrace.txt

The results are attached.  HTH.

Best, Arash

[-- Attachment #2: emacs_backtrace.txt --]
[-- Type: text/plain, Size: 794 bytes --]


Backtrace:
0000000400210fe3
00000004000da4e1
00000004000fc811
000000040027621a
0000000acbed7ff0
0000000acca123a7
0000000acc9c14ac
0000000acca10eb6
000000040019493a
0000000400197000
00000004001977a1
000000040019786a
0000000a6b357b39
000000040016bf6a
0000000a6b35b8e9
0000000a6b35ba67
000000040016bf6a
0000000a6b35e7df
000000040016bf6a
0000000a6b35d37c
000000040016bf6a
0000000a6b381ca5
000000040016bf6a
0000000a6b38171f
000000040016bf6a
0000000a6b59dfd2
000000040016bf6a
0000000a9903f073
000000040016bf6a
0000000a9903724d
000000040016bf6a
0000000a99033319
000000040016fe4b
000000040017258a
00000004000dae6a
000000040016a4d5
00000004000dbc55
000000040016a443
00000004000dad2d
00000004000e3f8f
00000004000e435c
00000004002883fe
00000004000012e6
00000004000013fe
0000000acafd733c
0000000acc9c26a9

[-- Attachment #3: addr2line.txt --]
[-- Type: text/plain, Size: 2063 bytes --]

?? ??:0
?? ??:0
w32_backtrace at Z:\emacs-build-test\src/w32fns.c:11152
 (inlined by) emacs_abort at Z:\emacs-build-test\src/w32fns.c:11191
terminate_due_to_signal at Z:\emacs-build-test\src/emacs.c:474
?? at Z:\emacs-build-test\src/sysdep.c:1810
_gnu_exception_handler at C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crt_handler.c:210
?? ??:0
?? ??:0
?? ??:0
?? ??:0
XBARE_SYMBOL at Z:\emacs-build-test\src/lisp.h:1152
 (inlined by) XSYMBOL at Z:\emacs-build-test\src/lisp.h:1164
 (inlined by) SYMBOL_NAME at Z:\emacs-build-test\src/lisp.h:2351
 (inlined by) print_object at Z:\emacs-build-test\src/print.c:2412
Fprin1 at Z:\emacs-build-test\src/print.c:776
print_error_message at Z:\emacs-build-test\src/print.c:1134
Ferror_message_string at Z:\emacs-build-test\src/print.c:1038
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
Ffuncall at Z:\emacs-build-test\src/eval.c:3022
?? ??:0
set_backtrace_args at Z:\emacs-build-test\src/eval.c:141
 (inlined by) eval_sub at Z:\emacs-build-test\src/eval.c:2517
Feval at Z:\emacs-build-test\src/eval.c:2389
top_level_2 at Z:\emacs-build-test\src/keyboard.c:1173
internal_condition_case at Z:\emacs-build-test\src/eval.c:1528
top_level_1 at Z:\emacs-build-test\src/keyboard.c:1185
internal_catch at Z:\emacs-build-test\src/eval.c:1215
command_loop at Z:\emacs-build-test\src/keyboard.c:1134
recursive_edit_1 at Z:\emacs-build-test\src/keyboard.c:742
Frecursive_edit at Z:\emacs-build-test\src/keyboard.c:825
main at Z:\emacs-build-test\src/emacs.c:2623
__tmainCRTStartup at C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:267
mainCRTStartup at C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:187
?? ??:0
?? ??:0

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
> Date: Thu, 01 Feb 2024 11:39:18 +0100
> 
> Arash Esbati <arash@gnu.org> writes:
> 
> > Ok, I will play with this and see if I can trigger the issue again; I'll
> > report back.
> 
> Here is the report.  Suppose the repo is inside Z:/emacs-build-test
> up-to-date with de020255a5c, I did:
> 
>   $ ./autogen.sh
>   $ ./configure --with-native-compilation
>   $ make
>   $ addr2line -C -f -i -p -e src/emacs.exe < lisp/emacs_backtrace.txt

Does this mean the build of the master branch still crashes for you?
I thought this was resolved long ago...  Or do you work around this in
some way?

> The results are attached.  HTH.

Thanks.  I think this means the problem with the backtrace addresses
has been solved, indeed: the frames which are printed with file name
and function name looks reasonable.  Those which remained "??" I think
are due to optimizations or something.  It would be educational to see
a corresponding backtrace from GDB, if you can capture it, since we
will be able then to compare the "??" portions with what GDB knows
about them.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> Does this mean the build of the master branch still crashes for you?
> I thought this was resolved long ago...  Or do you work around this in
> some way?

It did crash during this build, yes.  The solution provided once was to
set '-fno-optimize-sibling-calls', and this is what I usually do and it
works:

$ CFLAGS='-O2 -fno-optimize-sibling-calls' ./configure ...

> Thanks.  I think this means the problem with the backtrace addresses
> has been solved, indeed: the frames which are printed with file name
> and function name looks reasonable.  Those which remained "??" I think
> are due to optimizations or something.  It would be educational to see
> a corresponding backtrace from GDB, if you can capture it, since we
> will be able then to compare the "??" portions with what GDB knows
> about them.

Did you give me a recipe how to produce the backtrace from GDB?
Otherwise I'll go through this thread and look.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
> Date: Thu, 01 Feb 2024 12:11:25 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Does this mean the build of the master branch still crashes for you?
> > I thought this was resolved long ago...  Or do you work around this in
> > some way?
> 
> It did crash during this build, yes.  The solution provided once was to
> set '-fno-optimize-sibling-calls', and this is what I usually do and it
> works:
> 
> $ CFLAGS='-O2 -fno-optimize-sibling-calls' ./configure ...

Ah, okay.  So this is still the same version of GCC/libgccjit, and
downgrading them to an older version also solves the problem?

> > Thanks.  I think this means the problem with the backtrace addresses
> > has been solved, indeed: the frames which are printed with file name
> > and function name looks reasonable.  Those which remained "??" I think
> > are due to optimizations or something.  It would be educational to see
> > a corresponding backtrace from GDB, if you can capture it, since we
> > will be able then to compare the "??" portions with what GDB knows
> > about them.
> 
> Did you give me a recipe how to produce the backtrace from GDB?
> Otherwise I'll go through this thread and look.

I don't mind giving the instructions again:

  . run "make V=1"
  . find the last command before the crash
  . start GDB as "gdb ./emacs.exe" from the src subdirectory
  . if the crashed command was run from the lisp/ subdirectory, type
    at the GDB prompt "cd ../lisp"
  . finally, type "run" followed by the failing command line, which
    you can copy/paste from the output of "make V=1" above

This should run the failed command, and cause the crash.  Then type:

  (gdb) thread 1
  (gdb) bt

and post here everything that command produces.

Thanks.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> I'd like to see the backtrace from GDB and compare the addresses there
> with those in emacs_backtrace.txt.

I will try that later.

> Did you try to run the crashing command under GDB, as I described
> earlier?

Yes, I did that now, and the result is odd.  I did in the shell (master
branch, 0e4cc6a8bf):

  • git clean -fdx
  • ./autogen.sh
  • ./configure --with-native-compilation
  • make V=1

which ends with:

  *** "make all" failed with exit status 2.
  ***
  *** You could try to:
  *** - run "make bootstrap", which might fix the problem
  *** - run "make V=1", which displays the full commands invoked by make,
  ***   to further investigate the problem
  ***
  make[1]: *** [Makefile:414: advice-on-failure] Error 2
  make[1]: Leaving directory '/z/path/to/emacs-build-test'
  make: *** [Makefile:370: all] Error 2

Next, I inspected the build failures which look like this for example
(line breaks added manually)

  '../src/emacs.exe' -batch --no-site-file --no-site-lisp \
  --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" \
  --eval "(setq org--inhibit-version-check t)"  \
  -l comp -f batch-byte+native-compile obsolete/vip.el

  Backtrace:
  00007ff6ae7fa12e
  00007ff6ae6c8be1
  [...]
  00007ff6ae75e9aa
  ...
  make[3]: *** [Makefile:328: obsolete/vip.elc] Error 3

So I followed your advice and did:

  • cd src/
  • gdb ./emacs.exe
  • (gdb) cd ../lisp
  • (gdb) run -batch --no-site-file --no-site-lisp ... (as above)

which returns (line breaks added manually again):

  Starting program: z:\path\to\emacs-build-test\src\emacs.exe \
  -batch --no-site-file --no-site-lisp \
  --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" \
  --eval "(setq org--inhibit-version-check t)"  \
  -l comp -f batch-byte+native-compile obsolete/vip.el
  [New Thread 37728.0x6048]
  [New Thread 37728.0x73cc]
  [New Thread 37728.0x5e20]
  [New Thread 37728.0x84c8]
  [Thread 37728.0x84c8 exited with code 0]
  [Thread 37728.0x6048 exited with code 0]
  [Thread 37728.0x5e20 exited with code 0]
  [Thread 37728.0x73cc exited with code 0]
  [Inferior 1 (process 37728) exited normally]

Does this make sense?  It seems that under GDB, it works fine and it
breaks during the normal compilation.  I tried it for some other build
failures, they work under GDB.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 26 May 2023 15:05:01 +0200
> 
>   '../src/emacs.exe' -batch --no-site-file --no-site-lisp \
>   --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" \
>   --eval "(setq org--inhibit-version-check t)"  \
>   -l comp -f batch-byte+native-compile obsolete/vip.el
> 
>   Backtrace:
>   00007ff6ae7fa12e
>   00007ff6ae6c8be1
>   [...]
>   00007ff6ae75e9aa
>   ...
>   make[3]: *** [Makefile:328: obsolete/vip.elc] Error 3
> 
> So I followed your advice and did:
> 
>   • cd src/
>   • gdb ./emacs.exe
>   • (gdb) cd ../lisp
>   • (gdb) run -batch --no-site-file --no-site-lisp ... (as above)
> 
> which returns (line breaks added manually again):
> 
>   Starting program: z:\path\to\emacs-build-test\src\emacs.exe \
>   -batch --no-site-file --no-site-lisp \
>   --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" \
>   --eval "(setq org--inhibit-version-check t)"  \
>   -l comp -f batch-byte+native-compile obsolete/vip.el
>   [New Thread 37728.0x6048]
>   [New Thread 37728.0x73cc]
>   [New Thread 37728.0x5e20]
>   [New Thread 37728.0x84c8]
>   [Thread 37728.0x84c8 exited with code 0]
>   [Thread 37728.0x6048 exited with code 0]
>   [Thread 37728.0x5e20 exited with code 0]
>   [Thread 37728.0x73cc exited with code 0]
>   [Inferior 1 (process 37728) exited normally]
> 
> Does this make sense?  It seems that under GDB, it works fine and it
> breaks during the normal compilation.

Yes, it's unfortunate.

So now converting the addresses from emacs_backtrace.txt manually is
the only way forward, it seems.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> So now converting the addresses from emacs_backtrace.txt manually is
> the only way forward, it seems.

Thanks, so I did:

  • cd emacs-build-test/src
  • gdb ./emacs.exe
  • (gdb) list *0x00007ff6ae7fa12e (taken from emacs_backtrace.txt)

but GBD returns nothing.  Is there any other recipe to this?

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 26 May 2023 21:21:58 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > So now converting the addresses from emacs_backtrace.txt manually is
> > the only way forward, it seems.
> 
> Thanks, so I did:
> 
>   • cd emacs-build-test/src
>   • gdb ./emacs.exe
>   • (gdb) list *0x00007ff6ae7fa12e (taken from emacs_backtrace.txt)
> 
> but GBD returns nothing.  Is there any other recipe to this?

Is this what you get for all the addresses in emacs_backtrace.txt?
Some of them, especially the ones close to the top, are expected to
yield nothing, but not all of them.

Also, try doing the same after letting the program run to the
beginning of the 'main' function, like this:

  cd emacs-build-test/src
  gdb ./emacs.exe
  (gdb) start run -batch --no-site-file --no-site-lisp ...

(with the rest of the arguments you saw in the crashed command).

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> Is this what you get for all the addresses in emacs_backtrace.txt?
> Some of them, especially the ones close to the top, are expected to
> yield nothing, but not all of them.
>
> Also, try doing the same after letting the program run to the
> beginning of the 'main' function, like this:
>
>   cd emacs-build-test/src
>   gdb ./emacs.exe
>   (gdb) start run -batch --no-site-file --no-site-lisp ...
>
> (with the rest of the arguments you saw in the crashed command).

Ok, I tried it with the addresses emitted during compilation of
apropos.el and for some of them, the result of '(gdb) list *0xNNNN' is
attached.  Is this what you're looking for?  If so, I will do the
exercise for all of the addresses and post the result.

Best, Arash

[-- Attachment #2: addresses.txt --]
[-- Type: text/plain, Size: 3574 bytes --]

(gdb) list *0x00007ff60c14a12e
0x7ff60c14a12e is in emacs_abort (w32fns.c:11103).
11098         hm_kernel32 = LoadLibrary ("Kernel32.dll");
11099         s_pfn_CaptureStackBackTrace = (CaptureStackBackTrace_proc)
11100           get_proc_addr (hm_kernel32, "RtlCaptureStackBackTrace");
11101       }
11102     if (s_pfn_CaptureStackBackTrace)
11103       return s_pfn_CaptureStackBackTrace (0, min (BACKTRACE_LIMIT_MAX, limit),
11104                                           buffer, NULL);
11105     return 0;
11106   }
11107

(gdb) list *0x00007ff60c018be1
0x7ff60c018be1 is in terminate_due_to_signal (emacs.c:459).
454          have to unblock it if we want to really receive it.  */
455     #ifndef MSDOS
456       {
457         sigset_t unblocked;
458         sigemptyset (&unblocked);
459         sigaddset (&unblocked, sig);
460         pthread_sigmask (SIG_UNBLOCK, &unblocked, 0);
461       }
462     #endif
463

(gdb) list *0x00007ff60c039601
0x7ff60c039601 is at sysdep.c:1793.
1788    {
1789      deliver_process_signal (sig, handle_fatal_signal);
1790    }
1791
1792    static void
1793    deliver_fatal_thread_signal (int sig)
1794    {
1795      deliver_thread_signal (sig, handle_fatal_signal);
1796    }
1797

(gdb) list *0x00007ff60c1ae84a
0x7ff60c1ae84a is in _gnu_exception_handler (C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crt_handler.c:210).
205     C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crt_handler.c: No such file or directory.

(gdb) list *0x00007ff60c09e1b0
0x7ff60c09e1b0 is in styled_format (editfns.c:3460).
3455         and in that case, we won't know it here.  */
3456
3457      /* True if the output should be a multibyte string,
3458         which is true if any of the inputs is one.  */
3459      bool multibyte = multibyte_format;
3460      for (ptrdiff_t i = 1; !multibyte && i < nargs; i++)
3461        if (STRINGP (args[i]) && STRING_MULTIBYTE (args[i]))
3462          multibyte = true;
3463
3464      Lisp_Object quoting_style = message ? Ftext_quoting_style () : Qnil;

(gdb) list *0x00007ff60c0a80fa
0x7ff60c0a80fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff60c0a86ca
0x7ff60c0a86ca is in Fapply (eval.c:2635).
2630
2631      if (numargs == 0)
2632        return Ffuncall (max (1, nargs - 1), args);
2633      else if (numargs == 1)
2634        {
2635          args [nargs - 1] = XCAR (spread_arg);
2636          return Ffuncall (nargs, args);
2637        }
2638
2639      numargs += nargs - 2;

(gdb) list *0x00007ff60c0a80fa
0x7ff60c0a80fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff60c0a86ca
0x7ff60c0a86ca is in Fapply (eval.c:2635).
2630
2631      if (numargs == 0)
2632        return Ffuncall (max (1, nargs - 1), args);
2633      else if (numargs == 1)
2634        {
2635          args [nargs - 1] = XCAR (spread_arg);
2636          return Ffuncall (nargs, args);
2637        }
2638
2639      numargs += nargs - 2;

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
> Date: Sat, 27 May 2023 12:57:38 +0200
> 
> >   cd emacs-build-test/src
> >   gdb ./emacs.exe
> >   (gdb) start run -batch --no-site-file --no-site-lisp ...
> >
> > (with the rest of the arguments you saw in the crashed command).
> 
> Ok, I tried it with the addresses emitted during compilation of
> apropos.el and for some of them, the result of '(gdb) list *0xNNNN' is
> attached.  Is this what you're looking for?  If so, I will do the
> exercise for all of the addresses and post the result.

Yes, please.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

[-- Attachment #1: Type: text/plain, Size: 1814 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, please.

So these are the addresses I get for apropos.el:

--8<---------------cut here---------------start------------->8---
'../src/emacs.exe' -batch --no-site-file --no-site-lisp \
--eval "(setq load-prefer-newer t byte-compile-warnings 'all)" \
--eval "(setq org--inhibit-version-check t)"  \
-l comp -f batch-byte+native-compile apropos.el

Backtrace:
00007ff75f71a12e
00007ff75f5e8be1
00007ff75f609601
00007ff75f77e84a
00007ffa21d07ff0
00007ffa21f923d7
00007ffa21f4149c
00007ffa21f90f06
00007ff75f66e1b0
00007ff75f6780fa
00007ff75f6786ca
00007ff9addf77d1
00007ff75f6780fa
00007ff75f6786ca
00007ff9addf78e0
00007ff75f6780fa
00007ff9add42e69
00007ff75f6780fa
00007ff9add4573b
00007ff75f6780fa
00007ff9add45692
00007ff75f6780fa
00007ff9ade068d2
00007ff75f67b88e
00007ff75f6780fa
00007ff9ade0599a
00007ff75f6780fa
00007ff9ade0257b
00007ff75f67b88e
00007ff75f6c3cb8
00007ff75f6780fa
00007ff9ade189ab
00007ff9ade18bff
00007ff9ade18f9a
00007ff75f6780fa
00007ff9ade00e6c
00007ff75f6780fa
00007ff9ade00cfe
00007ff75f6780fa
00007ff9addf1fab
00007ff75f6780fa
00007ff9ade00d96
00007ff75f6780fa
00007ff9addfee0c
00007ff75f6780fa
00007ff9addff82a
00007ff75f6780fa
00007ff9addfd95d
00007ff75f6780fa
00007ff9ade1c585
00007ff75f6780fa
00007ff9ade1bfff
00007ff75f6780fa
00007ff9adedc32f
00007ff75f6780fa
00007ff9ae7cf03b
00007ff75f6780fa
00007ff9ae7c724e
00007ff75f6780fa
00007ff9ae7c32b0
00007ff75f67c2aa
00007ff75f67e9aa
...
make[3]: *** [Makefile:328: apropos.elc] Error 3
--8<---------------cut here---------------end--------------->8---

The corresponding results from GDB are attached.  The compilation breaks
for the following files:

apropos.el
checkdoc.el
obsolete/vip.el
progmodes/cc-langs.el

Do you need the information attached for all of the files above?

Best, Arash

[-- Attachment #2: apropos.el.txt --]
[-- Type: text/plain, Size: 14584 bytes --]

(gdb) list *0x00007ff75f71a12e
0x7ff75f71a12e is in emacs_abort (w32fns.c:11103).
11098         hm_kernel32 = LoadLibrary ("Kernel32.dll");
11099         s_pfn_CaptureStackBackTrace = (CaptureStackBackTrace_proc)
11100           get_proc_addr (hm_kernel32, "RtlCaptureStackBackTrace");
11101       }
11102     if (s_pfn_CaptureStackBackTrace)
11103       return s_pfn_CaptureStackBackTrace (0, min (BACKTRACE_LIMIT_MAX, limit),
11104                                           buffer, NULL);
11105     return 0;
11106   }
11107

(gdb) list *0x00007ff75f5e8be1
0x7ff75f5e8be1 is in terminate_due_to_signal (emacs.c:459).
454          have to unblock it if we want to really receive it.  */
455     #ifndef MSDOS
456       {
457         sigset_t unblocked;
458         sigemptyset (&unblocked);
459         sigaddset (&unblocked, sig);
460         pthread_sigmask (SIG_UNBLOCK, &unblocked, 0);
461       }
462     #endif
463

(gdb) list *0x00007ff75f609601
0x7ff75f609601 is at sysdep.c:1793.
1788    {
1789      deliver_process_signal (sig, handle_fatal_signal);
1790    }
1791
1792    static void
1793    deliver_fatal_thread_signal (int sig)
1794    {
1795      deliver_thread_signal (sig, handle_fatal_signal);
1796    }
1797

(gdb) list *0x00007ff75f77e84a
0x7ff75f77e84a is in _gnu_exception_handler (C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crt_handler.c:210).
205     C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crt_handler.c: No such file or directory.

(gdb) list *0x00007ff75f66e1b0
0x7ff75f66e1b0 is in styled_format (editfns.c:3460).
3455         and in that case, we won't know it here.  */
3456
3457      /* True if the output should be a multibyte string,
3458         which is true if any of the inputs is one.  */
3459      bool multibyte = multibyte_format;
3460      for (ptrdiff_t i = 1; !multibyte && i < nargs; i++)
3461        if (STRINGP (args[i]) && STRING_MULTIBYTE (args[i]))
3462          multibyte = true;
3463
3464      Lisp_Object quoting_style = message ? Ftext_quoting_style () : Qnil;

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6786ca
0x7ff75f6786ca is in Fapply (eval.c:2635).
2630
2631      if (numargs == 0)
2632        return Ffuncall (max (1, nargs - 1), args);
2633      else if (numargs == 1)
2634        {
2635          args [nargs - 1] = XCAR (spread_arg);
2636          return Ffuncall (nargs, args);
2637        }
2638
2639      numargs += nargs - 2;

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

0x7ff75f6786ca is in Fapply (eval.c:2635).
2630
2631      if (numargs == 0)
2632        return Ffuncall (max (1, nargs - 1), args);
2633      else if (numargs == 1)
2634        {
2635          args [nargs - 1] = XCAR (spread_arg);
2636          return Ffuncall (nargs, args);
2637        }
2638
2639      numargs += nargs - 2;

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f67b88e
0x7ff75f67b88e is in funcall_subr (eval.c:3055).
3050                case 3:
3051                  return subr->function.a3 (a[0], a[1], a[2]);
3052                case 4:
3053                  return subr->function.a4 (a[0], a[1], a[2], a[3]);
3054                case 5:
3055                  return subr->function.a5 (a[0], a[1], a[2], a[3], a[4]);
3056                case 6:
3057                  return subr->function.a6 (a[0], a[1], a[2], a[3], a[4], a[5]);
3058                case 7:
3059                  return subr->function.a7 (a[0], a[1], a[2], a[3], a[4], a[5],

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f67b88e
0x7ff75f67b88e is in funcall_subr (eval.c:3055).
3050                case 3:
3051                  return subr->function.a3 (a[0], a[1], a[2]);
3052                case 4:
3053                  return subr->function.a4 (a[0], a[1], a[2], a[3]);
3054                case 5:
3055                  return subr->function.a5 (a[0], a[1], a[2], a[3], a[4]);
3056                case 6:
3057                  return subr->function.a6 (a[0], a[1], a[2], a[3], a[4], a[5]);
3058                case 7:
3059                  return subr->function.a7 (a[0], a[1], a[2], a[3], a[4], a[5],

(gdb) list *0x00007ff75f6c3cb8
0x7ff75f6c3cb8 is in exec_byte_code (bytecode.c:809).
804                     goto setup_frame;
805                   }
806
807                 Lisp_Object val;
808                 if (SUBRP (call_fun) && !SUBR_NATIVE_COMPILED_DYNP (call_fun))
809                   val = funcall_subr (XSUBR (call_fun), call_nargs, call_args);
810                 else
811                   val = funcall_general (original_fun, call_nargs, call_args);
812
813                 lisp_eval_depth--;

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f6780fa
0x7ff75f6780fa is in Ffuncall (eval.c:3008).
3003      maybe_gc ();
3004
3005      if (debug_on_next_call)
3006        do_debug_on_call (Qlambda, count);
3007
3008      Lisp_Object val = funcall_general (args[0], nargs - 1, args + 1);
3009
3010      lisp_eval_depth--;
3011      if (backtrace_debug_on_exit (specpdl_ref_to_ptr (count)))
3012        val = call_debugger (list2 (Qexit, val));

(gdb) list *0x00007ff75f67c2aa
0x7ff75f67c2aa is in eval_sub (eval.c:147).
142     static void
143     set_backtrace_args (union specbinding *pdl, Lisp_Object *args, ptrdiff_t nargs)
144     {
145       eassert (pdl->kind == SPECPDL_BACKTRACE);
146       pdl->bt.args = args;
147       pdl->bt.nargs = nargs;
148     }
149
150     static void
151     set_backtrace_debug_on_exit (union specbinding *pdl, bool doe)

(gdb) list *0x00007ff75f67e9aa
0x7ff75f67e9aa is in Feval (eval.c:2375).
2370      (Lisp_Object form, Lisp_Object lexical)
2371    {
2372      specpdl_ref count = SPECPDL_INDEX ();
2373      specbind (Qinternal_interpreter_environment,
2374                CONSP (lexical) || NILP (lexical) ? lexical : list_of_t);
2375      return unbind_to (count, eval_sub (form));
2376    }
2377
2378    void
2379    grow_specpdl_allocation (void)

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
> Date: Sat, 27 May 2023 19:35:12 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Yes, please.
> 
> So these are the addresses I get for apropos.el:
> 
> --8<---------------cut here---------------start------------->8---
> '../src/emacs.exe' -batch --no-site-file --no-site-lisp \
> --eval "(setq load-prefer-newer t byte-compile-warnings 'all)" \
> --eval "(setq org--inhibit-version-check t)"  \
> -l comp -f batch-byte+native-compile apropos.el

Thanks.  It is a very strange crash, to say the least: we seem to
crash because we access an array out of its bounds or something.

Did you say that the problem goes away if you configure without
native-compilation?  If so, let's wait for Andrea to fix the problems
with that on master, and try again.

Or did you also see similar problems on the emacs-29 branch?

If this is not related to native-compilation, my first suspect is GCC
optimizations.  Try building with -Og or -O1, and see if that avoids
the problem.  Or just downgrade to GCC 12 and wait for the GCC folks
to get their act together.  (IME, using GCC version N.1 is not
recommended if you want to make sure your development environment
remains stable.  This is especially true for platforms like
MS-Windows, which are not the main target for GCC development.)

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[András Svraka]

MSYS2 might have run into a related problem. Currently Emacs 28.2 cannot be built on the Github Actions CI with GCC 13.1 and native comp enabled because the build gets stuck during native compilation of files.el (all the other elisp files are compiled). I’ve attached a backtrace from the CI.

Meanwhile several people reported successfully building Emacs on their own machines from the same MSYS2 that runs in the CI. There’s a discussion on Github [1]. Downgrading to GCC 12.2 helps, and we’re still investigating the effects of various compiler switches.

[1]: https://github.com/msys2/MINGW-packages/issues/17374

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[András Svraka]

[-- Attachment #1: Type: text/plain, Size: 30 bytes --]

Sorry, forgot the attachment.

[-- Attachment #2: files.eln.log --]
[-- Type: application/octet-stream, Size: 8968 bytes --]

Attaching to process 6872
[New Thread 6872.0x16a0]
[New Thread 6872.0x11e4]
Reading symbols from C:\_\B\src\build-UCRT64\src\emacs.exe...
(gdb) thread apply all bt

Thread 3 (Thread 6872.0x11e4):
#0  0x00007ffc3d263021 in ntdll!DbgBreakPoint () from C:\Windows\SYSTEM32\ntdll.dll
#1  0x00007ffc3d2901be in ntdll!DbgUiRemoteBreakin () from C:\Windows\SYSTEM32\ntdll.dll
#2  0x00007ffc3b6a4dd0 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#3  0x00007ffc3d23e3db in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#4  0x0000000000000000 in ?? ()

Thread 2 (Thread 6872.0x16a0):
#0  0x00007ffc3d262fc4 in ntdll!ZwWaitForWorkViaWorkerFactory () from C:\Windows\SYSTEM32\ntdll.dll
#1  0x00007ffc3d1cb77f in ntdll!RtlInitializeResource () from C:\Windows\SYSTEM32\ntdll.dll
#2  0x00007ffc3b6a4dd0 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#3  0x00007ffc3d23e3db in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#4  0x0000000000000000 in ?? ()

Thread 1 (Thread 6872.0xb48):
#0  0x00007ffc3a8911c4 in win32u!NtUserWaitMessage () from C:\Windows\System32\win32u.dll
#1  0x00007ffc3cbfb25c in USER32!EndDialog () from C:\Windows\System32\user32.dll
#2  0x00007ffc3cbfb797 in USER32!EndDialog () from C:\Windows\System32\user32.dll
#3  0x00007ffc3cc1ef66 in USER32!SoftModalMessageBox () from C:\Windows\System32\user32.dll
#4  0x00007ffc3cc1d8a1 in USER32!DrawStateA () from C:\Windows\System32\user32.dll
#5  0x00007ffc3cc1e695 in USER32!MessageBoxTimeoutW () from C:\Windows\System32\user32.dll
#6  0x00007ffc3cc1e488 in USER32!MessageBoxTimeoutA () from C:\Windows\System32\user32.dll
#7  0x00007ffc3cc1e09e in USER32!MessageBoxA () from C:\Windows\System32\user32.dll
#8  0x00007ff6747977d5 in emacs_abort ()
#9  0x00007ff674696308 in terminate_due_to_signal ()
#10 0x00007ff6746af949 in deliver_fatal_thread_signal ()
#11 0x00007ff6747f4992 in _gnu_exception_handler (exception_data=0x121dfa2d0) at C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crt_handler.c:213
#12 0x00007ffc3a7c55f0 in ucrtbase!__C_specific_handler () from C:\Windows\System32\ucrtbase.dll
#13 0x00007ffc3d2643af in ntdll!.chkstk () from C:\Windows\SYSTEM32\ntdll.dll
#14 0x00007ffc3d1f170e in ntdll!RtlVirtualUnwind2 () from C:\Windows\SYSTEM32\ntdll.dll
#15 0x00007ffc3d2633be in ntdll!KiUserExceptionDispatcher () from C:\Windows\SYSTEM32\ntdll.dll
#16 0x00007ff6747359d1 in print_object ()
#17 0x00007ff6747373a1 in Fprin1_to_string ()
#18 0x00007ffc0e2c267b in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_9 (par_0=0x19dd0292503, par_1=<optimized out>) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\byte-opt-9c5f25f5-718f7647.eln
#19 0x00007ff67471640f in Ffuncall ()
#20 0x00007ffc0e2c4007 in F627974652d6f7074696d697a652d666f726d_byte_optimize_form_0 (par_0=<optimized out>, par_1=0x0) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\byte-opt-9c5f25f5-718f764--Type <RET> for more, q to quit, c to continue without paging--c
7.eln
#21 0x00007ff67471640f in Ffuncall ()
#22 0x00007ffc0e2c3f7a in F627974652d6f7074696d697a652d6f6e652d666f726d_byte_optimize_one_form_0 (par_0=0x19dd0292503, par_1=0x0) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\byte-opt-9c5f25f5-718f7647.eln
#23 0x00007ff67471640f in Ffuncall ()
#24 0x00007ffc1196511a in F627974652d636f6d70696c652d746f702d6c6576656c_byte_compile_top_level_0 (par_0=0x19dd0292503, par_1=0x0, par_2=0xc0, par_3=0x19dcfaeaf13, par_4=0x0) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#25 0x00007ff6747180fe in funcall_subr ()
#26 0x00007ff67471640f in Ffuncall ()
#27 0x00007ffc119641d0 in F627974652d636f6d70696c652d6c616d626461_byte_compile_lambda_0 (par_0=<optimized out>, par_1=<optimized out>, par_2=0x0) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#28 0x00007ff67471640f in Ffuncall ()
#29 0x00007ffc11961371 in F627974652d636f6d70696c652d66696c652d666f726d2d6465666d756d626c65_byte_compile_file_form_defmumble_0 (par_0=0xffff81a75a3c5950, par_1=0x0, par_2=0x19dd029fd93, par_3=0x19dd0292953, par_4=0x0) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#30 0x00007ff6747180fe in funcall_subr ()
#31 0x00007ff67471640f in Ffuncall ()
#32 0x00007ff6747557b8 in exec_byte_code ()
#33 0x00007ff674719ba5 in funcall_lambda ()
#34 0x00007ff6747163c9 in Ffuncall ()
#35 0x00007ffc11976623 in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_84 (par_0=par_0@entry=0x19dd02929c3, par_1=par_1@entry=0xffff81a75a3c5950, par_2=par_2@entry=0x0, par_3=par_3@entry=0x19dd0292993, par_4=par_4@entry=0x0, par_5=0x19dd0292973) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#36 0x00007ffc11976727 in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_85 (par_0=par_0@entry=0x19dd02929c3, par_1=par_1@entry=0xffff81a75a3c5950, par_2=par_2@entry=0x0, par_3=0x19dd0292993, par_4=0x0) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#37 0x00007ffc11976ac1 in F627974652d636f6d70696c652d66696c652d666f726d2d646566616c696173_byte_compile_file_form_defalias_0 (par_0=0x19dd02929c3) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#38 0x00007ff67471640f in Ffuncall ()
#39 0x00007ffc1195fb44 in F627974652d636f6d70696c652d66696c652d666f726d_byte_compile_file_form_0 (par_0=0x19dd02929c3) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#40 0x00007ff67471640f in Ffuncall ()
#41 0x00007ffc1195fa66 in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_47 (par_0=0x19dd029fb03) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#42 0x00007ff67471640f in Ffuncall ()
#43 0x00007ffc119520a3 in F627974652d636f6d70696c652d726563757273652d746f706c6576656c_byte_compile_recurse_toplevel_0 (par_0=0x19dd029fd33, par_1=0x19dcfe50a45) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#44 0x00007ff67471640f in Ffuncall ()
#45 0x00007ffc1195facd in F627974652d636f6d70696c652d746f706c6576656c2d66696c652d666f726d_byte_compile_toplevel_file_form_0 (par_0=<optimized out>) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#46 0x00007ff67471640f in Ffuncall ()
#47 0x00007ffc1195db33 in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_45 (par_0=0x19dd012c355) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#48 0x00007ff67471640f in Ffuncall ()
#49 0x00007ffc1195e31f in F627974652d636f6d70696c652d66726f6d2d627566666572_byte_compile_from_buffer_0 (par_0=<optimized out>) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#50 0x00007ff67471640f in Ffuncall ()
#51 0x00007ffc1195c3c7 in F627974652d636f6d70696c652d66696c65_byte_compile_file_0 (par_0=<optimized out>, par_1=<optimized out>) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\bytecomp-12882072-4d43b0ac.eln
#52 0x00007ff67471640f in Ffuncall ()
#53 0x00007ff6747557b8 in exec_byte_code ()
#54 0x00007ff674719ba5 in funcall_lambda ()
#55 0x00007ff6747163c9 in Ffuncall ()
#56 0x00007ff674718950 in Fapply ()
#57 0x00007ff67471640f in Ffuncall ()
#58 0x00007ff6747557b8 in exec_byte_code ()
#59 0x00007ff674719ba5 in funcall_lambda ()
#60 0x00007ff6747163c9 in Ffuncall ()
#61 0x00007ffc0ec7b2e5 in F636f6d702d7370696c6c2d6c6170_comp_spill_lap_0 (par_0=0x19dcfab7c44) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\comp-7672a6ed-ac6bcf4e.eln
#62 0x00007ff67471640f in Ffuncall ()
#63 0x00007ffc0ecb121f in F636f6d702d2d6e61746976652d636f6d70696c65_comp__native_compile_0 (par_0=<optimized out>, par_1=<optimized out>, par_2=<optimized out>) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\comp-7672a6ed-ac6bcf4e.eln
#64 0x00007ff67471640f in Ffuncall ()
#65 0x00007ffc0ecb24b2 in F62617463682d6e61746976652d636f6d70696c65_batch_native_compile_0 (par_0=<optimized out>) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\comp-7672a6ed-ac6bcf4e.eln
#66 0x00007ff67471920f in eval_sub ()
#67 0x00007ff67471abdb in Feval ()
#68 0x00007ff67471640f in Ffuncall ()
#69 0x00007ff6747557b8 in exec_byte_code ()
#70 0x00007ff674719ba5 in funcall_lambda ()
#71 0x00007ff6747163c9 in Ffuncall ()
#72 0x00007ff6747557b8 in exec_byte_code ()
#73 0x00007ff674719ba5 in funcall_lambda ()
#74 0x00007ff6747163c9 in Ffuncall ()
#75 0x00007ff6747557b8 in exec_byte_code ()
#76 0x00007ff674719ba5 in funcall_lambda ()
#77 0x00007ff674719e89 in apply_lambda ()
#78 0x00007ff674718d14 in eval_sub ()
#79 0x00007ff67471abdb in Feval ()
#80 0x00007ff6747152dd in internal_condition_case ()
#81 0x00007ff6746977cd in top_level_1 ()
#82 0x00007ff67471524b in internal_catch ()
#83 0x00007ff6746969a5 in command_loop ()
#84 0x00007ff67469d1d4 in recursive_edit_1 ()
#85 0x00007ff67469d549 in Frecursive_edit ()
#86 0x00007ff674805ca6 in main ()

[-- Attachment #3: Type: text/plain, Size: 733 bytes --]



> On 2023. Jun 1., at 9:31, András Svraka <svraka.andras@gmail.com> wrote:
> 
> MSYS2 might have run into a related problem. Currently Emacs 28.2 cannot be built on the Github Actions CI with GCC 13.1 and native comp enabled because the build gets stuck during native compilation of files.el (all the other elisp files are compiled). I’ve attached a backtrace from the CI.
> 
> Meanwhile several people reported successfully building Emacs on their own machines from the same MSYS2 that runs in the CI. There’s a discussion on Github [1]. Downgrading to GCC 12.2 helps, and we’re still investigating the effects of various compiler switches.
> 
> [1]: https://github.com/msys2/MINGW-packages/issues/17374
> 
> 

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: András Svraka <svraka.andras@gmail.com>
> Date: Thu, 1 Jun 2023 09:37:26 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  Arash Esbati <arash@gnu.org>,
>  akrl@sdf.org
> 
> Thread 1 (Thread 6872.0xb48):
> #0  0x00007ffc3a8911c4 in win32u!NtUserWaitMessage () from C:\Windows\System32\win32u.dll
> #1  0x00007ffc3cbfb25c in USER32!EndDialog () from C:\Windows\System32\user32.dll
> #2  0x00007ffc3cbfb797 in USER32!EndDialog () from C:\Windows\System32\user32.dll
> #3  0x00007ffc3cc1ef66 in USER32!SoftModalMessageBox () from C:\Windows\System32\user32.dll
> #4  0x00007ffc3cc1d8a1 in USER32!DrawStateA () from C:\Windows\System32\user32.dll
> #5  0x00007ffc3cc1e695 in USER32!MessageBoxTimeoutW () from C:\Windows\System32\user32.dll
> #6  0x00007ffc3cc1e488 in USER32!MessageBoxTimeoutA () from C:\Windows\System32\user32.dll
> #7  0x00007ffc3cc1e09e in USER32!MessageBoxA () from C:\Windows\System32\user32.dll
> #8  0x00007ff6747977d5 in emacs_abort ()
> #9  0x00007ff674696308 in terminate_due_to_signal ()
> #10 0x00007ff6746af949 in deliver_fatal_thread_signal ()

This means Emacs crashed during native compilation, and is waiting for
"someone" to respond to the Abort dialog.  Since you are running CI
unattended, it might be a good idea to modify lisp/Makefile to pass

  --eval '(setq w32-disable-abort-dialog t)'

option on the batch compilation command line.

Why Emacs crashed is a separate issue:

> #11 0x00007ff6747f4992 in _gnu_exception_handler (exception_data=0x121dfa2d0) at C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crt_handler.c:213
> #12 0x00007ffc3a7c55f0 in ucrtbase!__C_specific_handler () from C:\Windows\System32\ucrtbase.dll
> #13 0x00007ffc3d2643af in ntdll!.chkstk () from C:\Windows\SYSTEM32\ntdll.dll
> #14 0x00007ffc3d1f170e in ntdll!RtlVirtualUnwind2 () from C:\Windows\SYSTEM32\ntdll.dll
> #15 0x00007ffc3d2633be in ntdll!KiUserExceptionDispatcher () from C:\Windows\SYSTEM32\ntdll.dll
> #16 0x00007ff6747359d1 in print_object ()
> #17 0x00007ff6747373a1 in Fprin1_to_string ()
> #18 0x00007ffc0e2c267b in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_9 (par_0=0x19dd0292503, par_1=<optimized out>) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\byte-opt-9c5f25f5-718f7647.eln
> #19 0x00007ff67471640f in Ffuncall ()
> #20 0x00007ffc0e2c4007 in F627974652d6f7074696d697a652d666f726d_byte_optimize_form_0 (par_0=<optimized out>, par_1=0x0) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\byte-opt-9c5f25f5-718f764--Type <RET> for more, q to quit, c to continue without paging--c
> 7.eln

This tells that the crash was inside print_object, which was called by
prin1-to-string, probably because Emacs tried to print some invalid
Lisp object.  Since there are no line numbers in the backtrace, I
cannot tell more.

I also understand that the crash is avoided by using lower
optimization levels, and perhaps other non-default GCC options could
be involved.  So the jury is still out on whether this is an Emacs bug
or a (MinGW) GCC bug.

Thanks.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: András Svraka <svraka.andras@gmail.com>
>> Date: Thu, 1 Jun 2023 09:37:26 +0200
>> Cc: Eli Zaretskii <eliz@gnu.org>,
>>  Arash Esbati <arash@gnu.org>,
>>  akrl@sdf.org
>> 
>> Thread 1 (Thread 6872.0xb48):
>> #0  0x00007ffc3a8911c4 in win32u!NtUserWaitMessage () from C:\Windows\System32\win32u.dll
>> #1  0x00007ffc3cbfb25c in USER32!EndDialog () from C:\Windows\System32\user32.dll
>> #2  0x00007ffc3cbfb797 in USER32!EndDialog () from C:\Windows\System32\user32.dll
>> #3  0x00007ffc3cc1ef66 in USER32!SoftModalMessageBox () from C:\Windows\System32\user32.dll
>> #4  0x00007ffc3cc1d8a1 in USER32!DrawStateA () from C:\Windows\System32\user32.dll
>> #5  0x00007ffc3cc1e695 in USER32!MessageBoxTimeoutW () from C:\Windows\System32\user32.dll
>> #6  0x00007ffc3cc1e488 in USER32!MessageBoxTimeoutA () from C:\Windows\System32\user32.dll
>> #7  0x00007ffc3cc1e09e in USER32!MessageBoxA () from C:\Windows\System32\user32.dll
>> #8  0x00007ff6747977d5 in emacs_abort ()
>> #9  0x00007ff674696308 in terminate_due_to_signal ()
>> #10 0x00007ff6746af949 in deliver_fatal_thread_signal ()
>
> This means Emacs crashed during native compilation, and is waiting for
> "someone" to respond to the Abort dialog.  Since you are running CI
> unattended, it might be a good idea to modify lisp/Makefile to pass
>
>   --eval '(setq w32-disable-abort-dialog t)'
>
> option on the batch compilation command line.
>
> Why Emacs crashed is a separate issue:
>
>> #11 0x00007ff6747f4992 in _gnu_exception_handler (exception_data=0x121dfa2d0) at C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crt_handler.c:213
>> #12 0x00007ffc3a7c55f0 in ucrtbase!__C_specific_handler () from C:\Windows\System32\ucrtbase.dll
>> #13 0x00007ffc3d2643af in ntdll!.chkstk () from C:\Windows\SYSTEM32\ntdll.dll
>> #14 0x00007ffc3d1f170e in ntdll!RtlVirtualUnwind2 () from C:\Windows\SYSTEM32\ntdll.dll
>> #15 0x00007ffc3d2633be in ntdll!KiUserExceptionDispatcher () from C:\Windows\SYSTEM32\ntdll.dll
>> #16 0x00007ff6747359d1 in print_object ()
>> #17 0x00007ff6747373a1 in Fprin1_to_string ()
>> #18 0x00007ffc0e2c267b in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_9 (par_0=0x19dd0292503, par_1=<optimized out>) from c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\byte-opt-9c5f25f5-718f7647.eln
>> #19 0x00007ff67471640f in Ffuncall ()
>> #20 0x00007ffc0e2c4007 in
>> F627974652d6f7074696d697a652d666f726d_byte_optimize_form_0
>> (par_0=<optimized out>, par_1=0x0) from
>> c:\_\B\src\build-UCRT64\native-lisp\28.2-16aa216e\byte-opt-9c5f25f5-718f764--Type
>> <RET> for more, q to quit, c to continue without paging--c
>> 7.eln
>
> This tells that the crash was inside print_object, which was called by
> prin1-to-string, probably because Emacs tried to print some invalid
> Lisp object.  Since there are no line numbers in the backtrace, I
> cannot tell more.
>
> I also understand that the crash is avoided by using lower
> optimization levels, and perhaps other non-default GCC options could
> be involved.  So the jury is still out on whether this is an Emacs bug
> or a (MinGW) GCC bug.

Didn't had the time to follow this thread carefully, but I think would
be interesting to know if someone did try a native compiled Emacs on GCC
13.1.

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[András Svraka]

> On 2023. Jun 1., at 10:49, Andrea Corallo <acorallo@gnu.org> wrote:
> 
> Didn't had the time to follow this thread carefully, but I think would
> be interesting to know if someone did try a native compiled Emacs on GCC
> 13.1.
> 
>  Andrea

It works on Windows with MSYS2, at least several people managed to build it on their machines. What we’re struggling with is building it in the GitHub CI for a release. 


András

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[András Svraka]

> On 2023. Jun 1., at 10:42, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> This means Emacs crashed during native compilation, and is waiting for
> "someone" to respond to the Abort dialog.  Since you are running CI
> unattended, it might be a good idea to modify lisp/Makefile to pass
> 
>  --eval '(setq w32-disable-abort-dialog t)'
> 
> option on the batch compilation command line.

Ran a new build on the CI with this option. Compilation of files.el now properly fails with the following errors:

  make[3]: *** [Makefile:298: ../../emacs-28.2/lisp/files.eln] Error 127
  make[2]: *** [Makefile:813: ../../emacs-28.2/lisp/files.eln] Error 2

After which the build continues, and files.eln is skipped. The first line refers to lisp/Makefile, the second to src/Makefile, which launches the former. The CI produced a dump as well. Is that of any help?

https://github.com/svraka/MINGW-packages/actions/runs/5144666663/

> This tells that the crash was inside print_object, which was called by
> prin1-to-string, probably because Emacs tried to print some invalid
> Lisp object.  Since there are no line numbers in the backtrace, I
> cannot tell more.

If I'm not mistaken, line numbers require a debug build. However, with a debug option compilation never got stuck.


András

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: András Svraka <svraka.andras@gmail.com>
> Date: Thu, 1 Jun 2023 17:30:39 +0200
> Cc: 63365@debbugs.gnu.org, arash@gnu.org, akrl@sdf.org
> 
> > This tells that the crash was inside print_object, which was called by
> > prin1-to-string, probably because Emacs tried to print some invalid
> > Lisp object.  Since there are no line numbers in the backtrace, I
> > cannot tell more.
> 
> If I'm not mistaken, line numbers require a debug build. However, with a debug option compilation never got stuck.

Even with optimizations?  And even with older versions of GCC?  And
even with the default GCC options determined by the configure script?

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> Did you say that the problem goes away if you configure without
> native-compilation?

Yes, no problems when --without-native-compilation is passed to
configure.

> If so, let's wait for Andrea to fix the problems with that on master,
> and try again.

I tried the latest master (a902156068), and the issue remains.

> Or did you also see similar problems on the emacs-29 branch?

Yes, issue remains with 0eba9cf651.

> If this is not related to native-compilation, my first suspect is GCC
> optimizations.  Try building with -Og or -O1, and see if that avoids
> the problem.

I will try the ideas above and report back.

> Or just downgrade to GCC 12 and wait for the GCC folks to get their
> act together.  (IME, using GCC version N.1 is not recommended if you
> want to make sure your development environment remains stable.  This
> is especially true for platforms like MS-Windows, which are not the
> main target for GCC development.)

It seams that downgrading to GCC 12 is the way.  Upgrading from GCC 11
to 12 went really smooth, so I didn't expect this kind of issues.  I
hope that someone more knowledgeable than me could report this to the
GCC folks so they can have a look at it.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
> Date: Thu, 08 Jun 2023 12:21:11 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Did you say that the problem goes away if you configure without
> > native-compilation?
> 
> Yes, no problems when --without-native-compilation is passed to
> configure.
> 
> > If so, let's wait for Andrea to fix the problems with that on master,
> > and try again.
> 
> I tried the latest master (a902156068), and the issue remains.
> 
> > Or did you also see similar problems on the emacs-29 branch?
> 
> Yes, issue remains with 0eba9cf651.

Andrea, can you suggest some ideas for what to try/test, or
alternatively how to prepare a concise test/reproducer for the GCC
folks?  AFAIU, GCC 13.1 does work on GNU/Linux to produce a working
Emacs with native-compilation, so this is probably Windows-specific.

Arash, are you building with that _FORTIFY_SOURCE option that was
discussed in bug#63752?  If so, please try without that (but still
with GCC 13.1).

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

[-- Attachment #1: Type: text/plain, Size: 1217 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arash Esbati <arash@gnu.org>
>> Cc: 63365@debbugs.gnu.org,  akrl@sdf.org
>> Date: Thu, 08 Jun 2023 12:21:11 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Did you say that the problem goes away if you configure without
>> > native-compilation?
>> 
>> Yes, no problems when --without-native-compilation is passed to
>> configure.
>> 
>> > If so, let's wait for Andrea to fix the problems with that on master,
>> > and try again.
>> 
>> I tried the latest master (a902156068), and the issue remains.
>> 
>> > Or did you also see similar problems on the emacs-29 branch?
>> 
>> Yes, issue remains with 0eba9cf651.
>
> Andrea, can you suggest some ideas for what to try/test, or
> alternatively how to prepare a concise test/reproducer for the GCC
> folks?  AFAIU, GCC 13.1 does work on GNU/Linux

I'm giving it a try mow to be sure.

> to produce a working
> Emacs with native-compilation, so this is probably Windows-specific.

I'd personally start producing an Emacs made only of bytecode but with
native compiler capabilies.  To achieve this one can hack the Makefile
removing the native code specific parts, something like the very much
untested attached.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: build-hack.diff --]
[-- Type: text/x-diff, Size: 1195 bytes --]

diff --git a/lisp/Makefile.in b/lisp/Makefile.in
index 1e0935f565f..635dd7d1450 100644
--- a/lisp/Makefile.in
+++ b/lisp/Makefile.in
@@ -278,23 +278,9 @@ TAGS:
 THEFILE = no-such-file
 .PHONY: $(THEFILE)c
 $(THEFILE)c:
-ifeq ($(HAVE_NATIVE_COMP),yes)
-	$(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) \
-		-l comp -f byte-compile-refresh-preloaded \
-		-f batch-byte+native-compile $(THEFILE)
-else
 	$(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) \
 		-l bytecomp -f byte-compile-refresh-preloaded \
 		-f batch-byte-compile $(THEFILE)
-endif
-
-ifeq ($(HAVE_NATIVE_COMP),yes)
-.PHONY: $(THEFILE)n
-$(THEFILE)n:
-	$(AM_V_ELN)$(emacs) $(BYTE_COMPILE_FLAGS) \
-		-l comp -f byte-compile-refresh-preloaded \
-		--eval '(batch-native-compile t)' $(THEFILE)
-endif
 
 # Files MUST be compiled one by one. If we compile several files in a
 # row (i.e., in the same instance of Emacs) we can't make sure that
@@ -323,11 +309,6 @@ .el.elc:
 	-l comp -f batch-byte-compile $<
 	TZ=UTC0 touch -t 197001010000 $@
 else
-.el.elc:
-	$(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) \
-	-l comp -f batch-byte+native-compile $<
-endif
-else
 .el.elc:
 	$(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) -f batch-byte-compile $<
 endif

[-- Attachment #3: Type: text/plain, Size: 426 bytes --]


I'd also nil `native-comp-jit-compilation' in the early init.

After that one should still be able to run the native compiler testsuite
with say:

./src/emacs -batch -l ert -l test/src/comp-tests.el -f ert-run-tests-batch-and-exit

We want probably start running first all tests but the bootstrap one and
if they pass run this as well.  At this point probably start reasoning
depending on the output.

Best Regards

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: Arash Esbati <arash@gnu.org>,  63365@debbugs.gnu.org
> Date: Thu, 08 Jun 2023 10:02:22 -0400
> 
> > Andrea, can you suggest some ideas for what to try/test, or
> > alternatively how to prepare a concise test/reproducer for the GCC
> > folks?  AFAIU, GCC 13.1 does work on GNU/Linux
> 
> I'm giving it a try mow to be sure.

Thanks, that'd be good.

> > to produce a working
> > Emacs with native-compilation, so this is probably Windows-specific.
> 
> I'd personally start producing an Emacs made only of bytecode but with
> native compiler capabilies.  To achieve this one can hack the Makefile
> removing the native code specific parts, something like the very much
> untested attached.

OK, but I thin Arash should try building without -D_FORTIFY_SOURCE=1
cpp option first.  Because if that works, it then becomes a general
MinGW GCC problem with using that option, and not something specific
to Emacs or libgccjit.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <acorallo@gnu.org>
>> Cc: Arash Esbati <arash@gnu.org>,  63365@debbugs.gnu.org
>> Date: Thu, 08 Jun 2023 10:02:22 -0400
>> 
>> > Andrea, can you suggest some ideas for what to try/test, or
>> > alternatively how to prepare a concise test/reproducer for the GCC
>> > folks?  AFAIU, GCC 13.1 does work on GNU/Linux
>> 
>> I'm giving it a try mow to be sure.
>
> Thanks, that'd be good.

I confirm Emacs 29 bootstraps fine native compiled with GCC+libgccjit
13.1 x86_64-pc-linux-gnu here.

>> > to produce a working
>> > Emacs with native-compilation, so this is probably Windows-specific.
>> 
>> I'd personally start producing an Emacs made only of bytecode but with
>> native compiler capabilies.  To achieve this one can hack the Makefile
>> removing the native code specific parts, something like the very much
>> untested attached.
>
> OK, but I thin Arash should try building without -D_FORTIFY_SOURCE=1
> cpp option first.  Because if that works, it then becomes a general
> MinGW GCC problem with using that option, and not something specific
> to Emacs or libgccjit.

Agree.

Thanks

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> Arash, are you building with that _FORTIFY_SOURCE option that was
> discussed in bug#63752?

No, never used that option.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> If this is not related to native-compilation, my first suspect is GCC
> optimizations.  Try building with -Og or -O1, and see if that avoids
> the problem.

Ok, I tried the different optimization levels and this is the result.  

Running the standard procedure

  $ git clean -fdx
  $ ./autogen.sh
  $ ./configure --with-native-compilation

Gives this in the summary

  What compiler should emacs be built with?  gcc  -g3 -O2 -gdwarf-2

which breaks during the make run.

Lowering the -O option works for these three scenarios and Emacs builds
successfully with GCC 13.1:

  $ git clean -fdx
  $ ./autogen.sh
  $ CFLAGS='-g3 -O0 -gdwarf-2' ./configure --with-native-compilation
  $ make

  $ git clean -fdx
  $ ./autogen.sh
  $ CFLAGS='-g3 -O1 -gdwarf-2' ./configure --with-native-compilation
  $ make

  $ git clean -fdx
  $ ./autogen.sh
  $ CFLAGS='-g3 -Og -gdwarf-2' ./configure --with-native-compilation
  $ make

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Cyril Arnould]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

I've been playing around with compiler options. From my findings, the
-foptimize-sibling-calls flag breaks the build:

./autogen.sh
CFLAGS='-g3 -O1 -gdwarf-2 -foptimize-sibling-calls' \
./configure --with-native-compilation
make

OTOH, using -O1 with every flag listed under -O2 *but*
-foptimize-sibling-calls results in the build succeeding. The following
works as well for me:

./autogen.sh
CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls' \
./configure --with-native-compilation
make

[-- Attachment #2: Type: text/html, Size: 2091 bytes --]

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Cyril Arnould <cyril.arnould@outlook.com>
> CC: "eliz@gnu.org" <eliz@gnu.org>, "63365@debbugs.gnu.org"
> 	<63365@debbugs.gnu.org>, "  akrl@sdf.org" <akrl@sdf.org>,
> 	András Svraka <svraka.andras@gmail.com>
> Date: Fri, 16 Jun 2023 09:04:16 +0000
> 
> I've been playing around with compiler options. From my findings, the
> 
> -foptimize-sibling-calls flag breaks the build:
> 
>  
> 
> ./autogen.sh
> 
> CFLAGS='-g3 -O1 -gdwarf-2 -foptimize-sibling-calls' \
> 
> ./configure --with-native-compilation
> 
> make
> 
>  
> 
> OTOH, using -O1 with every flag listed under -O2 *but*
> 
> -foptimize-sibling-calls results in the build succeeding. The following
> 
> works as well for me:
> 
>  
> 
> ./autogen.sh
> 
> CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls' \
> 
> ./configure --with-native-compilation
> 
> make

Andrea, any reason that this GCC switch should break
native-compilation?  Does the same problem happen on GNU/Linux?

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Cyril Arnould <cyril.arnould@outlook.com>
>> CC: "eliz@gnu.org" <eliz@gnu.org>, "63365@debbugs.gnu.org"
>> 	<63365@debbugs.gnu.org>, "  akrl@sdf.org" <akrl@sdf.org>,
>> 	András Svraka <svraka.andras@gmail.com>
>> Date: Fri, 16 Jun 2023 09:04:16 +0000
>> 
>> I've been playing around with compiler options. From my findings, the
>> 
>> -foptimize-sibling-calls flag breaks the build:
>> 
>>  
>> 
>> ./autogen.sh
>> 
>> CFLAGS='-g3 -O1 -gdwarf-2 -foptimize-sibling-calls' \
>> 
>> ./configure --with-native-compilation
>> 
>> make
>> 
>>  
>> 
>> OTOH, using -O1 with every flag listed under -O2 *but*
>> 
>> -foptimize-sibling-calls results in the build succeeding. The following
>> 
>> works as well for me:
>> 
>>  
>> 
>> ./autogen.sh
>> 
>> CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls' \
>> 
>> ./configure --with-native-compilation
>> 
>> make
>
> Andrea, any reason that this GCC switch should break
> native-compilation?

At this point I start to suspect that this is not strictly related to
native compilation.

The configure flags are used only for our regular C code and not for
Elisp native compilation.  So it might be that some kind of
misscompilaiton of our C code is happening only with
--with-native-compilation, but looks to me it's not our compiler or
libgccjit the issue here.

> Does the same problem happen on GNU/Linux?

I tried now on GCC 13.1.1

git clean -xfd && ./autogen.sh
CFLAGS='-g3 -O1 -gdwarf-2 -foptimize-sibling-calls'
./configure --with-native-compilation && make -j16

and have complete bootstrap succesfully on emacs-29 (65f355ea0a3), my
dev machine is still x86_64-pc-linux-gnu.

One idea might be asking GCC to log what optimize-sibling-calls is doing
and see if any of our C function supporting native compilation is
touched by this pass (I believe is called in GCC tree-tailcall).

CFLAGS='-g3 -O1 -gdwarf-2 -foptimize-sibling-calls -fdump-tree-tailr' && make bootstrap

should let GCC dump what this pass is for each C compilation unit, I'd
start looking at the output for comp.c (I've comp.c.045t.tailr1 and comp.c.125t.tailr2).  

Hope it helps!

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Andrea Corallo <acorallo@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Cyril Arnould <cyril.arnould@outlook.com>
>>> CC: "eliz@gnu.org" <eliz@gnu.org>, "63365@debbugs.gnu.org"
>>> 	<63365@debbugs.gnu.org>, "  akrl@sdf.org" <akrl@sdf.org>,
>>> 	András Svraka <svraka.andras@gmail.com>
>>> Date: Fri, 16 Jun 2023 09:04:16 +0000
>>> 
>>> I've been playing around with compiler options. From my findings, the
>>> 
>>> -foptimize-sibling-calls flag breaks the build:
>>> 
>>>  
>>> 
>>> ./autogen.sh
>>> 
>>> CFLAGS='-g3 -O1 -gdwarf-2 -foptimize-sibling-calls' \
>>> 
>>> ./configure --with-native-compilation
>>> 
>>> make
>>> 
>>>  
>>> 
>>> OTOH, using -O1 with every flag listed under -O2 *but*
>>> 
>>> -foptimize-sibling-calls results in the build succeeding. The following
>>> 
>>> works as well for me:
>>> 
>>>  
>>> 
>>> ./autogen.sh
>>> 
>>> CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls' \
>>> 
>>> ./configure --with-native-compilation
>>> 
>>> make
>>
>> Andrea, any reason that this GCC switch should break
>> native-compilation?
>
> At this point I start to suspect that this is not strictly related to
> native compilation.
>
> The configure flags are used only for our regular C code and not for
> Elisp native compilation.  So it might be that some kind of
> misscompilaiton of our C code is happening only with
> --with-native-compilation, but looks to me it's not our compiler or
> libgccjit the issue here.
>
>> Does the same problem happen on GNU/Linux?
>
> I tried now on GCC 13.1.1
>
> git clean -xfd && ./autogen.sh
> CFLAGS='-g3 -O1 -gdwarf-2 -foptimize-sibling-calls'
> ./configure --with-native-compilation && make -j16
>
> and have complete bootstrap succesfully on emacs-29 (65f355ea0a3), my
> dev machine is still x86_64-pc-linux-gnu.
>
> One idea might be asking GCC to log what optimize-sibling-calls is doing
> and see if any of our C function supporting native compilation is
> touched by this pass (I believe is called in GCC tree-tailcall).
>
> CFLAGS='-g3 -O1 -gdwarf-2 -foptimize-sibling-calls -fdump-tree-tailr' && make bootstrap
>
> should let GCC dump what this pass is for each C compilation unit, I'd
> start looking at the output for comp.c (I've comp.c.045t.tailr1 and comp.c.125t.tailr2).  
>
> Hope it helps!
>
>   Andrea

Just to add...

Looking at GCC source code, one should be able to grep within those
files for "Found tail call " or "Eliminated tail recursion in bb" as a
marker of some optimization happening there :)

Best Regards

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Cyril Arnould <cyril.arnould@outlook.com> writes:

> I've been playing around with compiler options. From my findings, the
>
> -foptimize-sibling-calls flag breaks the build:

Many thanks for digging into this.  I can confirm that Emacs builds with
-fno-optimize-sibling-calls in CFLAGS.  I have a script which (now)
does:

  CFLAGS='-O3 -g0 -pipe -fno-optimize-sibling-calls' \
  ./configure ...

and it works again.

Now the question is: Is this considered a GCC bug and we should close
this report as notabug?

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: "eliz@gnu.org" <eliz@gnu.org>,  "63365@debbugs.gnu.org"
>  <63365@debbugs.gnu.org>,  " akrl@sdf.org" <akrl@sdf.org>,
>   András Svraka
>  <svraka.andras@gmail.com>
> Date: Thu, 22 Jun 2023 22:34:48 +0200
> 
> Many thanks for digging into this.  I can confirm that Emacs builds with
> -fno-optimize-sibling-calls in CFLAGS.  I have a script which (now)
> does:
> 
>   CFLAGS='-O3 -g0 -pipe -fno-optimize-sibling-calls' \
>   ./configure ...
> 
> and it works again.
> 
> Now the question is: Is this considered a GCC bug and we should close
> this report as notabug?

AFAIR, last Andrea looked into this he was surprised by this finding
and didn't understand how -foptimize-sibling-calls could cause such
problems.  I hope Andrea will come up with some tests for you guys to
provide more information, so he does eventually understand the
cause(s).  I don't think we can in good faith claim this is a GCC bug
before we understand the problem better.

Meanwhile, I'd like to consider a PROBLEMS entry about this, if that's
needed.  Do I understand correctly that -fno-optimize-sibling-calls is
needed for successful building of Emacs 29.0.92 using MinGW64 GCC 13.1?
If so, I think we should have this in PROBLEMS.

Also, why are you using -O3?  That is not recommended when building
Emacs.  (But I understand that -foptimize-sibling-calls is in -O2, not
just in -O3, and so -fno-optimize-sibling-calls is needed even if
using -O2, right?)

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> I hope Andrea will come up with some tests for you guys to provide
> more information, so he does eventually understand the cause(s).  I
> don't think we can in good faith claim this is a GCC bug before we
> understand the problem better.

Fair enough.

> Meanwhile, I'd like to consider a PROBLEMS entry about this, if that's
> needed.  Do I understand correctly that -fno-optimize-sibling-calls is
> needed for successful building of Emacs 29.0.92 using MinGW64 GCC 13.1?

I'm not tracking 29 branch closely, but from my brief testing:

 • Emacs builds when running
   $ ./configure --with-native-compilation && make
   on the pretest tarball from
   https://alpha.gnu.org/gnu/emacs/pretest/emacs-29.0.92.tar.xz

 • Emacs doesn't build out of local git repo when running
   $ git switch emacs-29
   $ git clean -fdx
   $ ./autogen.sh
   $ ./configure --with-native-compilation && make

 • Emacs builds out of local git repo when running
   $ git switch emacs-29
   $ git clean -fdx
   $ ./autogen.sh
   $ CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls' \
     ./configure --with-native-compilation && make

> If so, I think we should have this in PROBLEMS.

If others can reproduce my experience, then it probably makes sense.

> Also, why are you using -O3?  That is not recommended when building
> Emacs.

I wasn't aware of that.  Can you elaborate why -O3 isn't recommended?
Or any pointer where I can read about it?

> (But I understand that -foptimize-sibling-calls is in -O2, not just in
> -O3, and so -fno-optimize-sibling-calls is needed even if using -O2,
> right?)

Correct, this is also what I read from this page and my experience so
far[1].

Best, Arash

Footnotes:
[1]  https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/Optimize-Options.html

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: Andrea Corallo <acorallo@gnu.org>,  cyril.arnould@outlook.com,
>   63365@debbugs.gnu.org,  svraka.andras@gmail.com
> Date: Fri, 23 Jun 2023 13:41:10 +0200
> 
> I'm not tracking 29 branch closely, but from my brief testing:
> 
>  • Emacs builds when running
>    $ ./configure --with-native-compilation && make
>    on the pretest tarball from
>    https://alpha.gnu.org/gnu/emacs/pretest/emacs-29.0.92.tar.xz
> 
>  • Emacs doesn't build out of local git repo when running
>    $ git switch emacs-29
>    $ git clean -fdx
>    $ ./autogen.sh
>    $ ./configure --with-native-compilation && make

This might mean that the problem happens when byte-compiling *.el
files -- in the tarball all the *.elc files are already present.  What
happens if you remove the *.elc files from the release tarball, and
then try building it?

> > Also, why are you using -O3?  That is not recommended when building
> > Emacs.
> 
> I wasn't aware of that.  Can you elaborate why -O3 isn't recommended?

In a nutshell, it bloats the code (due to excessive inlining), with no
real effect on speed.  The inner loops in Emacs are very large, and
thus the techniques used by -O3 to speed up code (loop unrolling etc.)
don't really work.  Moreover, they could make things worse because the
larger loops might no longer fit into the L1 cache of the CPU.

The -O3 is well suited to speed up relatively simple algorithms with
tight loops.  Emacs has very few of those, in the places that matter
for observable performance.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> This might mean that the problem happens when byte-compiling *.el
> files -- in the tarball all the *.elc files are already present.  What
> happens if you remove the *.elc files from the release tarball, and
> then try building it?

This is indeed the case: Removing the *.elc after unpacking the tarball
and running

  ./configure --with-native-compilation && make

doesn't build -- I will try ./configure --without-native-compilation
later.

> In a nutshell, it bloats the code (due to excessive inlining), with no
> real effect on speed.  The inner loops in Emacs are very large, and
> thus the techniques used by -O3 to speed up code (loop unrolling etc.)
> don't really work.  Moreover, they could make things worse because the
> larger loops might no longer fit into the L1 cache of the CPU.
>
> The -O3 is well suited to speed up relatively simple algorithms with
> tight loops.  Emacs has very few of those, in the places that matter
> for observable performance.

Thanks!  I will go with -O2 in future.

Best, Arash

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Deus Max]

On 2023-06-23T15:15 Fri, Eli Zaretskii wrote:

>> From: Arash Esbati <arash@gnu.org>
>> Cc: Andrea Corallo <acorallo@gnu.org>,  cyril.arnould@outlook.com,
>>   63365@debbugs.gnu.org,  svraka.andras@gmail.com
>> Date: Fri, 23 Jun 2023 13:41:10 +0200
>>
>> > Also, why are you using -O3?  That is not recommended when building
>> > Emacs.
>>
>> I wasn't aware of that.  Can you elaborate why -O3 isn't recommended?
>
> In a nutshell, it bloats the code (due to excessive inlining), with no
> real effect on speed.  The inner loops in Emacs are very large, and
> thus the techniques used by -O3 to speed up code (loop unrolling etc.)
> don't really work.  Moreover, they could make things worse because the
> larger loops might no longer fit into the L1 cache of the CPU.
>
> The -O3 is well suited to speed up relatively simple algorithms with
> tight loops.  Emacs has very few of those, in the places that matter
> for observable performance.

Interesting.
This recommendation and the explanation are worth documenting somewhere.
Shouldn't a new bug be opened on documenting the GCC -O3 recommendation?

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Deus Max <deusmax@gmx.com>
> Cc: Arash Esbati <arash@gnu.org>,  63365@debbugs.gnu.org,  acorallo@gnu.org,
>   svraka.andras@gmail.com,  cyril.arnould@outlook.com
> Date: Sat, 24 Jun 2023 12:17:11 +0300
> 
> On 2023-06-23T15:15 Fri, Eli Zaretskii wrote:
> 
> >> From: Arash Esbati <arash@gnu.org>
> >> Cc: Andrea Corallo <acorallo@gnu.org>,  cyril.arnould@outlook.com,
> >>   63365@debbugs.gnu.org,  svraka.andras@gmail.com
> >> Date: Fri, 23 Jun 2023 13:41:10 +0200
> >>
> >> > Also, why are you using -O3?  That is not recommended when building
> >> > Emacs.
> >>
> >> I wasn't aware of that.  Can you elaborate why -O3 isn't recommended?
> >
> > In a nutshell, it bloats the code (due to excessive inlining), with no
> > real effect on speed.  The inner loops in Emacs are very large, and
> > thus the techniques used by -O3 to speed up code (loop unrolling etc.)
> > don't really work.  Moreover, they could make things worse because the
> > larger loops might no longer fit into the L1 cache of the CPU.
> >
> > The -O3 is well suited to speed up relatively simple algorithms with
> > tight loops.  Emacs has very few of those, in the places that matter
> > for observable performance.
> 
> Interesting.
> This recommendation and the explanation are worth documenting somewhere.
> Shouldn't a new bug be opened on documenting the GCC -O3 recommendation?

I don't think it's our business to document this.  The default build
procedure correctly uses -O2.  People who use non-default compilation
switches should know what they are doing.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Deus Max]

On 2023-06-24T12:21 Sat, Eli Zaretskii wrote:

>> From: Deus Max <deusmax@gmx.com>
>> Cc: Arash Esbati <arash@gnu.org>,  63365@debbugs.gnu.org,  acorallo@gnu.org,
>>   svraka.andras@gmail.com,  cyril.arnould@outlook.com
>> Date: Sat, 24 Jun 2023 12:17:11 +0300
>>
>> On 2023-06-23T15:15 Fri, Eli Zaretskii wrote:
>>
>> >> From: Arash Esbati <arash@gnu.org>
>> >> Cc: Andrea Corallo <acorallo@gnu.org>,  cyril.arnould@outlook.com,
>> >>   63365@debbugs.gnu.org,  svraka.andras@gmail.com
>> >> Date: Fri, 23 Jun 2023 13:41:10 +0200
>> >>
>> >> > Also, why are you using -O3?  That is not recommended when building
>> >> > Emacs.
>> >>
>> >> I wasn't aware of that.  Can you elaborate why -O3 isn't recommended?
>> >
>> > In a nutshell, it bloats the code (due to excessive inlining), with no
>> > real effect on speed.  The inner loops in Emacs are very large, and
>> > thus the techniques used by -O3 to speed up code (loop unrolling etc.)
>> > don't really work.  Moreover, they could make things worse because the
>> > larger loops might no longer fit into the L1 cache of the CPU.
>> >
>> > The -O3 is well suited to speed up relatively simple algorithms with
>> > tight loops.  Emacs has very few of those, in the places that matter
>> > for observable performance.
>>
>> Interesting.
>> This recommendation and the explanation are worth documenting somewhere.
>> Shouldn't a new bug be opened on documenting the GCC -O3 recommendation?
>
> I don't think it's our business to document this.  The default build
> procedure correctly uses -O2.  People who use non-default compilation
> switches should know what they are doing.

Then whose business is it?


The default of course is correct, also it is not intuitive that -O3 is
wrong. People who...should know what they are doing, but a little
explanation goes a long way. It also helps newcomers catch up.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Deus Max <deusmax@gmx.com>
> Cc: Deus Max <deusmax@gmx.com>,  arash@gnu.org,  63365@debbugs.gnu.org,
>   acorallo@gnu.org,  svraka.andras@gmail.com,  cyril.arnould@outlook.com
> Date: Sat, 24 Jun 2023 17:41:30 +0300
> 
> >> Interesting.
> >> This recommendation and the explanation are worth documenting somewhere.
> >> Shouldn't a new bug be opened on documenting the GCC -O3 recommendation?
> >
> > I don't think it's our business to document this.  The default build
> > procedure correctly uses -O2.  People who use non-default compilation
> > switches should know what they are doing.
> 
> Then whose business is it?

That of the GCC developers, of course.  That's where the description
of -O3 and its practical implications belongs.

> The default of course is correct, also it is not intuitive that -O3 is
> wrong. People who...should know what they are doing, but a little
> explanation goes a long way. It also helps newcomers catch up.

Where do you suggest to put these factoids to make them even
marginally discoverable by those for whom you think they will be
useful?  If they are hidden among the rest of 100K lines of the ELisp
manual, how will anyone be able to find them?

That is why each piece of documentation should be in its natural
place.  When I want to know something about GCC optimization options,
I turn to the GCC manual, nowhere else.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arash Esbati <arash@gnu.org>
>> Cc: "eliz@gnu.org" <eliz@gnu.org>,  "63365@debbugs.gnu.org"
>>  <63365@debbugs.gnu.org>,  " akrl@sdf.org" <akrl@sdf.org>,
>>   András Svraka
>>  <svraka.andras@gmail.com>
>> Date: Thu, 22 Jun 2023 22:34:48 +0200
>> 
>> Many thanks for digging into this.  I can confirm that Emacs builds with
>> -fno-optimize-sibling-calls in CFLAGS.  I have a script which (now)
>> does:
>> 
>>   CFLAGS='-O3 -g0 -pipe -fno-optimize-sibling-calls' \
>>   ./configure ...
>> 
>> and it works again.
>> 
>> Now the question is: Is this considered a GCC bug and we should close
>> this report as notabug?
>
> AFAIR, last Andrea looked into this he was surprised by this finding
> and didn't understand how -foptimize-sibling-calls could cause such
> problems.  I hope Andrea will come up with some tests for you guys to
> provide more information, so he does eventually understand the
> cause(s).  I don't think we can in good faith claim this is a GCC bug
> before we understand the problem better.

Unfortunatelly I've trouble thinking to such a test.  

Maybe someone should compare the two binaries (with and without
-foptimize-sibling-calls) to understand which compilation unit (and
which function) differs in details.  Hopefully this could help us to
pinkpoint the function causing the issue?  Once that is done it's more
easy to understand what the compiler is doing.

Thanks

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: Arash Esbati <arash@gnu.org>,  cyril.arnould@outlook.com,
>   63365@debbugs.gnu.org,  svraka.andras@gmail.com
> Date: Sun, 25 Jun 2023 09:51:13 -0400
> 
> Maybe someone should compare the two binaries (with and without
> -foptimize-sibling-calls) to understand which compilation unit (and
> which function) differs in details.

How does one compare binaries in a useful way?  The Emacs binary is
AFAIR around 20MB even when stripped of all symbols.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <acorallo@gnu.org>
>> Cc: Arash Esbati <arash@gnu.org>,  cyril.arnould@outlook.com,
>>   63365@debbugs.gnu.org,  svraka.andras@gmail.com
>> Date: Sun, 25 Jun 2023 09:51:13 -0400
>> 
>> Maybe someone should compare the two binaries (with and without
>> -foptimize-sibling-calls) to understand which compilation unit (and
>> which function) differs in details.
>
> How does one compare binaries in a useful way?  The Emacs binary is
> AFAIR around 20MB even when stripped of all symbols.

That's a good question, for elf there are specific tools for that, even
just readelf can output function sizes and that's a good starting point.

For Windows I've idea (I'm assuming Windows is not elf based).

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: arash@gnu.org,  cyril.arnould@outlook.com,  63365@debbugs.gnu.org,
>   svraka.andras@gmail.com
> Date: Sun, 25 Jun 2023 14:11:15 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Maybe someone should compare the two binaries (with and without
> >> -foptimize-sibling-calls) to understand which compilation unit (and
> >> which function) differs in details.
> >
> > How does one compare binaries in a useful way?  The Emacs binary is
> > AFAIR around 20MB even when stripped of all symbols.
> 
> That's a good question, for elf there are specific tools for that, even
> just readelf can output function sizes and that's a good starting point.
> 
> For Windows I've idea (I'm assuming Windows is not elf based).

No, Windows doesn't use ELF.

Maybe we should start by narrowing the problem?  E.g., which Lisp
files cause the crashes, and which *.eln files, if any, are involved?

The C files more or less directly involved in byte-compilation are, I
think, eval.c, data.c, alloc.c, lread.c, bytecode.c.  If we think one
of these could be involved, it would be nice to find the one(s) that
cause the problem, for example, by selectively compiling only those
with -fno-optimize-sibling-calls, then removing them one by one from
the set of files compiled like that.

The investigation if this bug is harder because the problem doesn't
happen when running Emacs under GDB, and conversion of backtrace
addresses to file names and line numbers for some reason also doesn't
work.  So we need every smart idea we can come up with.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <acorallo@gnu.org>
>> Cc: arash@gnu.org,  cyril.arnould@outlook.com,  63365@debbugs.gnu.org,
>>   svraka.andras@gmail.com
>> Date: Sun, 25 Jun 2023 14:11:15 -0400
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Maybe someone should compare the two binaries (with and without
>> >> -foptimize-sibling-calls) to understand which compilation unit (and
>> >> which function) differs in details.
>> >
>> > How does one compare binaries in a useful way?  The Emacs binary is
>> > AFAIR around 20MB even when stripped of all symbols.
>> 
>> That's a good question, for elf there are specific tools for that, even
>> just readelf can output function sizes and that's a good starting point.
>> 
>> For Windows I've idea (I'm assuming Windows is not elf based).
>
> No, Windows doesn't use ELF.
>
> Maybe we should start by narrowing the problem?  E.g., which Lisp
> files cause the crashes, and which *.eln files, if any, are involved?
>
> The C files more or less directly involved in byte-compilation are, I
> think, eval.c, data.c, alloc.c, lread.c, bytecode.c.  If we think one
> of these could be involved, it would be nice to find the one(s) that
> cause the problem, for example, by selectively compiling only those
> with -fno-optimize-sibling-calls, then removing them one by one from
> the set of files compiled like that.

That would be a good starting point already, I'd suggest to add comp.c
to the investigated set as well.

Thanks

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Cyril Arnould]

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]

>> The C files more or less directly involved in byte-compilation are, I
>> think, eval.c, data.c, alloc.c, lread.c, bytecode.c.  If we think one
>> of these could be involved, it would be nice to find the one(s) that
>> cause the problem, for example, by selectively compiling only those
>> with -fno-optimize-sibling-calls, then removing them one by one from
>> the set of files compiled like that.
>
> That would be a good starting point already, I'd suggest to add comp.c
> to the investigated set as well.

I can take a look into this, but I'm unsure as to how exactly; I'm still
very new to the GNU Autotools. AFAIU, this would be done using
per-target Automake variables but there's no Makefile.am...

[-- Attachment #2: Type: text/html, Size: 2251 bytes --]

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Cyril Arnould <cyril.arnould@outlook.com>
> CC: "eliz@gnu.org" <eliz@gnu.org>, András Svraka
> 	<svraka.andras@gmail.com>, Arash Esbati <arash@gnu.org>,
> 	"63365@debbugs.gnu.org" <63365@debbugs.gnu.org>
> Date: Mon, 26 Jun 2023 22:04:19 +0000
> 
> >> The C files more or less directly involved in byte-compilation are, I
> >> think, eval.c, data.c, alloc.c, lread.c, bytecode.c.  If we think one
> >> of these could be involved, it would be nice to find the one(s) that
> >> cause the problem, for example, by selectively compiling only those
> >> with -fno-optimize-sibling-calls, then removing them one by one from
> >> the set of files compiled like that.
> >  
> > That would be a good starting point already, I'd suggest to add comp.c
> > to the investigated set as well.
>  
> I can take a look into this, but I'm unsure as to how exactly; I'm still
> very new to the GNU Autotools. AFAIU, this would be done using
> per-target Automake variables but there's no Makefile.am...

No, that would be a waste of your time.  It is much easier to do this
by hand.  To compile, say, lread.c, do this:

  $ cd src
  $ make lread.o -W lread.c CFLAGS='-O2 -fno-optimize-sibling-calls'
  $ make

The last "make command will produce an emacs.exe binary where lread.c
is compiled without the problematic optimization.

  

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Cyril Arnould]

[-- Attachment #1: Type: text/plain, Size: 1327 bytes --]

> No, that would be a waste of your time.  It is much easier to do this
> by hand.  To compile, say, lread.c, do this:
>
>   $ cd src
>   $ make lread.o -W lread.c CFLAGS='-O2 -fno-optimize-sibling-calls'
>   $ make
>
> The last "make command will produce an emacs.exe binary where lread.c
> is compiled without the problematic optimization.

I see, thanks. Is there a reason you left out the -g3 and -gdwarf-2
switches? To be sure, I've tried it with and without those, but I got
similar results so far: all of the combinations I've tried are
failing. I'm trying to widen the search to see if I can figure out which
file is the culprit.

> Maybe we should start by narrowing the problem?  E.g., which Lisp
> files cause the crashes, and which *.eln files, if any, are involved?

From the tests I've run it seems to me that there is absolutely no
consistency with which lisp files cause the crashes. Each of the builds
resulted in different lisp files failing. Now, when I run the make
command again after a failed attempt, the *same* lisp files will keep
failing to build over and over. However, I also noticed that if I run
the exact same build commands again from a clean checkout, different
lisp files will fail the second time around. Is it normal that there are
run-to-run variations with GCC?

[-- Attachment #2: Type: text/html, Size: 3281 bytes --]

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Cyril Arnould <cyril.arnould@outlook.com> writes:

>> No, that would be a waste of your time.  It is much easier to do this
>
>> by hand.  To compile, say, lread.c, do this:
>
>>  
>
>>   $ cd src
>
>>   $ make lread.o -W lread.c CFLAGS='-O2 -fno-optimize-sibling-calls'
>
>>   $ make
>
>>  
>
>> The last "make command will produce an emacs.exe binary where lread.c
>
>> is compiled without the problematic optimization.
>
>  
>
> I see, thanks. Is there a reason you left out the -g3 and -gdwarf-2
>
> switches? To be sure, I've tried it with and without those, but I got
>
> similar results so far: all of the combinations I've tried are
>
> failing. I'm trying to widen the search to see if I can figure out which
>
> file is the culprit.
>
>  
>
>> Maybe we should start by narrowing the problem?  E.g., which Lisp
>
>> files cause the crashes, and which *.eln files, if any, are involved?
>
>  
>
> From the tests I've run it seems to me that there is absolutely no
>
> consistency with which lisp files cause the crashes. Each of the builds
>
> resulted in different lisp files failing. Now, when I run the make
>
> command again after a failed attempt, the *same* lisp files will keep
>
> failing to build over and over. However, I also noticed that if I run
>
> the exact same build commands again from a clean checkout, different
>
> lisp files will fail the second time around. Is it normal that there are
>
> run-to-run variations with GCC?

I guess is due to the parallel nature of the build (and possibly its
initial state).

Using make -j1 (or even better make bootstrap -j1) should make it
reproducible.

  Andrea

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Cyril Arnould]

I believe the problem is centered around the mark_threads function (in
thread.c). I still have no idea if the issue is with emacs or GCC, but
I hope this helps narrow it down.

I found it by trying to suppress the sibling call optimization through
printf statements at the end of every function that showed differences
between the objdump files. With the additional printf calls, the
native-compilation build works together with the
-foptimize-sibling-calls flag.

After that, it was just a matter of narrowing it down. You can try it
for yourself with the following patch:

diff --git a/src/thread.c b/src/thread.c
index 040ca39511e..b522d146779 100644
--- a/src/thread.c
+++ b/src/thread.c
@@ -696,6 +696,7 @@ mark_threads_callback (void *ignore)
  mark_threads (void)
  {
    flush_stack_call_func (mark_threads_callback, NULL);
+  printf("");
  }

  void

Note that by now I've moved on to emacs-29.3 on GCC 14, however it
seems that the issue is still exactly the same.

bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Cyril Arnould <cyril.arnould@outlook.com> writes:

> I believe the problem is centered around the mark_threads function (in
> thread.c). I still have no idea if the issue is with emacs or GCC, but
> I hope this helps narrow it down.
>
> I found it by trying to suppress the sibling call optimization through
> printf statements at the end of every function that showed differences
> between the objdump files. With the additional printf calls, the
> native-compilation build works together with the
> -foptimize-sibling-calls flag.
>
> After that, it was just a matter of narrowing it down. You can try it
> for yourself with the following patch:
>
> diff --git a/src/thread.c b/src/thread.c
> index 040ca39511e..b522d146779 100644
> --- a/src/thread.c
> +++ b/src/thread.c
> @@ -696,6 +696,7 @@ mark_threads_callback (void *ignore)
>  mark_threads (void)
>  {
>    flush_stack_call_func (mark_threads_callback, NULL);
> +  printf("");
>  }
>
>  void
>
> Note that by now I've moved on to emacs-29.3 on GCC 14, however it
> seems that the issue is still exactly the same.

Interesting, would you mind instead of using 'printf' to try using
'__attribute__((optimize("O0")))' on the function to verify that the
issue is really an optimization?

Thanks

  Andrea

bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Cyril Arnould]

 > Interesting, would you mind instead of using 'printf' to try using
 > '__attribute__((optimize("O0")))' on the function to verify that the
 > issue is really an optimization?

With the following, the build fails:


diff --git a/src/thread.c b/src/thread.c
index 040ca39511e..ffdcf420af0 100644
--- a/src/thread.c
+++ b/src/thread.c
@@ -692,6 +692,7 @@ mark_threads_callback (void *ignore)
      }
  }

+__attribute__((optimize("O0")))
  void
  mark_threads (void)
  {


I hope this was correct? If I replace it with
'__attribute__((optimize("no-optimize-sibling-calls")))', the build
succeeds again.

I therefore tried to compile the entirety of thread.c with -O0
instead, upon which the build succeeded again. I'll try to add the
'__attribute__((optimize("O0")))' in more thread.c functions to see
which ones need it for a successful build (unless you have a better
suggestion?).

To summarize the current state:

thread.c compiled with -O2: build fails
thread.c compiled with -O2 -fno-optimize-sibling-calls: build succeeds
thread.c compiled with -O0: build succeeds

thread.c compiled with -O2 and
- mark_threads with printf: build succeeds
- mark_threads with 
__attribute__((optimize("no-optimize-sibling-calls"))): build succeeds
- mark_threads with __attribute__((optimize("O0"))): build fails

bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Cyril Arnould <cyril.arnould@outlook.com> writes:

>> Interesting, would you mind instead of using 'printf' to try using
>> '__attribute__((optimize("O0")))' on the function to verify that the
>> issue is really an optimization?
>
> With the following, the build fails:
>
>
> diff --git a/src/thread.c b/src/thread.c
> index 040ca39511e..ffdcf420af0 100644
> --- a/src/thread.c
> +++ b/src/thread.c
> @@ -692,6 +692,7 @@ mark_threads_callback (void *ignore)
>      }
>  }
>
> +__attribute__((optimize("O0")))
>  void
>  mark_threads (void)
>  {
>
>
> I hope this was correct? If I replace it with
> '__attribute__((optimize("no-optimize-sibling-calls")))', the build
> succeeds again.
>
> I therefore tried to compile the entirety of thread.c with -O0
> instead, upon which the build succeeded again. I'll try to add the
> '__attribute__((optimize("O0")))' in more thread.c functions to see
> which ones need it for a successful build (unless you have a better
> suggestion?).
>
> To summarize the current state:
>
> thread.c compiled with -O2: build fails
> thread.c compiled with -O2 -fno-optimize-sibling-calls: build succeeds
> thread.c compiled with -O0: build succeeds
>
> thread.c compiled with -O2 and
> - mark_threads with printf: build succeeds
> - mark_threads with
>   __attribute__((optimize("no-optimize-sibling-calls"))): build
>  succeeds
> - mark_threads with __attribute__((optimize("O0"))): build fails

Mmmh 🧐, must say it does not make much sense to me.  Are we sure these
results are reliably reproducible and we are not looking at some
statistic fluctuation of a non very reproducible issue?

Anyway if marking 'mark_threads' with
__attribute__((optimize("no-optimize-sibling-calls"))) solves the issue
in a stable way I think we should compare the assembly output of the two
functions.

Thanks!

  Andrea

bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Cyril Arnould]

[-- Attachment #1: Type: text/plain, Size: 8399 bytes --]

 > Mmmh 🧐, must say it does not make much sense to me.  Are we sure these
 > results are reliably reproducible and we are not looking at some
 > statistic fluctuation of a non very reproducible issue?

I understand your scepticism. However, I've ran every build 2-4 times,
and so far my reproducibility is at 100%. As in; with the same
configuration the build (as a whole) either always fails or is always
successful. It does seem random however which of the byte-compile
steps fail in the end. Maybe the -fno-optimize-sibling-calls just
makes the failure much more unlikely.

 > Anyway if marking 'mark_threads' with
 > __attribute__((optimize("no-optimize-sibling-calls"))) solves the issue
 > in a stable way I think we should compare the assembly output of the two
 > functions.

I've attached the corresponding objdump -S -d (including the original
.o files). This time I've modified thread.c so that 'mark_threads'
comes dead last. I ran the builds another 4 times each to make
sure. Moving the function in the source also moves it to the end of
the dumps, makes for easier diffing:

diff -ubBw thread.txt thread-attr-no-optimize-sibling-calls.txt
--- thread.txt  2024-05-15 17:59:49.003434900 +0200
+++ thread-attr-no-optimize-sibling-calls.txt   2024-05-15 
17:38:35.029292300 +0200
@@ -1,5 +1,5 @@

-../thread-last.o:     file format pe-x86-64
+../thread-attr-last.o:     file format pe-x86-64


  Disassembly of section .text:
@@ -2305,6 +2305,7 @@

  00000000000017d0 <mark_threads>:

+__attribute__((optimize("no-optimize-sibling-calls")))
  void
  mark_threads (void)
  {
@@ -2316,50 +2317,49 @@
      17d9:      57                      push   %rdi
      17da:      56                      push   %rsi
      17db:      53                      push   %rbx
-    17dc:      48 81 ec a8 00 00 00    sub    $0xa8,%rsp
-    17e3:      48 8d 2c 24             lea    (%rsp),%rbp
-    17e7:      0f 11 75 00             movups %xmm6,0x0(%rbp)
-    17eb:      0f 11 7d 10             movups %xmm7,0x10(%rbp)
-    17ef:      44 0f 11 45 20          movups %xmm8,0x20(%rbp)
-    17f4:      44 0f 11 4d 30          movups %xmm9,0x30(%rbp)
-    17f9:      44 0f 11 55 40          movups %xmm10,0x40(%rbp)
-    17fe:      44 0f 11 5d 50          movups %xmm11,0x50(%rbp)
-    1803:      44 0f 11 65 60          movups %xmm12,0x60(%rbp)
-    1808:      44 0f 11 6d 70          movups %xmm13,0x70(%rbp)
-    180d:      44 0f 11 b5 80 00 00    movups %xmm14,0x80(%rbp)
-    1814:      00
-    1815:      44 0f 11 bd 90 00 00    movups %xmm15,0x90(%rbp)
-    181c:      00
+    17dc:      48 81 ec c8 00 00 00    sub    $0xc8,%rsp
+    17e3:      48 8d 6c 24 20          lea    0x20(%rsp),%rbp
+    17e8:      0f 11 75 00             movups %xmm6,0x0(%rbp)
+    17ec:      0f 11 7d 10             movups %xmm7,0x10(%rbp)
+    17f0:      44 0f 11 45 20          movups %xmm8,0x20(%rbp)
+    17f5:      44 0f 11 4d 30          movups %xmm9,0x30(%rbp)
+    17fa:      44 0f 11 55 40          movups %xmm10,0x40(%rbp)
+    17ff:      44 0f 11 5d 50          movups %xmm11,0x50(%rbp)
+    1804:      44 0f 11 65 60          movups %xmm12,0x60(%rbp)
+    1809:      44 0f 11 6d 70          movups %xmm13,0x70(%rbp)
+    180e:      44 0f 11 b5 80 00 00    movups %xmm14,0x80(%rbp)
+    1815:      00
+    1816:      44 0f 11 bd 90 00 00    movups %xmm15,0x90(%rbp)
+    181d:      00
    flush_stack_call_func1 (func, arg);
-    181d:      31 d2                   xor    %edx,%edx
-    181f:      48 8d 0d da eb ff ff    lea -0x1426(%rip),%rcx        # 
400 <mark_threads_callback>
+    181e:      31 d2                   xor    %edx,%edx
+    1820:      48 8d 0d d9 eb ff ff    lea -0x1427(%rip),%rcx        # 
400 <mark_threads_callback>
+    1827:      e8 00 00 00 00          call   182c <mark_threads+0x5c>
+    182c:      90                      nop
    flush_stack_call_func (mark_threads_callback, NULL);
  }
-    1826:      0f 10 75 00             movups 0x0(%rbp),%xmm6
-    182a:      0f 10 7d 10             movups 0x10(%rbp),%xmm7
-    182e:      44 0f 10 45 20          movups 0x20(%rbp),%xmm8
-    1833:      44 0f 10 4d 30          movups 0x30(%rbp),%xmm9
-    1838:      44 0f 10 55 40          movups 0x40(%rbp),%xmm10
-    183d:      44 0f 10 5d 50          movups 0x50(%rbp),%xmm11
-    1842:      44 0f 10 65 60          movups 0x60(%rbp),%xmm12
-    1847:      44 0f 10 6d 70          movups 0x70(%rbp),%xmm13
-    184c:      44 0f 10 b5 80 00 00    movups 0x80(%rbp),%xmm14
-    1853:      00
-    1854:      44 0f 10 bd 90 00 00    movups 0x90(%rbp),%xmm15
-    185b:      00
-    185c:      48 81 c4 a8 00 00 00    add    $0xa8,%rsp
-    1863:      5b                      pop    %rbx
-    1864:      5e                      pop    %rsi
-    1865:      5f                      pop    %rdi
-    1866:      41 5c                   pop    %r12
-    1868:      41 5d                   pop    %r13
-    186a:      41 5e                   pop    %r14
-    186c:      41 5f                   pop    %r15
-    186e:      5d                      pop    %rbp
-    186f:      e9 00 00 00 00          jmp    1874 <mark_threads+0xa4>
-    1874:      90                      nop
-    1875:      90                      nop
-    1876:      90                      nop
+    182d:      0f 10 75 00             movups 0x0(%rbp),%xmm6
+    1831:      0f 10 7d 10             movups 0x10(%rbp),%xmm7
+    1835:      44 0f 10 45 20          movups 0x20(%rbp),%xmm8
+    183a:      44 0f 10 4d 30          movups 0x30(%rbp),%xmm9
+    183f:      44 0f 10 55 40          movups 0x40(%rbp),%xmm10
+    1844:      44 0f 10 5d 50          movups 0x50(%rbp),%xmm11
+    1849:      44 0f 10 65 60          movups 0x60(%rbp),%xmm12
+    184e:      44 0f 10 6d 70          movups 0x70(%rbp),%xmm13
+    1853:      44 0f 10 b5 80 00 00    movups 0x80(%rbp),%xmm14
+    185a:      00
+    185b:      44 0f 10 bd 90 00 00    movups 0x90(%rbp),%xmm15
+    1862:      00
+    1863:      48 81 c4 c8 00 00 00    add    $0xc8,%rsp
+    186a:      5b                      pop    %rbx
+    186b:      5e                      pop    %rsi
+    186c:      5f                      pop    %rdi
+    186d:      41 5c                   pop    %r12
+    186f:      41 5d                   pop    %r13
+    1871:      41 5e                   pop    %r14
+    1873:      41 5f                   pop    %r15
+    1875:      5d                      pop    %rbp
+    1876:      c3                      ret
      1877:      90                      nop
      1878:      90                      nop
      1879:      90                      nop


[-- Attachment #2: objdump.tar.gz --]
[-- Type: application/x-gzip, Size: 743990 bytes --]

bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Cyril Arnould <cyril.arnould@outlook.com> writes:

>> Mmmh 🧐, must say it does not make much sense to me.  Are we sure these
>> results are reliably reproducible and we are not looking at some
>> statistic fluctuation of a non very reproducible issue?
>
> I understand your scepticism. However, I've ran every build 2-4 times,
> and so far my reproducibility is at 100%. As in; with the same
> configuration the build (as a whole) either always fails or is always
> successful. It does seem random however which of the byte-compile
> steps fail in the end. Maybe the -fno-optimize-sibling-calls just
> makes the failure much more unlikely.

I was more confused then sceptic 🙂

>> Anyway if marking 'mark_threads' with
>> __attribute__((optimize("no-optimize-sibling-calls"))) solves the issue
>> in a stable way I think we should compare the assembly output of the two
>> functions.
>
> I've attached the corresponding objdump -S -d (including the original
> .o files). This time I've modified thread.c so that 'mark_threads'
> comes dead last. I ran the builds another 4 times each to make
> sure. Moving the function in the source also moves it to the end of
> the dumps, makes for easier diffing:

Okay this is getting really interesting, one of the two versions seems
to be actually optimized for recursion (dunno why we could not see it
from the pass dump) so your theory seems confirmed! 😀

If you could also share the two preprocessed versions of thread.c would
be of great help.  You should be able to do it adding to your original
GCC invokation like "-E -o thread.i".

Thanks!

  Andrea

bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Cyril Arnould]

[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]

 > If you could also share the two preprocessed versions of thread.c would
 > be of great help.  You should be able to do it adding to your original
 > GCC invokation like "-E -o thread.i".

What I ended up doing was:

cd src
make thread.o -W thread.c CFLAGS='-g3 -O2 -gdwarf-2 
-Wno-error=implicit-function-declaration -E -o thread.i'

I hope this is what you were after. The addition of
'-Wno-error=implicit-function-declaration' comes from issue #70889
btw. I've also edited the "E:/Git/emacs-debug-gcc14/emacs" paths in
the thread.i files by hand so they don't show up in the diff; they
were originally compiled in two separate folders. The resulting files
are almost exactly the same:


diff -ubBw thread.i thread-attr-no-optimize-sibling-calls.i
--- thread.i    2024-05-15 20:34:08 +0000
+++ thread-attr-no-optimize-sibling-calls.i     2024-05-15 20:33:44 +0000
@@ -152355,12 +152355,13 @@

  }

+__attribute__((optimize("no-optimize-sibling-calls")))
  void
  mark_threads (void)
  {
    flush_stack_call_func (mark_threads_callback,
-# 1181 "thread.c" 3 4
+# 1182 "thread.c" 3 4
                                                 ((void *)0)
-# 1181 "thread.c"
+# 1182 "thread.c"
                                                     );
  }


[-- Attachment #2: preproc.tar.gz --]
[-- Type: application/x-gzip, Size: 1320429 bytes --]

bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Cyril Arnould <cyril.arnould@outlook.com> writes:

>> If you could also share the two preprocessed versions of thread.c would
>> be of great help.  You should be able to do it adding to your original
>> GCC invokation like "-E -o thread.i".
>
> What I ended up doing was:
>
> cd src
> make thread.o -W thread.c CFLAGS='-g3 -O2 -gdwarf-2
> -Wno-error=implicit-function-declaration -E -o thread.i'
>
> I hope this is what you were after.

Hi Cyril,

it would be great if you could:

$ cd src
$ touch thread.c
$ make thread.o V=1

Get the exact GCC invocation and add there "-E -o thread.i".  This way
we are sure we are looking at the right thing.

Thanks for your patience

  Andrea

bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Cyril Arnould]

 > it would be great if you could:
 >
 > $ cd src
 > $ touch thread.c
 > $ make thread.o V=1
 >
 > Get the exact GCC invocation and add there "-E -o thread.i". This way
 > we are sure we are looking at the right thing.

I did as you asked, but a diff with the previously sent files shows
that the new ones are identical. At least I wasn't totally wrong with
my guess :) The exact invocation was:

gcc  -c -mtune=generic   -DUSE_CRT_DLL=1 -I 
/e/Git/emacs-debug-gcc14/emacs-attr/nt/inc -Demacs  -I. -I. -I../lib 
-I../lib  -mtune=generic    -isystem 
C:/msys64/mingw64/include/librsvg-2.0 -isystem 
C:/msys64/mingw64/include/gdk-pixbuf-2.0 -isystem 
C:/msys64/mingw64/include/webp -DLIBDEFLATE_DLL -isystem 
C:/msys64/mingw64/include/cairo -isystem 
C:/msys64/mingw64/include/freetype2 -isystem 
C:/msys64/mingw64/include/libpng16 -isystem 
C:/msys64/mingw64/include/harfbuzz -isystem 
C:/msys64/mingw64/include/glib-2.0 -isystem 
C:/msys64/mingw64/lib/glib-2.0/include -isystem 
C:/msys64/mingw64/include/pixman-1   -isystem 
C:/msys64/mingw64/include/libxml2        -isystem 
C:/msys64/mingw64/include/webp      -isystem 
C:/msys64/mingw64/include/harfbuzz -isystem 
C:/msys64/mingw64/include/freetype2 -isystem 
C:/msys64/mingw64/include/libpng16 -isystem 
C:/msys64/mingw64/include/glib-2.0 -isystem 
C:/msys64/mingw64/lib/glib-2.0/include   -MMD -MF deps/thread.d -MP     
-isystem C:/msys64/mingw64/include/p11-kit-1 -fno-common -Wall 
-Warith-conversion -Wdate-time -Wdisabled-optimization 
-Wdouble-promotion -Wduplicated-cond -Wextra -Wformat-signedness 
-Winit-self -Winvalid-pch -Wlogical-op -Wmissing-declarations 
-Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs 
-Wnull-dereference -Wold-style-definition -Wopenmp-simd -Wpacked 
-Wpointer-arith -Wstrict-prototypes -Wsuggest-attribute=noreturn 
-Wsuggest-final-methods -Wsuggest-final-types -Wtrampolines 
-Wuninitialized -Wunknown-pragmas -Wunused-macros -Wvariadic-macros 
-Wvector-operation-performance -Wwrite-strings -Warray-bounds=2 
-Wattribute-alias=2 -Wformat=2 -Wformat-truncation=2 
-Wimplicit-fallthrough=5 -Wshift-overflow=2 -Wuse-after-free=3 
-Wvla-larger-than=4031 -Wno-missing-field-initializers 
-Wno-override-init -Wno-sign-compare -Wno-type-limits 
-Wno-unused-parameter -Wno-format-nonliteral -Wno-bidi-chars 
-Wno-pointer-sign -g3 -O2 -gdwarf-2 
-Wno-error=implicit-function-declaration  thread.c -E -o thread.i

bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Cyril Arnould <cyril.arnould@outlook.com> writes:

>> it would be great if you could:
>>
>> $ cd src
>> $ touch thread.c
>> $ make thread.o V=1
>>
>> Get the exact GCC invocation and add there "-E -o thread.i". This way
>> we are sure we are looking at the right thing.
>
> I did as you asked, but a diff with the previously sent files shows
> that the new ones are identical.

That's very good, is important we are sure we look at the right thing.
I'll digest them later tomorrow and report then 😃

Thanks

  Andrea

bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Andrea Corallo <acorallo@gnu.org> writes:

> Cyril Arnould <cyril.arnould@outlook.com> writes:
>
>>> it would be great if you could:
>>>
>>> $ cd src
>>> $ touch thread.c
>>> $ make thread.o V=1
>>>
>>> Get the exact GCC invocation and add there "-E -o thread.i". This way
>>> we are sure we are looking at the right thing.
>>
>> I did as you asked, but a diff with the previously sent files shows
>> that the new ones are identical.
>
> That's very good, is important we are sure we look at the right thing.
> I'll digest them later tomorrow and report then 😃

Okay it's finally clear what is going on here.

In Emacs we use (in order to have all callee saved registers pushed on
the stack before entering in the garbage collector)
'__builtin_unwind_init'.  Our code reduced looks like this:


test.c ====================

extern void flush_stack_call_func1 (void (*func) (void *arg), void *arg);
extern void mark_threads (void);
extern void
mark_threads_callback (void *ignore);

static inline void
flush_stack_call_func (void (*func) (void *arg), void *arg)
{
  __builtin_unwind_init ();
  flush_stack_call_func1 (func, arg);
}

void
mark_threads (void)
{
  flush_stack_call_func (mark_threads_callback, ((void *)0));
}
====================

We want to call 'flush_stack_call_func1' being sure that all callee
saved registers are pushed on the stack (so that the GC can scan the
whole stack correctly).

The idea in pseudo asm is like:

mark_threads:
	[...]
	push	# these pushs are from 'builtin_unwind_init'
	push
	push
        [...]
	call flush_stack_call_func1
	pop	# these pops are from 'builtin_unwind_init'
	pop
	pop
	ret

Sibling call optimization makes this:

mark_threads:
	[...]
	push
	push
	push
        [...]
	pop
	pop
	pop
	jmp flush_stack_call_func1

Which indeed does not work for us.

I believe this is a GCC bug as the sibling call optimizer should not run
in functions making use of 'builtin_unwind_init', even if this one is
undocumented (otherwise I can't see its reason of being 🙂).

So I filed a GCC bug [1].

Despite the fact that this will or not be recognized as a bug (and in
case fixed), I think we'll have to fix this on our codebase disabling
the sibling call optimizations on the sentive function(s).

Also note, this bug is not a native-comp specifc one, it's just most
likely non triggerable on most configuration.  IOW I think we see it
only on mingw+nativecomp by chance.

Thanks

  Andrea

[1] <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115132>

bug#65727: bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: "63365@debbugs.gnu.org" <63365@debbugs.gnu.org>,  "eliz@gnu.org"
>  <eliz@gnu.org>,  65727@debbugs.gnu.org,  Arash Esbati <arash@gnu.org>,
>   András Svraka <svraka.andras@gmail.com>
> Date: Fri, 17 May 2024 08:06:56 -0400
> 
> I believe this is a GCC bug as the sibling call optimizer should not run
> in functions making use of 'builtin_unwind_init', even if this one is
> undocumented (otherwise I can't see its reason of being 🙂).
> 
> So I filed a GCC bug [1].
> 
> Despite the fact that this will or not be recognized as a bug (and in
> case fixed), I think we'll have to fix this on our codebase disabling
> the sibling call optimizations on the sentive function(s).

Agreed.  Can you install a patch along these lines?

Do we have other functions that could be sensitive to this issue?

bug#65727: bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <acorallo@gnu.org>
>> Cc: "63365@debbugs.gnu.org" <63365@debbugs.gnu.org>,  "eliz@gnu.org"
>>  <eliz@gnu.org>,  65727@debbugs.gnu.org,  Arash Esbati <arash@gnu.org>,
>>   András Svraka <svraka.andras@gmail.com>
>> Date: Fri, 17 May 2024 08:06:56 -0400
>> 
>> I believe this is a GCC bug as the sibling call optimizer should not run
>> in functions making use of 'builtin_unwind_init', even if this one is
>> undocumented (otherwise I can't see its reason of being 🙂).
>> 
>> So I filed a GCC bug [1].
>> 
>> Despite the fact that this will or not be recognized as a bug (and in
>> case fixed), I think we'll have to fix this on our codebase disabling
>> the sibling call optimizations on the sentive function(s).
>
> Agreed.  Can you install a patch along these lines?

Work in progress 👍

> Do we have other functions that could be sensitive to this issue?

All functions calling 'flush_stack_call_func' are impacted, but I think
I've a fix that touches only 'flush_stack_call_func'.

  Andrea

bug#63365: bug#65727: bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Andrea Corallo <acorallo@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <acorallo@gnu.org>
>>> Cc: "63365@debbugs.gnu.org" <63365@debbugs.gnu.org>,  "eliz@gnu.org"
>>>  <eliz@gnu.org>,  65727@debbugs.gnu.org,  Arash Esbati <arash@gnu.org>,
>>>   András Svraka <svraka.andras@gmail.com>
>>> Date: Fri, 17 May 2024 08:06:56 -0400
>>> 
>>> I believe this is a GCC bug as the sibling call optimizer should not run
>>> in functions making use of 'builtin_unwind_init', even if this one is
>>> undocumented (otherwise I can't see its reason of being 🙂).
>>> 
>>> So I filed a GCC bug [1].
>>> 
>>> Despite the fact that this will or not be recognized as a bug (and in
>>> case fixed), I think we'll have to fix this on our codebase disabling
>>> the sibling call optimizations on the sentive function(s).
>>
>> Agreed.  Can you install a patch along these lines?
>
> Work in progress 👍
>
>> Do we have other functions that could be sensitive to this issue?
>
> All functions calling 'flush_stack_call_func' are impacted, but I think
> I've a fix that touches only 'flush_stack_call_func'.

Okay I've installed 19c983ddedf on master, works here for me.

Cyril could you confirm works for you as well?

Thanks

  Andrea

bug#65727: bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Cyril Arnould]

 > Cyril could you confirm works for you as well?

It does. Thank you all for sticking this one out!

bug#63365: bug#65727: bug#63365: bug#65727: 30.0.50; Build failure in MSYS2 when --with-native-compilation

[Andrea Corallo]

Cyril Arnould <cyril.arnould@outlook.com> writes:

>> Cyril could you confirm works for you as well?
>
> It does. Thank you all for sticking this one out!

Thank you, it would be not soved without you.

Closing this then.

Thanks

  Andrea

PS @Eli not sure if more releases are planned on 29 and so if we want to
backport it.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Cyril Arnould <cyril.arnould@outlook.com>
> CC: "63365@debbugs.gnu.org" <63365@debbugs.gnu.org>,
> 	András Svraka <svraka.andras@gmail.com>, Andrea Corallo
> 	<acorallo@gnu.org>, Arash Esbati <arash@gnu.org>
> Date: Tue, 27 Jun 2023 19:28:12 +0000
> 
> >   $ cd src
> >   $ make lread.o -W lread.c CFLAGS='-O2 -fno-optimize-sibling-calls'
> >   $ make
> >
> > The last "make command will produce an emacs.exe binary where lread.c
> > is compiled without the problematic optimization.
> 
> I see, thanks. Is there a reason you left out the -g3 and -gdwarf-2
> switches?

No particular reason.

> To be sure, I've tried it with and without those, but I got
> similar results so far: all of the combinations I've tried are
> failing. I'm trying to widen the search to see if I can figure out which
> file is the culprit.

Since the problem disappears under GDB, it is probably memory-related.
So it figures that it behaves like a classic heisenbug.
Unfortunately, it means more combinations to try and more time to
waste...

> >From the tests I've run it seems to me that there is absolutely no
> consistency with which lisp files cause the crashes. Each of the builds
> resulted in different lisp files failing. Now, when I run the make
> command again after a failed attempt, the *same* lisp files will keep
> failing to build over and over. However, I also noticed that if I run
> the exact same build commands again from a clean checkout, different
> lisp files will fail the second time around. Is it normal that there are
> run-to-run variations with GCC?

See above: I'm not surprised.  Especially since MinGW64 uses ASLR.
However, using sequential builds, as suggested by Andrea, could
perhaps help.  I'd also consider building with -Wl,--disable-dynamicbase
for example like this (when you run Make in the src directory):

  make LD_SWITCH_SYSTEM='-Wl,--disable-dynamicbase'

Maybe this will make the bug less unpredictable, who knows?

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Cyril Arnould]

[-- Attachment #1: Type: text/plain, Size: 1099 bytes --]

Ok, I think I got it; it's src/thread.c. The following build succeeds:

git clean -xdf
git checkout emacs-29.0.92
./autogen.sh
./configure --with-native-compilation
cd src
make temacs.exe
make thread.o -W thread.c CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls'
cd ..
make

I first compile temacs.exe with CFLAGS at default and then rebuild
thread.c with -fno-optimize-sibling-calls. This way thread.c should be
the only file built with the flag. Conversely, compiling with
-fno-optimiize-sibling-calls everywhere *but* thread.c fails:

git clean -xdf
git checkout emacs-29.0.92
./autogen.sh
./configure --with-native-compilation
cd src
make temacs.exe CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls'
make thread.o -W thread.c
cd ..
make CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls'

> Using make -j1 (or even better make bootstrap -j1) should make it
> reproducible.
>   make LD_SWITCH_SYSTEM='-Wl,--disable-dynamicbase'
>
> Maybe this will make the bug less unpredictable, who knows?

Tried them, the failing lisp files still seem random.

[-- Attachment #2: Type: text/html, Size: 3218 bytes --]

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Eli Zaretskii]

> From: Cyril Arnould <cyril.arnould@outlook.com>
> CC: "63365@debbugs.gnu.org" <63365@debbugs.gnu.org>
> Date: Wed, 28 Jun 2023 23:16:58 +0000
> 
> Ok, I think I got it; it's src/thread.c. The following build succeeds:
> 
> git clean -xdf
> git checkout emacs-29.0.92
> ./autogen.sh
> ./configure --with-native-compilation
> cd src
> make temacs.exe
> make thread.o -W thread.c CFLAGS='-g3 -O2 -gdwarf-2 -fno-optimize-sibling-calls'
> cd ..
> make

OK, so I guess the next step is to compare the code produced from
thread.c between these two compilations.  Like this:

  cd src
  objdump -d -S thread-with.o > with.txt
  objdump -d -S thread-without.o > without.txt
  diff -ubBw with.txt without.txt

where the two *.o files are thread.o you get when compiling,
respectively, with and without -fno-optimize-sibling-calls option.

Then post here the two *.txt files and the diffs, and let's see where
this leads us.  (I hope the two *.txt files will not be identical...)

Thanks.

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Cyril Arnould]

[-- Attachment #1.1: Type: text/plain, Size: 291 bytes --]

The two are very different indeed. You can find the files including the
objects in the attachment. A lot of the differences come down to
different addresses though; I've tried to eliminate those differences by
replacing all addresses with **** in the separate files with suffix '-addr'.

[-- Attachment #1.2: Type: text/html, Size: 1522 bytes --]

[-- Attachment #2: thread-diff.tar.gz --]
[-- Type: application/x-gzip, Size: 787195 bytes --]

bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Andrea Corallo]

Cyril Arnould <cyril.arnould@outlook.com> writes:

> The two are very different indeed. You can find the files including the
>
> objects in the attachment. A lot of the differences come down to
>
> different addresses though; I've tried to eliminate those differences by
>
> replacing all addresses with **** in the separate files with suffix '-addr'.

Hi Cyril,

before getting into the diff of the binary given we can use objdump (I
understand it only now sorry) I suggest we compare function sizes with
objdump -t.

Also I'm assuming we are 100% sure the culprint is thread.o, given the
bug looks not very reproducible I'd repeat the test a couple of times to
be super sure we have identified the culprint.

Thanks for your time and patience

  Andrea

bug#63365: AW: bug#63365: 30.0.50; GCC 13.1 breaks building Emacs with native-compilation

[Cyril Arnould]

[-- Attachment #1.1: Type: text/plain, Size: 1777 bytes --]

> before getting into the diff of the binary given we can use objdump (I
> understand it only now sorry) I suggest we compare function sizes with
> objdump -t.

I'm not sure I follow, so let me be more explicit: The attached archive
contains the thread.o files from both runs, renamed to
'thread-foptimize-sibling-calls.o' and
'thread-fno-optimize-sibling-calls.o', respectively. This is mostly in
case someone wants to generate their own dumps.

Next, there's two text files for each that were generated using 'objdump
-d -S' as Eli suggested; 'foptimize-sibling-calls.txt' and
'fno-optimize-sibling-calls.txt'. There's also a diff of the two in
'diff.txt', generated with 'diff -ubBw'.

I noticed that in the diff, quite a lot of differences simply come down
to addresses, so I edited the objdumps of both files by hand by
replacing the addresses with ****. Those files are
'fno-optimize-sibling-calls-addr.txt' and
'foptimize-sibling-calls-addr.txt'. This greatly reduced their diff,
which is provided in 'diff-addr.txt'.

Now, as for objdump -t: I've attached the dumps and diff in this mail.

> Also I'm assuming we are 100% sure the culprint is thread.o, given the
> bug looks not very reproducible I'd repeat the test a couple of times to
> be super sure we have identified the culprint.

I did run it several times, I found it by doing a binary search over the
.c files in the src folder (i.e. I compiled half the .c files with the
optimization and half of them without it, then repeated with the
succeeding half). I can't recall a single run where the build succeeded
when thread.c was compiled with -foptimize-sibling-calls. Conversely,
the build so far never failed when thread.c was compiled with
-fno-optimize-sibling-calls.



[-- Attachment #1.2: Type: text/html, Size: 4079 bytes --]

[-- Attachment #2: foptimize-sibling-calls-dumpt.txt --]
[-- Type: text/plain, Size: 12810 bytes --]


thread-foptimize-sibling-calls.o:     file format pe-x86-64

SYMBOL TABLE:
[  0](sec -2)(fl 0x00)(ty    0)(scl 103) (nx 1) 0x0000000000000000 thread.c
File 
[  2](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 1) 0x0000000000000000 do_unwind_mutex_lock
AUX tagndx 0 ttlsiz 0x0 lnnos 0 next 0
[  4](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000010 record_thread_error
[  5](sec  3)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000000 last_thread_error
[  6](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000020 Fcurrent_thread
[  7](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000030 Fthread_last_error
[  8](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000050 Fthread_yield
[  9](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000100 Fcondition_notify
[ 10](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000210 mark_threads_callback
[ 11](sec  2)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000000 all_threads
[ 12](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000002c0 invoke_thread_function
[ 13](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000300 Fthread_signal
[ 14](sec  2)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000020 main_thread
[ 15](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000470 Fall_threads
[ 16](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000004b0 Fmake_thread
[ 17](sec 18)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata$.refptr.initialized
AUX scnlen 0x8 nreloc 1 nlnno 0 checksum 0x0 assoc 0 comdat 2
[ 19](sec 17)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata$.refptr.globals
AUX scnlen 0x8 nreloc 1 nlnno 0 checksum 0x0 assoc 0 comdat 2
[ 21](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000610 lisp_mutex_lock_for_thread
[ 22](sec  3)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000020 global_lock
[ 23](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000700 post_acquire_global_lock
[ 24](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000007b0 mutex_lock_callback
[ 25](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000007f0 thread_join_callback
[ 26](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000870 mutex_unlock_callback
[ 27](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000008c0 condition_wait_callback
[ 28](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000960 thread_signal_callback
[ 29](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000990 condition_notify_callback
[ 30](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000a10 really_call_select
[ 31](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000ac0 yield_callback
[ 32](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000b00 run_thread
[ 33](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000c80 Fcondition_name
[ 34](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000cd0 Fcondition_mutex
[ 35](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000d20 Fmutex_name
[ 36](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000d70 Fmutex_unlock
[ 37](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000e50 Fcondition_wait
[ 38](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000f50 Fthread_name
[ 39](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000fa0 Fthread_blocker
[ 40](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000ff0 Fmutex_lock
[ 41](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001110 Fthread_live_p
[ 42](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001160 Fthread_join
[ 43](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001280 Fmake_condition_variable
[ 44](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001320 Fmake_mutex
[ 45](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001390 maybe_reacquire_global_lock
[ 46](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000013f0 finalize_one_mutex
[ 47](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001400 finalize_one_condvar
[ 48](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001410 thread_select
[ 49](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000014f0 mark_threads
[ 50](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000015a0 unmark_main_thread
[ 51](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000015b0 finalize_one_thread
[ 52](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001630 thread_check_current_buffer
[ 53](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001680 main_thread_p
[ 54](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001690 in_current_thread
[ 55](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000016d0 init_threads
[ 56](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001730 syms_of_threads
[ 57](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000360 Sthread_yield
[ 58](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000300 Smake_thread
[ 59](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000002a0 Scurrent_thread
[ 60](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000240 Sthread_name
[ 61](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000001e0 Sthread_signal
[ 62](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000180 Sthread_live_p
[ 63](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000000c0 Sthread_join
[ 64](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000120 Sthread_blocker
[ 65](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000060 Sall_threads
[ 66](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000006c0 Smake_mutex
[ 67](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000660 Smutex_lock
[ 68](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000600 Smutex_unlock
[ 69](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000005a0 Smutex_name
[ 70](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000540 Smake_condition_variable
[ 71](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000004e0 Scondition_wait
[ 72](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000480 Scondition_notify
[ 73](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000420 Scondition_mutex
[ 74](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000003c0 Scondition_name
[ 75](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000000 Sthread_last_error
[ 76](sec  6)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000000c0 o_fwd.0
[ 77](sec  1)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .text
AUX scnlen 0x1877 nreloc 229 nlnno 0
[ 79](sec  2)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .data
AUX scnlen 0x280 nreloc 2 nlnno 0
[ 81](sec  3)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .bss
AUX scnlen 0x48 nreloc 0 nlnno 0
[ 83](sec  4)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .xdata
AUX scnlen 0x3ac nreloc 0 nlnno 0
[ 85](sec  5)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .pdata
AUX scnlen 0x228 nreloc 138 nlnno 0
[ 87](sec  6)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata
AUX scnlen 0x1df nreloc 1 nlnno 0
[ 89](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .subrs
AUX scnlen 0x718 nreloc 38 nlnno 0
[ 91](sec  8)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_info
AUX scnlen 0xf4fd nreloc 1639 nlnno 0
[ 93](sec  9)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_abbrev
AUX scnlen 0x9a3 nreloc 0 nlnno 0
[ 95](sec 10)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_loc
AUX scnlen 0x75b7 nreloc 22 nlnno 0
[ 97](sec 11)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_aranges
AUX scnlen 0x30 nreloc 2 nlnno 0
[ 99](sec 12)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_ranges
AUX scnlen 0xe70 nreloc 0 nlnno 0
[101](sec 13)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_macro
AUX scnlen 0xffa1e nreloc 1 nlnno 0
[103](sec 14)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_line
AUX scnlen 0x2f50 nreloc 13 nlnno 0
[105](sec 15)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_str
AUX scnlen 0xe5 nreloc 0 nlnno 0
[107](sec 16)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata$zzz
AUX scnlen 0x2b nreloc 0 nlnno 0
[109](sec 19)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_frame
AUX scnlen 0xb18 nreloc 92 nlnno 0
[111](sec  2)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000008 current_thread
[112](sec 18)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 .refptr.initialized
[113](sec 17)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 .refptr.globals
[114](sec  0)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 globals
[115](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 flush_stack_call_func1
[116](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 wrong_type_argument
[117](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 error
[118](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 mark_object
[119](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 mark_specpdl
[120](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 mark_c_stack
[121](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 mark_bytecode
[122](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 Ffuncall
[123](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 unbind_to
[124](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 list3
[125](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 kbd_buffer_store_event
[126](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 Fsignal
[127](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 Fcons
[128](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 allocate_pseudovector
[129](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 xmalloc
[130](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 init_bc_thread
[131](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_init
[132](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 code_convert_string_norecord
[133](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 xstrdup
[134](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_create
[135](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 emacs_abort
[136](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_wait
[137](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 specpdl_unrewind
[138](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 set_buffer_internal_2
[139](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_broadcast
[140](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_signal
[141](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 block_interrupt_signal
[142](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_mutex_unlock
[143](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 restore_signal_mask
[144](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_mutex_lock
[145](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_yield
[146](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_self
[147](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_set_name
[148](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 xzalloc
[149](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 push_handler
[150](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 internal_condition_case
[151](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 update_processes_for_thread_death
[152](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 xfree
[153](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 record_unwind_protect_void
[154](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_destroy
[155](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 free_bc_thread
[156](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_equal
[157](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_mutex_init
[158](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 defsubr
[159](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 staticpro
[160](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 intern_c_string_1
[161](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 Fprovide
[162](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 defvar_lisp
[163](sec  0)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 initialized



[-- Attachment #3: fno-optimize-sibling-calls-dumpt.txt --]
[-- Type: text/plain, Size: 12812 bytes --]


thread-fno-optimize-sibling-calls.o:     file format pe-x86-64

SYMBOL TABLE:
[  0](sec -2)(fl 0x00)(ty    0)(scl 103) (nx 1) 0x0000000000000000 thread.c
File 
[  2](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 1) 0x0000000000000000 do_unwind_mutex_lock
AUX tagndx 0 ttlsiz 0x0 lnnos 0 next 0
[  4](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000010 record_thread_error
[  5](sec  3)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000000 last_thread_error
[  6](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000020 Fcurrent_thread
[  7](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000030 Fthread_last_error
[  8](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000050 Fthread_yield
[  9](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000100 Fcondition_notify
[ 10](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000210 mark_threads_callback
[ 11](sec  2)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000000 all_threads
[ 12](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000002c0 invoke_thread_function
[ 13](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000300 Fthread_signal
[ 14](sec  2)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000020 main_thread
[ 15](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000470 Fall_threads
[ 16](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000004b0 Fmake_thread
[ 17](sec 18)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata$.refptr.initialized
AUX scnlen 0x8 nreloc 1 nlnno 0 checksum 0x0 assoc 0 comdat 2
[ 19](sec 17)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata$.refptr.globals
AUX scnlen 0x8 nreloc 1 nlnno 0 checksum 0x0 assoc 0 comdat 2
[ 21](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000610 lisp_mutex_lock_for_thread
[ 22](sec  3)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000020 global_lock
[ 23](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000700 post_acquire_global_lock
[ 24](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000007b0 mutex_lock_callback
[ 25](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000007f0 thread_join_callback
[ 26](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000880 mutex_unlock_callback
[ 27](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000008d0 condition_wait_callback
[ 28](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000970 thread_signal_callback
[ 29](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000009a0 condition_notify_callback
[ 30](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000a20 really_call_select
[ 31](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000ad0 yield_callback
[ 32](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000b10 run_thread
[ 33](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000c90 Fcondition_name
[ 34](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000ce0 Fcondition_mutex
[ 35](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000d30 Fmutex_name
[ 36](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000d80 Fmutex_unlock
[ 37](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000e60 Fcondition_wait
[ 38](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000f60 Fthread_name
[ 39](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000fb0 Fthread_blocker
[ 40](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001000 Fmutex_lock
[ 41](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001120 Fthread_live_p
[ 42](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001170 Fthread_join
[ 43](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001290 Fmake_condition_variable
[ 44](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001330 Fmake_mutex
[ 45](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000013a0 maybe_reacquire_global_lock
[ 46](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001400 finalize_one_mutex
[ 47](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001420 finalize_one_condvar
[ 48](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001440 thread_select
[ 49](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001520 mark_threads
[ 50](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000015d0 unmark_main_thread
[ 51](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000015e0 finalize_one_thread
[ 52](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001660 thread_check_current_buffer
[ 53](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000016b0 main_thread_p
[ 54](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000016c0 in_current_thread
[ 55](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000016f0 init_threads
[ 56](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001750 syms_of_threads
[ 57](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000360 Sthread_yield
[ 58](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000300 Smake_thread
[ 59](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000002a0 Scurrent_thread
[ 60](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000240 Sthread_name
[ 61](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000001e0 Sthread_signal
[ 62](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000180 Sthread_live_p
[ 63](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000000c0 Sthread_join
[ 64](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000120 Sthread_blocker
[ 65](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000060 Sall_threads
[ 66](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000006c0 Smake_mutex
[ 67](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000660 Smutex_lock
[ 68](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000600 Smutex_unlock
[ 69](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000005a0 Smutex_name
[ 70](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000540 Smake_condition_variable
[ 71](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000004e0 Scondition_wait
[ 72](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000480 Scondition_notify
[ 73](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000420 Scondition_mutex
[ 74](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000003c0 Scondition_name
[ 75](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000000 Sthread_last_error
[ 76](sec  6)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000000c0 o_fwd.0
[ 77](sec  1)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .text
AUX scnlen 0x1897 nreloc 229 nlnno 0
[ 79](sec  2)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .data
AUX scnlen 0x280 nreloc 2 nlnno 0
[ 81](sec  3)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .bss
AUX scnlen 0x48 nreloc 0 nlnno 0
[ 83](sec  4)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .xdata
AUX scnlen 0x3b4 nreloc 0 nlnno 0
[ 85](sec  5)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .pdata
AUX scnlen 0x228 nreloc 138 nlnno 0
[ 87](sec  6)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata
AUX scnlen 0x1df nreloc 1 nlnno 0
[ 89](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .subrs
AUX scnlen 0x718 nreloc 38 nlnno 0
[ 91](sec  8)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_info
AUX scnlen 0xf57c nreloc 1647 nlnno 0
[ 93](sec  9)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_abbrev
AUX scnlen 0x95e nreloc 0 nlnno 0
[ 95](sec 10)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_loc
AUX scnlen 0x753d nreloc 22 nlnno 0
[ 97](sec 11)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_aranges
AUX scnlen 0x30 nreloc 2 nlnno 0
[ 99](sec 12)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_ranges
AUX scnlen 0xdc0 nreloc 0 nlnno 0
[101](sec 13)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_macro
AUX scnlen 0xffa1e nreloc 1 nlnno 0
[103](sec 14)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_line
AUX scnlen 0x2ea1 nreloc 7 nlnno 0
[105](sec 15)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_str
AUX scnlen 0xe5 nreloc 0 nlnno 0
[107](sec 16)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata$zzz
AUX scnlen 0x2b nreloc 0 nlnno 0
[109](sec 19)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_frame
AUX scnlen 0xb10 nreloc 92 nlnno 0
[111](sec  2)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000008 current_thread
[112](sec 18)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 .refptr.initialized
[113](sec 17)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 .refptr.globals
[114](sec  0)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 globals
[115](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 flush_stack_call_func1
[116](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 wrong_type_argument
[117](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 error
[118](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 mark_object
[119](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 mark_specpdl
[120](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 mark_c_stack
[121](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 mark_bytecode
[122](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 Ffuncall
[123](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 unbind_to
[124](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 list3
[125](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 kbd_buffer_store_event
[126](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 Fsignal
[127](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 Fcons
[128](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 allocate_pseudovector
[129](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 xmalloc
[130](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 init_bc_thread
[131](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_init
[132](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 code_convert_string_norecord
[133](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 xstrdup
[134](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_create
[135](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 emacs_abort
[136](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_wait
[137](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 specpdl_unrewind
[138](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 set_buffer_internal_2
[139](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_broadcast
[140](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_signal
[141](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 block_interrupt_signal
[142](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_mutex_unlock
[143](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 restore_signal_mask
[144](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_mutex_lock
[145](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_yield
[146](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_self
[147](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_set_name
[148](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 xzalloc
[149](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 push_handler
[150](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 internal_condition_case
[151](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 update_processes_for_thread_death
[152](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 xfree
[153](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 record_unwind_protect_void
[154](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_cond_destroy
[155](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 free_bc_thread
[156](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_thread_equal
[157](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 sys_mutex_init
[158](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 defsubr
[159](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 staticpro
[160](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 intern_c_string_1
[161](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 Fprovide
[162](sec  0)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000000 defvar_lisp
[163](sec  0)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 initialized



[-- Attachment #4: diff-dumpt.txt --]
[-- Type: text/plain, Size: 8858 bytes --]

--- foptimize-sibling-calls-dumpt.txt	2023-06-29 10:49:09 +0000
+++ fno-optimize-sibling-calls-dumpt.txt	2023-06-29 11:08:25 +0000
@@ -1,5 +1,5 @@
 
-thread-foptimize-sibling-calls.o:     file format pe-x86-64
+thread-fno-optimize-sibling-calls.o:     file format pe-x86-64
 
 SYMBOL TABLE:
 [  0](sec -2)(fl 0x00)(ty    0)(scl 103) (nx 1) 0x0000000000000000 thread.c
@@ -28,37 +28,37 @@
 [ 23](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000700 post_acquire_global_lock
 [ 24](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000007b0 mutex_lock_callback
 [ 25](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000007f0 thread_join_callback
-[ 26](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000870 mutex_unlock_callback
-[ 27](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000008c0 condition_wait_callback
-[ 28](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000960 thread_signal_callback
-[ 29](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000990 condition_notify_callback
-[ 30](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000a10 really_call_select
-[ 31](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000ac0 yield_callback
-[ 32](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000b00 run_thread
-[ 33](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000c80 Fcondition_name
-[ 34](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000cd0 Fcondition_mutex
-[ 35](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000d20 Fmutex_name
-[ 36](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000d70 Fmutex_unlock
-[ 37](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000e50 Fcondition_wait
-[ 38](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000f50 Fthread_name
-[ 39](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000fa0 Fthread_blocker
-[ 40](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000ff0 Fmutex_lock
-[ 41](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001110 Fthread_live_p
-[ 42](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001160 Fthread_join
-[ 43](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001280 Fmake_condition_variable
-[ 44](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001320 Fmake_mutex
-[ 45](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001390 maybe_reacquire_global_lock
-[ 46](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000013f0 finalize_one_mutex
-[ 47](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001400 finalize_one_condvar
-[ 48](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001410 thread_select
-[ 49](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000014f0 mark_threads
-[ 50](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000015a0 unmark_main_thread
-[ 51](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000015b0 finalize_one_thread
-[ 52](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001630 thread_check_current_buffer
-[ 53](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001680 main_thread_p
-[ 54](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001690 in_current_thread
-[ 55](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000016d0 init_threads
-[ 56](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001730 syms_of_threads
+[ 26](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000880 mutex_unlock_callback
+[ 27](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000008d0 condition_wait_callback
+[ 28](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000970 thread_signal_callback
+[ 29](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x00000000000009a0 condition_notify_callback
+[ 30](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000a20 really_call_select
+[ 31](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000ad0 yield_callback
+[ 32](sec  1)(fl 0x00)(ty   20)(scl   3) (nx 0) 0x0000000000000b10 run_thread
+[ 33](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000c90 Fcondition_name
+[ 34](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000ce0 Fcondition_mutex
+[ 35](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000d30 Fmutex_name
+[ 36](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000d80 Fmutex_unlock
+[ 37](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000e60 Fcondition_wait
+[ 38](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000f60 Fthread_name
+[ 39](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000000fb0 Fthread_blocker
+[ 40](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001000 Fmutex_lock
+[ 41](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001120 Fthread_live_p
+[ 42](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001170 Fthread_join
+[ 43](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001290 Fmake_condition_variable
+[ 44](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001330 Fmake_mutex
+[ 45](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000013a0 maybe_reacquire_global_lock
+[ 46](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001400 finalize_one_mutex
+[ 47](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001420 finalize_one_condvar
+[ 48](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001440 thread_select
+[ 49](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001520 mark_threads
+[ 50](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000015d0 unmark_main_thread
+[ 51](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000015e0 finalize_one_thread
+[ 52](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001660 thread_check_current_buffer
+[ 53](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000016b0 main_thread_p
+[ 54](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000016c0 in_current_thread
+[ 55](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x00000000000016f0 init_threads
+[ 56](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 0) 0x0000000000001750 syms_of_threads
 [ 57](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000360 Sthread_yield
 [ 58](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000300 Smake_thread
 [ 59](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000002a0 Scurrent_thread
@@ -80,13 +80,13 @@
 [ 75](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x0000000000000000 Sthread_last_error
 [ 76](sec  6)(fl 0x00)(ty    0)(scl   3) (nx 0) 0x00000000000000c0 o_fwd.0
 [ 77](sec  1)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .text
-AUX scnlen 0x1877 nreloc 229 nlnno 0
+AUX scnlen 0x1897 nreloc 229 nlnno 0
 [ 79](sec  2)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .data
 AUX scnlen 0x280 nreloc 2 nlnno 0
 [ 81](sec  3)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .bss
 AUX scnlen 0x48 nreloc 0 nlnno 0
 [ 83](sec  4)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .xdata
-AUX scnlen 0x3ac nreloc 0 nlnno 0
+AUX scnlen 0x3b4 nreloc 0 nlnno 0
 [ 85](sec  5)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .pdata
 AUX scnlen 0x228 nreloc 138 nlnno 0
 [ 87](sec  6)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata
@@ -94,25 +94,25 @@
 [ 89](sec  7)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .subrs
 AUX scnlen 0x718 nreloc 38 nlnno 0
 [ 91](sec  8)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_info
-AUX scnlen 0xf4fd nreloc 1639 nlnno 0
+AUX scnlen 0xf57c nreloc 1647 nlnno 0
 [ 93](sec  9)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_abbrev
-AUX scnlen 0x9a3 nreloc 0 nlnno 0
+AUX scnlen 0x95e nreloc 0 nlnno 0
 [ 95](sec 10)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_loc
-AUX scnlen 0x75b7 nreloc 22 nlnno 0
+AUX scnlen 0x753d nreloc 22 nlnno 0
 [ 97](sec 11)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_aranges
 AUX scnlen 0x30 nreloc 2 nlnno 0
 [ 99](sec 12)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_ranges
-AUX scnlen 0xe70 nreloc 0 nlnno 0
+AUX scnlen 0xdc0 nreloc 0 nlnno 0
 [101](sec 13)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_macro
 AUX scnlen 0xffa1e nreloc 1 nlnno 0
 [103](sec 14)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_line
-AUX scnlen 0x2f50 nreloc 13 nlnno 0
+AUX scnlen 0x2ea1 nreloc 7 nlnno 0
 [105](sec 15)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_str
 AUX scnlen 0xe5 nreloc 0 nlnno 0
 [107](sec 16)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .rdata$zzz
 AUX scnlen 0x2b nreloc 0 nlnno 0
 [109](sec 19)(fl 0x00)(ty    0)(scl   3) (nx 1) 0x0000000000000000 .debug_frame
-AUX scnlen 0xb18 nreloc 92 nlnno 0
+AUX scnlen 0xb10 nreloc 92 nlnno 0
 [111](sec  2)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000008 current_thread
 [112](sec 18)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 .refptr.initialized
 [113](sec 17)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000000000 .refptr.globals