bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andy Moreton]

On Windows with the mingw64 32bit toolchain, the native branch build
fails when configured with "--with-nativecomp --with-wide-int".

Emacs crashes during .eln compilation. Attaching gdb to the crashing
emacs process does not produce a backtrace.

I suspect that this problem may not be Windows-specific, and should be 
reproducable on Linux, but I have not tried that.

     AndyM

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sat, 13 Feb 2021 17:57:09 +0000
> 
> On Windows with the mingw64 32bit toolchain, the native branch build
> fails when configured with "--with-nativecomp --with-wide-int".
> 
> Emacs crashes during .eln compilation. Attaching gdb to the crashing
> emacs process does not produce a backtrace.

What does GDB produce if you attach it?

Can you run a compilation from Emacs that runs under GDB?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andy Moreton]

On Sat 13 Feb 2021, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Sat, 13 Feb 2021 17:57:09 +0000
>> 
>> On Windows with the mingw64 32bit toolchain, the native branch build
>> fails when configured with "--with-nativecomp --with-wide-int".
>> 
>> Emacs crashes during .eln compilation. Attaching gdb to the crashing
>> emacs process does not produce a backtrace.
>
> What does GDB produce if you attach it?

Nothing - the windows tra handler, followed by gdb deciding that
something has died and giving up completely.

> Can you run a compilation from Emacs that runs under GDB?

How do I do that in a way that captures only the relevant subprocesses ?

   AndyM

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sat, 13 Feb 2021 20:23:20 +0000
> 
> > Can you run a compilation from Emacs that runs under GDB?
> 
> How do I do that in a way that captures only the relevant subprocesses ?

That's "set follow-exec-mode" in GDB, but I don't know if it works on
Windows.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andy Moreton]

On Sat 13 Feb 2021, Andy Moreton wrote:

> On Windows with the mingw64 32bit toolchain, the native branch build
> fails when configured with "--with-nativecomp --with-wide-int".
>
> Emacs crashes during .eln compilation. Attaching gdb to the crashing
> emacs process does not produce a backtrace.
>
> I suspect that this problem may not be Windows-specific, and should be
> reproducable on Linux, but I have not tried that.

I have also tried the mingw.org 32bit toolchain (i686-pc-mingw32) when
configured with with "--with-nativecomp --with-wide-int", and that
works.

Thus this bug seems to be a problem with the MSYS2 mingw32 32bit
toolchain (i686-w64-mingw32) only.

    AndyM

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sun, 14 Feb 2021 13:24:32 +0000
> 
> > I suspect that this problem may not be Windows-specific, and should be
> > reproducable on Linux, but I have not tried that.
> 
> I have also tried the mingw.org 32bit toolchain (i686-pc-mingw32) when
> configured with with "--with-nativecomp --with-wide-int", and that
> works.

Thanks, that's good news.

> Thus this bug seems to be a problem with the MSYS2 mingw32 32bit
> toolchain (i686-w64-mingw32) only.

It does seem that way, although I wonder what could that problem be...

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Sun, 14 Feb 2021 13:24:32 +0000
>> 
>> > I suspect that this problem may not be Windows-specific, and should be
>> > reproducable on Linux, but I have not tried that.
>> 
>> I have also tried the mingw.org 32bit toolchain (i686-pc-mingw32) when
>> configured with with "--with-nativecomp --with-wide-int", and that
>> works.
>
> Thanks, that's good news.
>
>> Thus this bug seems to be a problem with the MSYS2 mingw32 32bit
>> toolchain (i686-w64-mingw32) only.
>
> It does seem that way, although I wonder what could that problem be...

Well... I can confirm also i686-pc-linux-gnu "--with-nativecomp
--with-wide-int" looks broken for me.

I'm going to look into this the coming week.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andy Moreton]

On Sun 14 Feb 2021, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Sun, 14 Feb 2021 13:24:32 +0000
>> 
>> > I suspect that this problem may not be Windows-specific, and should be
>> > reproducable on Linux, but I have not tried that.
>> 
>> I have also tried the mingw.org 32bit toolchain (i686-pc-mingw32) when
>> configured with with "--with-nativecomp --with-wide-int", and that
>> works.
>
> Thanks, that's good news.
>
>> Thus this bug seems to be a problem with the MSYS2 mingw32 32bit
>> toolchain (i686-w64-mingw32) only.
>
> It does seem that way, although I wonder what could that problem be...

I have only just discovered this news item:

https://www.msys2.org/news/#2020-05-17-32-bit-msys2-no-longer-actively-supported

Perhaps that means that only the mingw.org (i686-w64-mingw32) toolchain
should be supported for 32bit builds on Windows.

    AndyM

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

On February 15, 2021 11:01:19 AM GMT+02:00, Andy Moreton <andrewjmoreton@gmail.com> wrote:
> On Sun 14 Feb 2021, Eli Zaretskii wrote:
> 
> >> From: Andy Moreton <andrewjmoreton@gmail.com>
> >> Date: Sun, 14 Feb 2021 13:24:32 +0000
> >> 
> >> > I suspect that this problem may not be Windows-specific, and
> should be
> >> > reproducable on Linux, but I have not tried that.
> >> 
> >> I have also tried the mingw.org 32bit toolchain (i686-pc-mingw32)
> when
> >> configured with with "--with-nativecomp --with-wide-int", and that
> >> works.
> >
> > Thanks, that's good news.
> >
> >> Thus this bug seems to be a problem with the MSYS2 mingw32 32bit
> >> toolchain (i686-w64-mingw32) only.
> >
> > It does seem that way, although I wonder what could that problem
> be...
> 
> I have only just discovered this news item:
> 
> https://www.msys2.org/news/#2020-05-17-32-bit-msys2-no-longer-actively-supported
> 
> Perhaps that means that only the mingw.org (i686-w64-mingw32)
> toolchain
> should be supported for 32bit builds on Windows.
> 
>     AndyM

Right, but Andrea said there were problems on Posix systems as well with this configuration, so it could be that something else is at work here.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> On February 15, 2021 11:01:19 AM GMT+02:00, Andy Moreton <andrewjmoreton@gmail.com> wrote:
>> On Sun 14 Feb 2021, Eli Zaretskii wrote:
>> 
>> >> From: Andy Moreton <andrewjmoreton@gmail.com>
>> >> Date: Sun, 14 Feb 2021 13:24:32 +0000
>> >> 
>> >> > I suspect that this problem may not be Windows-specific, and
>> should be
>> >> > reproducable on Linux, but I have not tried that.
>> >> 
>> >> I have also tried the mingw.org 32bit toolchain (i686-pc-mingw32)
>> when
>> >> configured with with "--with-nativecomp --with-wide-int", and that
>> >> works.
>> >
>> > Thanks, that's good news.
>> >
>> >> Thus this bug seems to be a problem with the MSYS2 mingw32 32bit
>> >> toolchain (i686-w64-mingw32) only.
>> >
>> > It does seem that way, although I wonder what could that problem
>> be...
>> 
>> I have only just discovered this news item:
>> 
>> https://www.msys2.org/news/#2020-05-17-32-bit-msys2-no-longer-actively-supported
>> 
>> Perhaps that means that only the mingw.org (i686-w64-mingw32)
>> toolchain
>> should be supported for 32bit builds on Windows.
>> 
>>     AndyM
>
> Right, but Andrea said there were problems on Posix systems as well with this configuration, so it could be that something else is at work here.

Confirm, I see a problem compiling closures, this should be the
responsible for the bootstrap to be broken (at least on my case).

Yesterday I've already reduced a small reproducer so I should be on the
good way to understand during the next debug session why this is
happening only on wide-int.

It will be interesting why is working in some configuration and not in
others.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> On February 15, 2021 11:01:19 AM GMT+02:00, Andy Moreton <andrewjmoreton@gmail.com> wrote:
>>> On Sun 14 Feb 2021, Eli Zaretskii wrote:
>>> 
>>> >> From: Andy Moreton <andrewjmoreton@gmail.com>
>>> >> Date: Sun, 14 Feb 2021 13:24:32 +0000
>>> >> 
>>> >> > I suspect that this problem may not be Windows-specific, and
>>> should be
>>> >> > reproducable on Linux, but I have not tried that.
>>> >> 
>>> >> I have also tried the mingw.org 32bit toolchain (i686-pc-mingw32)
>>> when
>>> >> configured with with "--with-nativecomp --with-wide-int", and that
>>> >> works.
>>> >
>>> > Thanks, that's good news.
>>> >
>>> >> Thus this bug seems to be a problem with the MSYS2 mingw32 32bit
>>> >> toolchain (i686-w64-mingw32) only.
>>> >
>>> > It does seem that way, although I wonder what could that problem
>>> be...
>>> 
>>> I have only just discovered this news item:
>>> 
>>> https://www.msys2.org/news/#2020-05-17-32-bit-msys2-no-longer-actively-supported
>>> 
>>> Perhaps that means that only the mingw.org (i686-w64-mingw32)
>>> toolchain
>>> should be supported for 32bit builds on Windows.
>>> 
>>>     AndyM
>>
>> Right, but Andrea said there were problems on Posix systems as well with this configuration, so it could be that something else is at work here.
>
> Confirm, I see a problem compiling closures, this should be the
> responsible for the bootstrap to be broken (at least on my case).
>
> Yesterday I've already reduced a small reproducer so I should be on the
> good way to understand during the next debug session why this is
> happening only on wide-int.
>
> It will be interesting why is working in some configuration and not in
> others.
>
>   Andrea

Andy could you point out the two libgccjit versions you used specifying
wich one was used in the successfull / failed experiment?

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, andrewjmoreton@gmail.com,
>         46495@debbugs.gnu.org
> Date: Tue, 16 Feb 2021 09:34:49 +0000
> 
> Andy could you point out the two libgccjit versions you used specifying
> wich one was used in the successfull / failed experiment?

Right, this is another potential reason for the difference.  AFAIK,
MinGW64 uses GCC 10.x, whereas mingw.org uses 9.2.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, andrewjmoreton@gmail.com,
>>         46495@debbugs.gnu.org
>> Date: Tue, 16 Feb 2021 09:34:49 +0000
>> 
>> Andy could you point out the two libgccjit versions you used specifying
>> wich one was used in the successfull / failed experiment?
>
> Right, this is another potential reason for the difference.  AFAIK,
> MinGW64 uses GCC 10.x, whereas mingw.org uses 9.2.

Hi Eli,

I did some GCC debugging (the crash is there) using my reduced
reproducer and clearly look (for my case at least) this is a libgccjit
bug that we trigger only generating code for 32bit wide-int.

GCC trunk is broken but as you've anticipated 9 is working (just
finished an Emacs bootstrap).

This evening I'll open a bug on the GCC bugzilla and link it here.
Would be nice to fix it before the end of stage4... :/

Regards

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Tue, 16 Feb 2021 16:30:46 +0000
> 
> I did some GCC debugging (the crash is there) using my reduced
> reproducer and clearly look (for my case at least) this is a libgccjit
> bug that we trigger only generating code for 32bit wide-int.

So GCC 10 has a bug when it generates code which manipulates 64-bit
integers, is that what you are saying?

> GCC trunk is broken but as you've anticipated 9 is working (just
> finished an Emacs bootstrap).

Thanks, this is good to know.  I think we should add an entry to
etc/PROBLEMS about this.

Does the buggy behavior of GCC 10 happen regardless of optimization
level?

> This evening I'll open a bug on the GCC bugzilla and link it here.
> Would be nice to fix it before the end of stage4... :/

Thanks.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> Date: Tue, 16 Feb 2021 16:30:46 +0000
>> 
>> I did some GCC debugging (the crash is there) using my reduced
>> reproducer and clearly look (for my case at least) this is a libgccjit
>> bug that we trigger only generating code for 32bit wide-int.
>
> So GCC 10 has a bug when it generates code which manipulates 64-bit
> integers, is that what you are saying?

It's not strictly related to the integer size.  On 32bit wide-int we
indeed we generate significantly different code respect to 64bit.  For
one case of this GCC manage to prove that a piece of code will deference
a null pointer (this code in reality is unreachable) and tries to add a
call to __builtin_trap () in place.  Unfortunately the libgccjit
front-end is not initializing this built-in declaration.  This is as far
as I've analyzed the problem for now.

>> GCC trunk is broken but as you've anticipated 9 is working (just
>> finished an Emacs bootstrap).
>
> Thanks, this is good to know.  I think we should add an entry to
> etc/PROBLEMS about this.

Will do, still wants to try 10 to be sure.

> Does the buggy behavior of GCC 10 happen regardless of optimization
> level?

I think -O0 should spot this as copy-prop is not running.  We might have
a better (more narrowed) ways to work around this but I need to
investigate more.  Will follow-up.

>> This evening I'll open a bug on the GCC bugzilla and link it here.
>> Would be nice to fix it before the end of stage4... :/
>
> Thanks.

Welcome

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>>> Date: Tue, 16 Feb 2021 16:30:46 +0000
>>> 
>>> I did some GCC debugging (the crash is there) using my reduced
>>> reproducer and clearly look (for my case at least) this is a libgccjit
>>> bug that we trigger only generating code for 32bit wide-int.
>>
>> So GCC 10 has a bug when it generates code which manipulates 64-bit
>> integers, is that what you are saying?
>
> It's not strictly related to the integer size.  On 32bit wide-int we
> indeed we generate significantly different code respect to 64bit.  For
> one case of this GCC manage to prove that a piece of code will deference
> a null pointer (this code in reality is unreachable) and tries to add a
> call to __builtin_trap () in place.  Unfortunately the libgccjit
> front-end is not initializing this built-in declaration.  This is as far
> as I've analyzed the problem for now.
>
>>> GCC trunk is broken but as you've anticipated 9 is working (just
>>> finished an Emacs bootstrap).
>>
>> Thanks, this is good to know.  I think we should add an entry to
>> etc/PROBLEMS about this.
>
> Will do, still wants to try 10 to be sure.
>
>> Does the buggy behavior of GCC 10 happen regardless of optimization
>> level?
>
> I think -O0 should spot this as copy-prop is not running.  We might have
> a better (more narrowed) ways to work around this but I need to
> investigate more.  Will follow-up.
>
>>> This evening I'll open a bug on the GCC bugzilla and link it here.
>>> Would be nice to fix it before the end of stage4... :/
>>
>> Thanks.
>
> Welcome
>
>   Andrea

Here the bugzilla bug with some description more:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99126

I think a good work-around might be to try to switch off the
'isolate-paths' pass.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andy Moreton]

On Tue 16 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:

> Andy could you point out the two libgccjit versions you used specifying
> wich one was used in the successfull / failed experiment?

It seems they both now fail (and I have lost the build log from the
successful experiment, so I don't have a note of the commit used).

mingw.org (i686-w64-mingw32)
----------------------------
I retried the mingw.org build configured with "--with-nativecomp
--with-wide-int" (starting from "git clean -xdf"), and that now fails
at commit 31416495ad9b2c84473f72ad99e2adc87dd66e5a.

My mingw.org installation has libgccjit packages:
  libgccjit-9.2.0-2-mingw32-dev.tar.xz
  libgccjit-9.2.0-2-mingw32-dll-0.tar.xz
  libgccjit-9.2.0-2-mingw32-info.tar.xz

These were obtained from a link on the ticket at:
  https://osdn.net/projects/mingw/ticket/41070

That was part of the work done after Eli requested libjcc support.


MSYS2 (i686-w64-mingw32)
------------------------
Running "pacman -Qs gccjit" shows:

local/mingw-w64-i686-libgccjit 10.2.0-6 (mingw-w64-i686-toolchain)
    GNU Compiler Collection (libgccjit) for MinGW-w64
local/mingw-w64-x86_64-libgccjit 10.2.0-6 (mingw-w64-x86_64-toolchain)
    GNU Compiler Collection (libgccjit) for MinGW-w64

As noted in an earlier message upthread, the MSYS2 developers have
stopped support for the 32bit builds.


   AndyM

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> On Tue 16 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>
>> Andy could you point out the two libgccjit versions you used specifying
>> wich one was used in the successfull / failed experiment?
>
> It seems they both now fail (and I have lost the build log from the
> successful experiment, so I don't have a note of the commit used).

That's odd.  The culprit of the bug I've isolated is a crash in
libgccjit so should be easy to recognize (we should never crash in
libgccjit).

Anyway I've an half cooked patch to work around this issue, I guess I'll
test it this evening.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1455 bytes --]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Andy Moreton <andrewjmoreton@gmail.com> writes:
>
>> On Tue 16 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>>
>>> Andy could you point out the two libgccjit versions you used specifying
>>> wich one was used in the successfull / failed experiment?
>>
>> It seems they both now fail (and I have lost the build log from the
>> successful experiment, so I don't have a note of the commit used).
>
> That's odd.  The culprit of the bug I've isolated is a crash in
> libgccjit so should be easy to recognize (we should never crash in
> libgccjit).
>
> Anyway I've an half cooked patch to work around this issue, I guess I'll
> test it this evening.

A fix like the attached is working around the GCC bug.

The only annoyance of this approach is that libgccjit for each
compilation is outputting to console:
"libgccjit.so: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]"

We probably also want to check at configure time and raise an error in
case configuring --with-nativecomp --with-wide-int but with a (really)
old libgccjit that does not provide
'gcc_jit_context_add_command_line_option'.

Works for me bootstraping Emacs on i686-pc-linux-gnu + GCC10 but I can't
test the Windows side, Andy would you mind giving it a try?

Thanks

  Andrea


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 46495.patch --]
[-- Type: text/x-diff, Size: 2665 bytes --]

diff --git a/src/comp.c b/src/comp.c
index 5e95161030..bb3cd83036 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -56,6 +56,7 @@
 #undef gcc_jit_block_end_with_return
 #undef gcc_jit_block_end_with_void_return
 #undef gcc_jit_context_acquire
+#undef gcc_jit_context_add_command_line_option
 #undef gcc_jit_context_add_driver_option
 #undef gcc_jit_context_compile_to_file
 #undef gcc_jit_context_dump_reproducer_to_file
@@ -124,6 +125,8 @@ DEF_DLL_FN (const char *, gcc_jit_context_get_first_error,
 DEF_DLL_FN (gcc_jit_block *, gcc_jit_function_new_block,
             (gcc_jit_function *func, const char *name));
 DEF_DLL_FN (gcc_jit_context *, gcc_jit_context_acquire, (void));
+DEF_DLL_FN (void, gcc_jit_context_add_command_line_option,
+            (gcc_jit_context *ctxt, const char *optname));
 DEF_DLL_FN (void, gcc_jit_context_add_driver_option,
             (gcc_jit_context *ctxt, const char *optname));
 DEF_DLL_FN (gcc_jit_field *, gcc_jit_context_new_field,
@@ -312,6 +315,7 @@ init_gccjit_functions (void)
   LOAD_DLL_FN (library, gcc_jit_struct_set_fields);
   LOAD_DLL_FN (library, gcc_jit_type_get_const);
   LOAD_DLL_FN (library, gcc_jit_type_get_pointer);
+  LOAD_DLL_FN_OPT (library, gcc_jit_context_add_command_line_option);
   LOAD_DLL_FN_OPT (library, gcc_jit_context_add_driver_option);
   LOAD_DLL_FN_OPT (library, gcc_jit_global_set_initializer);
   LOAD_DLL_FN_OPT (library, gcc_jit_version_major);
@@ -330,6 +334,7 @@ #define gcc_jit_block_end_with_jump fn_gcc_jit_block_end_with_jump
 #define gcc_jit_block_end_with_return fn_gcc_jit_block_end_with_return
 #define gcc_jit_block_end_with_void_return fn_gcc_jit_block_end_with_void_return
 #define gcc_jit_context_acquire fn_gcc_jit_context_acquire
+#define gcc_jit_context_add_command_line_option fn_gcc_jit_context_add_command_line_option
 #define gcc_jit_context_add_driver_option fn_gcc_jit_context_add_driver_option
 #define gcc_jit_context_compile_to_file fn_gcc_jit_context_compile_to_file
 #define gcc_jit_context_dump_reproducer_to_file fn_gcc_jit_context_dump_reproducer_to_file
@@ -4400,6 +4405,15 @@ DEFUN ("comp--compile-ctxt-to-file", Fcomp__compile_ctxt_to_file,
     if (!EQ (HASH_VALUE (func_h, i), Qunbound))
       compile_function (HASH_VALUE (func_h, i));
 
+#if defined (WIDE_EMACS_INT)						\
+  && (defined (LIBGCCJIT_HAVE_gcc_jit_context_add_command_line_option)	\
+      || defined (WINDOWSNT))
+  Lisp_Object version = Fcomp_libgccjit_version ();
+  if (!NILP (version) && XFIXNUM (XCAR (version)) >= 10)
+    gcc_jit_context_add_command_line_option (comp.ctxt,
+					     "-fdisable-tree-isolate-paths");
+#endif
+
   add_driver_options ();
 
   if (comp.debug)

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andy Moreton]

On Wed 17 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:

> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs@gnu.org> writes:
>
>> Andy Moreton <andrewjmoreton@gmail.com> writes:
>>
>>> On Tue 16 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>>>
>>>> Andy could you point out the two libgccjit versions you used specifying
>>>> wich one was used in the successfull / failed experiment?
>>>
>>> It seems they both now fail (and I have lost the build log from the
>>> successful experiment, so I don't have a note of the commit used).
>>
>> That's odd.  The culprit of the bug I've isolated is a crash in
>> libgccjit so should be easy to recognize (we should never crash in
>> libgccjit).
>>
>> Anyway I've an half cooked patch to work around this issue, I guess I'll
>> test it this evening.
>
> A fix like the attached is working around the GCC bug.
>
> The only annoyance of this approach is that libgccjit for each
> compilation is outputting to console:
> "libgccjit.so: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]"
>
> We probably also want to check at configure time and raise an error in
> case configuring --with-nativecomp --with-wide-int but with a (really)
> old libgccjit that does not provide
> 'gcc_jit_context_add_command_line_option'.
>
> Works for me bootstraping Emacs on i686-pc-linux-gnu + GCC10 but I can't
> test the Windows side, Andy would you mind giving it a try?

I tried doing clean builds (after "git clean -xdf" each time ) both
without and then with this patch. The i686-pc-mingw32 toolchain is:
   gcc version 9.2.0 (MinGW.org GCC Build-2).

Without your patch, the i686-pc-mingw32 build succeeded. As I have seen
some builds succeed and some fail, there is perhaps another issue to
look into after this one.

With your patch, the i686-pc-mingw32 build succeeded. Running the
resulting emacs still produced some crashes in the async compile processes.

Program received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 6952.0x1e8c]
0x757bff43 in KERNELBASE!DebugBreak () from C:\WINDOWS\System32\KernelBase.dll
#0  0x757bff43 in KERNELBASE!DebugBreak () from C:\WINDOWS\System32\KernelBase.dll
#1  0x012d2778 in emacs_abort () at c:/emacs/git/emacs/native/src/w32fns.c:10914
#2  0x0112509c in terminate_due_to_signal (sig=11, backtrace_limit=40) at c:/emacs/git/emacs/native/src/emacs.c:416
#3  0x01158fd2 in handle_fatal_signal (sig=11) at c:/emacs/git/emacs/native/src/sysdep.c:1762
#4  0x01158fac in deliver_thread_signal (sig=11, handler=0x1158fb9 <handle_fatal_signal>) at c:/emacs/git/emacs/native/src/sysdep.c:1754
#5  0x01159007 in deliver_fatal_thread_signal (sig=11) at c:/emacs/git/emacs/native/src/sysdep.c:1774
#6  0x010010d1 in __register_frame_info ()
#7  0x012d2693 in my_exception_handler (exception_data=0xbfc764) at c:/emacs/git/emacs/native/src/w32fns.c:10862
#8  0x7581e6e2 in UnhandledExceptionFilter () from C:\WINDOWS\System32\KernelBase.dll
#9  0x00bfc764 in ?? ()
#10 0x77364c91 in ntdll!RtlCaptureStackContext () from C:\WINDOWS\SYSTEM32\ntdll.dll
#11 0x77327684 in ntdll!RtlGetAppContainerNamedObjectPath () from C:\WINDOWS\SYSTEM32\ntdll.dll
#12 0x00000000 in ?? ()

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> On Wed 17 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>
>> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
>> text editors" <bug-gnu-emacs@gnu.org> writes:
>>
>>> Andy Moreton <andrewjmoreton@gmail.com> writes:
>>>
>>>> On Tue 16 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>>>>
>>>>> Andy could you point out the two libgccjit versions you used specifying
>>>>> wich one was used in the successfull / failed experiment?
>>>>
>>>> It seems they both now fail (and I have lost the build log from the
>>>> successful experiment, so I don't have a note of the commit used).
>>>
>>> That's odd.  The culprit of the bug I've isolated is a crash in
>>> libgccjit so should be easy to recognize (we should never crash in
>>> libgccjit).
>>>
>>> Anyway I've an half cooked patch to work around this issue, I guess I'll
>>> test it this evening.
>>
>> A fix like the attached is working around the GCC bug.
>>
>> The only annoyance of this approach is that libgccjit for each
>> compilation is outputting to console:
>> "libgccjit.so: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]"
>>
>> We probably also want to check at configure time and raise an error in
>> case configuring --with-nativecomp --with-wide-int but with a (really)
>> old libgccjit that does not provide
>> 'gcc_jit_context_add_command_line_option'.
>>
>> Works for me bootstraping Emacs on i686-pc-linux-gnu + GCC10 but I can't
>> test the Windows side, Andy would you mind giving it a try?
>
> I tried doing clean builds (after "git clean -xdf" each time ) both
> without and then with this patch. The i686-pc-mingw32 toolchain is:
>    gcc version 9.2.0 (MinGW.org GCC Build-2).

The patch is not affecting GCC9 as initially was reported to be working.
If you suspect also 9 is affected you can trivially modify the patch to
take effect on 9 too.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andy Moreton]

On Fri 19 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:

> The patch is not affecting GCC9 as initially was reported to be working.
> If you suspect also 9 is affected you can trivially modify the patch to
> take effect on 9 too.

I tried a clean rebuild after modifying the patch to appy for gcc9.
No difference - the compile processes still crash.

Debugging this further will have to wait for Eli or some other
interested person who uses this platform to investigate this issue.

    AndyM

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> On Fri 19 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>
>> The patch is not affecting GCC9 as initially was reported to be working.
>> If you suspect also 9 is affected you can trivially modify the patch to
>> take effect on 9 too.
>
> I tried a clean rebuild after modifying the patch to appy for gcc9.
> No difference - the compile processes still crash.
>
> Debugging this further will have to wait for Eli or some other
> interested person who uses this platform to investigate this issue.

Agree.  BTW it might be as well that a new bug was introduced with the
recent changes, I'll have another 32bit wide-int try in the weekend.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Fri, 19 Feb 2021 16:01:36 +0000
> 
> Debugging this further will have to wait for Eli or some other
> interested person who uses this platform to investigate this issue.

Didn't you say the crashes didn't happen with mingw.org's libgccjit?
Or did they start happening recently?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andy Moreton]

On Fri 19 Feb 2021, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Fri, 19 Feb 2021 16:01:36 +0000
>> 
>> Debugging this further will have to wait for Eli or some other
>> interested person who uses this platform to investigate this issue.
>
> Didn't you say the crashes didn't happen with mingw.org's libgccjit?
> Or did they start happening recently?

Originally I thought that mingw.org's libgccjit was working, but later
builds showed that not to be true.

    AndyM

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Fri, 19 Feb 2021 17:35:46 +0000
> 
> > Didn't you say the crashes didn't happen with mingw.org's libgccjit?
> > Or did they start happening recently?
> 
> Originally I thought that mingw.org's libgccjit was working, but later
> builds showed that not to be true.

I see, thanks.

Btw, one other way of getting GDB attach to the crashing program "just
in time" is to use the Windows AeDebug support in GDB.  It is
described in the node "Cygwin Native" in the GDB manual, under
"signal-event ID".

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> On Fri 19 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>
>> The patch is not affecting GCC9 as initially was reported to be working.
>> If you suspect also 9 is affected you can trivially modify the patch to
>> take effect on 9 too.
>
> I tried a clean rebuild after modifying the patch to appy for gcc9.
> No difference - the compile processes still crash.
>
> Debugging this further will have to wait for Eli or some other
> interested person who uses this platform to investigate this issue.
>
>     AndyM

So with 39792cf629 I've pushed the work-around limiting it to GCC10 (as
in trunk is freshly fixed).

I tried again to bootstrap on 32bit wide-int and it works.

The compiler testsuite is segfaulting for me... so I guess I'll have
something more to debug.  This might be the reason of the problem you
are observing.  I'll follow-up on this.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo <akrl@sdf.org> writes:

> Andy Moreton <andrewjmoreton@gmail.com> writes:
>
>> On Fri 19 Feb 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>>
>>> The patch is not affecting GCC9 as initially was reported to be working.
>>> If you suspect also 9 is affected you can trivially modify the patch to
>>> take effect on 9 too.
>>
>> I tried a clean rebuild after modifying the patch to appy for gcc9.
>> No difference - the compile processes still crash.
>>
>> Debugging this further will have to wait for Eli or some other
>> interested person who uses this platform to investigate this issue.
>>
>>     AndyM
>
> So with 39792cf629 I've pushed the work-around limiting it to GCC10 (as
> in trunk is freshly fixed).
>
> I tried again to bootstrap on 32bit wide-int and it works.
>
> The compiler testsuite is segfaulting for me... so I guess I'll have
> something more to debug.  This might be the reason of the problem you
> are observing.  I'll follow-up on this.

I've tested now the current branch c6c7b30e4b compiling 32bit
--with-wide-int.

I have the impression that all 32bit --with-wide-int specific issues
have been solved here or in the other related threads.

All the compiler testsuite is running clean for me and I think we should
classify the segfault Eli observed running `comp-tests-bootstrap' as a
libgccjit bug specific to that version (libgccjit should *never* crash
and if does it's a bug in the library).

Eli, how do you find the current branch stability on 32bit wide-int?

Is there anything left we should look into for this bug?

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> Cc: 46495@debbugs.gnu.org
> Date: Fri, 26 Mar 2021 10:18:55 +0000
> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Eli, how do you find the current branch stability on 32bit wide-int?

It looks in good shape, AFAICT.

One issue left, which I'm pursuing locally, is the one with strange
gaps in the C backtraces.  You can see the traces of that on the
gdb-patches mailing list.  When I finish that investigation, hopefully
next week, we will be able to see whether this is some bug in the
native-comp branch or just lack of debug info; in the latter case I
will probably ask for changing the default debug-level in comp.el, at
least on MS-Windows.

Stay tuned.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 46495@debbugs.gnu.org
>> Date: Fri, 26 Mar 2021 10:18:55 +0000
>> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> Eli, how do you find the current branch stability on 32bit wide-int?
>
> It looks in good shape, AFAICT.

Very nice.

> One issue left, which I'm pursuing locally, is the one with strange
> gaps in the C backtraces.  You can see the traces of that on the
> gdb-patches mailing list.  When I finish that investigation, hopefully
> next week, we will be able to see whether this is some bug in the
> native-comp branch or just lack of debug info; in the latter case I
> will probably ask for changing the default debug-level in comp.el, at
> least on MS-Windows.
>
> Stay tuned.

Sure, I've seen some of that (interesting reading), thanks for looking
into this.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> Date: Fri, 26 Mar 2021 14:27:21 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> 
> One issue left, which I'm pursuing locally, is the one with strange
> gaps in the C backtraces.  You can see the traces of that on the
> gdb-patches mailing list.

The GDB experts seem to indicate that the prologue of functions we
produce is different from the prologue normally produced by GCC from C
code on x86.  See

  https://sourceware.org/pipermail/gdb-patches/2021-March/177334.html

Is that possible? are we doing something that could affect the
prologue of the natively-compiled Lisp code?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Fri, 26 Mar 2021 14:27:21 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> 
>> One issue left, which I'm pursuing locally, is the one with strange
>> gaps in the C backtraces.  You can see the traces of that on the
>> gdb-patches mailing list.
>
> The GDB experts seem to indicate that the prologue of functions we
> produce is different from the prologue normally produced by GCC from C
> code on x86.  See
>
>   https://sourceware.org/pipermail/gdb-patches/2021-March/177334.html
>
> Is that possible?

Well if this is what you observe it certainly can be.  There might very
well be something in how/what libgccjit generates that is causing this
difference.

> are we doing something that could affect the
> prologue of the natively-compiled Lisp code?

I suspect this is a libgccjit thing, without knowing where the
difference is coming from it's hard to predict if there's a workaround
we can put in place in the Emacs codebase, but I suspect there's not.

If the generated code is correct I think gdb should recognize it
improving its unwinder, otherwise if this is a libgccjit bug we should
open a PR on bugzilla.

Perhaps if the case is the later one can try a simpler testcase to
report it using the test we have in the configure or libgccjit hello
world [1].  This might help also analysing how this different frame is
formed.

Thanks

  Andrea

[1] <https://gcc.gnu.org/onlinedocs/jit/intro/tutorial01.html>

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Mon, 29 Mar 2021 16:15:56 +0000
> 
> >   https://sourceware.org/pipermail/gdb-patches/2021-March/177334.html
> >
> > Is that possible?
> 
> Well if this is what you observe it certainly can be.  There might very
> well be something in how/what libgccjit generates that is causing this
> difference.

David, can you please chime in and help us understand what is going on
here?

> > are we doing something that could affect the
> > prologue of the natively-compiled Lisp code?
> 
> I suspect this is a libgccjit thing, without knowing where the
> difference is coming from it's hard to predict if there's a workaround
> we can put in place in the Emacs codebase, but I suspect there's not.
> 
> If the generated code is correct I think gdb should recognize it
> improving its unwinder, otherwise if this is a libgccjit bug we should
> open a PR on bugzilla.

GDB's native platform is ELF-based, so its unwinders' support for
COFF-PE format used by MS-Windows is known to be less thorough.

> Perhaps if the case is the later one can try a simpler testcase to
> report it using the test we have in the configure or libgccjit hello
> world [1].  This might help also analysing how this different frame is
> formed.

I think if David doesn't show us the light, my best bet is to
recompile the Lisp code with debug info and see if that resolves the
problem and/or allows us to understand why the code with no debug info
produces these effects.

Thanks.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[David Malcolm]

On Mon, 2021-03-29 at 19:59 +0300, Eli Zaretskii wrote:
> > From: Andrea Corallo <akrl@sdf.org>
> > Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> > Date: Mon, 29 Mar 2021 16:15:56 +0000
> > 
> > >   
> > > https://sourceware.org/pipermail/gdb-patches/2021-March/177334.html
> > > 
> > > Is that possible?
> > 
> > Well if this is what you observe it certainly can be.  There might
> > very
> > well be something in how/what libgccjit generates that is causing
> > this
> > difference.
> 
> David, can you please chime in and help us understand what is going
> on
> here?

In theory, libgccjit uses the same code-generation code and the same
debuginfo-generation code as a regular gcc invocation.

Some ideas:

(a) Is libgccjit being invoked multiple times in-process?
If so, I wonder if you're running into a latent bug in state-handling
in libgccjit that's affecting either code generation or debuginfo
generation.  Maybe something isn't being reset that should have been?

Is there a way to get emacs to only do one libgccjit context
compilation per process, and see if that affects things?

(b) Alternatively, maybe a simple bug in libgccjit that affects the
first in-process invocation?

(c) Alternatively, maybe a bug in the emacs jit stuff that's corrupting
the stack somehow?

Some debugging hooks are here:
  https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html#debugging


> > > are we doing something that could affect the
> > > prologue of the natively-compiled Lisp code?
> > 
> > I suspect this is a libgccjit thing, without knowing where the
> > difference is coming from it's hard to predict if there's a
> > workaround
> > we can put in place in the Emacs codebase, but I suspect there's
> > not.
> > 
> > If the generated code is correct I think gdb should recognize it
> > improving its unwinder, otherwise if this is a libgccjit bug we
> > should
> > open a PR on bugzilla.

In particular, this may be helpful for that:
https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html#c.gcc_jit_context_dump_reproducer_to_file


> GDB's native platform is ELF-based, so its unwinders' support for
> COFF-PE format used by MS-Windows is known to be less thorough.
> 
> > Perhaps if the case is the later one can try a simpler testcase to
> > report it using the test we have in the configure or libgccjit
> > hello
> > world [1].  This might help also analysing how this different frame
> > is
> > formed.
> 
> I think if David doesn't show us the light, my best bet is to
> recompile the Lisp code with debug info and see if that resolves the
> problem and/or allows us to understand why the code with no debug
> info
> produces these effects.

I'd try turning on debuginfo generation to see if that helps.

Hope this is helpful
Dave

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

David Malcolm <dmalcolm@redhat.com> writes:

> On Mon, 2021-03-29 at 19:59 +0300, Eli Zaretskii wrote:
>> > From: Andrea Corallo <akrl@sdf.org>
>> > Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> > Date: Mon, 29 Mar 2021 16:15:56 +0000
>> > 
>> > >   
>> > > https://sourceware.org/pipermail/gdb-patches/2021-March/177334.html
>> > > 
>> > > Is that possible?
>> > 
>> > Well if this is what you observe it certainly can be.  There might
>> > very
>> > well be something in how/what libgccjit generates that is causing
>> > this
>> > difference.
>> 
>> David, can you please chime in and help us understand what is going
>> on
>> here?
>
> In theory, libgccjit uses the same code-generation code and the same
> debuginfo-generation code as a regular gcc invocation.
>
> Some ideas:
>
> (a) Is libgccjit being invoked multiple times in-process?
> If so, I wonder if you're running into a latent bug in state-handling
> in libgccjit that's affecting either code generation or debuginfo
> generation.  Maybe something isn't being reset that should have been?
>
> Is there a way to get emacs to only do one libgccjit context
> compilation per process, and see if that affects things?

Hi Dave,

libgccjit here is invoked once.

> (b) Alternatively, maybe a simple bug in libgccjit that affects the
> first in-process invocation?

Shouldn't this likely affect configurations other than Windows i386?

> (c) Alternatively, maybe a bug in the emacs jit stuff that's corrupting
> the stack somehow?

I think as suggested compiling a simple function on Eli's system like
the hello word may give a strong indication on if this is Emacs related
or not.

Regards

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, andrewjmoreton@gmail.com,
>         46495@debbugs.gnu.org
> Date: Mon, 29 Mar 2021 20:46:27 +0000
> 
> > (c) Alternatively, maybe a bug in the emacs jit stuff that's corrupting
> > the stack somehow?
> 
> I think as suggested compiling a simple function on Eli's system like
> the hello word may give a strong indication on if this is Emacs related
> or not.

I have the hello world example from the libgccjit tutorial compiled
here, but I'm not sure what would you like me to test there.

Here's what I did:

  . compiled the example with -gdwarf-4 -g3
  . started GDB and ran the program until it calls the 'greet'
    function
  . used "si" and "ni" commands to step into 'greet' and through the
    functions it calls

The below is the transcript of that GDB session.  Note how a valid
backtrace sometimes become incomplete and uses "??" for some functions
that are correctly identified by their names at other times.

Does this provide the evidence you wanted to see?  Does it mean that
GDB is sometimes unable to determine the beginning of libgccjit
generated functions, at least on MS-Windows?

  D:\usr\eli\libgccjit>gdb ./tut01-hello-world_d.exe
  GNU gdb (GDB) 10.1
  Copyright (C) 2020 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.
  Type "show copying" and "show warranty" for details.
  This GDB was configured as "i686-pc-mingw32".
  Type "show configuration" for configuration details.
  For bug reporting instructions, please see:
  <https://www.gnu.org/software/gdb/bugs/>.
  Find the GDB manual and other documentation resources online at:
      <http://www.gnu.org/software/gdb/documentation/>.

  For help, type "help".
  Type "apropos word" to search for commands related to "word"...
  Reading symbols from ./tut01-hello-world_d.exe...
  (gdb) start
  Temporary breakpoint 1 at 0x4015c2: file tut01-hello-world.c, line 81.
  Starting program: D:\usr\eli\libgccjit\tut01-hello-world_d.exe

  Temporary breakpoint 1, main (argc=1, argv=0x3e2978) at tut01-hello-world.c:81
  81        ctxt = gcc_jit_context_acquire ();
  (gdb) until 117
  main (argc=1, argv=0x3e2978) at tut01-hello-world.c:117
  117       greet ("world");
  (gdb) si
  0x004016d3      117       greet ("world");
  (gdb)
  0x004016d7      117       greet ("world");
  (gdb)
  0x66a41280 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb) bt
  #0  0x66a41280 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #1  0x004016d9 in main (argc=1, argv=0x3e2978) at tut01-hello-world.c:117
  (gdb) ni
  0x66a41281 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb)
  0x66a41283 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb) si
  0x66a41286 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb) si
  0x66a41289 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb) si
  0x66a4128d in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb) si
  0x66a41294 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb)
  0x66a41ad8 in printf ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb) bt
  #0  0x66a41ad8 in printf ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #1  0x66a41299 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #2  0x004016d9 in main (argc=1, argv=0x3e2978) at tut01-hello-world.c:117
  (gdb) si
  0x77c4186a in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c4186c in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c41871 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37420 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37425 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3742b in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3742c in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37430 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37434 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37438 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3743a in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3743b in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3743c in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3743d in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37440 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37443 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37444 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c37444 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x66a41299 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #2  0x004016d9 in main (argc=1, argv=0x3e2978) at tut01-hello-world.c:117
  (gdb) si
  0x77c37447 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3744e in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37451 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c37454 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3745a in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c41876 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c4187b in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c4187c in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c4187e in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3b914 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3b916 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3b917 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3b919 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3b91c in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3b91f in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3b921 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c3b921 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x77c41883 in printf () from C:\WINDOWS\system32\msvcrt.dll
  #2  0x00000001 in ?? ()
  #3  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #4  0x00000000 in ?? ()
  (gdb) si
  0x77c3b924 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3b925 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5bb in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5bd in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5be in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5c0 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5c3 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5c4 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5cb in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5ce in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5e3 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x77c3a5e5 in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) si
  0x7c901000 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) bt
  #0  0x7c901000 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  #1  0x77c3a5eb in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  #2  0x77c3b92a in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  #3  0x77c41883 in printf () from C:\WINDOWS\system32\msvcrt.dll
  #4  0x00000001 in ?? ()
  #5  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #6  0x00000000 in ?? ()
  (gdb) ni
  0x7c901007 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c90100b in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c90100f in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901060 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901063 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901066 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901080 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901083 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901088 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c90108d in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901092 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901094 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c901097 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c90109e in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c9010a1 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c9010a4 in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c9010ab in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x7c9010ad in ntdll!RtlEnterCriticalSection ()
     from C:\WINDOWS\system32\ntdll.dll
  (gdb) ni
  0x77c3a5eb in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c3a5eb in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x77c3b92a in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  #2  0x77c41883 in printf () from C:\WINDOWS\system32\msvcrt.dll
  #3  0x00000001 in ?? ()
  #4  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #5  0x00000000 in ?? ()
  (gdb) ni
  0x77c3a5ec in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c3a5ed in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c3b92a in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c3b92a in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x77c41883 in printf () from C:\WINDOWS\system32\msvcrt.dll
  #2  0x00000001 in ?? ()
  #3  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #4  0x00000000 in ?? ()
  (gdb) ni
  0x77c3b92b in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c3b92c in msvcrt!_lock () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c41883 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c41883 in printf () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x00000001 in ?? ()
  #2  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #3  0x00000000 in ?? ()
  (gdb) ni
  0x77c41884 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c41885 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c41889 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c4188a in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c4188f in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c4188f in printf () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #2  0x00000000 in ?? ()
  (gdb) ni
  0x77c41892 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c41895 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c41896 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c41899 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c4189a in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c4189f in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c418a2 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c418a3 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c418a6 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  hello world
  0x77c418ab in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c418ab in printf () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x00000001 in ?? ()
  #2  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #3  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #4  0x66a43044 in ?? ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #5  0x66a41299 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #6  0x004016d9 in main (argc=1, argv=0x3e2978) at tut01-hello-world.c:117
  (gdb) ni
  0x77c418ae in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c418b2 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c418b7 in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c418ba in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c3745b in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c3745b in strerror () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x66a41299 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #2  0x004016d9 in main (argc=1, argv=0x3e2978) at tut01-hello-world.c:117
  (gdb) ni
  0x77c3745e in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c37465 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c37466 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c37467 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c37468 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c37469 in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c3746a in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c3746b in strerror () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) ni
  0x77c418bf in printf () from C:\WINDOWS\system32\msvcrt.dll
  (gdb) bt
  #0  0x77c418bf in printf () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x66a41299 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #2  0x004016d9 in main (argc=1, argv=0x3e2978) at tut01-hello-world.c:117
  (gdb) ni
  0x66a41299 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb) bt
  #0  0x66a41299 in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  #1  0x004016d9 in main (argc=1, argv=0x3e2978) at tut01-hello-world.c:117
  (gdb) ni
  0x66a4129a in greet ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-wf9bcr\fake.so
  (gdb) ni
  main (argc=1, argv=0x3e2978) at tut01-hello-world.c:118
  118       fflush (stdout);
  (gdb) bt
  #0  main (argc=1, argv=0x3e2978) at tut01-hello-world.c:118
  (gdb) c
  Continuing.
  [Inferior 1 (process 7076) exited normally]
  (gdb)

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> Date: Tue, 30 Mar 2021 10:50:05 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
> 
> Here's what I did:
> 
>   . compiled the example with -gdwarf-4 -g3
>   . started GDB and ran the program until it calls the 'greet'
>     function
>   . used "si" and "ni" commands to step into 'greet' and through the
>     functions it calls

I have now compiled a slightly modified version of the hello world
example, after adding this to the source:

  gcc_jit_context_set_bool_option (
				   ctxt,
				   GCC_JIT_BOOL_OPTION_DEBUGINFO,
				   1);

The results under debugger are slightly better, for example, the
arguments to JIT functions are shown:

  (gdb) bt
  #0  0x6ac81280 in greet (name=0x406088 "world")
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-0z6iua\fake.so

However, the "??" thingies and the incomplete backtraces are still
present, here are two examples:

  #0  0x77c418ab in printf () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x00000001 in ?? ()
  #2  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #3  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #4  0x6ac83044 in ?? ()
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-0z6iua\fake.so
  #5  0x6ac81299 in greet (name=0x406088 "world")
     from C:\DOCUME~1\Zaretzky\LOCALS~1\Temp\libgccjit-0z6iua\fake.so
  #6  0x004016f5 in main (argc=1, argv=0x3e2978) at tut01d-hello-world.c:121

  #0  0x77c41883 in printf () from C:\WINDOWS\system32\msvcrt.dll
  #1  0x00000001 in ?? ()
  #2  0x77c5fca0 in msvcrt!_iob () from C:\WINDOWS\system32\msvcrt.dll
  #3  0x00000000 in ?? ()

This is quite disappointing.  It probably means that debugging Emacs
with natively-compiled Lisp will be much harder on MS-Windows, if you
happen to be in a situation (whose causes are still unclear to me,
probably some minor variations of the prologue code?) with these
incomplete backtraces.

I also have 3 followup questions:

 1) When GCC_JIT_BOOL_OPTION_DEBUGINFO is used, what kind of debug
    info is generated by libgccjit? is it DWARF2 or stabs or something
    else?  Using "objdump -W", it looks like DWARF2, but if so, why
    isn't GDB able to produce a decent backtrace?

 2) If the debug info created under GCC_JIT_BOOL_OPTION_DEBUGINFO is
    not DWARF2, is it possible to use
    gcc_jit_context_add_command_line_option to request "-gdwarf-4" or
    similar, like we do with the command-line GCC, so that the debug
    info for the JIT code is more detailed and accurate?

 3) I see in my temporary directory subdirectories, created when I run
    the example program, with files fake.s and fake.so.  Are they
    supposed to be left there, or are they supposed to be deleted
    when the program exits?

Thanks.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> Date: Tue, 30 Mar 2021 12:06:38 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: akrl@sdf.org, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> 
>  3) I see in my temporary directory subdirectories, created when I run
>     the example program, with files fake.s and fake.so.  Are they
>     supposed to be left there, or are they supposed to be deleted
>     when the program exits?

These temporary files behave strangely, to say the least.  Just
running the tut01-hello-world example program produces a new temporary
directory each time, and deposits a fake.so file there.  If I run a
variant of that which I built after adding

  gcc_jit_context_set_bool_option (
				   ctxt,
				   GCC_JIT_BOOL_OPTION_DEBUGINFO,
				   1);

then the temporary directory isn't created, or maybe it's deleted when
the program exits.  I think the latter is the case, because the
directory is visible if I step through the program with a debugger,
but disappears when the program exits.

David, what's the story with these temporary directories?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[David Malcolm]

On Wed, 2021-03-31 at 11:13 +0300, Eli Zaretskii wrote:
> > Date: Tue, 30 Mar 2021 12:06:38 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: akrl@sdf.org, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> > 
> >  3) I see in my temporary directory subdirectories, created when I
> > run
> >     the example program, with files fake.s and fake.so.  Are they
> >     supposed to be left there, or are they supposed to be deleted
> >     when the program exits?
> 
> These temporary files behave strangely, to say the least.  Just
> running the tut01-hello-world example program produces a new
> temporary
> directory each time, and deposits a fake.so file there.  If I run a
> variant of that which I built after adding
> 
>   gcc_jit_context_set_bool_option (
>                                    ctxt,
>                                    GCC_JIT_BOOL_OPTION_DEBUGINFO,
>                                    1);
> 
> then the temporary directory isn't created, or maybe it's deleted
> when
> the program exits.  I think the latter is the case, because the
> directory is visible if I step through the program with a debugger,
> but disappears when the program exits.
> 
> David, what's the story with these temporary directories?

They're meant to be cleaned up automatically by libgccjit: on
gcc_jit_result_release for a successful compilation, or at the end of
gcc_jit_context_compile* for a failed compilation.

The removal code is in gcc::jit::tempdir::~tempdir in gcc/jit/jit-
tempdir.c, maybe there's a bug there? (perhaps only affecting Windows?)
It calls unlink on everything it "knows" about, and them rmdir on the
directory.  I see now that I'm not checking for errors on those unlink
and rmdir calls.

Is the code setting GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES to true,
perhaps?  That forcibly keeps them around:
  https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html#c.GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES


Dave

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

David Malcolm <dmalcolm@redhat.com> writes:

> On Wed, 2021-03-31 at 11:13 +0300, Eli Zaretskii wrote:
>> > Date: Tue, 30 Mar 2021 12:06:38 +0300
>> > From: Eli Zaretskii <eliz@gnu.org>
>> > Cc: akrl@sdf.org, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> > 
>> >  3) I see in my temporary directory subdirectories, created when I
>> > run
>> >     the example program, with files fake.s and fake.so.  Are they
>> >     supposed to be left there, or are they supposed to be deleted
>> >     when the program exits?
>> 
>> These temporary files behave strangely, to say the least.  Just
>> running the tut01-hello-world example program produces a new
>> temporary
>> directory each time, and deposits a fake.so file there.  If I run a
>> variant of that which I built after adding
>> 
>>   gcc_jit_context_set_bool_option (
>>                                    ctxt,
>>                                    GCC_JIT_BOOL_OPTION_DEBUGINFO,
>>                                    1);
>> 
>> then the temporary directory isn't created, or maybe it's deleted
>> when
>> the program exits.  I think the latter is the case, because the
>> directory is visible if I step through the program with a debugger,
>> but disappears when the program exits.
>> 
>> David, what's the story with these temporary directories?
>
> They're meant to be cleaned up automatically by libgccjit: on
> gcc_jit_result_release for a successful compilation, or at the end of
> gcc_jit_context_compile* for a failed compilation.

I suspect we are just missing the call to 'gcc_jit_result_release'.  I'm
having a look.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, andrewjmoreton@gmail.com,
>         46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 13:13:39 +0000
> 
> >>   gcc_jit_context_set_bool_option (
> >>                                    ctxt,
> >>                                    GCC_JIT_BOOL_OPTION_DEBUGINFO,
> >>                                    1);
> >> 
> >> then the temporary directory isn't created, or maybe it's deleted
> >> when
> >> the program exits.  I think the latter is the case, because the
> >> directory is visible if I step through the program with a debugger,
> >> but disappears when the program exits.
> >> 
> >> David, what's the story with these temporary directories?
> >
> > They're meant to be cleaned up automatically by libgccjit: on
> > gcc_jit_result_release for a successful compilation, or at the end of
> > gcc_jit_context_compile* for a failed compilation.
> 
> I suspect we are just missing the call to 'gcc_jit_result_release'.  I'm
> having a look.

The tutorial does call gcc_jit_result_release.

I think this only happens in the tutorial example, not in Emacs.
Sorry if that was unclear.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: David Malcolm <dmalcolm@redhat.com>
> Cc: akrl@sdf.org, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 09:03:33 -0400
> 
> > David, what's the story with these temporary directories?
> 
> They're meant to be cleaned up automatically by libgccjit: on
> gcc_jit_result_release for a successful compilation, or at the end of
> gcc_jit_context_compile* for a failed compilation.
> 
> The removal code is in gcc::jit::tempdir::~tempdir in gcc/jit/jit-
> tempdir.c, maybe there's a bug there? (perhaps only affecting Windows?)
> It calls unlink on everything it "knows" about, and them rmdir on the
> directory.  I see now that I'm not checking for errors on those unlink
> and rmdir calls.

maybe you are relying on the Posix semantics whereby you can remove
files that are still in use, and the OS will remove them when the last
user closes the file?  That's not going to work on Windows, you need
to close first and remove later.

When does the object go out of scope, and its destructor called?

> Is the code setting GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES to true,
> perhaps?

No.  the code is just tut01d-hello-world.c as it is given here:

  https://gcc.gnu.org/onlinedocs/jit/intro/tutorial01.html

The only variation (as described in my previous messages) is that I
added

  gcc_jit_context_set_bool_option (
				   ctxt,
				   GCC_JIT_BOOL_OPTION_DEBUGINFO,
				   1);

in one of the variants I compiled.  But the variant with the above
added actually behaves correctly and removes the temporary directory
upon exit.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

Andrea, if I want to recompile the Lisp files with comp-debug set to
1, is there a better way than removing the existing *.eln files, then
compiling them again with a command such as (for preloaded files):

  emacs -batch -l comp -eval "(setq comp-debug 1)" -f batch-byte-native-compile-for-bootstrap FILE

and for non-preloaded ones:

  emacs -batch -l comp -eval "(setq comp-debug 1)" -f batch-native-compile FILE

Is that the right procedure?

And another question: if the preloaded files are recompiled as above,
do I also need to re-dump Emacs, or will the existing emacs.pdmp still
be valid for running Emacs?

And finally, in what directory will the pseudo C code files be dumped,
when using comp-debug = 1? in the same directory as the corresponding
.eln files, or somewhere else?

Thanks.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Andrea, if I want to recompile the Lisp files with comp-debug set to
> 1, is there a better way than removing the existing *.eln files, then
> compiling them again with a command such as (for preloaded files):
>
>   emacs -batch -l comp -eval "(setq comp-debug 1)" -f batch-byte-native-compile-for-bootstrap FILE
>
> and for non-preloaded ones:
>
>   emacs -batch -l comp -eval "(setq comp-debug 1)" -f batch-native-compile FILE
>
> Is that the right procedure?

Removing the .eln should not be necessary, thinking about actually if
it's removed I guess emacs will not be able to start and recompile the
file as not able to resurrect from dump.

`batch-native-compile' or `batch-byte-native-compile-for-bootstrap'
should be equivalent here as the second is just a way to do only byte
compilation for non dumped files when we are not using NATIVE_FULL_AOT.

> And another question: if the preloaded files are recompiled as above,
> do I also need to re-dump Emacs, or will the existing emacs.pdmp still
> be valid for running Emacs?

Dump is needed if the .eln file changes position or content (compiled
functions) so generally speaking yes, redumping is a good idea.

> And finally, in what directory will the pseudo C code files be dumped,
> when using comp-debug = 1? in the same directory as the corresponding
> .eln files, or somewhere else?

Correct, same directory of the eln.

Thanks

   Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Tue, 30 Mar 2021 14:33:04 +0000
> 
> Removing the .eln should not be necessary, thinking about actually if
> it's removed I guess emacs will not be able to start and recompile the
> file as not able to resurrect from dump.
> 
> `batch-native-compile' or `batch-byte-native-compile-for-bootstrap'
> should be equivalent here as the second is just a way to do only byte
> compilation for non dumped files when we are not using NATIVE_FULL_AOT.

Then I guess I'm missing something: how does Emacs know whether a
given .eln file should be saved in native-lisp/ or in
~/.emacs.d/eln-cache/?

> > And finally, in what directory will the pseudo C code files be dumped,
> > when using comp-debug = 1? in the same directory as the corresponding
> > .eln files, or somewhere else?
> 
> Correct, same directory of the eln.

Thanks.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> Date: Tue, 30 Mar 2021 14:33:04 +0000
>> 
>> Removing the .eln should not be necessary, thinking about actually if
>> it's removed I guess emacs will not be able to start and recompile the
>> file as not able to resurrect from dump.
>> 
>> `batch-native-compile' or `batch-byte-native-compile-for-bootstrap'
>> should be equivalent here as the second is just a way to do only byte
>> compilation for non dumped files when we are not using NATIVE_FULL_AOT.
>
> Then I guess I'm missing something: how does Emacs know whether a
> given .eln file should be saved in native-lisp/ or in
> ~/.emacs.d/eln-cache/?

Ops apologies you are correct, `batch-byte-native-compile-for-bootstrap'
also select as destination folder the `native-lisp' directory in the
build tree.  It is correct to invoke
`batch-byte-native-compile-for-bootstrap' if we want the .eln to be
deposed there.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Tue, 30 Mar 2021 19:19:12 +0000
> 
> >> `batch-native-compile' or `batch-byte-native-compile-for-bootstrap'
> >> should be equivalent here as the second is just a way to do only byte
> >> compilation for non dumped files when we are not using NATIVE_FULL_AOT.
> >
> > Then I guess I'm missing something: how does Emacs know whether a
> > given .eln file should be saved in native-lisp/ or in
> > ~/.emacs.d/eln-cache/?
> 
> Ops apologies you are correct, `batch-byte-native-compile-for-bootstrap'
> also select as destination folder the `native-lisp' directory in the
> build tree.  It is correct to invoke
> `batch-byte-native-compile-for-bootstrap' if we want the .eln to be
> deposed there.

OK, thanks.

Another nit: the doc string of comp-debug says:

  0 no debugging output.
    This is the recommended value unless you are debugging the compiler itself.
  1 emit debug symbols and dump pseudo C code.
  2 dump gcc passes and libgccjit log file.
  3 dump libgccjit reproducers.

But comp.c does this:

  if (comp.debug)
      gcc_jit_context_set_bool_option (comp.ctxt,
				       GCC_JIT_BOOL_OPTION_DEBUGINFO,
				       1);
  if (comp.debug > 2)
    {
      logfile = emacs_fopen ("libgccjit.log", "w");
      gcc_jit_context_set_logfile (comp.ctxt,
				   logfile,
				   0, 0);
      gcc_jit_context_set_bool_option (comp.ctxt,
				       GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES,
				       1);
      gcc_jit_context_set_bool_option (comp.ctxt,
				       GCC_JIT_BOOL_OPTION_DUMP_EVERYTHING,
				       1);
    }
  [...]
  if (comp.debug)
      gcc_jit_context_dump_to_file (comp.ctxt,
				    format_string ("%s.c", SSDATA (ebase_name)),
				    1);

AFAIU, this means the libgccjit log file is only output with
comp-debug 3 and higher?  Also, does comp-debug = 3 indeed cause the
reproducer to be written, or is that controlled independently by
comp-libgccjit-reproducer?

And finally, what do you think about moving the
gcc_jit_context_dump_to_file call to comp-debug 2 or higher?  IOW,
make level 1 just add debug info to the *.eln files?  Especially if we
are going to make comp-debug = 1 the default (as I think we should),
wouldn't it be better than the current setup?  Or maybe we should
introduce an intermediate debug level between the current 0 and 1?
And if we make that change, do we also need to bump the ABI number?

Thanks.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> Date: Tue, 30 Mar 2021 19:19:12 +0000
>>
>> >> `batch-native-compile' or `batch-byte-native-compile-for-bootstrap'
>> >> should be equivalent here as the second is just a way to do only byte
>> >> compilation for non dumped files when we are not using NATIVE_FULL_AOT.
>> >
>> > Then I guess I'm missing something: how does Emacs know whether a
>> > given .eln file should be saved in native-lisp/ or in
>> > ~/.emacs.d/eln-cache/?
>>
>> Ops apologies you are correct, `batch-byte-native-compile-for-bootstrap'
>> also select as destination folder the `native-lisp' directory in the
>> build tree.  It is correct to invoke
>> `batch-byte-native-compile-for-bootstrap' if we want the .eln to be
>> deposed there.
>
> OK, thanks.
>
> Another nit: the doc string of comp-debug says:
>
>   0 no debugging output.
>     This is the recommended value unless you are debugging the compiler itself.
>   1 emit debug symbols and dump pseudo C code.
>   2 dump gcc passes and libgccjit log file.
>   3 dump libgccjit reproducers.
>
> But comp.c does this:
>
>   if (comp.debug)
>       gcc_jit_context_set_bool_option (comp.ctxt,
> 				       GCC_JIT_BOOL_OPTION_DEBUGINFO,
> 				       1);
>   if (comp.debug > 2)
>     {
>       logfile = emacs_fopen ("libgccjit.log", "w");
>       gcc_jit_context_set_logfile (comp.ctxt,
> 				   logfile,
> 				   0, 0);
>       gcc_jit_context_set_bool_option (comp.ctxt,
> 				       GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES,
> 				       1);
>       gcc_jit_context_set_bool_option (comp.ctxt,
> 				       GCC_JIT_BOOL_OPTION_DUMP_EVERYTHING,
> 				       1);
>     }
>   [...]
>   if (comp.debug)
>       gcc_jit_context_dump_to_file (comp.ctxt,
> 				    format_string ("%s.c", SSDATA (ebase_name)),
> 				    1);
>
> AFAIU, this means the libgccjit log file is only output with
> comp-debug 3 and higher?

Correct, the docstring wasn't correct :/ aa159bf696 should update it to
reflect the current situation.

> Also, does comp-debug = 3 indeed cause the
> reproducer to be written, or is that controlled independently by
> comp-libgccjit-reproducer?

The reproducer is controlled independently by
`comp-libgccjit-reproducer', the idea behind this is that we want to be
able to produce reproducers independently from the debug settings (we
might have a ligccjit bug that is related to one debug option).

>
> And finally, what do you think about moving the
> gcc_jit_context_dump_to_file call to comp-debug 2 or higher?  IOW,
> make level 1 just add debug info to the *.eln files?  Especially if we
> are going to make comp-debug = 1 the default (as I think we should),
> wouldn't it be better than the current setup?  Or maybe we should
> introduce an intermediate debug level between the current 0 and 1?

My proposal would be like:

0 none
1 debug symbols
2 debug symbols + pseudo C file
3 debug symbols + pseudo C file + GCC passes + libgccjit log file

I think the backtrace issue you are facing is clearly a gdb unwinder
limitation on Windows that should be fixed in gdb.  OTOH I understand we
can't update gdb for all users that might want to help debugging an
issue, but I don't like very much the idea to have comp-debug 1 as
default on every system.  What about having 1 as default only on
Windows?  WDYT?

> And if we make that change, do we also need to bump the ABI number?

I should not be necessary.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 08:47:37 +0000
> 
> > AFAIU, this means the libgccjit log file is only output with
> > comp-debug 3 and higher?
> 
> Correct, the docstring wasn't correct :/ aa159bf696 should update it to
> reflect the current situation.

Thanks.

> > Also, does comp-debug = 3 indeed cause the
> > reproducer to be written, or is that controlled independently by
> > comp-libgccjit-reproducer?
> 
> The reproducer is controlled independently by
> `comp-libgccjit-reproducer', the idea behind this is that we want to be
> able to produce reproducers independently from the debug settings (we
> might have a ligccjit bug that is related to one debug option).

Right, so comp-debug = 3 alone doesn't trigger the reproducer, right?

> My proposal would be like:
> 
> 0 none
> 1 debug symbols
> 2 debug symbols + pseudo C file
> 3 debug symbols + pseudo C file + GCC passes + libgccjit log file
> 
> I think the backtrace issue you are facing is clearly a gdb unwinder
> limitation on Windows that should be fixed in gdb.

It's more like a missing feature in GDB on platforms that don't use
ELF binary format.  I don't hold my breath to have the situation
improved, since this is so far a rare situation when debugging
programs: no one expects debugging code without debug symbols to be
easy.

> OTOH I understand we can't update gdb for all users that might want
> to help debugging an issue, but I don't like very much the idea to
> have comp-debug 1 as default on every system.  What about having 1
> as default only on Windows?  WDYT?

In general, I'd prefer these settings to be the same on all platforms,
although it isn't a strong feeling in this case.  But I'm curious why
you are so against doing that by default on other systems.  Can you
explain?

Anyway, I've compiled the relevant files with comp-debug = 1, and the
problems with backtraces are now completely gone.  So I think we can
more or less close this issue (modulo changing the meanings of
comp-debug and the decision regarding the default value).

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 08:47:37 +0000
>> 
>> > AFAIU, this means the libgccjit log file is only output with
>> > comp-debug 3 and higher?
>> 
>> Correct, the docstring wasn't correct :/ aa159bf696 should update it to
>> reflect the current situation.
>
> Thanks.
>
>> > Also, does comp-debug = 3 indeed cause the
>> > reproducer to be written, or is that controlled independently by
>> > comp-libgccjit-reproducer?
>> 
>> The reproducer is controlled independently by
>> `comp-libgccjit-reproducer', the idea behind this is that we want to be
>> able to produce reproducers independently from the debug settings (we
>> might have a ligccjit bug that is related to one debug option).
>
> Right, so comp-debug = 3 alone doesn't trigger the reproducer, right?

That's correct.

>> My proposal would be like:
>> 
>> 0 none
>> 1 debug symbols
>> 2 debug symbols + pseudo C file
>> 3 debug symbols + pseudo C file + GCC passes + libgccjit log file
>> 
>> I think the backtrace issue you are facing is clearly a gdb unwinder
>> limitation on Windows that should be fixed in gdb.
>
> It's more like a missing feature in GDB on platforms that don't use
> ELF binary format.  I don't hold my breath to have the situation
> improved, since this is so far a rare situation when debugging
> programs: no one expects debugging code without debug symbols to be
> easy.
>
>> OTOH I understand we can't update gdb for all users that might want
>> to help debugging an issue, but I don't like very much the idea to
>> have comp-debug 1 as default on every system.  What about having 1
>> as default only on Windows?  WDYT?
>
> In general, I'd prefer these settings to be the same on all platforms,
> although it isn't a strong feeling in this case.  But I'm curious why
> you are so against doing that by default on other systems.  Can you
> explain?

Nothing special, just that that as on GNU/Linux we can produce
backtraces without debug symbols so seams to me it would be a waste of
space to have them always present just for this reason.

> Anyway, I've compiled the relevant files with comp-debug = 1, and the
> problems with backtraces are now completely gone.  So I think we can
> more or less close this issue (modulo changing the meanings of
> comp-debug and the decision regarding the default value).

Sounds good, if my proposal of the new 4 values is acceptable for you
and we decide on the default one I can proceed with that.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 12:16:14 +0000
> 
> >> OTOH I understand we can't update gdb for all users that might want
> >> to help debugging an issue, but I don't like very much the idea to
> >> have comp-debug 1 as default on every system.  What about having 1
> >> as default only on Windows?  WDYT?
> >
> > In general, I'd prefer these settings to be the same on all platforms,
> > although it isn't a strong feeling in this case.  But I'm curious why
> > you are so against doing that by default on other systems.  Can you
> > explain?
> 
> Nothing special, just that that as on GNU/Linux we can produce
> backtraces without debug symbols so seams to me it would be a waste of
> space to have them always present just for this reason.

Well, using comp-debug = 1 produces minor improvements also on
GNU/Linux, as it shows the values of arguments of the
natively-compiled functions instead of just "()".  Example:

  #75 0x01260cad in funcall_subr (subr=0x1730200 <Sfuncall_interactively>,
      numargs=3, args=0x82f028) at eval.c:3064
  #76 0x01260805 in Ffuncall (nargs=4, args=0x82f020) at eval.c:3009
  #77 0x0125f238 in Fapply (nargs=3, args=0x82f1a8) at eval.c:2639
  #78 0x01250e54 in Fcall_interactively (function=XIL(0x44e24ec),
      record_flag=XIL(0), keys=XIL(0xa00000000678d168)) at callint.c:353
  #79 0x70f1ae41 in F636f6d6d616e642d65786563757465_command_execute_0 (
      par_0=72230124, par_1=0, par_2=0, par_3=0)  <<<<<<<<<<<<<<<<<<<<<<<<<
      at d:/gnu/git/emacs/native-comp/native-lisp/28.0.50-c99426db/simple-fab5b0cf-db78b289.c:23993

But I'm okay with leaving it at zero except on Windows for now, we can
always revisit this later.

> > Anyway, I've compiled the relevant files with comp-debug = 1, and the
> > problems with backtraces are now completely gone.  So I think we can
> > more or less close this issue (modulo changing the meanings of
> > comp-debug and the decision regarding the default value).
> 
> Sounds good, if my proposal of the new 4 values is acceptable for you
> and we decide on the default one I can proceed with that.

Yes, let's make that change.

Please also look into the issue with temporary *.el files I reported
earlier.

Thanks.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 12:16:14 +0000
>> 
>> >> OTOH I understand we can't update gdb for all users that might want
>> >> to help debugging an issue, but I don't like very much the idea to
>> >> have comp-debug 1 as default on every system.  What about having 1
>> >> as default only on Windows?  WDYT?
>> >
>> > In general, I'd prefer these settings to be the same on all platforms,
>> > although it isn't a strong feeling in this case.  But I'm curious why
>> > you are so against doing that by default on other systems.  Can you
>> > explain?
>> 
>> Nothing special, just that that as on GNU/Linux we can produce
>> backtraces without debug symbols so seams to me it would be a waste of
>> space to have them always present just for this reason.
>
> Well, using comp-debug = 1 produces minor improvements also on
> GNU/Linux, as it shows the values of arguments of the
> natively-compiled functions instead of just "()".  Example:
>
>   #75 0x01260cad in funcall_subr (subr=0x1730200 <Sfuncall_interactively>,
>       numargs=3, args=0x82f028) at eval.c:3064
>   #76 0x01260805 in Ffuncall (nargs=4, args=0x82f020) at eval.c:3009
>   #77 0x0125f238 in Fapply (nargs=3, args=0x82f1a8) at eval.c:2639
>   #78 0x01250e54 in Fcall_interactively (function=XIL(0x44e24ec),
>       record_flag=XIL(0), keys=XIL(0xa00000000678d168)) at callint.c:353
>   #79 0x70f1ae41 in F636f6d6d616e642d65786563757465_command_execute_0 (
>       par_0=72230124, par_1=0, par_2=0, par_3=0)  <<<<<<<<<<<<<<<<<<<<<<<<<
>       at d:/gnu/git/emacs/native-comp/native-lisp/28.0.50-c99426db/simple-fab5b0cf-db78b289.c:23993
>
> But I'm okay with leaving it at zero except on Windows for now, we can
> always revisit this later.
>
>> > Anyway, I've compiled the relevant files with comp-debug = 1, and the
>> > problems with backtraces are now completely gone.  So I think we can
>> > more or less close this issue (modulo changing the meanings of
>> > comp-debug and the decision regarding the default value).
>> 
>> Sounds good, if my proposal of the new 4 values is acceptable for you
>> and we decide on the default one I can proceed with that.
>
> Yes, let's make that change.

Okay 53ca0d9844 reworks the levels and change the default value on
Windows, I think `comp-debug' should do what we want now.

>
> Please also look into the issue with temporary *.el files I reported
> earlier.

Yep coming on that.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Tue, 30 Mar 2021 19:19:12 +0000
> 
> >> `batch-native-compile' or `batch-byte-native-compile-for-bootstrap'
> >> should be equivalent here as the second is just a way to do only byte
> >> compilation for non dumped files when we are not using NATIVE_FULL_AOT.
> >
> > Then I guess I'm missing something: how does Emacs know whether a
> > given .eln file should be saved in native-lisp/ or in
> > ~/.emacs.d/eln-cache/?
> 
> Ops apologies you are correct, `batch-byte-native-compile-for-bootstrap'
> also select as destination folder the `native-lisp' directory in the
> build tree.  It is correct to invoke
> `batch-byte-native-compile-for-bootstrap' if we want the .eln to be
> deposed there.

For some reason, compiling files with batch-native-compile does NOT
produce the message that I'm used to see with
batch-byte-native-compile-for-bootstrap:

  libgccjit-0.dll: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]

Why is that? is that normal, or does it indicate some subtle problem
with compiling by this method?  The exact command I used is, for
example,

   emacs -batch -l comp -eval "(setq comp-debug 1)" -f batch-native-compile ../lisp/emacs-lisp/seq.el

Am I doing something wrong?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> Date: Wed, 31 Mar 2021 13:01:36 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
> 
> For some reason, compiling files with batch-native-compile does NOT
> produce the message that I'm used to see with
> batch-byte-native-compile-for-bootstrap:
> 
>   libgccjit-0.dll: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]

Oh, I see: it's because batch-native-compile runs the final
compilation phase in a subprocess.  But doesn't that mean the
non-default setting of comp-debug will not have its effect in this
case?

Btw, I find the doc strings of the various top-level native-compile
functions not very helpful when I need to understand under what
circumstances they run the compilation asynchronously.  I always need
to read the code, and read it carefully, to figure that out.  Can this
please be improved?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Wed, 31 Mar 2021 13:01:36 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
>> 
>> For some reason, compiling files with batch-native-compile does NOT
>> produce the message that I'm used to see with
>> batch-byte-native-compile-for-bootstrap:
>> 
>>   libgccjit-0.dll: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]
>
> Oh, I see: it's because batch-native-compile runs the final
> compilation phase in a subprocess.  But doesn't that mean the
> non-default setting of comp-debug will not have its effect in this
> case?

`comp-debug' at that point is captured in the compilation context
`comp-ctxt', we have a slot for that in the structure.

> Btw, I find the doc strings of the various top-level native-compile
> functions not very helpful when I need to understand under what
> circumstances they run the compilation asynchronously.  I always need
> to read the code, and read it carefully, to figure that out.  Can this
> please be improved?

Sure, if you have a list of the one that you have found troublesome
(and/or some suggestion) I'll try to improve them.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 10:33:43 +0000
> 
> >>   libgccjit-0.dll: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]
> >
> > Oh, I see: it's because batch-native-compile runs the final
> > compilation phase in a subprocess.  But doesn't that mean the
> > non-default setting of comp-debug will not have its effect in this
> > case?
> 
> `comp-debug' at that point is captured in the compilation context
> `comp-ctxt', we have a slot for that in the structure.

OK, thanks.

> > Btw, I find the doc strings of the various top-level native-compile
> > functions not very helpful when I need to understand under what
> > circumstances they run the compilation asynchronously.  I always need
> > to read the code, and read it carefully, to figure that out.  Can this
> > please be improved?
> 
> Sure, if you have a list of the one that you have found troublesome
> (and/or some suggestion) I'll try to improve them.

I think these:

  comp--native-compile, native-compile, batch-native-compile,
  batch-byte-native-compile-for-bootstrap

The first of these is an internal function, but it's used a lot, and
its doc string describes most of what it does -- except the async
aspect.

One question I have, that perhaps will be answered by the enhanced doc
strings, is this: how to run a batch compilation of a non-preloaded
file such that no subprocesses at all are used?  There's
comp-async-jobs-number, but a zero value doesn't disable async
compilation, it means something else.

Another question is: the documentation sometimes mentions async
compilation and sometimes mentions deferred compilation -- but these
are not the same, right?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 10:33:43 +0000
>> 
>> >>   libgccjit-0.dll: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]
>> >
>> > Oh, I see: it's because batch-native-compile runs the final
>> > compilation phase in a subprocess.  But doesn't that mean the
>> > non-default setting of comp-debug will not have its effect in this
>> > case?
>> 
>> `comp-debug' at that point is captured in the compilation context
>> `comp-ctxt', we have a slot for that in the structure.
>
> OK, thanks.
>
>> > Btw, I find the doc strings of the various top-level native-compile
>> > functions not very helpful when I need to understand under what
>> > circumstances they run the compilation asynchronously.  I always need
>> > to read the code, and read it carefully, to figure that out.  Can this
>> > please be improved?
>> 
>> Sure, if you have a list of the one that you have found troublesome
>> (and/or some suggestion) I'll try to improve them.
>
> I think these:
>
>   comp--native-compile, native-compile, batch-native-compile,
>   batch-byte-native-compile-for-bootstrap
>
> The first of these is an internal function, but it's used a lot, and
> its doc string describes most of what it does -- except the async
> aspect.
>
> One question I have, that perhaps will be answered by the enhanced doc
> strings, is this: how to run a batch compilation of a non-preloaded
> file such that no subprocesses at all are used?

ATM there's no way.  The idea is that we typically don't want to run in
the same process as libgccjit leaks memory (and contribute to memory
fragmentation).  We do it only for
`batch-byte-native-compile-for-bootstrap' as we know that the Emacs
process will not last long.

> There's
> comp-async-jobs-number, but a zero value doesn't disable async
> compilation, it means something else.
>
> Another question is: the documentation sometimes mentions async
> compilation and sometimes mentions deferred compilation -- but these
> are not the same, right?

I'd say:

- async compilation is the mechanism that allow for compiling
  asynchronously.

- deferred is the mechanism that allow for loading bytecode, triggering
  an async compilation and eventually perform the swap of the function
  definition when the async compilation is done.


Regards

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 12:24:07 +0000
> 
> > One question I have, that perhaps will be answered by the enhanced doc
> > strings, is this: how to run a batch compilation of a non-preloaded
> > file such that no subprocesses at all are used?
> 
> ATM there's no way.  The idea is that we typically don't want to run in
> the same process as libgccjit leaks memory (and contribute to memory
> fragmentation).  We do it only for
> `batch-byte-native-compile-for-bootstrap' as we know that the Emacs
> process will not last long.

But batch-native-compile is also going to exit once the compilation
ends, won't it?  Or is the difference that
batch-byte-native-compile-for-bootstrap only ever compiles a single
file?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 12:24:07 +0000
>> 
>> > One question I have, that perhaps will be answered by the enhanced doc
>> > strings, is this: how to run a batch compilation of a non-preloaded
>> > file such that no subprocesses at all are used?
>> 
>> ATM there's no way.  The idea is that we typically don't want to run in
>> the same process as libgccjit leaks memory (and contribute to memory
>> fragmentation).  We do it only for
>> `batch-byte-native-compile-for-bootstrap' as we know that the Emacs
>> process will not last long.
>
> But batch-native-compile is also going to exit once the compilation
> ends, won't it?  Or is the difference that
> batch-byte-native-compile-for-bootstrap only ever compiles a single
> file?

Originally I created `batch-byte-native-compile-for-bootstrap' only to
integrate with the build system and support the NATIVE_FULL_AOT thing
while `batch-native-compile' is supposed to be more of a general purpose
one.  Your point is correct tho.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 10:33:43 +0000
> 
> > Oh, I see: it's because batch-native-compile runs the final
> > compilation phase in a subprocess.  But doesn't that mean the
> > non-default setting of comp-debug will not have its effect in this
> > case?
> 
> `comp-debug' at that point is captured in the compilation context
> `comp-ctxt', we have a slot for that in the structure.

Each FILE compiled via batch-native-compile leaves in my temp
directory a file whose name is
emacs-int-compile-FILE-XXXXX-YYYYY-NNNNN.el.  I see this file created
in comp-final, but where is it deleted?  Should we delete it once
call-process returns?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 10:33:43 +0000
>> 
>> > Oh, I see: it's because batch-native-compile runs the final
>> > compilation phase in a subprocess.  But doesn't that mean the
>> > non-default setting of comp-debug will not have its effect in this
>> > case?
>> 
>> `comp-debug' at that point is captured in the compilation context
>> `comp-ctxt', we have a slot for that in the structure.
>
> Each FILE compiled via batch-native-compile leaves in my temp
> directory a file whose name is
> emacs-int-compile-FILE-XXXXX-YYYYY-NNNNN.el.  I see this file created
> in comp-final, but where is it deleted?  Should we delete it once
> call-process returns?

Is the application responsible for cleaning up temporary files in '/tmp?
If yes then yes... I guess we should :)

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 12:25:52 +0000
> 
> > Each FILE compiled via batch-native-compile leaves in my temp
> > directory a file whose name is
> > emacs-int-compile-FILE-XXXXX-YYYYY-NNNNN.el.  I see this file created
> > in comp-final, but where is it deleted?  Should we delete it once
> > call-process returns?
> 
> Is the application responsible for cleaning up temporary files in '/tmp?
> If yes then yes... I guess we should :)

I think it's good practice, yes.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 12:25:52 +0000
>> 
>> > Each FILE compiled via batch-native-compile leaves in my temp
>> > directory a file whose name is
>> > emacs-int-compile-FILE-XXXXX-YYYYY-NNNNN.el.  I see this file created
>> > in comp-final, but where is it deleted?  Should we delete it once
>> > call-process returns?
>> 
>> Is the application responsible for cleaning up temporary files in '/tmp?
>> If yes then yes... I guess we should :)
>
> I think it's good practice, yes.

8e524f4591 fix this.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 18:52:55 +0000
> 
> >> Is the application responsible for cleaning up temporary files in '/tmp?
> >> If yes then yes... I guess we should :)
> >
> > I think it's good practice, yes.
> 
> 8e524f4591 fix this.

Thanks, confirmed.

Btw, I have now 3 different *.eln files for comp.el in the same
subdirectory of native-lisp:

  comp-7672a6ed-ad0cbb8b.eln
  comp-7672a6ed-58fb0518.eln
  comp-7672a6ed-9f0b1563.eln

Is this expected?  What does the second hash depend on?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 18:52:55 +0000
>> 
>> >> Is the application responsible for cleaning up temporary files in '/tmp?
>> >> If yes then yes... I guess we should :)
>> >
>> > I think it's good practice, yes.
>> 
>> 8e524f4591 fix this.
>
> Thanks, confirmed.
>
> Btw, I have now 3 different *.eln files for comp.el in the same
> subdirectory of native-lisp:
>
>   comp-7672a6ed-ad0cbb8b.eln
>   comp-7672a6ed-58fb0518.eln
>   comp-7672a6ed-9f0b1563.eln
>
> Is this expected?  What does the second hash depend on?

The second hash is the one based on the source file content.

I see why, every time we compile a new eln we call
`comp-clean-up-stale-eln' to remove the old .eln.  But we exclude from
the clean-up the eln system directory (the last in
`comp-eln-laod-path').

I don't remember if the reason is that when Emacs is installed this is
typically read only or there's some other reason.  Probably we should
remove this limitation and handle correctly the case where we are not
able of removing the file if necessary.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 19:22:54 +0000
> 
> >   comp-7672a6ed-ad0cbb8b.eln
> >   comp-7672a6ed-58fb0518.eln
> >   comp-7672a6ed-9f0b1563.eln
> >
> > Is this expected?  What does the second hash depend on?
> 
> The second hash is the one based on the source file content.
> 
> I see why, every time we compile a new eln we call
> `comp-clean-up-stale-eln' to remove the old .eln.  But we exclude from
> the clean-up the eln system directory (the last in
> `comp-eln-laod-path').

So with these 3 files in that directory, which one will be loaded by
Emacs, and how will Emacs know which to load?

> I don't remember if the reason is that when Emacs is installed this is
> typically read only or there's some other reason.  Probably we should
> remove this limitation and handle correctly the case where we are not
> able of removing the file if necessary.

Probably.  But didn't you try that recently and found that something
broke as result?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 19:22:54 +0000
>> 
>> >   comp-7672a6ed-ad0cbb8b.eln
>> >   comp-7672a6ed-58fb0518.eln
>> >   comp-7672a6ed-9f0b1563.eln
>> >
>> > Is this expected?  What does the second hash depend on?
>> 
>> The second hash is the one based on the source file content.
>> 
>> I see why, every time we compile a new eln we call
>> `comp-clean-up-stale-eln' to remove the old .eln.  But we exclude from
>> the clean-up the eln system directory (the last in
>> `comp-eln-laod-path').
>
> So with these 3 files in that directory, which one will be loaded by
> Emacs, and how will Emacs know which to load?

Emacs will scan and hash the source file and load the correct one if
present.

>> I don't remember if the reason is that when Emacs is installed this is
>> typically read only or there's some other reason.  Probably we should
>> remove this limitation and handle correctly the case where we are not
>> able of removing the file if necessary.
>
> Probably.  But didn't you try that recently and found that something
> broke as result?

Yeah, you see now how terrible is my memory :) please see my other mail
on this.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Wed, 31 Mar 2021 19:43:17 +0000
> 
> > So with these 3 files in that directory, which one will be loaded by
> > Emacs, and how will Emacs know which to load?
> 
> Emacs will scan and hash the source file and load the correct one if
> present.

And if the source file isn't available?

In any case, AFAIU I can safely delete the 2 older *.eln files because
they will never be used, right?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> Date: Wed, 31 Mar 2021 19:43:17 +0000
>> 
>> > So with these 3 files in that directory, which one will be loaded by
>> > Emacs, and how will Emacs know which to load?
>> 
>> Emacs will scan and hash the source file and load the correct one if
>> present.
>
> And if the source file isn't available?

No eln will be loaded.

> In any case, AFAIU I can safely delete the 2 older *.eln files because
> they will never be used, right?

Correct, unless you have another binary that is using one of these eln
as preloaded, indeed this should not be the case as comp.el is not
preloaded.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Thu, 01 Apr 2021 07:07:53 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Andrea Corallo <akrl@sdf.org>
> >> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> >> Date: Wed, 31 Mar 2021 19:43:17 +0000
> >> 
> >> > So with these 3 files in that directory, which one will be loaded by
> >> > Emacs, and how will Emacs know which to load?
> >> 
> >> Emacs will scan and hash the source file and load the correct one if
> >> present.
> >
> > And if the source file isn't available?
> 
> No eln will be loaded.

Ouch!  That's a general issue, then, not just when there are multiple
copies of *.eln for the same .el file, right?  IOW, if Emacs is
installed such that the *.el files are not available, it will not load
the *.eln files, only the *.elc files.  I think we should at least
produce a run-time warning about that.

If the .el file is available, but was compressed, will Emacs
scan it after uncompressing to verify the signature?  This is the
usual way Emacs is installed: we compress all the *.el files.

> > In any case, AFAIU I can safely delete the 2 older *.eln files because
> > they will never be used, right?
> 
> Correct, unless you have another binary that is using one of these eln
> as preloaded, indeed this should not be the case as comp.el is not
> preloaded.

But even if the file is preloaded, the fact that it is in the same
hashed subdirectory of native-lisp/ means it can be used with any
binary whose ABI is compatible.  Right?  Or are you saying that the
source-content hash of the .eln file is recorded in the .pdmp file?

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> Date: Thu, 01 Apr 2021 07:07:53 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Andrea Corallo <akrl@sdf.org>
>> >> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> >> Date: Wed, 31 Mar 2021 19:43:17 +0000
>> >> 
>> >> > So with these 3 files in that directory, which one will be loaded by
>> >> > Emacs, and how will Emacs know which to load?
>> >> 
>> >> Emacs will scan and hash the source file and load the correct one if
>> >> present.
>> >
>> > And if the source file isn't available?
>> 
>> No eln will be loaded.
>
> Ouch!  That's a general issue, then, not just when there are multiple
> copies of *.eln for the same .el file, right?  IOW, if Emacs is
> installed such that the *.el files are not available, it will not load
> the *.eln files, only the *.elc files.  I think we should at least
> produce a run-time warning about that.

Correct, okay the warning should not be a problem (taking note into my
todo).

Do we really have installations with no source code?

> If the .el file is available, but was compressed, will Emacs
> scan it after uncompressing to verify the signature?  This is the
> usual way Emacs is installed: we compress all the *.el files.

Yes it does that :) (I tipically use Emacs installed).

>> > In any case, AFAIU I can safely delete the 2 older *.eln files because
>> > they will never be used, right?
>> 
>> Correct, unless you have another binary that is using one of these eln
>> as preloaded, indeed this should not be the case as comp.el is not
>> preloaded.
>
> But even if the file is preloaded, the fact that it is in the same
> hashed subdirectory of native-lisp/ means it can be used with any
> binary whose ABI is compatible.  Right?  Or are you saying that the
> source-content hash of the .eln file is recorded in the .pdmp file?

When the file was preloaded when reviving from dump it is looked-up
using its filename, as you suggests includes the `comp-abi-hash' in it.

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Thu, 01 Apr 2021 08:18:08 +0000
> 
> > Ouch!  That's a general issue, then, not just when there are multiple
> > copies of *.eln for the same .el file, right?  IOW, if Emacs is
> > installed such that the *.el files are not available, it will not load
> > the *.eln files, only the *.elc files.  I think we should at least
> > produce a run-time warning about that.
> 
> Correct, okay the warning should not be a problem (taking note into my
> todo).
> 
> Do we really have installations with no source code?

Good question.  I don't know the answer, but I guess we will learn,
when the warning is implemented.

> > But even if the file is preloaded, the fact that it is in the same
> > hashed subdirectory of native-lisp/ means it can be used with any
> > binary whose ABI is compatible.  Right?  Or are you saying that the
> > source-content hash of the .eln file is recorded in the .pdmp file?
> 
> When the file was preloaded when reviving from dump it is looked-up
> using its filename, as you suggests includes the `comp-abi-hash' in it.

Right, noted.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Michael Welsh Duggan]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>>> Date: Thu, 01 Apr 2021 07:07:53 +0000
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> >> From: Andrea Corallo <akrl@sdf.org>
>>> >> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>>> >> Date: Wed, 31 Mar 2021 19:43:17 +0000
>>> >> 
>>> >> > So with these 3 files in that directory, which one will be loaded by
>>> >> > Emacs, and how will Emacs know which to load?
>>> >> 
>>> >> Emacs will scan and hash the source file and load the correct one if
>>> >> present.
>>> >
>>> > And if the source file isn't available?
>>> 
>>> No eln will be loaded.
>>
>> Ouch!  That's a general issue, then, not just when there are multiple
>> copies of *.eln for the same .el file, right?  IOW, if Emacs is
>> installed such that the *.el files are not available, it will not load
>> the *.eln files, only the *.elc files.  I think we should at least
>> produce a run-time warning about that.
>
> Correct, okay the warning should not be a problem (taking note into my
> todo).
>
> Do we really have installations with no source code?

At least in the Debian distribution, the package containing the .el
files are not a direct dependency but a recommended dependency.

-- 
Michael Welsh Duggan
(md5i@md5i.com)

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>>> Date: Thu, 01 Apr 2021 07:07:53 +0000
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> >> From: Andrea Corallo <akrl@sdf.org>
>>> >> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>>> >> Date: Wed, 31 Mar 2021 19:43:17 +0000
>>> >> 
>>> >> > So with these 3 files in that directory, which one will be loaded by
>>> >> > Emacs, and how will Emacs know which to load?
>>> >> 
>>> >> Emacs will scan and hash the source file and load the correct one if
>>> >> present.
>>> >
>>> > And if the source file isn't available?
>>> 
>>> No eln will be loaded.
>>
>> Ouch!  That's a general issue, then, not just when there are multiple
>> copies of *.eln for the same .el file, right?  IOW, if Emacs is
>> installed such that the *.el files are not available, it will not load
>> the *.eln files, only the *.elc files.  I think we should at least
>> produce a run-time warning about that.
>
> Correct, okay the warning should not be a problem (taking note into my
> todo).

Okay dc393517ca adds the warning emittion.  I've also added a customize
`comp-warning-on-missing-source' to silence that in case.

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo <akrl@sdf.org> writes:

> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs@gnu.org> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: Andrea Corallo <akrl@sdf.org>
>>>> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>>>> Date: Thu, 01 Apr 2021 07:07:53 +0000
>>>> 
>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>> 
>>>> >> From: Andrea Corallo <akrl@sdf.org>
>>>> >> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>>>> >> Date: Wed, 31 Mar 2021 19:43:17 +0000
>>>> >> 
>>>> >> > So with these 3 files in that directory, which one will be loaded by
>>>> >> > Emacs, and how will Emacs know which to load?
>>>> >> 
>>>> >> Emacs will scan and hash the source file and load the correct one if
>>>> >> present.
>>>> >
>>>> > And if the source file isn't available?
>>>> 
>>>> No eln will be loaded.
>>>
>>> Ouch!  That's a general issue, then, not just when there are multiple
>>> copies of *.eln for the same .el file, right?  IOW, if Emacs is
>>> installed such that the *.el files are not available, it will not load
>>> the *.eln files, only the *.elc files.  I think we should at least
>>> produce a run-time warning about that.
>>
>> Correct, okay the warning should not be a problem (taking note into my
>> todo).
>
> Okay dc393517ca adds the warning emittion.  I've also added a customize
> `comp-warning-on-missing-source' to silence that in case.
>
>   Andrea

Not an expert of the bug tracker here but I think this is still open.
In case should we close it?

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
> Date: Sat, 10 Apr 2021 16:49:15 +0000
> 
> > Okay dc393517ca adds the warning emittion.  I've also added a customize
> > `comp-warning-on-missing-source' to silence that in case.
> >
> >   Andrea
> 
> Not an expert of the bug tracker here but I think this is still open.
> In case should we close it?

Right, closing.

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: andrewjmoreton@gmail.com, dmalcolm@redhat.com, 46495@debbugs.gnu.org
>>> Date: Wed, 31 Mar 2021 18:52:55 +0000
>>> 
>>> >> Is the application responsible for cleaning up temporary files in '/tmp?
>>> >> If yes then yes... I guess we should :)
>>> >
>>> > I think it's good practice, yes.
>>> 
>>> 8e524f4591 fix this.
>>
>> Thanks, confirmed.
>>
>> Btw, I have now 3 different *.eln files for comp.el in the same
>> subdirectory of native-lisp:
>>
>>   comp-7672a6ed-ad0cbb8b.eln
>>   comp-7672a6ed-58fb0518.eln
>>   comp-7672a6ed-9f0b1563.eln
>>
>> Is this expected?  What does the second hash depend on?
>
> The second hash is the one based on the source file content.
>
> I see why, every time we compile a new eln we call
> `comp-clean-up-stale-eln' to remove the old .eln.  But we exclude from
> the clean-up the eln system directory (the last in
> `comp-eln-laod-path').
>
> I don't remember if the reason is that when Emacs is installed this is
> typically read only or there's some other reason.  Probably we should
> remove this limitation and handle correctly the case where we are not
> able of removing the file if necessary.

Okay apparently we did it already in the past (be22cda7be) but I
reverted the commit with d0280ce1b1.  The commit message for this says:

"Older binaries might still need those .eln if they where preloaded."

I guess the issue is clear now and we should be able to better tackle it
once we depose the preloaded .eln in the preloaded subfolder (I think
I'll be on that this weekend).

Thanks

  Andrea

bug#46495: 28.0.50; [native-comp] Build fails for 32bit --with-wide-int

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: dmalcolm@redhat.com, andrewjmoreton@gmail.com, 46495@debbugs.gnu.org
>> Date: Tue, 30 Mar 2021 19:19:12 +0000
>> 
>> >> `batch-native-compile' or `batch-byte-native-compile-for-bootstrap'
>> >> should be equivalent here as the second is just a way to do only byte
>> >> compilation for non dumped files when we are not using NATIVE_FULL_AOT.
>> >
>> > Then I guess I'm missing something: how does Emacs know whether a
>> > given .eln file should be saved in native-lisp/ or in
>> > ~/.emacs.d/eln-cache/?
>> 
>> Ops apologies you are correct, `batch-byte-native-compile-for-bootstrap'
>> also select as destination folder the `native-lisp' directory in the
>> build tree.  It is correct to invoke
>> `batch-byte-native-compile-for-bootstrap' if we want the .eln to be
>> deposed there.
>
> For some reason, compiling files with batch-native-compile does NOT
> produce the message that I'm used to see with
> batch-byte-native-compile-for-bootstrap:
>
>   libgccjit-0.dll: note: disable pass tree-isolate-paths for functions in the range of [0, 4294967295]

I think is probably because `batch-native-compile' run the compilation
as a subprocess so even if GCC is printing to stderr (or stdout) you
don't see it in your terminal.

  Andrea