bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[StrawberryTea]

Hello Emacs developers,

I am trying to native compile Emacs with the -flto flag patched into the
native-comp-compiler-options variable. However, when I do this, I get
the following error:

/usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/bin/ld: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccW8pLE7.o: plugin needed to handle lto object
/usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/bin/ld: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccW8pLE7.o: plugin needed to handle lto object
lto1: internal compiler error: original not compressed with zstd
0x11b434d internal_error(char const*, ...)
    ???:0
0x128df56 cgraph_node::get_untransformed_body()
    ???:0
0x128d4c3 cgraph_node::expand()
    ???:0
0x19c264f symbol_table::compile()
    ???:0
0x19b6f50 lto_main()
    ???:0
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <https://bugs.gentoo.org/> for instructions.
make[4]: *** [/var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccZwblRQ.mk:2: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccq4BEJb.ltrans0.ltrans.o] Error 1
lto-wrapper: fatal error: make returned 2 exit status
compilation terminated.
collect2: fatal error: lto-wrapper returned 1 exit status
compilation terminated.
libgccjit.so: error: error invoking gcc driver

The full build log is here: https://0x0.st/HhAc.txt
Build environment: https://0x0.st/HhAT.txt
emerge --info output (I use Gentoo): https://0x0.st/HhAm.txt

Sincerely,
StrawberryTea

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Eli Zaretskii]

> From: StrawberryTea <look@strawberrytea.xyz>
> Date: Sat, 09 Mar 2024 21:19:57 -0600
> 
> Hello Emacs developers,
> 
> I am trying to native compile Emacs with the -flto flag patched into the
> native-comp-compiler-options variable. However, when I do this, I get
> the following error:
> 
> /usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/bin/ld: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccW8pLE7.o: plugin needed to handle lto object
> /usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/bin/ld: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccW8pLE7.o: plugin needed to handle lto object
> lto1: internal compiler error: original not compressed with zstd
> 0x11b434d internal_error(char const*, ...)
>     ???:0
> 0x128df56 cgraph_node::get_untransformed_body()
>     ???:0
> 0x128d4c3 cgraph_node::expand()
>     ???:0
> 0x19c264f symbol_table::compile()
>     ???:0
> 0x19b6f50 lto_main()
>     ???:0
> Please submit a full bug report, with preprocessed source (by using -freport-bug).
> Please include the complete backtrace with any bug report.
> See <https://bugs.gentoo.org/> for instructions.
> make[4]: *** [/var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccZwblRQ.mk:2: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccq4BEJb.ltrans0.ltrans.o] Error 1
> lto-wrapper: fatal error: make returned 2 exit status
> compilation terminated.
> collect2: fatal error: lto-wrapper returned 1 exit status
> compilation terminated.
> libgccjit.so: error: error invoking gcc driver

Why do you think this is an Emacs problem?  The error messages seem to
imply the problem is with your installation of libgccjit or
GCC/Binutils in general.  The request to submit a bug report cites a
Gentoo bug address, so I think you should report it there first.

Paul, does Emacs even support -flto builds?

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[StrawberryTea]

Hello Eli,

Well, I wanted to ask here because I wanted to check if anyone else is
able to compile elisp files with -flto in native-comp-compiler-options.
I believe that the error is related to the native-comp branch, as a
Gentoo developer from the libgccjit mailing list has already confirmed
that libgccjit is compiled with lto support. The matter at hand is not
about Emacs's inherent support for -flto, but rather if the native-comp
branch can compile Elisp using the -flto option.

Sincerely,
StrawberryTea

Eli Zaretskii <eliz@gnu.org> writes:

>> From: StrawberryTea <look@strawberrytea.xyz>
>> Date: Sat, 09 Mar 2024 21:19:57 -0600
>>
>> Hello Emacs developers,
>>
>> I am trying to native compile Emacs with the -flto flag patched into the
>> native-comp-compiler-options variable. However, when I do this, I get
>> the following error:
>>
>> /usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/bin/ld: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccW8pLE7.o: plugin needed to handle lto object
>> /usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/bin/ld: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccW8pLE7.o: plugin needed to handle lto object
>> lto1: internal compiler error: original not compressed with zstd
>> 0x11b434d internal_error(char const*, ...)
>>     ???:0
>> 0x128df56 cgraph_node::get_untransformed_body()
>>     ???:0
>> 0x128d4c3 cgraph_node::expand()
>>     ???:0
>> 0x19c264f symbol_table::compile()
>>     ???:0
>> 0x19b6f50 lto_main()
>>     ???:0
>> Please submit a full bug report, with preprocessed source (by using -freport-bug).
>> Please include the complete backtrace with any bug report.
>> See <https://bugs.gentoo.org/> for instructions.
>> make[4]: *** [/var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccZwblRQ.mk:2: /var/tmp/portage/app-editors/emacs-30.0.9999/temp/ccq4BEJb.ltrans0.ltrans.o] Error 1
>> lto-wrapper: fatal error: make returned 2 exit status
>> compilation terminated.
>> collect2: fatal error: lto-wrapper returned 1 exit status
>> compilation terminated.
>> libgccjit.so: error: error invoking gcc driver
>
> Why do you think this is an Emacs problem?  The error messages seem to
> imply the problem is with your installation of libgccjit or
> GCC/Binutils in general.  The request to submit a bug report cites a
> Gentoo bug address, so I think you should report it there first.
>
> Paul, does Emacs even support -flto builds?

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Eli Zaretskii]

> From: StrawberryTea <look@strawberrytea.xyz>
> Cc: Paul Eggert <eggert@cs.ucla.edu>,  69689@debbugs.gnu.org
> Date: Sun, 10 Mar 2024 12:32:25 -0500
> 
> Well, I wanted to ask here because I wanted to check if anyone else is
> able to compile elisp files with -flto in native-comp-compiler-options.
> I believe that the error is related to the native-comp branch, as a
> Gentoo developer from the libgccjit mailing list has already confirmed
> that libgccjit is compiled with lto support. The matter at hand is not
> about Emacs's inherent support for -flto, but rather if the native-comp
> branch can compile Elisp using the -flto option.

I don't see how this is a question for us, given the error messages
that you cite in your OP.  It clearly says that something is missing
in your GCC/Binutils installation, and therefore libgccjit fails to
run the GCC driver.  How can that be a problem with Emacs?

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Paul Eggert]

On 2024-03-10 10:32, StrawberryTea wrote:
> Well, I wanted to ask here because I wanted to check if anyone else is
> able to compile elisp files with -flto in native-comp-compiler-options.
> I believe that the error is related to the native-comp branch, as a
> Gentoo developer

Google searches suggest that "plugin needed to handle lto object" is an 
issue that arises in Gentoo builds of other projects. The basic issue is 
that if you're going to build everything from scratch, you must build 
the infrastructure correctly first, and that can include linker plugins. 
Sorry I can't help more, as I don't use Gentoo.


On 2024-03-09 22:24, Eli Zaretskii wrote:
 > Paul, does Emacs even support -flto builds?

Yes, with './configure --enable-link-time-optimization'. This should 
work fine, although the build time is greater and there are a few 
false-alarm warnings from GCC - at least they were false alarms the last 
time I checked, many moons ago. (I normally don't use 
--enable-link-time-optimization as it makes --enable-gcc-warnings way 
too slow.)

I don't think -flto per se is the issue here, though it's hard to be 
sure since we weren't given a precise recipe for reproducing the bug 
(and I don't use Gentoo so it'd be hard for me to reproduce it anyway).

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Eli Zaretskii]

> Date: Sun, 10 Mar 2024 14:08:48 -0700
> Cc: 69689@debbugs.gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> 
> On 2024-03-10 10:32, StrawberryTea wrote:
> > Well, I wanted to ask here because I wanted to check if anyone else is
> > able to compile elisp files with -flto in native-comp-compiler-options.
> > I believe that the error is related to the native-comp branch, as a
> > Gentoo developer
> 
> Google searches suggest that "plugin needed to handle lto object" is an 
> issue that arises in Gentoo builds of other projects. The basic issue is 
> that if you're going to build everything from scratch, you must build 
> the infrastructure correctly first, and that can include linker plugins. 
> Sorry I can't help more, as I don't use Gentoo.

So I think this report should be sent to Gentoo, not to us.

> On 2024-03-09 22:24, Eli Zaretskii wrote:
>  > Paul, does Emacs even support -flto builds?
> 
> Yes, with './configure --enable-link-time-optimization'.

configure.ac says "experimental", so I guess it's not yet reliable
enough?

It also uses the getconf command (to figure out the number of CPU
cores?), which AFAIK is for glibc-only systems, right?

In any case, the OP didn't say whether he used this configure-time
option.

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Paul Eggert]

On 3/11/24 06:38, Eli Zaretskii wrote:

> configure.ac says "experimental", so I guess it's not yet reliable
> enough?

It was added by Dmitry Antipov in 2012. I never use it, and didn't even 
recall it until I looked at configure.ac recently.

I've not have good luck with -lto elsewhere. It can make linking slow 
and it tends to expose compiler bugs. Unless someone's using it 
regularly I would consider it experimental.


> It also uses the getconf command (to figure out the number of CPU
> cores?), which AFAIK is for glibc-only systems, right?

Yes, but if getconf is missing 'configure' is supposed to do something 
reasonable anyway.

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

StrawberryTea <look@strawberrytea.xyz> writes:

> Hello Emacs developers,
>
> I am trying to native compile Emacs with the -flto flag patched into the
> native-comp-compiler-options variable. However, when I do this, I get
> the following error:

Is the gcc lto plugin really installed?

I set the some option without having any of these errors: 
(setopt native-comp-compiler-options '("-flto"))

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Andrea Corallo]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 3/11/24 06:38, Eli Zaretskii wrote:
>
>> configure.ac says "experimental", so I guess it's not yet reliable
>> enough?
>
> It was added by Dmitry Antipov in 2012. I never use it, and didn't
> even recall it until I looked at configure.ac recently.
>
> I've not have good luck with -lto elsewhere. It can make linking slow
> and it tends to expose compiler bugs. Unless someone's using it
> regularly I would consider it experimental.

Also as a note.

I doubt using native compilation with LTO could bring measurable
advantages.

All the Emacs C primitives we call from elns have a function pointer
indirection as they can be redefined, so GCC can't inline them.  Few
support functions are called directly but I would be surprised of any
measurable effect there.

Best Regards

  Andrea

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[StrawberryTea]

Andrea Corallo <acorallo@gnu.org> writes:

> Paul Eggert <eggert@cs.ucla.edu> writes:
>
>> On 3/11/24 06:38, Eli Zaretskii wrote:
>>
>>> configure.ac says "experimental", so I guess it's not yet reliable
>>> enough?
>>
>> It was added by Dmitry Antipov in 2012. I never use it, and didn't
>> even recall it until I looked at configure.ac recently.
>>
>> I've not have good luck with -lto elsewhere. It can make linking slow
>> and it tends to expose compiler bugs. Unless someone's using it
>> regularly I would consider it experimental.
>
> Also as a note.
>
> I doubt using native compilation with LTO could bring measurable
> advantages.
>
> All the Emacs C primitives we call from elns have a function pointer
> indirection as they can be redefined, so GCC can't inline them.  Few
> support functions are called directly but I would be surprised of any
> measurable effect there.
>
> Best Regards
>
>   Andrea

I see. I am going to report this to the GCC bug tracker to try and get
it fixed. I would like to use LTO with native compilation, even if it
would not bring any measurable advantages. And I feel there has to be
some GCC person that knows how to fix this.

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

StrawberryTea <look@strawberrytea.xyz> writes:

>>
>>   Andrea
>
> I see. I am going to report this to the GCC bug tracker to try and get
> it fixed. I would like to use LTO with native compilation, even if it
> would not bring any measurable advantages. And I feel there has to be
> some GCC person that knows how to fix this.

Did you read my mail? I am able to compile elisp files with -flto in
native-comp-compiler-options.

The errors must be somewhere on your side.

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[StrawberryTea]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

> StrawberryTea <look@strawberrytea.xyz> writes:
>
>>>
>>>   Andrea
>>
>> I see. I am going to report this to the GCC bug tracker to try and get
>> it fixed. I would like to use LTO with native compilation, even if it
>> would not bring any measurable advantages. And I feel there has to be
>> some GCC person that knows how to fix this.
>
> Did you read my mail? I am able to compile elisp files with -flto in
> native-comp-compiler-options.
>
> The errors must be somewhere on your side.
Oh, I see. I did not read your mail. I am sorry for that. Thank you for
the information. I will add this to the Gentoo bug report.

bug#69689: 30.0.50; cannot native compile with -flto in native-comp-compiler-options

[Andrea Corallo]

StrawberryTea <look@strawberrytea.xyz> writes:

> Björn Bidar <bjorn.bidar@thaodan.de> writes:
>
>> StrawberryTea <look@strawberrytea.xyz> writes:
>>
>>>>
>>>>   Andrea
>>>
>>> I see. I am going to report this to the GCC bug tracker to try and get
>>> it fixed. I would like to use LTO with native compilation, even if it
>>> would not bring any measurable advantages. And I feel there has to be
>>> some GCC person that knows how to fix this.
>>
>> Did you read my mail? I am able to compile elisp files with -flto in
>> native-comp-compiler-options.
>>
>> The errors must be somewhere on your side.
> Oh, I see. I did not read your mail. I am sorry for that. Thank you for
> the information. I will add this to the Gentoo bug report.

Hello all,

I'm closing this as I think this is not an Emacs bug.  Happy to re-open
if we discover it is :)

Thanks

  Andrea