bug#45103: 28.0.50; feature/native-comp:Trying to load incoherent dumped eln file

bug#45103: 28.0.50; feature/native-comp:Trying to load incoherent dumped eln file

[Y Qu]

[-- Attachment #1: Type: text/plain, Size: 2330 bytes --]

Dear maintainers:

I try to use pdumper together with native-comp but fail.

To reproduce:

    $ emacs --batch -Q -eval '(dump-emacs-portable "~/.emacs.d/.cache/dumps/emacs.pdmp")'
    $ emacs --dump-file=~/.emacs.d/.cache/dumps/emacs.pdmp

You can get the following message:

    emacs: Trying to load incoherent dumped eln file /usr/sbin/../lib/emacs/28.0.50/native-lisp/28.0.50-x86_64-pc-linux-gnu-8fc0b1a455d90f52078950afaab3b40e/lisp-mode-410874a46ab8852855281f021ca61fe7-0babcd5810b0347a56a4b722059f6199.eln

This issue appears on build from both chaotic-aur/emacs-pgtk-native-comp and archlinuxcn/emacs-native-comp.

Thank you for your time!

Ying Qu

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo version 1.17.4)
 of 2020-12-05 built on pkg816084a8e53
Repository revision: fc535f14c9621bcb43b7e140b9f47d53b017f27b
Repository branch: makepkg
Windowing system distributor 'System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
 --with-sound=alsa --with-modules --without-gconf --without-gsettings
 --with-nativecomp --with-pgtk --with-x-toolkit=gtk3 --without-xaw3d
 --without-m17n-flt --with-cairo --with-xwidgets
 --without-compress-install 'CFLAGS=-march=x86-64 -mtune=generic -O2
 -pipe -fstack-protector-strong -fno-plt -g -fuse-ld=gold -g
 -fuse-ld=gold' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GLIB NOTIFY INOTIFY ACL
GNUTLS LIBXML2 FREETYPE HARFBUZZ LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3
PGTK XIM MODULES NATIVE_COMP THREADS XWIDGETS LIBSYSTEMD JSON PDUMPER
LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

[-- Attachment #2: Type: text/html, Size: 8561 bytes --]

bug#45103: 28.0.50; feature/native-comp:Trying to load incoherent dumped eln file

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

retitle [feature/native-comp] support re-dumping support
thanks

Y Qu <yqu212@gmail.com> writes:

> Dear maintainers:
>
> I try to use pdumper together with native-comp but fail.
>
> To reproduce:
>
>     $ emacs --batch -Q -eval '(dump-emacs-portable "~/.emacs.d/.cache/dumps/emacs.pdmp")'
>     $ emacs --dump-file=~/.emacs.d/.cache/dumps/emacs.pdmp
>
> You can get the following message:
>
>     emacs: Trying to load incoherent dumped eln file
> /usr/sbin/../lib/emacs/28.0.50/native-lisp/28.0.50-x86_64-pc-linux-gnu-8fc0b1a455d90f52078950afaab3b40e/lisp-mode-410874a46ab8852855281f021ca61fe7-0babcd5810b0347a56a4b722059f6199.eln
>
>
> This issue appears on build from both chaotic-aur/emacs-pgtk-native-comp and archlinuxcn/emacs-native-comp.
>
> Thank you for your time!
>
> Ying Qu

Hi Ying Qu,

I guess you are the first trying to re-dump a native compiled Emacs
session.  Despite being in principle possible there's certanly something
to look at.

In general my understanding is that ATM we do not support officially
Emacs re-dumps, so also on a vanilla build it may work or not.

As a consequence I'm not sure this is a requirement for this feature
ATM.  Perhaps we'll get some other opinion on this.

Thanks

  Andrea

bug#45103: 28.0.50; feature/native-comp:Trying to load incoherent dumped eln file

[Y Qu]

[-- Attachment #1: Type: text/plain, Size: 1960 bytes --]

Hi, Andrea:

Thanks for your quick response!

There is indeed a demand for this feature. You can find a discussion in Chinese  on https://emacs-china.org/t/pdump-native-comp/15234.

I'd love to see this feature officially supported. Pdumper has been a feature of Emacs 27. After you merge native-comp branch into master branch in the future, I believe there will definitely be more and more people to try this combination.

Ying

________________________________
发件人: Andrea Corallo <akrl@sdf.org>
发送时间: 2020年12月8日星期二 00:05
收件人: Y Qu
抄送: 45103@debbugs.gnu.org
主题: Re: bug#45103: 28.0.50; feature/native-comp:Trying to load incoherent dumped eln file

retitle [feature/native-comp] support re-dumping support
thanks

Y Qu <yqu212@gmail.com> writes:

> Dear maintainers:
>
> I try to use pdumper together with native-comp but fail.
>
> To reproduce:
>
>     $ emacs --batch -Q -eval '(dump-emacs-portable "~/.emacs.d/.cache/dumps/emacs.pdmp")'
>     $ emacs --dump-file=~/.emacs.d/.cache/dumps/emacs.pdmp
>
> You can get the following message:
>
>     emacs: Trying to load incoherent dumped eln file
> /usr/sbin/../lib/emacs/28.0.50/native-lisp/28.0.50-x86_64-pc-linux-gnu-8fc0b1a455d90f52078950afaab3b40e/lisp-mode-410874a46ab8852855281f021ca61fe7-0babcd5810b0347a56a4b722059f6199.eln
>
>
> This issue appears on build from both chaotic-aur/emacs-pgtk-native-comp and archlinuxcn/emacs-native-comp.
>
> Thank you for your time!
>
> Ying Qu

Hi Ying Qu,

I guess you are the first trying to re-dump a native compiled Emacs
session.  Despite being in principle possible there's certanly something
to look at.

In general my understanding is that ATM we do not support officially
Emacs re-dumps, so also on a vanilla build it may work or not.

As a consequence I'm not sure this is a requirement for this feature
ATM.  Perhaps we'll get some other opinion on this.

Thanks

  Andrea


[-- Attachment #2: Type: text/html, Size: 4221 bytes --]

bug#45103: [feature/native-comp] support re-dumping support

[Y Qu]

[-- Attachment #1: Type: text/plain, Size: 2251 bytes --]

This response is just to retitle.

________________________________
发件人: Y Qu <yqu212@gmail.com>
发送时间: 2020年12月8日星期二 06:43
收件人: Andrea Corallo
抄送: 45103@debbugs.gnu.org
主题: Re: bug#45103: 28.0.50; feature/native-comp:Trying to load incoherent dumped eln file

Hi, Andrea:

Thanks for your quick response!

There is indeed a demand for this feature. You can find a discussion in Chinese  on https://emacs-china.org/t/pdump-native-comp/15234.

I'd love to see this feature officially supported. Pdumper has been a feature of Emacs 27. After you merge native-comp branch into master branch in the future, I believe there will definitely be more and more people to try this combination.

Ying

________________________________
发件人: Andrea Corallo <akrl@sdf.org>
发送时间: 2020年12月8日星期二 00:05
收件人: Y Qu
抄送: 45103@debbugs.gnu.org
主题: Re: bug#45103: 28.0.50; feature/native-comp:Trying to load incoherent dumped eln file

retitle [feature/native-comp] support re-dumping support
thanks

Y Qu <yqu212@gmail.com> writes:

> Dear maintainers:
>
> I try to use pdumper together with native-comp but fail.
>
> To reproduce:
>
>     $ emacs --batch -Q -eval '(dump-emacs-portable "~/.emacs.d/.cache/dumps/emacs.pdmp")'
>     $ emacs --dump-file=~/.emacs.d/.cache/dumps/emacs.pdmp
>
> You can get the following message:
>
>     emacs: Trying to load incoherent dumped eln file
> /usr/sbin/../lib/emacs/28.0.50/native-lisp/28.0.50-x86_64-pc-linux-gnu-8fc0b1a455d90f52078950afaab3b40e/lisp-mode-410874a46ab8852855281f021ca61fe7-0babcd5810b0347a56a4b722059f6199.eln
>
>
> This issue appears on build from both chaotic-aur/emacs-pgtk-native-comp and archlinuxcn/emacs-native-comp.
>
> Thank you for your time!
>
> Ying Qu

Hi Ying Qu,

I guess you are the first trying to re-dump a native compiled Emacs
session.  Despite being in principle possible there's certanly something
to look at.

In general my understanding is that ATM we do not support officially
Emacs re-dumps, so also on a vanilla build it may work or not.

As a consequence I'm not sure this is a requirement for this feature
ATM.  Perhaps we'll get some other opinion on this.

Thanks

  Andrea



[-- Attachment #2: Type: text/html, Size: 4944 bytes --]

bug#45103: [feature/native-comp] add re-dumping support

[Elric Milon via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo <akrl@sdf.org> writes:

> Hi Ying Qu,
>
> I guess you are the first trying to re-dump a native compiled 
> Emacs
> session.  Despite being in principle possible there's certanly 
> something
> to look at.
>
> In general my understanding is that ATM we do not support 
> officially
> Emacs re-dumps, so also on a vanilla build it may work or not.
>
> As a consequence I'm not sure this is a requirement for this 
> feature
> ATM.  Perhaps we'll get some other opinion on this.

Hi Andrea,

Even if not supported, it did work (some workarounds required at 
times)
and it's used. I had been using it before switching to native-comp 
via
spacemacs' dumping support:

https://github.com/syl20bnr/spacemacs/blob/develop/EXPERIMENTAL.org#spacemacs-dumps-using-the-portable-dumper

I think it would be really nice to be able to do something similar 
again.

I spent some time attempting to make it work again by avoiding
re-dumping (starting from bare temacs) but at some point I gave up 
(It's
been a while but, if memory serves, I found C and lisp code 
clearly
not expecting for anyone to want to dump anything else than 
loadup.el).
If that where an option then support for re-dumping would not be
needed anymore.

Maybe it would be a good idea to add a guard to prevent re-dumping 
so
that users are aware they are doing something that will never work 
and
it's not something they are doing wrong (I spent a lot of time on 
this
myself :)).

Cheers!

bug#45103: [feature/native-comp] add re-dumping support

[Andrea Corallo]

Y Qu <yqu212@gmail.com> writes:

> Dear maintainers:
>
> I try to use pdumper together with native-comp but fail.
>
> To reproduce:
>
>     $ emacs --batch -Q -eval '(dump-emacs-portable "~/.emacs.d/.cache/dumps/emacs.pdmp")'
>     $ emacs --dump-file=~/.emacs.d/.cache/dumps/emacs.pdmp
>
> You can get the following message:
>
>     emacs: Trying to load incoherent dumped eln file
> /usr/sbin/../lib/emacs/28.0.50/native-lisp/28.0.50-x86_64-pc-linux-gnu-8fc0b1a455d90f52078950afaab3b40e/lisp-mode-410874a46ab8852855281f021ca61fe7-0babcd5810b0347a56a4b722059f6199.eln
>
>
> This issue appears on build from both chaotic-aur/emacs-pgtk-native-comp and archlinuxcn/emacs-native-comp.
>
> Thank you for your time!
>
> Ying Qu

Hello,

on the line of "better later than ever" I believe I implemented the fix
for this bug with f7d88f4a047 now on master.

$ emacs --dump-file -batch ~/emacs.pdm
$ emacs --dump-file ~/emacs.pdmp

Works for me here therefore I'm closing this bug.

Happy to reopen if necessary.

Thanks!

  Andrea

bug#45103: [feature/native-comp] add re-dumping support

[Andrea Corallo]

Andrea Corallo <acorallo@gnu.org> writes:

> Y Qu <yqu212@gmail.com> writes:
>
>> Dear maintainers:
>>
>> I try to use pdumper together with native-comp but fail.
>>
>> To reproduce:
>>
>>     $ emacs --batch -Q -eval '(dump-emacs-portable "~/.emacs.d/.cache/dumps/emacs.pdmp")'
>>     $ emacs --dump-file=~/.emacs.d/.cache/dumps/emacs.pdmp
>>
>> You can get the following message:
>>
>>     emacs: Trying to load incoherent dumped eln file
>> /usr/sbin/../lib/emacs/28.0.50/native-lisp/28.0.50-x86_64-pc-linux-gnu-8fc0b1a455d90f52078950afaab3b40e/lisp-mode-410874a46ab8852855281f021ca61fe7-0babcd5810b0347a56a4b722059f6199.eln
>>
>>
>> This issue appears on build from both chaotic-aur/emacs-pgtk-native-comp and archlinuxcn/emacs-native-comp.
>>
>> Thank you for your time!
>>
>> Ying Qu
>
> Hello,
>
> on the line of "better later than ever" I believe I implemented the fix
> for this bug with f7d88f4a047 now on master.
>
> $ emacs --dump-file -batch ~/emacs.pdm

Sorry wanted to write:
$ emacs --batch -Q -eval '(dump-emacs-portable "~/emacs.pdmp")'

> $ emacs --dump-file ~/emacs.pdmp
>
> Works for me here therefore I'm closing this bug.
>
> Happy to reopen if necessary.
>
> Thanks!
>
>   Andrea

bug#45103: [feature/native-comp] add re-dumping support

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Where is the file stored? Should there be a variable to override the
location of this file?

I think ideally Emacs could load the file if found in the set location.

bug#45103: [feature/native-comp] add re-dumping support

[Eli Zaretskii]

> Cc: yqu212@gmail.com, 45103@debbugs.gnu.org
> Date: Sun, 29 Oct 2023 16:44:44 +0200
> From:  Björn Bidar via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> 
> Where is the file stored? Should there be a variable to override the
> location of this file?

How would a variable help when we are installing the file as part of
"make install"?

And how would this variable help, if Emacs must load the dump file
before it can do anything else?

> I think ideally Emacs could load the file if found in the set location.

It does.  And we also have the --dump-file=FILE command-line option,
in case someone wants Emacs to load a specific file.

bug#45103: [feature/native-comp] add re-dumping support

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: yqu212@gmail.com, 45103@debbugs.gnu.org
>> Date: Sun, 29 Oct 2023 16:44:44 +0200
>> From:  Björn Bidar via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> 
>> Where is the file stored? Should there be a variable to override the
>> location of this file?
>
> How would a variable help when we are installing the file as part of
> "make install"?
>
I was talking about the user generated pdmp as described in the bug,
not the one generated during install.

> And how would this variable help, if Emacs must load the dump file
> before it can do anything else?
>

My idea was more in the line of producing the pdmp after the
installation, store in side the user config or cache directory and load
it each time Emacs starts.
But if understand correctly to check correctly for each platform for
these directories would be to early.

bug#45103: [feature/native-comp] add re-dumping support

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: yqu212@gmail.com,  acorallo@gnu.org,  45103@debbugs.gnu.org
> Date: Wed, 01 Nov 2023 02:22:27 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Cc: yqu212@gmail.com, 45103@debbugs.gnu.org
> >> Date: Sun, 29 Oct 2023 16:44:44 +0200
> >> From:  Björn Bidar via "Bug reports for GNU Emacs,
> >>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> >> 
> >> 
> >> Where is the file stored? Should there be a variable to override the
> >> location of this file?
> >
> > How would a variable help when we are installing the file as part of
> > "make install"?
> >
> I was talking about the user generated pdmp as described in the bug,
> not the one generated during install.

If the produced pdmp file is not written in the places where Emacs
looks for it at startup, then it can only be loaded via --dump-file.

> > And how would this variable help, if Emacs must load the dump file
> > before it can do anything else?
> 
> My idea was more in the line of producing the pdmp after the
> installation, store in side the user config or cache directory and load
> it each time Emacs starts.

If you are talking about _producing_ a dump file, then I think this
discussion is premature, since we don't yet support re-dumping well
enough, and not at all in interactive sessions.  We should postpone
this until we have the known problems fixed, because only then the
various implications will be known.