bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

Something strange is going on on this GNU/Linux system, unlike other
systems where I build Emacs with native-compilation: when dumping,
some of the preloaded files seem to be loaded in their byte-compiled
form, not native-compiled form.  The files are the ones loaded at the
very beginning of 'loadup':

  Loading emacs-lisp/byte-run...
  Loading emacs-lisp/backquote...
  Loading subr...
  Loading version...
  Loading widget...
  Loading custom...
  Loading emacs-lisp/map-ynp...
  Loading international/mule...
  Loading international/mule-conf...
  Loading env...
  Loading format...
  Loading bindings...
  Loading window...
  Loading files...

All the preloaded files after these ones are loaded as native-compiled
Lisp, as expected.

I forced native compilation of the offending files, and verified the
corresponding *.eln files are refreshed in the native-lisp directory,
but the problem persists.

Why does this happen, and how do I fix it?


In GNU Emacs 28.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.22.30, cairo version 1.15.10)
 of 2021-05-22 built on fencepost.gnu.org
Repository revision: a3de48687eb28121f3dbfc20be19bd06c4cd6e98
Repository branch: master
System Description: Trisquel GNU/Linux Etiona (9.0)

Configured using:
 'configure --with-gif=no --with-tiff=no --with-jpeg=no
 --with-xpm=ifavailable --with-modules --enable-checking=yes,glyphs
 --with-native-compilation 'CFLAGS=-O0 -g3''

Configured features:
ACL CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ LIBOTF
LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG
SECCOMP SOUND THREADS TOOLKIT_SCROLL_BARS X11 XDBE XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source eieio eieio-core eieio-loaddefs
password-cache json map text-property-search time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils comp
comp-cstr warnings subr-x rx cl-seq cl-macs cl-extra help-mode seq
cl-loaddefs cl-lib term/xterm xterm byte-opt gv bytecomp byte-compile
cconv iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 80499 8044)
 (symbols 48 7882 1)
 (strings 32 21664 3880)
 (string-bytes 1 739515)
 (vectors 16 14774)
 (vector-slots 8 245438 10812)
 (floats 8 29 316)
 (intervals 56 200 0)
 (buffers 992 11))

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Something strange is going on on this GNU/Linux system, unlike other
> systems where I build Emacs with native-compilation: when dumping,
> some of the preloaded files seem to be loaded in their byte-compiled
> form, not native-compiled form.  The files are the ones loaded at the
> very beginning of 'loadup':
>
>   Loading emacs-lisp/byte-run...
>   Loading emacs-lisp/backquote...
>   Loading subr...
>   Loading version...
>   Loading widget...
>   Loading custom...
>   Loading emacs-lisp/map-ynp...
>   Loading international/mule...
>   Loading international/mule-conf...
>   Loading env...
>   Loading format...
>   Loading bindings...
>   Loading window...
>   Loading files...
>
> All the preloaded files after these ones are loaded as native-compiled
> Lisp, as expected.
>
> I forced native compilation of the offending files, and verified the
> corresponding *.eln files are refreshed in the native-lisp directory,
> but the problem persists.
>
> Why does this happen, and how do I fix it?

Hi Eli,

I just did a fresh build on GNU/Linux and I cannot reproduce this.

===
Loading emacs-lisp/byte-run (native compiled elisp)...
Loading emacs-lisp/backquote (native compiled elisp)...
Loading subr (native compiled elisp)...
Loading version (native compiled elisp)...
Loading widget (native compiled elisp)...
Loading custom (native compiled elisp)...
Loading emacs-lisp/map-ynp (native compiled elisp)...
Loading international/mule (native compiled elisp)...
Loading international/mule-conf (native compiled elisp)...
Loading env (native compiled elisp)...
Loading format (native compiled elisp)...
Loading bindings (native compiled elisp)...
Loading window (native compiled elisp)...
Loading files (native compiled elisp)...
Loading emacs-lisp/macroexp (native compiled elisp)...
===

Maybe related to the fix for bug#48497 and some state of your build?

Regards

  Andrea

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

On May 24, 2021 9:55:10 AM GMT+03:00, Andrea Corallo <akrl@sdf.org> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Something strange is going on on this GNU/Linux system, unlike other
> > systems where I build Emacs with native-compilation: when dumping,
> > some of the preloaded files seem to be loaded in their byte-compiled
> > form, not native-compiled form.  The files are the ones loaded at
> the
> > very beginning of 'loadup':
> >
> >   Loading emacs-lisp/byte-run...
> >   Loading emacs-lisp/backquote...
> >   Loading subr...
> >   Loading version...
> >   Loading widget...
> >   Loading custom...
> >   Loading emacs-lisp/map-ynp...
> >   Loading international/mule...
> >   Loading international/mule-conf...
> >   Loading env...
> >   Loading format...
> >   Loading bindings...
> >   Loading window...
> >   Loading files...
> >
> > All the preloaded files after these ones are loaded as
> native-compiled
> > Lisp, as expected.
> >
> > I forced native compilation of the offending files, and verified the
> > corresponding *.eln files are refreshed in the native-lisp
> directory,
> > but the problem persists.
> >
> > Why does this happen, and how do I fix it?
> 
> Hi Eli,
> 
> I just did a fresh build on GNU/Linux and I cannot reproduce this.
> 
> ===
> Loading emacs-lisp/byte-run (native compiled elisp)...
> Loading emacs-lisp/backquote (native compiled elisp)...
> Loading subr (native compiled elisp)...
> Loading version (native compiled elisp)...
> Loading widget (native compiled elisp)...
> Loading custom (native compiled elisp)...
> Loading emacs-lisp/map-ynp (native compiled elisp)...
> Loading international/mule (native compiled elisp)...
> Loading international/mule-conf (native compiled elisp)...
> Loading env (native compiled elisp)...
> Loading format (native compiled elisp)...
> Loading bindings (native compiled elisp)...
> Loading window (native compiled elisp)...
> Loading files (native compiled elisp)...
> Loading emacs-lisp/macroexp (native compiled elisp)...
> ===
> 
> Maybe related to the fix for bug#48497 and some state of your build?
> 
> Regards
> 
>   Andrea

What state could that be?

And what do I do to escape from that state?

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> On May 24, 2021 9:55:10 AM GMT+03:00, Andrea Corallo <akrl@sdf.org> wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Something strange is going on on this GNU/Linux system, unlike other
>> > systems where I build Emacs with native-compilation: when dumping,
>> > some of the preloaded files seem to be loaded in their byte-compiled
>> > form, not native-compiled form.  The files are the ones loaded at
>> the
>> > very beginning of 'loadup':
>> >
>> >   Loading emacs-lisp/byte-run...
>> >   Loading emacs-lisp/backquote...
>> >   Loading subr...
>> >   Loading version...
>> >   Loading widget...
>> >   Loading custom...
>> >   Loading emacs-lisp/map-ynp...
>> >   Loading international/mule...
>> >   Loading international/mule-conf...
>> >   Loading env...
>> >   Loading format...
>> >   Loading bindings...
>> >   Loading window...
>> >   Loading files...
>> >
>> > All the preloaded files after these ones are loaded as
>> native-compiled
>> > Lisp, as expected.
>> >
>> > I forced native compilation of the offending files, and verified the
>> > corresponding *.eln files are refreshed in the native-lisp
>> directory,
>> > but the problem persists.
>> >
>> > Why does this happen, and how do I fix it?
>> 
>> Hi Eli,
>> 
>> I just did a fresh build on GNU/Linux and I cannot reproduce this.
>> 
>> ===
>> Loading emacs-lisp/byte-run (native compiled elisp)...
>> Loading emacs-lisp/backquote (native compiled elisp)...
>> Loading subr (native compiled elisp)...
>> Loading version (native compiled elisp)...
>> Loading widget (native compiled elisp)...
>> Loading custom (native compiled elisp)...
>> Loading emacs-lisp/map-ynp (native compiled elisp)...
>> Loading international/mule (native compiled elisp)...
>> Loading international/mule-conf (native compiled elisp)...
>> Loading env (native compiled elisp)...
>> Loading format (native compiled elisp)...
>> Loading bindings (native compiled elisp)...
>> Loading window (native compiled elisp)...
>> Loading files (native compiled elisp)...
>> Loading emacs-lisp/macroexp (native compiled elisp)...
>> ===
>> 
>> Maybe related to the fix for bug#48497 and some state of your build?
>> 
>> Regards
>> 
>>   Andrea
>
> What state could that be?

Hard to say, mine is just a guess as I cannot reproduce it here.

> And what do I do to escape from that state?

make bootstrap or even better restart from a git clean?

  Andrea

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

On May 24, 2021 10:47:06 AM GMT+03:00, Andrea Corallo <akrl@sdf.org> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> 
> >> Maybe related to the fix for bug#48497 and some state of your
> build?
> >> 
> >> Regards
> >> 
> >>   Andrea
> >
> > What state could that be?
> 
> Hard to say, mine is just a guess as I cannot reproduce it here.
> 
> > And what do I do to escape from that state?
> 
> make bootstrap or even better restart from a git clean?

That would erase all the traces of the problem.  I would prefer to try to understand why it happens, there could be a subtle bug here.

Suggestions for where to look for the reasons of the problem?

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> On May 24, 2021 10:47:06 AM GMT+03:00, Andrea Corallo <akrl@sdf.org> wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> 
>> >> Maybe related to the fix for bug#48497 and some state of your
>> build?
>> >> 
>> >> Regards
>> >> 
>> >>   Andrea
>> >
>> > What state could that be?
>> 
>> Hard to say, mine is just a guess as I cannot reproduce it here.
>> 
>> > And what do I do to escape from that state?
>> 
>> make bootstrap or even better restart from a git clean?
>
> That would erase all the traces of the problem.  I would prefer to try to understand why it happens, there could be a subtle bug here.
>
> Suggestions for where to look for the reasons of the problem?

I'd start verifying what given the el source file
`comp-el-to-eln-rel-filename' suggests as eln filename and if this is
present as native compiled file (I guess so if you recompiled manually).

The next step would be verifying why when loading in lread.c
'maybe_swap_for_eln' does not decide to load the eln file in place of
the elc one.

  Andrea

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 48578@debbugs.gnu.org
> Date: Mon, 24 May 2021 08:48:39 +0000
> 
> > Suggestions for where to look for the reasons of the problem?
> 
> I'd start verifying what given the el source file
> `comp-el-to-eln-rel-filename' suggests as eln filename and if this is
> present as native compiled file (I guess so if you recompiled manually).
> 
> The next step would be verifying why when loading in lread.c
> 'maybe_swap_for_eln' does not decide to load the eln file in place of
> the elc one.

Thanks.  The problem is in comp-el-to-eln-rel-filename, in this
fragment at its beginning:

  /* Use `file-truename' or fall back to `expand-file-name' when the
     first is not available (bug#44701).

     `file-truename' is not available only for a short phases of the
     bootstrap before file.el is loaded, given we do not symlink
     inside the build directory this should work.  */
  filename = NILP (Ffboundp (intern_c_string ("file-truename")))
    ? Fexpand_file_name (filename, Qnil)
    : CALL1I (file-truename, filename);

During loadup, file-truename is not available until files.elc/eln is
loaded.  But the assumption above that it isn't a problem because "we
do not symlink inside the build directory" in false in my case,
because my home directory, where I build Emacs, is itself a symlink:

  (expand-file-name "~") => "/home/e/eliz"
  (file-truename "~") => "/srv/data/home/e/eliz"

So every one of the *.eln files loaded before and including files.elc
will fail in the above logic and will produce a different path_hash,
thus failing the attempt to find the correct .eln file.

We could use 'realpath' (and a suitable emulation on WINDOWSNT)
instead of file-truename.

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 48578@debbugs.gnu.org
>> Date: Mon, 24 May 2021 08:48:39 +0000
>>
>> > Suggestions for where to look for the reasons of the problem?
>>
>> I'd start verifying what given the el source file
>> `comp-el-to-eln-rel-filename' suggests as eln filename and if this is
>> present as native compiled file (I guess so if you recompiled manually).
>>
>> The next step would be verifying why when loading in lread.c
>> 'maybe_swap_for_eln' does not decide to load the eln file in place of
>> the elc one.
>
> Thanks.  The problem is in comp-el-to-eln-rel-filename, in this
> fragment at its beginning:
>
>   /* Use `file-truename' or fall back to `expand-file-name' when the
>      first is not available (bug#44701).
>
>      `file-truename' is not available only for a short phases of the
>      bootstrap before file.el is loaded, given we do not symlink
>      inside the build directory this should work.  */
>   filename = NILP (Ffboundp (intern_c_string ("file-truename")))
>     ? Fexpand_file_name (filename, Qnil)
>     : CALL1I (file-truename, filename);
>
> During loadup, file-truename is not available until files.elc/eln is
> loaded.  But the assumption above that it isn't a problem because "we
> do not symlink inside the build directory" in false in my case,
> because my home directory, where I build Emacs, is itself a symlink:
>
>   (expand-file-name "~") => "/home/e/eliz"
>   (file-truename "~") => "/srv/data/home/e/eliz"
>
> So every one of the *.eln files loaded before and including files.elc
> will fail in the above logic and will produce a different path_hash,
> thus failing the attempt to find the correct .eln file.
>
> We could use 'realpath' (and a suitable emulation on WINDOWSNT)
> instead of file-truename.

Hi Eli,

I see, it would then look something like the attached I guess.

It does bootstrap on GNU/Linux but I'm no expert on how to fill the
WINDOWSNT ifdef branch.

Thanks

  Andrea


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 48578.patch --]
[-- Type: text/x-diff, Size: 1380 bytes --]

diff --git a/src/comp.c b/src/comp.c
index 340ed85038..e6debefa1d 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -4008,15 +4008,23 @@ DEFUN ("comp-el-to-eln-rel-filename", Fcomp_el_to_eln_rel_filename,
 {
   CHECK_STRING (filename);
 
-  /* Use `file-truename' or fall back to `expand-file-name' when the
-     first is not available (bug#44701).
-
-     `file-truename' is not available only for a short phases of the
-     bootstrap before file.el is loaded, given we do not symlink
-     inside the build directory this should work.  */
-  filename = NILP (Ffboundp (intern_c_string ("file-truename")))
-    ? Fexpand_file_name (filename, Qnil)
-    : CALL1I (file-truename, filename);
+  /* Use `file-truename' or fall back to 'realpath' `expand-file-name'
+     when the first is not available.  (`file-truename' is not
+     available only for a short phases of the bootstrap before file.el
+     is loaded).  */
+
+  if (NILP (Ffboundp (intern_c_string ("file-truename"))))
+    {
+#ifndef WINDOWSNT
+      char *file_normalized = realpath (SSDATA (filename), NULL);
+#else
+      char *file_normalized = ??? ;
+#endif
+      filename = Fexpand_file_name (build_string (file_normalized), Qnil);
+      xfree (file_normalized);
+    }
+  else
+    filename = CALL1I (file-truename, filename);
 
   if (NILP (Ffile_exists_p (filename)))
     xsignal1 (Qfile_missing, filename);

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 48578@debbugs.gnu.org
> Date: Tue, 25 May 2021 12:34:27 +0000
> 
> +  /* Use `file-truename' or fall back to 'realpath' `expand-file-name'
> +     when the first is not available.  (`file-truename' is not
> +     available only for a short phases of the bootstrap before file.el
> +     is loaded).  */
> +
> +  if (NILP (Ffboundp (intern_c_string ("file-truename"))))
> +    {
> +#ifndef WINDOWSNT
> +      char *file_normalized = realpath (SSDATA (filename), NULL);
> +#else
> +      char *file_normalized = ??? ;
> +#endif
> +      filename = Fexpand_file_name (build_string (file_normalized), Qnil);
> +      xfree (file_normalized);
> +    }
> +  else
> +    filename = CALL1I (file-truename, filename);
>  
>    if (NILP (Ffile_exists_p (filename)))
>      xsignal1 (Qfile_missing, filename);

Thanks, I will try that.  But why not use realpath at all times, even
if file-truename is available?  They do the same thing, AFAIK, no?

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1340 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 48578@debbugs.gnu.org
>> Date: Tue, 25 May 2021 12:34:27 +0000
>> 
>> +  /* Use `file-truename' or fall back to 'realpath' `expand-file-name'
>> +     when the first is not available.  (`file-truename' is not
>> +     available only for a short phases of the bootstrap before file.el
>> +     is loaded).  */
>> +
>> +  if (NILP (Ffboundp (intern_c_string ("file-truename"))))
>> +    {
>> +#ifndef WINDOWSNT
>> +      char *file_normalized = realpath (SSDATA (filename), NULL);
>> +#else
>> +      char *file_normalized = ??? ;
>> +#endif
>> +      filename = Fexpand_file_name (build_string (file_normalized), Qnil);
>> +      xfree (file_normalized);
>> +    }
>> +  else
>> +    filename = CALL1I (file-truename, filename);
>>  
>>    if (NILP (Ffile_exists_p (filename)))
>>      xsignal1 (Qfile_missing, filename);
>
> Thanks, I will try that.  But why not use realpath at all times, even
> if file-truename is available?  They do the same thing, AFAIK, no?

AFAIK yes but the file-truename implementation didn't look sufficiently
trivial to be sure they are really equivalent with the quick skim I had.

I agree with the idea of using always one (realpath), should be also
easier to ideantify issues in case.

Attached the updated patch.

  Andrea


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 48578.patch --]
[-- Type: text/x-diff, Size: 1232 bytes --]

diff --git a/src/comp.c b/src/comp.c
index 340ed85038..e961161fc6 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -4008,15 +4008,18 @@ DEFUN ("comp-el-to-eln-rel-filename", Fcomp_el_to_eln_rel_filename,
 {
   CHECK_STRING (filename);
 
-  /* Use `file-truename' or fall back to `expand-file-name' when the
-     first is not available (bug#44701).
-
-     `file-truename' is not available only for a short phases of the
-     bootstrap before file.el is loaded, given we do not symlink
-     inside the build directory this should work.  */
-  filename = NILP (Ffboundp (intern_c_string ("file-truename")))
-    ? Fexpand_file_name (filename, Qnil)
-    : CALL1I (file-truename, filename);
+  /* Use `file-truename' or fall back to 'realpath' `expand-file-name'
+     when the first is not available.  (`file-truename' is not
+     available only for a short phases of the bootstrap before file.el
+     is loaded).  */
+
+#ifndef WINDOWSNT
+  char *file_normalized = realpath (SSDATA (filename), NULL);
+#else
+  char *file_normalized = ??? ;
+#endif
+  filename = Fexpand_file_name (build_string (file_normalized), Qnil);
+  xfree (file_normalized);
 
   if (NILP (Ffile_exists_p (filename)))
     xsignal1 (Qfile_missing, filename);

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1546 bytes --]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: 48578@debbugs.gnu.org
>>> Date: Tue, 25 May 2021 12:34:27 +0000
>>> 
>>> +  /* Use `file-truename' or fall back to 'realpath' `expand-file-name'
>>> +     when the first is not available.  (`file-truename' is not
>>> +     available only for a short phases of the bootstrap before file.el
>>> +     is loaded).  */
>>> +
>>> +  if (NILP (Ffboundp (intern_c_string ("file-truename"))))
>>> +    {
>>> +#ifndef WINDOWSNT
>>> +      char *file_normalized = realpath (SSDATA (filename), NULL);
>>> +#else
>>> +      char *file_normalized = ??? ;
>>> +#endif
>>> +      filename = Fexpand_file_name (build_string (file_normalized), Qnil);
>>> +      xfree (file_normalized);
>>> +    }
>>> +  else
>>> +    filename = CALL1I (file-truename, filename);
>>>  
>>>    if (NILP (Ffile_exists_p (filename)))
>>>      xsignal1 (Qfile_missing, filename);
>>
>> Thanks, I will try that.  But why not use realpath at all times, even
>> if file-truename is available?  They do the same thing, AFAIK, no?
>
> AFAIK yes but the file-truename implementation didn't look sufficiently
> trivial to be sure they are really equivalent with the quick skim I had.
>
> I agree with the idea of using always one (realpath), should be also
> easier to ideantify issues in case.
>
> Attached the updated patch.

Well... attached updating also the comment.

  Andrea


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 48578.patch --]
[-- Type: text/x-diff, Size: 1097 bytes --]

diff --git a/src/comp.c b/src/comp.c
index 340ed85038..8e40ea0f25 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -4008,15 +4008,15 @@ DEFUN ("comp-el-to-eln-rel-filename", Fcomp_el_to_eln_rel_filename,
 {
   CHECK_STRING (filename);
 
-  /* Use `file-truename' or fall back to `expand-file-name' when the
-     first is not available (bug#44701).
-
-     `file-truename' is not available only for a short phases of the
-     bootstrap before file.el is loaded, given we do not symlink
-     inside the build directory this should work.  */
-  filename = NILP (Ffboundp (intern_c_string ("file-truename")))
-    ? Fexpand_file_name (filename, Qnil)
-    : CALL1I (file-truename, filename);
+  /* `file-truename' is not available only during bootstrap before
+     file.el is loaded.  */
+#ifndef WINDOWSNT
+  char *file_normalized = realpath (SSDATA (filename), NULL);
+#else
+  char *file_normalized = ??? ;
+#endif
+  filename = Fexpand_file_name (build_string (file_normalized), Qnil);
+  xfree (file_normalized);
 
   if (NILP (Ffile_exists_p (filename)))
     xsignal1 (Qfile_missing, filename);

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Basil L. Contovounesios]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Thanks, I will try that.  But why not use realpath at all times, even
>> if file-truename is available?  They do the same thing, AFAIK, no?
>
> AFAIK yes but the file-truename implementation didn't look sufficiently
> trivial to be sure they are really equivalent with the quick skim I had.

The main difference is that file-truename heeds file name handlers, no?

Thanks,

-- 
Basil

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Cc: Andrea Corallo <akrl@sdf.org>,  48578@debbugs.gnu.org
> Date: Tue, 25 May 2021 19:13:02 +0100
> 
> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs@gnu.org> writes:
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> >> Thanks, I will try that.  But why not use realpath at all times, even
> >> if file-truename is available?  They do the same thing, AFAIK, no?
> >
> > AFAIK yes but the file-truename implementation didn't look sufficiently
> > trivial to be sure they are really equivalent with the quick skim I had.
> 
> The main difference is that file-truename heeds file name handlers, no?

Yes, but is that an issue when building Emacs or, more generally,
native-compiling *.el files?

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Basil L. Contovounesios]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Basil L. Contovounesios" <contovob@tcd.ie>
>> Cc: Andrea Corallo <akrl@sdf.org>,  48578@debbugs.gnu.org
>> Date: Tue, 25 May 2021 19:13:02 +0100
>> 
>> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
>> text editors" <bug-gnu-emacs@gnu.org> writes:
>> 
>> > Eli Zaretskii <eliz@gnu.org> writes:
>> >
>> >> Thanks, I will try that.  But why not use realpath at all times, even
>> >> if file-truename is available?  They do the same thing, AFAIK, no?
>> >
>> > AFAIK yes but the file-truename implementation didn't look sufficiently
>> > trivial to be sure they are really equivalent with the quick skim I had.
>> 
>> The main difference is that file-truename heeds file name handlers, no?
>
> Yes, but is that an issue when building Emacs or, more generally,
> native-compiling *.el files?

I don't know, but I wouldn't have guessed so.

-- 
Basil

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, 48578@debbugs.gnu.org
> Date: Tue, 25 May 2021 13:22:39 +0000
> 
> diff --git a/src/comp.c b/src/comp.c
> index 340ed85038..8e40ea0f25 100644
> --- a/src/comp.c
> +++ b/src/comp.c
> @@ -4008,15 +4008,15 @@ DEFUN ("comp-el-to-eln-rel-filename", Fcomp_el_to_eln_rel_filename,
>  {
>    CHECK_STRING (filename);
>  
> -  /* Use `file-truename' or fall back to `expand-file-name' when the
> -     first is not available (bug#44701).
> -
> -     `file-truename' is not available only for a short phases of the
> -     bootstrap before file.el is loaded, given we do not symlink
> -     inside the build directory this should work.  */
> -  filename = NILP (Ffboundp (intern_c_string ("file-truename")))
> -    ? Fexpand_file_name (filename, Qnil)
> -    : CALL1I (file-truename, filename);
> +  /* `file-truename' is not available only during bootstrap before
> +     file.el is loaded.  */
> +#ifndef WINDOWSNT
> +  char *file_normalized = realpath (SSDATA (filename), NULL);
> +#else
> +  char *file_normalized = ??? ;
> +#endif
> +  filename = Fexpand_file_name (build_string (file_normalized), Qnil);
> +  xfree (file_normalized);
>  
>    if (NILP (Ffile_exists_p (filename)))
>      xsignal1 (Qfile_missing, filename);

Thanks, this works.  But I have a few questions/comments:

  . why do you use Fexpand_file_name _after_ realpath? shouldn't we
    call it before realpath instead?
  . we need to encode the file name before passing it to realpath, and
    decode the result, right?
  . realpath could potentially return NULL, and we should check for
    that, and return the result of Fexpand_file_name if it does
  . we don't need to support remote files here via file-handlers,
    right?

If you agree with the above, I will prepare a proper patch and install
it, including the necessary changes for WINDOWSNT.

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, 48578@debbugs.gnu.org
>> Date: Tue, 25 May 2021 13:22:39 +0000
>> 
>> diff --git a/src/comp.c b/src/comp.c
>> index 340ed85038..8e40ea0f25 100644
>> --- a/src/comp.c
>> +++ b/src/comp.c
>> @@ -4008,15 +4008,15 @@ DEFUN ("comp-el-to-eln-rel-filename", Fcomp_el_to_eln_rel_filename,
>>  {
>>    CHECK_STRING (filename);
>>  
>> -  /* Use `file-truename' or fall back to `expand-file-name' when the
>> -     first is not available (bug#44701).
>> -
>> -     `file-truename' is not available only for a short phases of the
>> -     bootstrap before file.el is loaded, given we do not symlink
>> -     inside the build directory this should work.  */
>> -  filename = NILP (Ffboundp (intern_c_string ("file-truename")))
>> -    ? Fexpand_file_name (filename, Qnil)
>> -    : CALL1I (file-truename, filename);
>> +  /* `file-truename' is not available only during bootstrap before
>> +     file.el is loaded.  */
>> +#ifndef WINDOWSNT
>> +  char *file_normalized = realpath (SSDATA (filename), NULL);
>> +#else
>> +  char *file_normalized = ??? ;
>> +#endif
>> +  filename = Fexpand_file_name (build_string (file_normalized), Qnil);
>> +  xfree (file_normalized);
>>  
>>    if (NILP (Ffile_exists_p (filename)))
>>      xsignal1 (Qfile_missing, filename);
>
> Thanks, this works.  But I have a few questions/comments:
>
>   . why do you use Fexpand_file_name _after_ realpath? shouldn't we
>     call it before realpath instead?

Right that's correct.

>   . we need to encode the file name before passing it to realpath, and
>     decode the result, right?

Correct

>   . realpath could potentially return NULL, and we should check for
>     that, and return the result of Fexpand_file_name if it does

Of course

>   . we don't need to support remote files here via file-handlers,
>     right?

Correct, so far we do not support it.

> If you agree with the above, I will prepare a proper patch and install
> it, including the necessary changes for WINDOWSNT.

Agreed, thanks.

  Andrea

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 48578@debbugs.gnu.org
> Date: Thu, 27 May 2021 11:57:22 +0000
> 
> >   . why do you use Fexpand_file_name _after_ realpath? shouldn't we
> >     call it before realpath instead?
> 
> Right that's correct.
> 
> >   . we need to encode the file name before passing it to realpath, and
> >     decode the result, right?
> 
> Correct
> 
> >   . realpath could potentially return NULL, and we should check for
> >     that, and return the result of Fexpand_file_name if it does
> 
> Of course
> 
> >   . we don't need to support remote files here via file-handlers,
> >     right?
> 
> Correct, so far we do not support it.
> 
> > If you agree with the above, I will prepare a proper patch and install
> > it, including the necessary changes for WINDOWSNT.
> 
> Agreed, thanks.

Thanks, I installed a fix along these lines, please review and
comment.

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 48578@debbugs.gnu.org
>> Date: Thu, 27 May 2021 11:57:22 +0000
>> 
>> >   . why do you use Fexpand_file_name _after_ realpath? shouldn't we
>> >     call it before realpath instead?
>> 
>> Right that's correct.
>> 
>> >   . we need to encode the file name before passing it to realpath, and
>> >     decode the result, right?
>> 
>> Correct
>> 
>> >   . realpath could potentially return NULL, and we should check for
>> >     that, and return the result of Fexpand_file_name if it does
>> 
>> Of course
>> 
>> >   . we don't need to support remote files here via file-handlers,
>> >     right?
>> 
>> Correct, so far we do not support it.
>> 
>> > If you agree with the above, I will prepare a proper patch and install
>> > it, including the necessary changes for WINDOWSNT.
>> 
>> Agreed, thanks.
>
> Thanks, I installed a fix along these lines, please review and
> comment.

Hi Eli,

it looks good to me.

Thanks for fixing this.

  Andrea

 

bug#48578: 28.0.50; Native-compiled files of some preloaded files not loaded at dump time

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 48578@debbugs.gnu.org
> Date: Sat, 29 May 2021 07:02:45 +0000
> 
> > Thanks, I installed a fix along these lines, please review and
> > comment.
> 
> Hi Eli,
> 
> it looks good to me.
> 
> Thanks for fixing this.

Great, so I'm closing this bug.