bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

If you use ~/.emacs instead of ~.emacs.d/init.el, and happen to compile
.emacs, Emacs loads .emacs.elc (as it has always done), but the native
compiler does *not* know how to compile it (because it does not know how to
locate its source), so it gives a warning:

022-11-18 02:54:01+0100 Warning (comp): Cannot look up eln file as no
source file was found for d:/Home/.emacs.elc

and then it assigns the user-init-file to the native-compiled warnings.el,

"d:/Home/.emacs.d/native/29.0.50-a4a10996/warnings-28e75f4d-02da775e.eln"

which now, with the fix for bug#59334, ends up pointing to warnings.el:

ELISP> user-init-file
"d:/Devel/emacs/repo/trunk/lisp/emacs-lisp/warnings.el"

[-- Attachment #2: Type: text/html, Size: 1569 bytes --]

bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Fri, 18 Nov 2022 14:17:36 +0100
> 
> If you use ~/.emacs instead of ~.emacs.d/init.el, and happen to compile .emacs, Emacs loads .emacs.elc
> (as it has always done), but the native compiler does *not* know how to compile it (because it does not know
> how to locate its source), so it gives a warning:
> 
> 022-11-18 02:54:01+0100 Warning (comp): Cannot look up eln file as no source file was found for
> d:/Home/.emacs.elc

I think the solution to that is to special-case ".emacs", in
maybe_swap_for_eln, as a file name that doesn't have to have the .el
extension for this purpose.

> and then it assigns the user-init-file to the native-compiled warnings.el,
> 
> "d:/Home/.emacs.d/native/29.0.50-a4a10996/warnings-28e75f4d-02da775e.eln"
> 
> which now, with the fix for bug#59334, ends up pointing to warnings.el:
> 
> ELISP> user-init-file
> "d:/Devel/emacs/repo/trunk/lisp/emacs-lisp/warnings.el"

That is a separate bug, IMO.  Is this because in Fload we set 'found'
to the file name of warnings.el/.eln in this case?  If so, we should
prevent this nonsense.

bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 1600 bytes --]

On Fri, Nov 18, 2022 at 3:24 PM Eli Zaretskii <eliz@gnu.org> wrote:


> That is a separate bug, IMO.  Is this because in Fload we set 'found'
> to the file name of warnings.el/.eln in this case?  If so, we should
> prevent this nonsense.
>

The bug is the consequence of maybe_swap_for_eln calling `display-warning',
which initiates a recursive Fload while user-init-file is still t, so Fload
"records" the current file name.

The easiest fix in my view is to use delayed warnings and avoid the
recursive Fload.

diff --git i/src/lread.c w/src/lread.c
index c28324dc35..2a57f72194 100644
--- i/src/lread.c
+++ w/src/lread.c
@@ -1742,10 +1742,13 @@ maybe_swap_for_eln (bool no_native, Lisp_Object
*filename, int *fd,
                                               Qnil, Qnil)))
                return;
-             call2 (intern_c_string ("display-warning"),
-                    Qcomp,
-                    CALLN (Fformat,
-                           build_string ("Cannot look up eln file as "
-                                         "no source file was found for
%s"),
-                           *filename));
+             Vdelayed_warnings_list
+               = Fcons (list2
+                        (Qcomp,
+                         CALLN (Fformat,
+                                build_string ("Cannot look up eln "
+                                              "file as no source file "
+                                              "was found for %s"),
+                                *filename)),
+                        Vdelayed_warnings_list);
              return;
            }

[-- Attachment #2: Type: text/html, Size: 2735 bytes --]

bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Sat, 19 Nov 2022 10:13:58 +0100
> Cc: 59358@debbugs.gnu.org
> 
> On Fri, Nov 18, 2022 at 3:24 PM Eli Zaretskii <eliz@gnu.org> wrote:
>  
>  That is a separate bug, IMO.  Is this because in Fload we set 'found'
>  to the file name of warnings.el/.eln in this case?  If so, we should
>  prevent this nonsense.
> 
> The bug is the consequence of maybe_swap_for_eln calling `display-warning', which initiates a recursive
> Fload while user-init-file is still t, so Fload "records" the current file name.
> 
> The easiest fix in my view is to use delayed warnings and avoid the recursive Fload.

Fine by me.

Andrea, do you agree?

bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Juanma Barranquero <lekktu@gmail.com>
>> Date: Sat, 19 Nov 2022 10:13:58 +0100
>> Cc: 59358@debbugs.gnu.org
>> 
>> On Fri, Nov 18, 2022 at 3:24 PM Eli Zaretskii <eliz@gnu.org> wrote:
>>  
>>  That is a separate bug, IMO.  Is this because in Fload we set 'found'
>>  to the file name of warnings.el/.eln in this case?  If so, we should
>>  prevent this nonsense.
>> 
>> The bug is the consequence of maybe_swap_for_eln calling
>> `display-warning', which initiates a recursive
>> Fload while user-init-file is still t, so Fload "records" the current file name.
>> 
>> The easiest fix in my view is to use delayed warnings and avoid the
>> recursive Fload.
>
> Fine by me.
>
> Andrea, do you agree?

Agreed.

Thanks

  Andrea

bug#59358:

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 251 bytes --]

The more serious bug is fixed with this commit:

8ec8decd9f  of  2022-11-20
Avoid native compiler setting user-init-file to warnings.el (bug#59358)

I'll open a new bug report with the first issue (the native compiler being
unable to compile .emacs).

[-- Attachment #2: Type: text/html, Size: 566 bytes --]

bug#59424: bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 544 bytes --]

On Fri, Nov 18, 2022 at 3:24 PM Eli Zaretskii <eliz@gnu.org> wrote:


> I think the solution to that is to special-case ".emacs", in
> maybe_swap_for_eln, as a file name that doesn't have to have the .el
> extension for this purpose.
>

I've been fiddling with maybe_swap_for_eln for a while to try to
special-case .emacs and make comp native-compile it, but I reached a point
where it was getting a bit out of hand in the kludgey department.

I think that's better implemented by someone more knowledgeable with the
native-compiler machinery.

[-- Attachment #2: Type: text/html, Size: 1042 bytes --]

bug#59424: bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Sun, 27 Nov 2022 20:01:25 +0100
> Cc: 59424@debbugs.gnu.org
> 
> On Fri, Nov 18, 2022 at 3:24 PM Eli Zaretskii <eliz@gnu.org> wrote:
>  
>  I think the solution to that is to special-case ".emacs", in
>  maybe_swap_for_eln, as a file name that doesn't have to have the .el
>  extension for this purpose.
> 
> I've been fiddling with maybe_swap_for_eln for a while to try to special-case .emacs and make comp
> native-compile it, but I reached a point where it was getting a bit out of hand in the kludgey department.
> 
> I think that's better implemented by someone more knowledgeable with the native-compiler machinery.

I'm probably missing something, because it looks like this is the place to
do it:

  /* Search eln in the eln-cache directories.  */
  Lisp_Object eln_path_tail = Vnative_comp_eln_load_path;
  Lisp_Object src_name =
    Fsubstring (*filename, Qnil, make_fixnum (-1));  <<<<<<<<<<<<<<<<

The idea is to use

    Fsubstring (*filename, Qnil, make_fixnum (-3))

instead of the above if '*filename' ends in ".emacs.elc" (which you can
establish by using the same suffix_p that is used above this code to verify
the ".elc" extension).

But since you say there's more here than meets the eye, I'm wondering what
did I miss?

bug#59424: bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 1955 bytes --]

This patch below is what I did, *just* to get comp not to complain about
the "missing" .emacs.el.

Then something should be done to make it to compile .emacs, and I'm not
sure where or how.

I can't shake the feeling that I'm thrashing around and someone who knows
the code could do it faster and cleaner.


diff --git i/src/comp.c w/src/comp.c
index b6072a866e..a8e67bbeb2 100644
--- i/src/comp.c
+++ w/src/comp.c
@@ -4473,7 +4473,10 @@ DEFUN ("comp-el-to-eln-rel-filename",
Fcomp_el_to_eln_rel_filename,
   Lisp_Object separator = build_string ("-");
   Lisp_Object path_hash = comp_hash_string (filename);
-  filename = concat2 (Ffile_name_nondirectory (Fsubstring (filename, Qnil,
-   make_fixnum (-3))),
-      separator);
+  if (suffix_p (filename, ".emacs"))
+    filename = concat2 (Ffile_name_nondirectory (filename), separator);
+  else
+    filename = concat2 (Ffile_name_nondirectory (Fsubstring (filename,
Qnil,
+     make_fixnum (-3))),
+ separator);
   Lisp_Object hash = concat3 (path_hash, separator, content_hash);
   return concat3 (filename, hash, build_string (NATIVE_ELISP_SUFFIX));
diff --git i/src/lread.c w/src/lread.c
index 0a6e4201e4..b0b08ba872 100644
--- i/src/lread.c
+++ w/src/lread.c
@@ -1716,7 +1716,10 @@ maybe_swap_for_eln (bool no_native, Lisp_Object
*filename, int *fd,
     Fremhash (*filename, V_comp_no_native_file_h);

-  if (no_native
-      || load_no_native
-      || !suffix_p (*filename, ".elc"))
+  if (no_native || load_no_native)
+    return;
+
+  bool _emacs = suffix_p (*filename, ".emacs.elc");
+
+  if (!_emacs && !suffix_p (*filename, ".elc"))
     return;

@@ -1724,5 +1727,5 @@ maybe_swap_for_eln (bool no_native, Lisp_Object
*filename, int *fd,
   Lisp_Object eln_path_tail = Vnative_comp_eln_load_path;
   Lisp_Object src_name =
-    Fsubstring (*filename, Qnil, make_fixnum (-1));
+    Fsubstring (*filename, Qnil, make_fixnum (_emacs ? -4 : -1));
   if (NILP (Ffile_exists_p (src_name)))
     {

[-- Attachment #2: Type: text/html, Size: 2433 bytes --]

bug#59424: bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Sun, 27 Nov 2022 20:53:27 +0100
> Cc: akrl@sdf.org, 59424@debbugs.gnu.org
> 
> This patch below is what I did, *just* to get comp not to complain about the "missing" .emacs.el.
> 
> Then something should be done to make it to compile .emacs, and I'm not sure where or how.

So the problem is not to find the source of .emacs.elc, the problem is also
to compile .emacs natively into .emacs.eln?  I though the problem was only
the former.

> I can't shake the feeling that I'm thrashing around and someone who knows the code could do it faster and
> cleaner.

I'm not sure a cleaner way exists, but I will let Andrea chime in.

Note that there's also maybe_defer_native_compilation.

bug#59424: bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 283 bytes --]

Both: comp complains that it doesn't find .emacs.el; and it doesn't compile
.emacs

With my changes, it "knows" that the source should be .emacs, and it's able
to generate a suitable name for it (a /path/to/.emacs-xxxxxxxx-yyyyyyyy.eln
name in the hash). But it does not compile it.

[-- Attachment #2: Type: text/html, Size: 477 bytes --]

bug#59424: bug#59358: 29.0.50; failing to load .eln for init file sets user-init-file to warnings.el

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Juanma Barranquero <lekktu@gmail.com>
>> Date: Sun, 27 Nov 2022 20:53:27 +0100
>> Cc: akrl@sdf.org, 59424@debbugs.gnu.org
>> 
>> This patch below is what I did, *just* to get comp not to complain about the "missing" .emacs.el.
>> 
>> Then something should be done to make it to compile .emacs, and I'm not sure where or how.
>
> So the problem is not to find the source of .emacs.elc, the problem is also
> to compile .emacs natively into .emacs.eln?  I though the problem was only
> the former.
>
>> I can't shake the feeling that I'm thrashing around and someone who knows the code could do it faster and
>> cleaner.
>
> I'm not sure a cleaner way exists, but I will let Andrea chime in.
>
> Note that there's also maybe_defer_native_compilation.

Hello all,

yes I think Juanma you should have a look into
'maybe_defer_native_compilation'.

This will be called when the first byte compiled function defined in
your .emacs (if any) is being loaded.  As you can see
'maybe_defer_native_compilation' tries to reconstruct the src filename,
so I guess this need some tweaking for this specific case.

Best Regards

  Andrea