bug#66534: 30.0.50; [PATCH] Expand file-name of ~/.emacs before attempt to load it

bug#66534: 30.0.50; [PATCH] Expand file-name of ~/.emacs before attempt to load it

[Christoph via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 882 bytes --]

Hi,

I noticed that during startup, emacs tries to load ~/.emacs (with
different extensions) many times over.  You can see this by using
strace and tracing the openat syscall.  The problem is, that `load'
does not expand the ~/ in the filename passed to it.  So it does not
recognize the file as being absolute and tries to resolve it using
the load-path.

While resolving the path in the openp function in lread.c,
`expand-file-name' is used with the default directory being the
elements of the load-path.  Since for `expand-file-name', ~/.emacs is
an absolute path, it returns the path unchanged, and load tries to
load ~/.emacs many times over.  I am not sure if the behavior of
`load' should also be considered a bug, but since all the other paths
of init files are resolved using `expand-file-name', I guess the same
should be done for the ~/.emacs path as well.

-- Christoph

[-- Attachment #2: 0001-Expand-file-name-of-.emacs-before-attempt-to-load-it.patch --]
[-- Type: text/x-patch, Size: 2783 bytes --]

From 42bc428470f0b6907cfc81718f35083a2df30d12 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Christoph=20G=C3=B6ttschkes?= <just.mychris@googlemail.com>
Date: Fri, 13 Oct 2023 20:19:48 +0200
Subject: [PATCH] Expand file-name of ~/.emacs before attempt to load it.

* lisp/startup.el (command-line):
Expand the name of the ~/.emacs (or its other variant)
initialization file-name before passing the absolute path to load.
Otherwise, load does not recognize the path to be absolute and uses the
load-path to find the file.
---
 lisp/startup.el | 40 +++++++++++++++++++++-------------------
 1 file changed, 21 insertions(+), 19 deletions(-)

diff --git a/lisp/startup.el b/lisp/startup.el
index 6329e3ea8d0..73e27439e50 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -1493,25 +1493,27 @@ command-line
     ;; Load that user's init file, or the default one, or none.
     (startup--load-user-init-file
      (lambda ()
-       (cond
-	((eq startup-init-directory xdg-dir) nil)
-        ((eq system-type 'ms-dos)
-         (concat "~" init-file-user "/_emacs"))
-        ((not (eq system-type 'windows-nt))
-         (concat "~" init-file-user "/.emacs"))
-        ;; Else deal with the Windows situation.
-        ((directory-files "~" nil "\\`\\.emacs\\(\\.elc?\\)?\\'")
-         ;; Prefer .emacs on Windows.
-         "~/.emacs")
-        ((directory-files "~" nil "\\`_emacs\\(\\.elc?\\)?\\'")
-         ;; Also support _emacs for compatibility, but warn about it.
-         (push `(initialization
-                 ,(format-message
-                   "`_emacs' init file is deprecated, please use `.emacs'"))
-               delayed-warnings-list)
-         "~/_emacs")
-        (t ;; But default to .emacs if _emacs does not exist.
-         "~/.emacs")))
+       (if (eq startup-init-directory xdg-dir)
+           nil
+         (expand-file-name
+          (cond
+           ((eq system-type 'ms-dos)
+            (concat "~" init-file-user "/_emacs"))
+           ((not (eq system-type 'windows-nt))
+            (concat "~" init-file-user "/.emacs"))
+           ;; Else deal with the Windows situation.
+           ((directory-files "~" nil "\\`\\.emacs\\(\\.elc?\\)?\\'")
+            ;; Prefer .emacs on Windows.
+            "~/.emacs")
+           ((directory-files "~" nil "\\`_emacs\\(\\.elc?\\)?\\'")
+            ;; Also support _emacs for compatibility, but warn about it.
+            (push `(initialization
+                    ,(format-message
+                      "`_emacs' init file is deprecated, please use `.emacs'"))
+                  delayed-warnings-list)
+            "~/_emacs")
+           (t ;; But default to .emacs if _emacs does not exist.
+            "~/.emacs")))))
      (lambda ()
        (expand-file-name
         "init.el"
-- 
2.42.0

bug#66534: 30.0.50; [PATCH] Expand file-name of ~/.emacs before attempt to load it

[Eli Zaretskii]

> Date: Fri, 13 Oct 2023 20:41:16 +0200
> From:  Christoph via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> I noticed that during startup, emacs tries to load ~/.emacs (with
> different extensions) many times over.  You can see this by using
> strace and tracing the openat syscall.  The problem is, that `load'
> does not expand the ~/ in the filename passed to it.  So it does not
> recognize the file as being absolute and tries to resolve it using
> the load-path.
> 
> While resolving the path in the openp function in lread.c,
> `expand-file-name' is used with the default directory being the
> elements of the load-path.  Since for `expand-file-name', ~/.emacs is
> an absolute path, it returns the path unchanged, and load tries to
> load ~/.emacs many times over.

I don't understand what you are saying here.  The last sentence is
incorrect, as evidenced by the following:

  (expand-file-name "~/.emacs" "/tmp")
   => "/home/eliz/.emacs"

IOW, "~/.emacs" is indeed treated by Emacs as an absolute file name,
but expand-file-name does NOT return "~/.emacs" unchanged.

So please explain what exactly is the problem you see here, and in
particular what issues that problem causes in your case.

Thanks.

bug#66534: 30.0.50; [PATCH] Expand file-name of ~/.emacs before attempt to load it

[Andreas Schwab]

On Okt 13 2023, Christoph via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:

> I noticed that during startup, emacs tries to load ~/.emacs (with
> different extensions) many times over.  You can see this by using
> strace and tracing the openat syscall.  The problem is, that `load'
> does not expand the ~/ in the filename passed to it.  So it does not
> recognize the file as being absolute and tries to resolve it using
> the load-path.

That's probably a bug in complete_filename_p.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#66534: 30.0.50; [PATCH] Expand file-name of ~/.emacs before attempt to load it

[Christoph via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sat, Oct 14, 2023 at 8:54 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > Date: Fri, 13 Oct 2023 20:41:16 +0200
> > From:  Christoph via "Bug reports for GNU Emacs,
> >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> >
> > I noticed that during startup, emacs tries to load ~/.emacs (with
> > different extensions) many times over.  You can see this by using
> > strace and tracing the openat syscall.  The problem is, that `load'
> > does not expand the ~/ in the filename passed to it.  So it does not
> > recognize the file as being absolute and tries to resolve it using
> > the load-path.
> >
> > While resolving the path in the openp function in lread.c,
> > `expand-file-name' is used with the default directory being the
> > elements of the load-path.  Since for `expand-file-name', ~/.emacs is
> > an absolute path, it returns the path unchanged, and load tries to
> > load ~/.emacs many times over.
>
> I don't understand what you are saying here.  The last sentence is
> incorrect, as evidenced by the following:
>
>   (expand-file-name "~/.emacs" "/tmp")
>    => "/home/eliz/.emacs"
>
> IOW, "~/.emacs" is indeed treated by Emacs as an absolute file name,
> but expand-file-name does NOT return "~/.emacs" unchanged.

What I meant is "unchanged" in the regard to the default
directory. You can give any default directory to expand-file-name and
it will always return the first argument in its expanded form, if the
expanded form of the first argument is an absolute path.

>
> So please explain what exactly is the problem you see here, and in
> particular what issues that problem causes in your case.
>
> Thanks.

If you start Emacs without an ~/.emacs file, and with using .emacs.d
and not .config/emacs, I think there is a difference as well, Emacs
will try to load ~/.emacs `(length load-path)' times.  If I start
Emacs with strace -e openat I see the following lines 147 times:

openat(AT_FDCWD, "/home/chris/.emacs", O_RDONLY|O_CLOEXEC) = -1 ENOENT
(No such file or directory)
openat(AT_FDCWD, "/home/chris/.emacs.gz", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/chris/.emacs.so", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/chris/.emacs.so.gz", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/chris/.emacs.elc", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/chris/.emacs.elc.gz", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/chris/.emacs.el", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/chris/.emacs.el.gz", O_RDONLY|O_CLOEXEC) = -1
ENOENT (No such file or directory)

But Emacs should only try to load the file once.  The problem is that
the path to ~/.emacs is not expanded before it is handed to `load' and
load basically does:

(dolist (path load-path)
  (low-level-load (expand-file-name "~/.emacs" path)))

where low-level-load is some magic function which really loads a file.
There is off course more going on, like iterating over the extension
and such, but I want to keep it simple, since I am not familiar with
it at all.

`load' does check if the given path is absolute and
changes its behavior.  If the path is absolute, `load' does not take the
load-path into account, but paths beginning with "~/" are not recognized as
beeing absolute, hence the paths should be expanded before passing them
to `load'.

Because all the other paths to init files (early-init.el and init.el)
are expanded before they are passed to `load', I think "~/.emacs"
should be expanded as well.

There is also the question if `load' itself should be able to handle
paths that begin with "~/" correctly, or if the user is expected to
expand paths before passing them to `load'.

Regards, Christoph

bug#66534: 30.0.50; [PATCH] Expand file-name of ~/.emacs before attempt to load it

[Christoph via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sat, Oct 14, 2023 at 9:25 AM Andreas Schwab <schwab@linux-m68k.org> wrote:
>
> On Okt 13 2023, Christoph via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:
>
> > I noticed that during startup, emacs tries to load ~/.emacs (with
> > different extensions) many times over.  You can see this by using
> > strace and tracing the openat syscall.  The problem is, that `load'
> > does not expand the ~/ in the filename passed to it.  So it does not
> > recognize the file as being absolute and tries to resolve it using
> > the load-path.
>
> That's probably a bug in complete_filename_p.
>

I am not so sure if complete_filename_p should be able to recognize
these kinds of paths, or if `load' should expand the filename before
using it.  I am not familiar with all supported platforms, but it might
be tricky to check if a filename is absolute, without expanding it
beforehand.

For instance, what about filenames that start with a series of "../".
I guess they should be considered to be absolute, if the path goes up
to the root.

On the other hand, the documentation of `load' does not mention
absolute paths at all and states, This function searches the
directories on `load-path'.  So I can't say what the expected behaviour
of load is with paths that aren't expanded, but are absolute.

I am hesitant with changing the C code, because I am not very familiar
with it and just started to get into it, so I thought it might be more
appropriate to change it on the Lisp side, since all the other paths
to init files are fully expanded before passing them to load.

But yes, as I wrote before, I think this behaviour of load could be
considered a bug as well.

-- Christoph

bug#66534: 30.0.50; [PATCH] Expand file-name of ~/.emacs before attempt to load it

[Christoph via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sat, Oct 14, 2023 at 8:54 AM Eli Zaretskii <eliz@gnu.org> wrote:
> So please explain what exactly is the problem you see here, and in
> particular what issues that problem causes in your case.

Do you think the patch makes sense like that, or should I rather look
into the native side, as suggested by Andreas?