bug#75022: 31.0.50; --init-directory sets user-emacs-directory after user eln-cache is set

bug#75022: 31.0.50; --init-directory sets user-emacs-directory after user eln-cache is set

[Jordan Isaacs]

In startup.el, the user-emacs-directory and the user eln-cache are set up before processing command line args.
The user eln-cache is then gets updated after the early-init.el is loaded. This leaves a gap where early-init.el is
loaded without the correct eln-cache directory. This means there will always be a cache miss if the early-init.eln
file was in the `--init-directory`.

To fix this, either `--init-directory` needs to be read before setting up the user eln-cache, or a call to `startup--update-eln-cache` needs to be called after command line args are processed since user-emacs-directory may have changed (from `--init-directory`).

bug#75022: 31.0.50; --init-directory sets user-emacs-directory after user eln-cache is set

[Eli Zaretskii]

> Date: Sun, 22 Dec 2024 03:57:01 -0800
> From: "Jordan Isaacs" <jordan@snowytrees.dev>
> 
> In startup.el, the user-emacs-directory and the user eln-cache are set up before processing command line args.
> The user eln-cache is then gets updated after the early-init.el is loaded. This leaves a gap where early-init.el is
> loaded without the correct eln-cache directory. This means there will always be a cache miss if the early-init.eln
> file was in the `--init-directory`.
> 
> To fix this, either `--init-directory` needs to be read before setting up the user eln-cache, or a call to `startup--update-eln-cache` needs to be called after command line args are processed since user-emacs-directory may have changed (from `--init-directory`).

You are not supposed to have in early-init.el stuff that needs to be
natively-compiled.  early-init.el is for changing the few variables
which affect the initialization stage, and that doesn't need native
code.  You are supposed to have most of your configuration in the
normal init file, not in early-init.el.

As for the --init-directory, it doesn't affect the search for your
init files, it only affects the value of user-emacs-directory (for
looking up other files).

So I don't think I understand why you are asking for changes in how
the startup code works in Emacs.  Please tell more about your use
case.

bug#75022: 31.0.50; --init-directory sets user-emacs-directory after user eln-cache is set

[Jordan Isaacs]

> As for the --init-directory, it doesn't affect the search for your
> init files, it only affects the value of user-emacs-directory (for
> looking up other files).

The issue is the `--init-directory` updated the user-emacs-directory, but there
was no corresponding call to update the eln-cache directory (startup--update-eln-cache).
There are comments throughout startup.el about updating the cache directory whenever
user-emacs-directory is changed. It is missing that call post command line evaluation.
 
> So I don't think I understand why you are asking for changes in how
> the startup code works in Emacs.  Please tell more about your use
> case.

I have a read-only directory of my init.el and early-init.el along with the
byte compiled versions of them. So I call emacs with --init-directory:

/nix/store/ln4pwf5blaqiynydrcn344zn7kbfn6vs-emacs-config
├── early-init.el -> /nix/store/qq073crkwrjvpfrfnsbsqq502c8k0a5y-emacs-config-init-1/share/emacs/site-lisp/early-init.el
├── early-init.elc -> /nix/store/qq073crkwrjvpfrfnsbsqq502c8k0a5y-emacs-config-init-1/share/emacs/site-lisp/early-init.elc
├── eln-cache
│   └── 31.0.50-0186c17f
│       ├── early-init-ca07bdae-ee82ea37.eln -> /nix/store/qq073crkwrjvpfrfnsbsqq502c8k0a5y-emacs-config-init-1/share/emacs/native-lisp/31.0.50-0186c17f/early-init-ca07bdae-ee82ea37.eln
│       └── init-ab609663-d324ef3d.eln -> /nix/store/qq073crkwrjvpfrfnsbsqq502c8k0a5y-emacs-config-init-1/share/emacs/native-lisp/31.0.50-0186c17f/init-ab609663-d324ef3d.eln
├── init.el -> /nix/store/qq073crkwrjvpfrfnsbsqq502c8k0a5y-emacs-config-init-1/share/emacs/site-lisp/init.el
└── init.elc -> /nix/store/qq073crkwrjvpfrfnsbsqq502c8k0a5y-emacs-config-init-1/share/emacs/site-lisp/init.elc

However, when I remove any mutable eln-cache directory I get the following message:

"Error muted by safe_call: (apply native--compile-async ("/nix/store/ln4pwf5blaqiynydrcn344zn7kbfn6vs-emacs-config/early-init.el" nil late)) signaled (error "Cannot find suitable directory for output in ‘native-comp-eln-load-path’.")"

Or, when I do give it a mutable eln-cache directory, it ends up recompiling it in the background doing unnecessary work:

/home/jd/.local/state/emacs/eln-cache/31.0.50-0186c17f/
└── early-init-ed4380c6-4ba9e9ac.eln

When I set add  `/nix/store/ln4pwf5blaqiynydrcn344zn7kbfn6vs-emacs-config/eln-cache` to `EMACSNATIVELOADPATH` everything works great. But ideally, startup.el should respect `--init-directory` and update the eln-cache location for you
before loading early-init.el

bug#75022: 31.0.50; --init-directory sets user-emacs-directory after user eln-cache is set

[Eli Zaretskii]

> Date: Sun, 22 Dec 2024 05:12:47 -0800
> From: "Jordan Isaacs" <jordan@snowytrees.dev>
> Cc: 75022@debbugs.gnu.org
> 
> > So I don't think I understand why you are asking for changes in how
> > the startup code works in Emacs.  Please tell more about your use
> > case.
> 
> I have a read-only directory of my init.el and early-init.el along with the
> byte compiled versions of them. So I call emacs with --init-directory:

I'm asking why did you compile early-init.el.  What do you have in it
that can benefit from byte-compilation, let alone native compilation?

bug#75022: 31.0.50; --init-directory sets user-emacs-directory after user eln-cache is set

[Jordan Isaacs]

Because it is read only, there is no reason not to byte compile everything. I just make it part of the build step when packaging up my configuration.

The reasoning for why I byte compile though seems irrelevant, it’s a bug with a simple fix. The eln-cache path isn’t staying in sync with user-emacs-directory. 

    (let ((old-face-font-rescale-alist face-font-rescale-alist))
      (unwind-protect
	  (command-line)
+  (when (featurep 'native-compile)
+     (startup--update-eln-cache))