bug#43476: feature/native-comp; path for .eln files when running with --no-init-file

bug#43476: feature/native-comp; path for .eln files when running with --no-init-file

[Tom Gillespie]

Hi Andrea,
   Not quite a bug, but an inconsistency in the behavior of
comp-eln-load-path compared to the behavior of load-path. Best!
Tom

While investigating the json-mode kill-buffer bug I encountered an
issue which seems like it might cause confusion at some point.
comp-eln-load-path is set at startup and materialized.

When Emacs is launched with -q, load-path does not contain the
materialized locations of paths inside user-emacs-directory until
after a call to package-initialize.

I think that there is a similar need for comp-eln-load-path, which is
that it needs to exclude the user eln cache path when emacs is
launched with -q so that it is possible to debug issues arising from
stale/bad eln files. There will be a similar issue with site-lisp and
-Q.

I don't know whether stale files could cause a problem given how you
hash to generate the names for the eln files, but I'm imagining cases
where someone upgraded gcc or libgccjit and something changed. I think
that not setting the user path when Emacs is run with -q is consistent
with Emacs behavior for load-path.

Right now a user has to explicitly pop the old eln cache directory and
then update the new directory if they reset user-emacs-directory as I
do in the longer reproduction for the json-mode bug (reproduced
below).

#+begin_src bash
emacs -q \
--eval "(setq user-emacs-directory \"/tmp/test/.emacs.d/\")" \
--eval "(when (boundp 'comp-eln-load-path) (pop comp-eln-load-path)
(add-to-list 'comp-eln-load-path (concat user-emacs-directory \"eln-cache\")))"
#+end_src

I don't have a good solution for this, but wanted to raise the issue
since I anticipate that there will be quite a bit of confusion if the
user eln cache continues to point to the path ~/.emacs.d/eln-cache
instead of either not being set or updating to be inside
user-emacs-directory the first time the value is needed.

bug#43476: feature/native-comp; path for .eln files when running with --no-init-file

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Tom Gillespie <tgbugs@gmail.com> writes:

> Hi Andrea,
>    Not quite a bug, but an inconsistency in the behavior of
> comp-eln-load-path compared to the behavior of load-path. Best!
> Tom
>
> While investigating the json-mode kill-buffer bug I encountered an
> issue which seems like it might cause confusion at some point.
> comp-eln-load-path is set at startup and materialized.
>
> When Emacs is launched with -q, load-path does not contain the
> materialized locations of paths inside user-emacs-directory until
> after a call to package-initialize.
>
> I think that there is a similar need for comp-eln-load-path, which is
> that it needs to exclude the user eln cache path when emacs is
> launched with -q so that it is possible to debug issues arising from
> stale/bad eln files. There will be a similar issue with site-lisp and
> -Q.
>
> I don't know whether stale files could cause a problem given how you
> hash to generate the names for the eln files, but I'm imagining cases
> where someone upgraded gcc or libgccjit and something changed. I think
> that not setting the user path when Emacs is run with -q is consistent
> with Emacs behavior for load-path.
>
> Right now a user has to explicitly pop the old eln cache directory and
> then update the new directory if they reset user-emacs-directory as I
> do in the longer reproduction for the json-mode bug (reproduced
> below).
>
> #+begin_src bash
> emacs -q \
> --eval "(setq user-emacs-directory \"/tmp/test/.emacs.d/\")" \
> --eval "(when (boundp 'comp-eln-load-path) (pop comp-eln-load-path)
> (add-to-list 'comp-eln-load-path (concat user-emacs-directory \"eln-cache\")))"
> #+end_src
>
> I don't have a good solution for this, but wanted to raise the issue
> since I anticipate that there will be quite a bit of confusion if the
> user eln cache continues to point to the path ~/.emacs.d/eln-cache
> instead of either not being set or updating to be inside
> user-emacs-directory the first time the value is needed.

Hi Tom,

I think the fundamental issue is that we need a eln-cache directory to
operate anyway.  So either we assume that eln compilation is correct and
transparent (current approach) or we need to set the eln-cache directory
to be in some temporary directory for every -q run.  This would indeed
require the recompilation a bunch of compilation units at each -q
startup.

I'm personally for keeping the current approach as the second one could
be annoying especially on non very powerful machines.  Still a power
user (like you :) can decide not to trust the correctness assumption and
hack it around as you have showed.

As ATM I don't have any better idea on this I'd be for closing this bug
unless some idea/action is suggested, what do you think about?

Thanks

  Andrea

bug#43476: feature/native-comp; path for .eln files when running with --no-init-file

[Tom Gillespie]

Hi Andrea,
    I think it is ok to close this. I don't see an easy solution here.
My workaround works, and the way that the eln files are named seems
like it is safe against most common issues (I have no idea what the
effect of changing versions of libgccjit would actually be, it was
just a hypothetical). I think the only practical thing that could be
done is to add a note to the manual about how to change the location
of the eln-cache at startup. Otherwise, this bug can serve as a record
in case anyone encounters an issue. Best!
Tom

On Sat, Oct 10, 2020 at 5:48 AM Andrea Corallo <akrl@sdf.org> wrote:
>
> Tom Gillespie <tgbugs@gmail.com> writes:
>
> > Hi Andrea,
> >    Not quite a bug, but an inconsistency in the behavior of
> > comp-eln-load-path compared to the behavior of load-path. Best!
> > Tom
> >
> > While investigating the json-mode kill-buffer bug I encountered an
> > issue which seems like it might cause confusion at some point.
> > comp-eln-load-path is set at startup and materialized.
> >
> > When Emacs is launched with -q, load-path does not contain the
> > materialized locations of paths inside user-emacs-directory until
> > after a call to package-initialize.
> >
> > I think that there is a similar need for comp-eln-load-path, which is
> > that it needs to exclude the user eln cache path when emacs is
> > launched with -q so that it is possible to debug issues arising from
> > stale/bad eln files. There will be a similar issue with site-lisp and
> > -Q.
> >
> > I don't know whether stale files could cause a problem given how you
> > hash to generate the names for the eln files, but I'm imagining cases
> > where someone upgraded gcc or libgccjit and something changed. I think
> > that not setting the user path when Emacs is run with -q is consistent
> > with Emacs behavior for load-path.
> >
> > Right now a user has to explicitly pop the old eln cache directory and
> > then update the new directory if they reset user-emacs-directory as I
> > do in the longer reproduction for the json-mode bug (reproduced
> > below).
> >
> > #+begin_src bash
> > emacs -q \
> > --eval "(setq user-emacs-directory \"/tmp/test/.emacs.d/\")" \
> > --eval "(when (boundp 'comp-eln-load-path) (pop comp-eln-load-path)
> > (add-to-list 'comp-eln-load-path (concat user-emacs-directory \"eln-cache\")))"
> > #+end_src
> >
> > I don't have a good solution for this, but wanted to raise the issue
> > since I anticipate that there will be quite a bit of confusion if the
> > user eln cache continues to point to the path ~/.emacs.d/eln-cache
> > instead of either not being set or updating to be inside
> > user-emacs-directory the first time the value is needed.
>
> Hi Tom,
>
> I think the fundamental issue is that we need a eln-cache directory to
> operate anyway.  So either we assume that eln compilation is correct and
> transparent (current approach) or we need to set the eln-cache directory
> to be in some temporary directory for every -q run.  This would indeed
> require the recompilation a bunch of compilation units at each -q
> startup.
>
> I'm personally for keeping the current approach as the second one could
> be annoying especially on non very powerful machines.  Still a power
> user (like you :) can decide not to trust the correctness assumption and
> hack it around as you have showed.
>
> As ATM I don't have any better idea on this I'd be for closing this bug
> unless some idea/action is suggested, what do you think about?
>
> Thanks
>
>   Andrea

bug#43476: feature/native-comp; path for .eln files when running with --no-init-file

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Tom Gillespie <tgbugs@gmail.com> writes:

> Hi Andrea,
>     I think it is ok to close this. I don't see an easy solution here.
> My workaround works, and the way that the eln files are named seems
> like it is safe against most common issues (I have no idea what the
> effect of changing versions of libgccjit would actually be, it was
> just a hypothetical). I think the only practical thing that could be
> done is to add a note to the manual about how to change the location
> of the eln-cache at startup.

I agree this is something we'll have to mention.

closing, thanks!

  Andrea