bug#73318: 31.0.50; with-native-compilation=aot breaks exec -a emacs

[Eli Zaretskii <eliz@gnu.org>]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: Ship Mints <shipmints@gmail.com>,  larsi@gnus.org,  acorallo@gnu.org,
>    73318@debbugs.gnu.org
> Date: Tue, 17 Sep 2024 18:31:05 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I don't remember the details, sorry.  You are welcome to look up the
> > past discussions in the archives.  I think they were triggered by look
> > up of the pdumper file, but the results of that are also used by the
> > code which decides where to look for the *.eln files.
> 
> I looked up /proc/self/exe in the archives and the only mention is
> https://lists.gnu.org/archive/html/emacs-devel/2019-05/msg00951.html

A more interesting discussion starts here:

  https://lists.gnu.org/archive/html/emacs-devel/2019-01/msg00635.html

That discussion is about finding the pdumper file, but the side effect
of looking for pdumper file is the directory where we think the Emacs
executable file is.  That discussion mentions several issues related
to finding the leading directories of the Emacs executable.

Another useful read is here:

  https://stackoverflow.com/questions/1023306/finding-current-executables-path-without-proc-self-exe

> With all due humility, I think I personally am enough of an expert on
> Linux minutiae to say that /proc/self/exe will be substantially more
> reliable than using argv[0].

And I will see your humility and raise ya.  Please describe your ideas
for the patch before actually writing the code.  Because there's more
here than meets the eye.  Some issues the related code needs to
handle:

  . what if /proc/self/exe is unreadable? AFAIK, on some systems you
    need special privileges to follow its symlink
  . what if /proc/self/exe points to a file name that is a symlink, or
    some of its leading directories are symlinks?
  . what if Emacs is invoked via a script which is in the correct
    installation directory, but the actual binary the script invokes
    is not in the expected location relative to the native-lisp/
    directory where we have the preloaded *.eln files?

The existing code handles all these cases, and some others.  We could
perhaps _add_ the use of /proc/self/exe to what we have, but we'd need
to be sure that it doesn't break for the above situations.

I also don't understand why your script insists on removing the
leading directories from argv[0] of Emacs.  Is there any problem for
you to modify your script such that the leading directories would
still be present in argv[0]?

And finally, your description of the original issue seems to omit some
crucial details (or maybe I'm missing something):

> 1. Compile and install Emacs with --with-native-compilation=aot, e.g.:
> prefix=~/prefix
> mkdir $prefix
> ./configure --with-native-compilation=aot --prefix=$prefix
> make -j64 && make install
> 2. Run emacs with "exec -a" to change its argv[0]:
> sh -c "exec -a emacs $prefix/bin/emacs -Q --batch"
> 3. Observe an error like:
> Error using execdir /usr/local/home/sbaugh/workspaces/24833141-bffb-3c99-a9d6-c366d37c4f5e/+share+/app/emacs/bin/:
> emacs: /usr/local/home/sbaugh/workspaces/24833141-bffb-3c99-a9d6-c366d37c4f5e/+share+/app/emacs/bin/../native-lisp/31.0.50-a88a37f5/preloaded/minibuffer-b2d9c221-284ab177.eln: cannot open shared object file: No such file or directory

Where did the /usr/local/home/sbaugh/workspaces/24833141-bffb-3c99-a9d6-c366d37c4f5e/+share+/app/
part come from?  I'm guessing that /usr/local/home/sbaugh/ is the
expansion of "~" in your case, but where did the rest come from if
your $prefix is just "~/prefix"?

When Emacs does not find its executable file using argv[0], it assumes
that the executable is in PATH_EXEC/../../../../bin/.  Since you are
running an installed Emacs, that should have worked, unless you also
somehow changed the relative path from $prefix/bin to the directory
where the native-lisp/ directory is installed.  Why didn't it work?

Bottom line: I think there are still unclear aspects of what happened
in your case, and using /proc/self/exe to fix that is not as simple as
it might seem, especially since we don't yet understand fully what
failed and why.