Re: Finding the dump (redux)

[Eli Zaretskii <eliz@gnu.org>]

> Cc: emacs-devel@gnu.org
> From: Ali Bahrami <ali_gnu2@emvision.com>
> Date: Sat, 17 Apr 2021 18:15:13 -0600
> 
> >>        % ln -s /usr/bin/emacs-gtk ~/bin/myemacs
> > 
> > Why can't you put the *.pdmp files in the same directory where you
> > keep the emacs-* executables (/usr/bin, AFAIU)?  That is already
> > supported by the current code, I think.
> 
> I know the current code does support that, but it doesn't fit the
> long lived rules we have for what belongs in /usr/bin.

Then place a symlink emacs.pdmp there, and have the actual pdumper
file where you want it, under any name you want.  Or move/copy the
emacs-* executables to another directory, make the 'emacs' symlink
resolve to those emacs-* files, and have the pdumper files in the same
place.  Or configure with a different value of $libdir when you build
each emacs-* variant, and then have the corresponding emacs.pdmp file
in the directory under /usr/lib that is private to that variant.  Or
use the --dump-file command-line option.

There are many possible solutions that already work, so why insist on
something that doesn't work?  That it happened to work with unexec is
just sheer luck: the upstream Emacs project never explicitly supported
such configurations.

> A related idea that's been floated before would be for the
> executable to carry the default dump data within itself.

That idea didn't fly because it meant we again need to comply to
various binary formats, which change with time out of our control.
We'd eventually get into the same trouble as with unexec: the
corresponding developers will refuse supporting the tricks we play for
that to work, exactly as glibc dropped support for malloc hooks we
needed to support unexec.

More generally, that doesn't solve the general problem of how Emacs
finds files it needs to start.  Even if the dump data is in the
executable itself, there could be other files that are similarly
needed at startup.  We already have that with the native-compilation
feature: the *.eln files produced from the preloaded Lisp packages
need to be located at startup, otherwise Emacs will be unable to
start.  We cannot possibly put everything inside the Emacs executable,
even if we wanted to.

> > But realpath(argv[0]) can produce to a file in another directory,
> > because realpath expands all the symlinks, not just that of the
> > basename.  Does it make sense to look up the .pdmp file in the
> > directory of the original argv[0] when it is a symlink?
> 
> It's an interesting question, and I think can be argued
> either way.

Exactly.  So who's to say which way is TRT?  Whatever we decide, there
could be another distro out there which will argue that the opposite
makes sense because "it worked for them until now".

And if you are thinking about trying both, then (a) there's still the
question of order (which could affect the correctness), and (b) it
makes the startup slower, and soon enough people will start
complaining about that.

> I can imagine a scenario where it might be useful to
> say "yes". It might offer a pretty slick way for end users
> to create arbitrary pdmp files and associate them to specific
> purposes. Suppose for instance that I want to use a special 'X'
> dump file when working on "Project X" code. I could create a special
> name for that emacs variant as a symlink to the basic emacs-gtk
> in my personal bin:
> 
>     % ln -s /usr/bin/emacs-gtk ~/bin/emacsX
> 
> Then, if I were to create ~/bin/emacsX.pdmp, and if emacs were
> willing to see it as a pdmp file to be loaded, then I could
> run my special emacsX, and get the standard emacs (from the
> symlink) using my specialized X pdmp.

We support the --dump-file command-line option for this purpose: using
that you can have the pdumper file under any file name you want, all
you need is a shell script or an alias that would add that option.

> The reverse question is, what harm does it do to look in PATH_EXEC
> for both names?

See above: it makes startup slower, and also runs the risk of picking
up the wrong pdumper file and failing the startup altogether.

> >>        - Move the repeated code into functions, to clean up load_pdump().
> >>
> >>        - Save the basename from the result of calling realpath() during the
> >>          search of the executable directory.
> >>
> >>        - During the PATH_EXEC stage, use the saved realpath basename to
> >>          add a check for that name.
> > 
> > This is not enough, if we want to support *.pdmp files that have
> > arbitrary names.  For example, when Emacs is invoked as "../emacs" (or
> > any other relative file name which includes slashes), we currently
> > don't expand symlinks, so with your proposal "emacs" and "../emacs"
> > will behave differently.
> 
> I'm not sure I understand. I have the proposed bits installed
> on my desktop right now, and this does work as I expect.
> 
>      % cd /usr/bin
>      % ../bin/emacs
> 
> As does
> 
>      % emacs

That's because you are running Emacs installed, so it looks for the
pdumper file in the hardcoded place under PATH_EXEC, no matter what.
I was alluding to the case that you run Emacs uninstalled, when the
pdumper file is in the same directory where the Emacs binary lives.

> I don't see any code in load_pdump() that special cases
> the case that includes slashes

Look in load_pdump_find_executable, and you will see it.

> > IOW, supporting arbitrarily-named *.pdmp files requires more thorough
> > revision of the logic in load_pdump than just some simple refactoring.
> > Especially as Emacs 28 will have the native-compilation feature,
> > whereby it also needs to find at startup the directory with the *.eln
> > files that correspond to the preloaded Lisp files; see bug#44128.
> 
> I think you're right that the above isn't enough for arbitrary names,
> but it's not trying to be. As long as it doesn't interfere with that
> later effort, there shouldn't be a conflict. My goal is just to make
> sure that out of the box default behavior works, without the sort of
> mysterious failure (to a naive user) that symlinks cause today.

But in this case _you_ are the distro, so you determine how OOTB
works.  It isn't carved in stone, and the arrangements you used in the
past don't need to be repeated verbatim in the future, because "times
are a-changing".

Having said all of the above, since we are currently working on
related issues on the native-compilation branch, it is possible that
we eventually will teach Emacs to support also the arrangement you
want to work in your case.  But I make no promises, and in any case
this will not hit the street before Emacs 28.1, which is probably
still a year or more in the future.  We don't expect another 27.x
release, and even if there is such a release, it will probably be to
fix some very grave bug, so unsuitable for extending existing
features.  So it's your call whether to wait for Emacs 28 in the hope
that maybe it fixes your problem, or redesign your deployment now to
use some arrangement that already works.