bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Jonas Bernoulli]

I usually run emacs without installing it, via a symlink, which is
placed on PATH and which points at the executable in the checkout.
Due to reasons that aren't important here, I recently had to replace the
symlink with a wrapper script, and that resulted in these messages being
send to stderr.

  Loading loadup.el (source)...
  Dump mode: nil
  Using load-path (...)
  Loading emacs-lisp/debug-early...
  ...
  Finding pointers to doc strings...done

Running the executable directly also results in these messages.

I then changed my wrapper script to use an intermediate symlink, with
the resulted that the messages were not being shown for a while again,
but today I updated emacs again and now the messages are always shown.
Using a symlink no longer suppresses them.

Are this messages shown intentionally?  Is it already possible to
suppress them?  If not, could something be added to allow suppressing
them?

(I prefer to run the uninstalled executable, because that allows me
to jump to a definition and then immediately view its history.)

     Thanks,
     Jonas

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Date: Sun, 25 Sep 2022 17:15:18 +0200
> 
> I usually run emacs without installing it, via a symlink, which is
> placed on PATH and which points at the executable in the checkout.
> Due to reasons that aren't important here, I recently had to replace the
> symlink with a wrapper script, and that resulted in these messages being
> send to stderr.
> 
>   Loading loadup.el (source)...
>   Dump mode: nil
>   Using load-path (...)
>   Loading emacs-lisp/debug-early...
>   ...
>   Finding pointers to doc strings...done
> 
> Running the executable directly also results in these messages.
> 
> I then changed my wrapper script to use an intermediate symlink, with
> the resulted that the messages were not being shown for a while again,
> but today I updated emacs again and now the messages are always shown.
> Using a symlink no longer suppresses them.
> 
> Are this messages shown intentionally?  Is it already possible to
> suppress them?  If not, could something be added to allow suppressing
> them?

Is your Emacs built with or without native-compilation?

The messages mean that Emacs either cannot find its pdumper file, or
cannot find the preloaded *.eln files, and therefore loads all the
preloaded files explicitly during startup.  The question is why.  My
guess is that the way you invoke Emacs is not supported by the startup
code.  This became more complicated with the pdumper build, and yet
more with the native-compilation build, because where formerly Emacs
needed only to find its executable file, it now needs also to find the
pdumper file and all the preloaded *.eln files.

> (I prefer to run the uninstalled executable, because that allows me
> to jump to a definition and then immediately view its history.)

Then may I suggest a shell alias?  Emacs has special support for
running from the build tree, but in your case it somehow doesn't
realize that.

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Jonas Bernoulli]

>> Due to reasons that aren't important here, I recently had to replace the
>> symlink with a wrapper script, and that resulted in these messages being
>> send to stderr.

I was very wrong about that not being important here.

Installing emacs using guix sets EMACSLOADPATH in a file that is sourced
by the shells startup file.  To me it looks like they are abusing this
variable; it makes it impossible to use an emacs that wasn't installed
using guix, without something to counter their modification.  I did that
with a wrapper script.

The value that is set somewhere in the shell's init files is something
like "/home/jonas/.guix-home/profile/share/emacs/site-lisp".  It doesn't
contain an empty element, which would "stand for the default value of
`load-path'", according to (info "(emacs)General Variables").

The side startup file is located inside this directory and contains

  (when (require 'guix-emacs nil t)
    (guix-emacs-autoload-packages)
    (advice-add 'package-load-all-descriptors :after
                #'guix-emacs-load-package-descriptors))

The file "guix-emacs.el" is located in the same directory.  So the
purpose of setting EMACSLOADPATH seems to be to allow using `require' to
load that file.  This doesn't seem right to me, they could just as well
extend the load-path inside "site-lisp.el", or load their additional
init file using `load'.

To deal with the incomplete EMACSLOADPATH, as set in the shell
environment, they use a wrapper script named "emacs" and located on PATH:

  #!/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash
  export XDG_DATA_DIRS="/gnu/store/h6qdfd4sjdg28jq16600hcjky0cfw9gx-shared-mime-info-1.15/share:/gnu/store/bnsf9il448hl5xjavbhq3rcx355svz2v-glib-2.70.2/share:/gnu/store/7njbpmsgwmx59xn2r6q097iy2wwscb44-gtk+-3.24.30/share:/gnu/store/w0ssipd06il3kvyvqihpmw4lwbhwwfq3-emacs-28.1/share${XDG_DATA_DIRS:+:}$XDG_DATA_DIRS"
  export GTK_PATH="/gnu/store/7njbpmsgwmx59xn2r6q097iy2wwscb44-gtk+-3.24.30/lib/gtk-3.0${GTK_PATH:+:}$GTK_PATH"
  export PATH="$PATH${PATH:+:}/gnu/store/0c1yfbxyv877mlgychfgvmk5ha2jqh52-gzip-1.10/bin:/gnu/store/8fpk2cja3f07xls48jfnpgrzrljpqivr-coreutils-8.32/bin"
  export EMACSLOADPATH="$EMACSLOADPATH${EMACSLOADPATH:+:}/gnu/store/w0ssipd06il3kvyvqihpmw4lwbhwwfq3-emacs-28.1/share/emacs/28.1/lisp"
  exec -a "$0" "/gnu/store/w0ssipd06il3kvyvqihpmw4lwbhwwfq3-emacs-28.1/bin/.emacs-28.1-real" "$@"

I am under the impression that they should stop setting EMACSLOADPATH
and should either invent their own variable to replace that or load
"guix-emacs.el" using simpler means as I suggested above.

What do you think?  I intend to raise the issue with the guix folks but
would first like to hear your opinion.

If I unset EMACSLOADPATH, then everything works as expected.

> Emacs has special support for running from the build tree, but in your
> case it somehow doesn't realize that.

As a side-note, the wrapper script that I previously used looked like
this:

  #!/bin/sh
  export EMACSLOADPATH="\
  /home/jonas/.guix-home/profile/share/emacs/site-lisp:\
  /home/jonas/src/emacs/emacs/lisp"
  exec -a "$0" "/home/jonas/src/emacs/emacs/src/emacs" "$@"

The build tree is at "/home/jonas/src/emacs/emacs"; I think the value I
set above is correct (correct me if I am wrong), so it seems that when
running from the build tree EMACSLOADPATH has to be unset; explicitly
doubling down on the defaults doesn't work.  I also tried with an empty
element.

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: 58073@debbugs.gnu.org
> Date: Sun, 25 Sep 2022 20:30:05 +0200
> 
> Installing emacs using guix sets EMACSLOADPATH in a file that is sourced
> by the shells startup file.  To me it looks like they are abusing this
> variable; it makes it impossible to use an emacs that wasn't installed
> using guix, without something to counter their modification.  I did that
> with a wrapper script.
> 
> The value that is set somewhere in the shell's init files is something
> like "/home/jonas/.guix-home/profile/share/emacs/site-lisp".  It doesn't
> contain an empty element, which would "stand for the default value of
> `load-path'", according to (info "(emacs)General Variables").
> 
> The side startup file is located inside this directory and contains
> 
>   (when (require 'guix-emacs nil t)
>     (guix-emacs-autoload-packages)
>     (advice-add 'package-load-all-descriptors :after
>                 #'guix-emacs-load-package-descriptors))
> 
> The file "guix-emacs.el" is located in the same directory.  So the
> purpose of setting EMACSLOADPATH seems to be to allow using `require' to
> load that file.  This doesn't seem right to me, they could just as well
> extend the load-path inside "site-lisp.el", or load their additional
> init file using `load'.
> 
> To deal with the incomplete EMACSLOADPATH, as set in the shell
> environment, they use a wrapper script named "emacs" and located on PATH:
> 
>   #!/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash
>   export XDG_DATA_DIRS="/gnu/store/h6qdfd4sjdg28jq16600hcjky0cfw9gx-shared-mime-info-1.15/share:/gnu/store/bnsf9il448hl5xjavbhq3rcx355svz2v-glib-2.70.2/share:/gnu/store/7njbpmsgwmx59xn2r6q097iy2wwscb44-gtk+-3.24.30/share:/gnu/store/w0ssipd06il3kvyvqihpmw4lwbhwwfq3-emacs-28.1/share${XDG_DATA_DIRS:+:}$XDG_DATA_DIRS"
>   export GTK_PATH="/gnu/store/7njbpmsgwmx59xn2r6q097iy2wwscb44-gtk+-3.24.30/lib/gtk-3.0${GTK_PATH:+:}$GTK_PATH"
>   export PATH="$PATH${PATH:+:}/gnu/store/0c1yfbxyv877mlgychfgvmk5ha2jqh52-gzip-1.10/bin:/gnu/store/8fpk2cja3f07xls48jfnpgrzrljpqivr-coreutils-8.32/bin"
>   export EMACSLOADPATH="$EMACSLOADPATH${EMACSLOADPATH:+:}/gnu/store/w0ssipd06il3kvyvqihpmw4lwbhwwfq3-emacs-28.1/share/emacs/28.1/lisp"
>   exec -a "$0" "/gnu/store/w0ssipd06il3kvyvqihpmw4lwbhwwfq3-emacs-28.1/bin/.emacs-28.1-real" "$@"
> 
> I am under the impression that they should stop setting EMACSLOADPATH
> and should either invent their own variable to replace that or load
> "guix-emacs.el" using simpler means as I suggested above.

I tend to agree.  I don't understand why they "invade" EMACSLOADPATH:
that is supposed to be left to the users.

> > Emacs has special support for running from the build tree, but in your
> > case it somehow doesn't realize that.
> 
> As a side-note, the wrapper script that I previously used looked like
> this:
> 
>   #!/bin/sh
>   export EMACSLOADPATH="\
>   /home/jonas/.guix-home/profile/share/emacs/site-lisp:\
>   /home/jonas/src/emacs/emacs/lisp"
>   exec -a "$0" "/home/jonas/src/emacs/emacs/src/emacs" "$@"
> 
> The build tree is at "/home/jonas/src/emacs/emacs"; I think the value I
> set above is correct (correct me if I am wrong), so it seems that when
> running from the build tree EMACSLOADPATH has to be unset; explicitly
> doubling down on the defaults doesn't work.  I also tried with an empty
> element.

Your EMACSLOADPATH is not entirely correct, I think: it doesn't
include the subdirectories of /home/jonas/src/emacs/emacs/lisp.

All in all, when you run Emacs either from the source tree, or from
the place where it was configured to be installed, there should be no
need to set EMACSLOADPATH, and doing so without being VERY careful
could indeed get you in trouble.

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:
>> I am under the impression that they should stop setting EMACSLOADPATH
>> and should either invent their own variable to replace that or load
>> "guix-emacs.el" using simpler means as I suggested above.
>
> I tend to agree.  I don't understand why they "invade" EMACSLOADPATH:
> that is supposed to be left to the users.
>
>> > Emacs has special support for running from the build tree, but in your
>> > case it somehow doesn't realize that.
>> 
>> As a side-note, the wrapper script that I previously used looked like
>> this:
>> 
>>   #!/bin/sh
>>   export EMACSLOADPATH="\
>>   /home/jonas/.guix-home/profile/share/emacs/site-lisp:\
>>   /home/jonas/src/emacs/emacs/lisp"
>>   exec -a "$0" "/home/jonas/src/emacs/emacs/src/emacs" "$@"
>> 
>> The build tree is at "/home/jonas/src/emacs/emacs"; I think the value I
>> set above is correct (correct me if I am wrong), so it seems that when
>> running from the build tree EMACSLOADPATH has to be unset; explicitly
>> doubling down on the defaults doesn't work.  I also tried with an empty
>> element.
>
> Your EMACSLOADPATH is not entirely correct, I think: it doesn't
> include the subdirectories of /home/jonas/src/emacs/emacs/lisp.
>
> All in all, when you run Emacs either from the source tree, or from
> the place where it was configured to be installed, there should be no
> need to set EMACSLOADPATH, and doing so without being VERY careful
> could indeed get you in trouble.

It probably will be a while until I submit a patch for Guix (am very
new to the distro).  Meanwhile I'll keep using a wrapper script.  In
case someone else runs into the same problem until I get around to it,
here is what I currently use:

  #!/bin/sh
  unset EMACSLOADPATH
  exec -a "emacs" "/home/jonas/src/emacs/emacs/src/emacs" "$@"

One problem with that approach is that the wrapper cannot be named
"emacs".  If it is named "emacs", then that somehow trips up Emacs
and it loads all the preloaded files explicitly during startup (so
it appears that it cannot find the pdumper file in this case).

Eli, this isn't terribly important, but I was wondering if there is
something you could do so the name of the wrapper does not matter?
There might be other legitimate uses of a wrapper around the binary
from the build directory, other than "I currently have not other
choice because of what my distro does".

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: 58073@debbugs.gnu.org
> Date: Sun, 02 Oct 2022 16:50:19 +0200
> 
>   #!/bin/sh
>   unset EMACSLOADPATH
>   exec -a "emacs" "/home/jonas/src/emacs/emacs/src/emacs" "$@"
> 
> One problem with that approach is that the wrapper cannot be named
> "emacs".  If it is named "emacs", then that somehow trips up Emacs
> and it loads all the preloaded files explicitly during startup (so
> it appears that it cannot find the pdumper file in this case).
> 
> Eli, this isn't terribly important, but I was wondering if there is
> something you could do so the name of the wrapper does not matter?
> There might be other legitimate uses of a wrapper around the binary
> from the build directory, other than "I currently have not other
> choice because of what my distro does".

One thing I can suggest is to use the --dump-file=FILE command-line
option inside the script to direct Emacs to its .pdmp file.  If you
don't do that, Emacs tries to intuit it using the value of argv[0] it
receives from the OS, and I guess your naming somehow trips that?

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> One problem with that approach is that the wrapper cannot be named
>> "emacs".
>
> One thing I can suggest is to use the --dump-file=FILE command-line
> option inside the script to direct Emacs to its .pdmp file.  If you
> don't do that, Emacs tries to intuit it using the value of argv[0] it
> receives from the OS, and I guess your naming somehow trips that?

Thanks for the hint, that might come in handy.  However, I think I made
a mistake when testing earlier; it now works when the wrapper is named
"emacs", despite not using --dump-file=FILE.

Please close, and sorry for the noise.

     Cheers,
     Jonas

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Stefan Kangas]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Please close,

Done.

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: 58073@debbugs.gnu.org
> Date: Sun, 02 Oct 2022 22:55:58 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> One problem with that approach is that the wrapper cannot be named
> >> "emacs".
> >
> > One thing I can suggest is to use the --dump-file=FILE command-line
> > option inside the script to direct Emacs to its .pdmp file.  If you
> > don't do that, Emacs tries to intuit it using the value of argv[0] it
> > receives from the OS, and I guess your naming somehow trips that?
> 
> Thanks for the hint, that might come in handy.  However, I think I made
> a mistake when testing earlier; it now works when the wrapper is named
> "emacs", despite not using --dump-file=FILE.
> 
> Please close, and sorry for the noise.

Done.

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Jonas Bernoulli]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> One problem with that approach is that the wrapper cannot be named
>>> "emacs".
>>
>> One thing I can suggest is to use the --dump-file=FILE command-line
>> option inside the script to direct Emacs to its .pdmp file.  If you
>> don't do that, Emacs tries to intuit it using the value of argv[0] it
>> receives from the OS, and I guess your naming somehow trips that?
>
> Thanks for the hint, that might come in handy.  However, I think I made
> a mistake when testing earlier; it now works when the wrapper is named
> "emacs", despite not using --dump-file=FILE.

For the record, I figured out why it "works now but didn't work before".

  cat
  #!/bin/sh
  unset EMACSLOADPATH
  exec -a "emacs" "/home/jonas/src/emacs/emacs/src/emacs" "$@"

*appears* to work as intended (by me), but in actuality it uses the
libraries from where the package is installed, *not* from the source
repository.  I didn't inspect closely enough before to notice this.
Also, the name of the wrapper script does not actually matter.  (Use
of --dump-file doesn't help.)

Of course that doesn't work if Emacs hasn't actually been installed.

For completeness sake I should mention that I have once more confirmed
that using a symlink works (the libraries from the repo are used), but
an alias does not work (the files from the installation are used).

> Please close, and sorry for the noise.

No need to reopen, as far as I am concerned, but I thought it was
worth mentioning this here.

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: 58073@debbugs.gnu.org
> Date: Mon, 03 Oct 2022 18:03:50 +0200
> 
>   cat
>   #!/bin/sh
>   unset EMACSLOADPATH
>   exec -a "emacs" "/home/jonas/src/emacs/emacs/src/emacs" "$@"
> 
> *appears* to work as intended (by me), but in actuality it uses the
> libraries from where the package is installed, *not* from the source
> repository.  I didn't inspect closely enough before to notice this.
> Also, the name of the wrapper script does not actually matter.  (Use
> of --dump-file doesn't help.)

Sorry, I don't understand: you installed Emacs _and_ invoke it from
the source tree?  Why would you do that?

> Of course that doesn't work if Emacs hasn't actually been installed.
> 
> For completeness sake I should mention that I have once more confirmed
> that using a symlink works (the libraries from the repo are used), but
> an alias does not work (the files from the installation are used).
> 
> > Please close, and sorry for the noise.
> 
> No need to reopen, as far as I am concerned, but I thought it was
> worth mentioning this here.

Now I'm confused: is the problem solved or isn't it?

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jonas Bernoulli <jonas@bernoul.li>
>> Cc: 58073@debbugs.gnu.org
>> Date: Mon, 03 Oct 2022 18:03:50 +0200
>> 
>>   cat
>>   #!/bin/sh
>>   unset EMACSLOADPATH
>>   exec -a "emacs" "/home/jonas/src/emacs/emacs/src/emacs" "$@"
>> 
>> *appears* to work as intended (by me), but in actuality it uses the
>> libraries from where the package is installed, *not* from the source
>> repository.  I didn't inspect closely enough before to notice this.
>> Also, the name of the wrapper script does not actually matter.  (Use
>> of --dump-file doesn't help.)
>
> Sorry, I don't understand: you installed Emacs _and_ invoke it from
> the source tree?  Why would you do that?

It's more complicated like that but let's say: if I hack on an Emacs
package, then I use Emacs from source, since I might hack to look at
the git history of an Emacs function I use in package; but if what I
do has nothing to do with Emacs, then I use the installed Emacs.

>> Of course that doesn't work if Emacs hasn't actually been installed.
>> 
>> For completeness sake I should mention that I have once more confirmed
>> that using a symlink works (the libraries from the repo are used), but
>> an alias does not work (the files from the installation are used).
>> 
>> > Please close, and sorry for the noise.
>> 
>> No need to reopen, as far as I am concerned, but I thought it was
>> worth mentioning this here.
>
> Now I'm confused: is the problem solved or isn't it?

I no longer need to use a wrapper to unset EMACSLOADPATH, because I now
unset that variable in a file that is being sourced not only by my login
shell but also by my display manager.  I can now launch emacs from a
program launcher that does not inherit the environment from a shell but
from the display manager.

(I had to switch display managers to accomplish that.  Previously I used
Gdm3 but that has fully embraced systemd and wayland by now and does not
source any of the files (such as ~/.xsession) that could traditionally
be used to set variable for all of an X11 session.

So after the switch to another display manager, I can now use a plain
old symlink again, and that works perfectly; the elisp libraries from
the repository are being used, when I launch the emacs binary that is
located inside emacs.git through a symlink.

Before I switched away from Gdm3, I had to use a wrapper script, but as
we found out that does not actually work.  For some reason that ends up
using the elisp libraries from where "make install" put them.  Or if
that has never been run, then emacs fails because it cannot find the
libraries where it (IMO falsely) expects to find them.  The same happens
when using a shell alias.

The question now is whether *you* consider this a bug worth fixing.
I would say it is, but I won't insist on it.

I hope that is a bit clearer now.

bug#58073: 29.0.50; Uninstalled emacs sends startup messages to stderr

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: 58073@debbugs.gnu.org
> Date: Mon, 03 Oct 2022 20:51:05 +0200
> 
> Before I switched away from Gdm3, I had to use a wrapper script, but as
> we found out that does not actually work.  For some reason that ends up
> using the elisp libraries from where "make install" put them.  Or if
> that has never been run, then emacs fails because it cannot find the
> libraries where it (IMO falsely) expects to find them.  The same happens
> when using a shell alias.
> 
> The question now is whether *you* consider this a bug worth fixing.
> I would say it is, but I won't insist on it.

It may or may not be a bug; it could be a feature.

When Emacs starts up it needs to decide where to find the pdumper file
and where to find the *.eln files.  It does that by looking at the
directory where its executable lives, if it can determine that via
argv[0].  But symlinks could trick that, and if you invoke Emacs
through wrapper scripts or aliases, that could make the above
impossible, because there are no valid leading directories in argv[0].
In that case, the startup code uses Plan B: it goes by its configured
installation location.  And since you have the same Emacs both
installed and uninstalled, you get what you see.  Or something like
that: to see what really happens, you need to step with a debugger
through the code of the load_pdump function, see where it finds the
pdumper file, and where it decides to look for *.eln (by constructing
the name in emacs_executable, whose leading directory is then used in
dump_do_dump_relocation under 'case RELOC_NATIVE_COMP_UNIT').