bug#34707: Failure to find the associated pdump file

bug#34707: Failure to find the associated pdump file

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I have a symlink from bin/emacs to emacs-git/build-feb-24/src/emacs.
It contains master from Feb 24, built.

When I run that by typing `emacs', with some other directory current,
it crashes: it gets error and exits when it tries to run
tty-set-up-initial-frame-faces, from init_display.
That is supposed to have been set up by associated Lisp files
which have not been run.

I think the cause is that Emacs failed to find the pdump file.

    Apparently that failure gives no error message!
    It must give an error message, since you can't get anything
    like what you want if you don't get the pdump file.

There is an annoying secondary problem: it exits leaving the tty set
up for Emacs (no echo).  It ought to restore the outside-Emacs
terminal settings before exiting.

The reason it fails to find the pdump file
is that it fails to implement this case

    If the executable file is a symbolic link, the program can find the
    link target and use its containing directory instead of the link's
    containing directory.

which is implemented for associated Lisp files in this code in emacs.c

	  /* If the Emacs executable is actually a link,
	     next try the dir that the link points into.  */
	  tem = Ffile_symlink_p (name);
	  if (!NILP (tem))
	    {
	      name = Fexpand_file_name (tem, dir);
	      dir = Ffile_name_directory (name);
	    }
	  else
	    break;

but that is not implemented for pdump files.


-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#34707: Failure to find the associated pdump file

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Fri, 01 Mar 2019 22:27:54 -0500
> 
> I have a symlink from bin/emacs to emacs-git/build-feb-24/src/emacs.
> It contains master from Feb 24, built.
> 
> When I run that by typing `emacs', with some other directory current,
> it crashes: it gets error and exits when it tries to run
> tty-set-up-initial-frame-faces, from init_display.
> That is supposed to have been set up by associated Lisp files
> which have not been run.
> 
> I think the cause is that Emacs failed to find the pdump file.
> 
>     Apparently that failure gives no error message!
>     It must give an error message, since you can't get anything
>     like what you want if you don't get the pdump file.

It is supposed to give an error message saying that it could not load
the dump file because it was not built for this Emacs executable.
Could you please step with a debugger through the function load_pdump,
called from the 'main' function, and see why that doesn't happen for
you?

> There is an annoying secondary problem: it exits leaving the tty set
> up for Emacs (no echo).  It ought to restore the outside-Emacs
> terminal settings before exiting.

If Emacs already set up the terminal, then I think it did find the
pdump file and loaded it successfully, because loading the pdump file
happens before the call to init_display.  So I don't think I have a
clear idea of what happens on your system.

> The reason it fails to find the pdump file
> is that it fails to implement this case
> 
>     If the executable file is a symbolic link, the program can find the
>     link target and use its containing directory instead of the link's
>     containing directory.

That is correct (and we will have to implement that before Emacs 27
goes to print), but I think there's something else at work in your
case.  Stepping through the code with a debugger should clarify the
situation.

Also, can you show the details of the crash, like the signal and the
C-level backtrace?

bug#34707: Failure to find the associated pdump file

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >     If the executable file is a symbolic link, the program can find the
  > >     link target and use its containing directory instead of the link's
  > >     containing directory.

  > That is correct (and we will have to implement that before Emacs 27
  > goes to print)

I guess I should postpone trying the new version until that is done.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#34707: Failure to find the associated pdump file

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: 34707@debbugs.gnu.org
> Date: Sat, 02 Mar 2019 21:58:54 -0500
> 
>   > >     If the executable file is a symbolic link, the program can find the
>   > >     link target and use its containing directory instead of the link's
>   > >     containing directory.
> 
>   > That is correct (and we will have to implement that before Emacs 27
>   > goes to print)
> 
> I guess I should postpone trying the new version until that is done.

You could use the new --dump-file command-line option.  Or maybe
symlink the pdump file as well -- does that work?

bug#34707: Failure to find the associated pdump file

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You could use the new --dump-file command-line option.  Or maybe
  > symlink the pdump file as well -- does that work?

For the moment I will use my previous build
in the circumstances where the new one won't start.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#34707: Failure to find the associated pdump file

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

load_pdump returns to main with PDUMPER_LOAD_FILE_NOT_FOUND.
I do not see any code there that ought to exit with an error message.
If there is some, where is it?

So main continues and tries to initialize Emacs.

Don't we want this?

    #ifdef HAVE_PDUMPER
      if (attempt_load_pdump)
	{
	  load_pdump (argc, argv);
	  fatal ("cannot find portable dumper memory file");
	}
    #endif

However, it should show the file name it looked for.
Why is it useful for load_pdump return if it fails?


Regarding the secondary bug:

  > If Emacs already set up the terminal,

It has not done so yet.

					  then I think it did find the
  > pdump file and loaded it successfully, because loading the pdump file
  > happens before the call to init_display.

main continues and calls init_display.

That gets to init_display_interactive, which calls init_tty at line 6138.
That turns off echo, right?  Then it calls init_faces_initial at line 6193,
which tries to call tty-set-up-initial-frame-faces.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#34707: Failure to find the associated pdump file

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: 34707@debbugs.gnu.org
> Date: Sat, 02 Mar 2019 21:58:54 -0500
> 
> load_pdump returns to main with PDUMPER_LOAD_FILE_NOT_FOUND.
> I do not see any code there that ought to exit with an error message.
> If there is some, where is it?

If the file is not found, Emacs behaves like temacs: it loads the Lisp
files before proceeding.  It sounds like in your case it didn't do tat
because it found no Lisp files, is that right?  Then perhaps this is
the situation we should diagnose.

bug#34707: Failure to find the associated pdump file

[Eli Zaretskii]

> Date: Sun, 03 Mar 2019 05:41:50 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 34707@debbugs.gnu.org
> 
> > From: Richard Stallman <rms@gnu.org>
> > Cc: 34707@debbugs.gnu.org
> > Date: Sat, 02 Mar 2019 21:58:54 -0500
> > 
> > load_pdump returns to main with PDUMPER_LOAD_FILE_NOT_FOUND.
> > I do not see any code there that ought to exit with an error message.
> > If there is some, where is it?
> 
> If the file is not found, Emacs behaves like temacs: it loads the Lisp
> files before proceeding.  It sounds like in your case it didn't do tat
> because it found no Lisp files, is that right?  Then perhaps this is
> the situation we should diagnose.

It turned out this was a bug, which also precluded invoking temacs
interactively, something that has always worked.  I have now fixed
this.

bug#34707: Failure to find the associated pdump file

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > load_pdump returns to main with PDUMPER_LOAD_FILE_NOT_FOUND.
  > > I do not see any code there that ought to exit with an error message.
  > > If there is some, where is it?

  > If the file is not found, Emacs behaves like temacs: it loads the Lisp
  > files before proceeding.  It sounds like in your case it didn't do tat
  > because it found no Lisp files, is that right?

It did find the Lisp directory -- that code is old and knows how to
follow the symlink.  But that's not the issue, because
"tty-set-up-initial-frame-faces" is supposed to be preloaded.

I think you said this problem is fixed now.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#34707: Failure to find the associated pdump file

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: 34707@debbugs.gnu.org
> Date: Sun, 03 Mar 2019 22:26:56 -0500
> 
>   > If the file is not found, Emacs behaves like temacs: it loads the Lisp
>   > files before proceeding.  It sounds like in your case it didn't do tat
>   > because it found no Lisp files, is that right?
> 
> It did find the Lisp directory -- that code is old and knows how to
> follow the symlink.  But that's not the issue, because
> "tty-set-up-initial-frame-faces" is supposed to be preloaded.
> 
> I think you said this problem is fixed now.

Yes, it was an unrelated bug, which went unnoticed because no one
tried "emacs -nw" in the situation where the pdump file cannot be
found.

bug#34707: Failure to find the associated pdump file

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: 34707@debbugs.gnu.org
> Date: Sat, 02 Mar 2019 21:58:54 -0500
> 
> Regarding the secondary bug:
> 
>   > If Emacs already set up the terminal,
> 
> It has not done so yet.
> 
> 					  then I think it did find the
>   > pdump file and loaded it successfully, because loading the pdump file
>   > happens before the call to init_display.
> 
> main continues and calls init_display.
> 
> That gets to init_display_interactive, which calls init_tty at line 6138.
> That turns off echo, right?  Then it calls init_faces_initial at line 6193,
> which tries to call tty-set-up-initial-frame-faces.

Thanks, I fixed that part.

bug#34707: Failure to find the associated pdump file

[Lars Ingebrigtsen]

I'm able to reproduce this bug easily enough:

[larsi@stories ~]$ ln -s ~/src/emacs/trunk/src/emacs /tmp/sym-emacs
[larsi@stories ~]$ /tmp/sym-emacs
Loading loadup.el (source)...
dump mode: nil
Using load-path (/usr/local/share/emacs/27.0.50/site-lisp /usr/local/share/emacs/site-lisp /home/larsi/src/emacs/trunk/lisp /home/larsi/src/emacs/trunk/lisp/emacs-lisp /home/larsi/src/emacs/trunk/lisp/progmodes /home/larsi/src/emacs/trunk/lisp/language /home/larsi/src/emacs/trunk/lisp/international /home/larsi/src/emacs/trunk/lisp/textmodes /home/larsi/src/emacs/trunk/lisp/vc)
Loading emacs-lisp/byte-run...
Loading emacs-lisp/byte-run...done
Loading emacs-lisp/backquote...
[etc]

Running ~/src/emacs/trunk/src/emacs directly works, but it's not
uncommon to have symlinks to the binary (somebody else just complained
to me about this)...  When Emacs is able to find the rest of itself,
it's odd that it's not finding the pdump file, surely?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#34707: Failure to find the associated pdump file

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 17 Jun 2019 20:56:13 +0200
> Cc: 34707@debbugs.gnu.org
> 
> When Emacs is able to find the rest of itself, it's odd that it's
> not finding the pdump file, surely?

It's odd only if you don't consider the details.  Emacs finds the rest
of itself by consing Lisp strings, but that technique is unusable when
looking for the pdump file, because we must find it before we can
allocate memory etc., so we could set the heap flags correctly.  See
the comments in 'main'.  It's a typical bootstrap problem.

Solution is surely possible, it's just that it isn't "more of the
same", i.e. we cannot simply reuse the code which finds, say,
data-directory and similar.

bug#34707: Failure to find the associated pdump file

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Running ~/src/emacs/trunk/src/emacs directly works, but it's not
  > uncommon to have symlinks to the binary (somebody else just complained
  > to me about this)...  When Emacs is able to find the rest of itself,
  > it's odd that it's not finding the pdump file, surely?

The code to find the Lisp files handles the symlink case,
and it still works just as it has worked for years.

However, finding the pdump file uses new code,
and that code doesn't try to handle this symlink case.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#34707: Failure to find the associated pdump file

[Eli Zaretskii]

This bug was fixed by Daniel in commit b9ac4f8.  Closing.