bug#47800: [native-comp] could not resolve realpath of "emacs"

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Dario Gjorgjevski]

Hi,

I install and symlink Emacs via Stow; i.e., Emacs is installed in
$HOME/.local/stow/emacs and then stowed into $HOME/.local.  Therefore,

  $HOME/.local/bin/emacs -> ../stow/emacs/bin/emacs
  $HOME/.local/libexec/emacs/28.0.50 -> ../../stow/emacs/libexec/emacs/28.0.50

and so on.  Since commit 0c1fc9d, running "emacs" in the terminal gives
me an error:

  could not resolve realpath of "emacs": No such file or directory

Also,

  $ which emacs
  $HOME/.local/bin/emacs

However, running "$HOME/.local/bin/emacs" works just fine, as does
"$HOME/.local/stow/emacs/bin/emacs".  "emacsclient" also runs fine.

There are no problems prior to commit 0c1fc9d.

Best regards,
Dario

-- 
$ keyserver=hkps://hkps.pool.sks-keyservers.net
$ keyid=744A4F0B4F1C9371
$ gpg --keyserver $keyserver --search-keys $keyid

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Phil Sainty]

Hi Dario.  See also:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44128

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Dario Gjorgjevski]

> Hi Dario.  See also:
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44128

Thanks Phil.  What Eli wrote in message#28 there describes the issue
very well.  I would also like to point out that

  $ cd $HOME/.local/bin
  $ emacs

works.  (Note that it's "emacs" and not "./emacs", but there are no
problems with the latter either.)  I guess this can be closed in favor
of bug#44128.

Best regards,
Dario

-- 
$ keyserver=hkps://hkps.pool.sks-keyservers.net
$ keyid=744A4F0B4F1C9371
$ gpg --keyserver $keyserver --search-keys $keyid

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Eli Zaretskii]

> From: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>
> Date: Thu, 15 Apr 2021 16:09:26 +0200
> 
> I install and symlink Emacs via Stow; i.e., Emacs is installed in
> $HOME/.local/stow/emacs and then stowed into $HOME/.local.  Therefore,
> 
>   $HOME/.local/bin/emacs -> ../stow/emacs/bin/emacs
>   $HOME/.local/libexec/emacs/28.0.50 -> ../../stow/emacs/libexec/emacs/28.0.50
> 
> and so on.  Since commit 0c1fc9d, running "emacs" in the terminal gives
> me an error:
> 
>   could not resolve realpath of "emacs": No such file or directory

I think this is a duplicate of bug#44128, in its latest incarnation.

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Eli Zaretskii]

> From: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>
> Date: Thu, 15 Apr 2021 16:26:43 +0200
> Cc: 47800@debbugs.gnu.org
> 
> > Hi Dario.  See also:
> >
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44128
> 
> Thanks Phil.  What Eli wrote in message#28 there describes the issue
> very well.

Does the untested patch below fix the problem?

diff --git a/src/emacs.c b/src/emacs.c
index a256564..7eaaa28 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -846,7 +846,11 @@ load_pdump_find_executable (char const *argv0, ptrdiff_t *candidate_size)
       struct stat st;
       if (file_access_p (candidate, X_OK)
 	  && stat (candidate, &st) == 0 && S_ISREG (st.st_mode))
-	return candidate;
+	{
+	  if (lstat (candidate, &st) == 0 && S_ISLNK (st.st_mode))
+	    return realpath (candidate, NULL);
+	  return candidate;
+	}
       *candidate = '\0';
     }
   while (*path++ != '\0');

bug#44128: bug#47800: [native-comp] could not resolve realpath of "emacs"

[Phil Sainty]

[-- Attachment #1: Type: text/plain, Size: 2275 bytes --]

On 16/04/21 3:08 am, Eli Zaretskii wrote (on bug 47800):
> Does the untested patch below fix the problem?

That didn't change the outcomes for me.

I've just tried some very basic debugging with the attached patch
(yours plus some debug output) which gives me the following outputs:


$ cd /tmp

$ ls emacs
ls: cannot access 'emacs': No such file or directory


$ emacs --version
load_pdump_find_executable: candidate: /home/phil/bin/emacs
=> file: /home/phil/bin/emacs
=> (link) realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
load_pdump: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
real_filename( /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50 )
=> realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
set_invocation_vars: emacs
real_filename( emacs )
=> realpath: (null)
emacs: could not resolve realpath of "emacs": No such file or directory


$ mkdir emacs

$ emacs --version
load_pdump_find_executable: candidate: /home/phil/bin/emacs
=> file: /home/phil/bin/emacs
=> (link) realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
load_pdump: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
real_filename( /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50 )
=> realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
set_invocation_vars: emacs
real_filename( emacs )
=> realpath: /tmp/emacs
emacs: /tmp/../native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln: cannot open shared object file: No such file or directory


$ /home/phil/emacs/native-comp/usr/local/bin/emacs --version
load_pdump: /home/phil/emacs/native-comp/usr/local/bin/emacs
real_filename( /home/phil/emacs/native-comp/usr/local/bin/emacs )
=> realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
set_invocation_vars: /home/phil/emacs/native-comp/usr/local/bin/emacs
real_filename( /home/phil/emacs/native-comp/usr/local/bin/emacs )
=> realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
GNU Emacs 28.0.50
Copyright (C) 2021 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.


[-- Attachment #2: bug47800.patch --]
[-- Type: text/x-patch, Size: 2212 bytes --]

diff --git a/src/emacs.c b/src/emacs.c
index a256564..0503504 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -446,6 +446,7 @@ terminate_due_to_signal (int sig, int backtrace_limit)
 static char *
 real_filename (char *filename)
 {
+  fprintf (stderr, "real_filename( %s )\n", filename);
   char *real_name;
 #ifdef WINDOWSNT
   /* w32_my_exename resolves symlinks internally, so no need to
@@ -453,6 +454,7 @@ real_filename (char *filename)
   real_name = xstrdup (filename);
 #else
   real_name = realpath (filename, NULL);
+  fprintf (stderr, "=> realpath: %s\n", real_name);
   if (!real_name)
     fatal ("could not resolve realpath of \"%s\": %s",
 	   filename, strerror (errno));
@@ -495,6 +497,7 @@ set_invocation_vars (char *argv0, char const *original_pwd)
   if (! NILP (handler))
     raw_name = concat2 (slash_colon, raw_name);
 
+  fprintf (stderr, "set_invocation_vars: %s\n", SSDATA (raw_name));
   char *filename = real_filename (SSDATA (raw_name));
   raw_name = build_unibyte_string (filename);
   xfree (filename);
@@ -843,10 +846,18 @@ load_pdump_find_executable (char const *argv0, ptrdiff_t *candidate_size)
       memcpy (candidate + 0, path_part, path_part_length);
       candidate[path_part_length] = DIRECTORY_SEP;
       memcpy (candidate + path_part_length + 1, argv0, argv0_length + 1);
+      fprintf (stderr, "load_pdump_find_executable: candidate: %s\n", candidate);
       struct stat st;
       if (file_access_p (candidate, X_OK)
 	  && stat (candidate, &st) == 0 && S_ISREG (st.st_mode))
-	return candidate;
+	{
+	  fprintf (stderr, "=> file: %s\n", candidate);
+	  if (lstat (candidate, &st) == 0 && S_ISLNK (st.st_mode)) {
+	    fprintf (stderr, "=> (link) realpath: %s\n", realpath (candidate, NULL));
+	    return realpath (candidate, NULL);
+	  }
+	  return candidate;
+	}
       *candidate = '\0';
     }
   while (*path++ != '\0');
@@ -912,6 +923,7 @@ load_pdump (int argc, char **argv, char const *original_pwd)
      the dump in the hardcoded location.  */
   if (dump_file && *dump_file)
     {
+      fprintf (stderr, "load_pdump: %s\n", dump_file);
       char *real_exename = real_filename (dump_file);
       xfree (dump_file);
       dump_file = real_exename;

bug#44128: #44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> Cc: 44128@debbugs.gnu.org
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Fri, 16 Apr 2021 17:41:52 +1200
> 
> On 16/04/21 3:08 am, Eli Zaretskii wrote (on bug 47800):
> > Does the untested patch below fix the problem?
> 
> That didn't change the outcomes for me.

But it moves one step closer, AFAICT: the real file name of the Emacs
executable is now correctly identified.

> I've just tried some very basic debugging with the attached patch
> (yours plus some debug output) which gives me the following outputs:
> 
> 
> $ cd /tmp
> 
> $ ls emacs
> ls: cannot access 'emacs': No such file or directory
> 
> 
> $ emacs --version
> load_pdump_find_executable: candidate: /home/phil/bin/emacs
> => file: /home/phil/bin/emacs
> => (link) realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> load_pdump: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> real_filename( /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50 )
> => realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> set_invocation_vars: emacs
> real_filename( emacs )
> => realpath: (null)
> emacs: could not resolve realpath of "emacs": No such file or directory
> 
> 
> $ mkdir emacs
> 
> $ emacs --version
> load_pdump_find_executable: candidate: /home/phil/bin/emacs
> => file: /home/phil/bin/emacs
> => (link) realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> load_pdump: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> real_filename( /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50 )
> => realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> set_invocation_vars: emacs
> real_filename( emacs )
> => realpath: /tmp/emacs
> emacs: /tmp/../native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln: cannot open shared object file: No such file or directory
> 
> 
> $ /home/phil/emacs/native-comp/usr/local/bin/emacs --version
> load_pdump: /home/phil/emacs/native-comp/usr/local/bin/emacs
> real_filename( /home/phil/emacs/native-comp/usr/local/bin/emacs )
> => realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> set_invocation_vars: /home/phil/emacs/native-comp/usr/local/bin/emacs
> real_filename( /home/phil/emacs/native-comp/usr/local/bin/emacs )
> => realpath: /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> GNU Emacs 28.0.50

Thanks, but this doesn't provide enough information to understand how
this Emacs was configured and installed, and AFAICS you didn't supply
that information up-thread.  So:

  . how was Emacs configured? with what value of --prefix (or other
    related directory variables, like libdir etc)?
  . was Emacs installed, and if so, with what command?
  . what is the full absolute file name of the native-lisp/ directory
    that this Emacs binary was supposed to find and load?
  . what is the full absolute file name of the pdumper file for this
    Emacs binary?

P.S. To avoid confusion, please use the Subject line I used, not the
one which mentions bug#48700.

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Dario Gjorgjevski]

[-- Attachment #1: Type: text/plain, Size: 325 bytes --]

> Does the untested patch below fix the problem?

Hi Eli,

Thanks for the patch.  Unfortunately it does not help as the raw_name
variable in set_invocation_vars is still not resolved according to PATH.

The patch I am attaching to this message _does_ resolve the issue, but I
am not sure about Windows.

Best regards,
Dario


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Resolve raw_name according to PATH --]
[-- Type: text/x-diff, Size: 6243 bytes --]

diff --git a/src/emacs.c b/src/emacs.c
index a256564..16fdc3c 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -460,8 +460,81 @@ real_filename (char *filename)
   return real_name;
 }
 
-/* Set `invocation-name' `invocation-directory'.  */
+/* Find a name (absolute or relative) of the Emacs executable whose
+   name (as passed into this program) is ARGV0.  Called early in
+   initialization by portable dumper loading code, so avoid Lisp and
+   associated machinery.  Return a heap-allocated string giving a name
+   of the Emacs executable, or an empty heap-allocated string or NULL
+   if not found.  Store into *CANDIDATE_SIZE a lower bound on the size
+   of any heap allocation.  */
+static char *
+find_executable (char const *argv0, ptrdiff_t *candidate_size)
+{
+  *candidate_size = 0;
+
+  /* Use xstrdup etc. to allocate storage, so as to call our private
+     implementation of malloc, since the caller calls our free.  */
+#ifdef WINDOWSNT
+  char *prog_fname = w32_my_exename ();
+  return prog_fname ? xstrdup (prog_fname) : NULL;
+#else  /* !WINDOWSNT */
+  char *candidate = NULL;
+
+  /* If the executable name contains a slash, we have some kind of
+     path already, so just copy it.  */
+  eassert (argv0);
+  if (strchr (argv0, DIRECTORY_SEP))
+    return xstrdup (argv0);
+  ptrdiff_t argv0_length = strlen (argv0);
+
+  const char *path = getenv ("PATH");
+  if (!path)
+    {
+      /* Default PATH is implementation-defined, so we don't know how
+         to conduct the search.  */
+      return NULL;
+    }
+
+  /* Actually try each concatenation of a path element and the
+     executable basename.  */
+  do
+    {
+      static char const path_sep[] = { SEPCHAR, '\0' };
+      ptrdiff_t path_part_length = strcspn (path, path_sep);
+      const char *path_part = path;
+      path += path_part_length;
+      if (path_part_length == 0)
+        {
+          path_part = ".";
+          path_part_length = 1;
+        }
+      ptrdiff_t needed = path_part_length + 1 + argv0_length + 1;
+      if (*candidate_size <= needed)
+	{
+	  xfree (candidate);
+	  candidate = xpalloc (NULL, candidate_size,
+			       needed - *candidate_size + 1, -1, 1);
+	}
+      memcpy (candidate + 0, path_part, path_part_length);
+      candidate[path_part_length] = DIRECTORY_SEP;
+      memcpy (candidate + path_part_length + 1, argv0, argv0_length + 1);
+      struct stat st;
+      if (file_access_p (candidate, X_OK)
+	  && stat (candidate, &st) == 0 && S_ISREG (st.st_mode))
+	{
+	  if (lstat (candidate, &st) == 0 && S_ISLNK (st.st_mode))
+	    return realpath (candidate, NULL);
+	  return candidate;
+	}
+      *candidate = '\0';
+    }
+  while (*path++ != '\0');
+
+  return candidate;
+#endif	/* !WINDOWSNT */
+}
 
+/* Set `invocation-name' `invocation-directory'.  */
 static void
 set_invocation_vars (char *argv0, char const *original_pwd)
 {
@@ -486,7 +559,9 @@ set_invocation_vars (char *argv0, char const *original_pwd)
       raw_name = build_unibyte_string (argv0);
   }
 #else
-  raw_name = build_unibyte_string (argv0);
+  ptrdiff_t bufsize;
+  char *executable = find_executable (argv0, &bufsize);
+  raw_name = build_unibyte_string (executable);
 #endif
 
   /* Add /: to the front of the name
@@ -785,76 +860,6 @@ dump_error_to_string (int result)
     }
 }
 
-/* Find a name (absolute or relative) of the Emacs executable whose
-   name (as passed into this program) is ARGV0.  Called early in
-   initialization by portable dumper loading code, so avoid Lisp and
-   associated machinery.  Return a heap-allocated string giving a name
-   of the Emacs executable, or an empty heap-allocated string or NULL
-   if not found.  Store into *CANDIDATE_SIZE a lower bound on the size
-   of any heap allocation.  */
-static char *
-load_pdump_find_executable (char const *argv0, ptrdiff_t *candidate_size)
-{
-  *candidate_size = 0;
-
-  /* Use xstrdup etc. to allocate storage, so as to call our private
-     implementation of malloc, since the caller calls our free.  */
-#ifdef WINDOWSNT
-  char *prog_fname = w32_my_exename ();
-  return prog_fname ? xstrdup (prog_fname) : NULL;
-#else  /* !WINDOWSNT */
-  char *candidate = NULL;
-
-  /* If the executable name contains a slash, we have some kind of
-     path already, so just copy it.  */
-  eassert (argv0);
-  if (strchr (argv0, DIRECTORY_SEP))
-    return xstrdup (argv0);
-  ptrdiff_t argv0_length = strlen (argv0);
-
-  const char *path = getenv ("PATH");
-  if (!path)
-    {
-      /* Default PATH is implementation-defined, so we don't know how
-         to conduct the search.  */
-      return NULL;
-    }
-
-  /* Actually try each concatenation of a path element and the
-     executable basename.  */
-  do
-    {
-      static char const path_sep[] = { SEPCHAR, '\0' };
-      ptrdiff_t path_part_length = strcspn (path, path_sep);
-      const char *path_part = path;
-      path += path_part_length;
-      if (path_part_length == 0)
-        {
-          path_part = ".";
-          path_part_length = 1;
-        }
-      ptrdiff_t needed = path_part_length + 1 + argv0_length + 1;
-      if (*candidate_size <= needed)
-	{
-	  xfree (candidate);
-	  candidate = xpalloc (NULL, candidate_size,
-			       needed - *candidate_size + 1, -1, 1);
-	}
-      memcpy (candidate + 0, path_part, path_part_length);
-      candidate[path_part_length] = DIRECTORY_SEP;
-      memcpy (candidate + path_part_length + 1, argv0, argv0_length + 1);
-      struct stat st;
-      if (file_access_p (candidate, X_OK)
-	  && stat (candidate, &st) == 0 && S_ISREG (st.st_mode))
-	return candidate;
-      *candidate = '\0';
-    }
-  while (*path++ != '\0');
-
-  return candidate;
-#endif	/* !WINDOWSNT */
-}
-
 static void
 load_pdump (int argc, char **argv, char const *original_pwd)
 {
@@ -906,7 +911,7 @@ load_pdump (int argc, char **argv, char const *original_pwd)
      encoding the system natively uses for filesystem access, so
      there's no need for character set conversion.  */
   ptrdiff_t bufsize;
-  dump_file = load_pdump_find_executable (argv[0], &bufsize);
+  dump_file = find_executable (argv[0], &bufsize);
 
   /* If we couldn't find our executable, go straight to looking for
      the dump in the hardcoded location.  */

[-- Attachment #3: Type: text/plain, Size: 127 bytes --]

-- 
$ keyserver=hkps://hkps.pool.sks-keyservers.net
$ keyid=744A4F0B4F1C9371
$ gpg --keyserver $keyserver --search-keys $keyid

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Dario Gjorgjevski]

I suppose one should also check the return value of find_executable
before using it for raw_name.

-- 
$ keyserver=hkps://hkps.pool.sks-keyservers.net
$ keyid=744A4F0B4F1C9371
$ gpg --keyserver $keyserver --search-keys $keyid

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Dario Gjorgjevski]

Also, the more I look at the patch I attached above, the worse it seems,
so...

But I can at least confirm that the issue is that raw_name is not
resolved according to PATH prior to the call of real_filename in

  char *filename = real_filename (SSDATA (raw_name));

Best regards,
Dario

-- 
$ keyserver=hkps://hkps.pool.sks-keyservers.net
$ keyid=744A4F0B4F1C9371
$ gpg --keyserver $keyserver --search-keys $keyid

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dario Gjorgjevski <dario.gjorgjevski@gmail.com> writes:

Hi Dario, thanks for the patch.

> Also, the more I look at the patch I attached above, the worse it seems,
> so...

Why?

> But I can at least confirm that the issue is that raw_name is not
> resolved according to PATH prior to the call of real_filename in
>
>   char *filename = real_filename (SSDATA (raw_name));

I agree that's the issue.

Thanks

  Andrea

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Dario Gjorgjevski]

Hi Andrea,

>> Also, the more I look at the patch I attached above, the worse it seems,
>> so...
>
> Why?

These are the points that make me suspicious:

  1. I am not sure if the subsequent call to Ffind_file_name_handler
     would work correctly.
  2. There is already a search over exec_path happening immediately
     afterwards.
  3. Moreover, I am not sure what real_filename would do when a handler
     is found and raw_name is prefixed by slash-colon.

Best regards,
Dario

-- 
$ keyserver=hkps://hkps.pool.sks-keyservers.net
$ keyid=744A4F0B4F1C9371
$ gpg --keyserver $keyserver --search-keys $keyid

bug#47800: [native-comp] could not resolve realpath of "emacs"

[wilde]

Hi *,

FWIW: when /proc/self/exe exists it is a symlink to the currently
running executable, that should make the implementation of
find_executable() in these cases straight forward.

Basically:
  realpath ("/proc/self/exe", NULL);
should do the job.

/proc/self/run exists per default on most (all?) GNU/Linux Systems and
on my i386 NetBSD 9.1 per default.  In case of *BSD this might be a
Linux compatibility feature but I'm not sure -- needs checking.

cheers
sascha

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Phil Sainty]

On 16/04/21 9:27 pm, Andrea Corallo wrote:
> Dario Gjorgjevski <dario.gjorgjevski@gmail.com> writes:
>> But I can at least confirm that the issue is that raw_name is not
>> resolved according to PATH prior to the call of real_filename in
>>
>>   char *filename = real_filename (SSDATA (raw_name));
> 
> I agree that's the issue.

My (perhaps naive) impression is that set_invocation_vars should
be using load_pdump_find_executable (or a copy of the result that
it already established) to establish what argv0 is referring to?

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46790#38 may have
been driving at the same point.

In https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44128#49 I'm
seeing load_pdump_find_executable successfully figuring out the
genuine path to the executable every time.

bug#44128: #44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Phil Sainty]

On 16/04/21 6:41 pm, Eli Zaretskii wrote:
> Thanks, but this doesn't provide enough information to understand how
> this Emacs was configured and installed, and AFAICS you didn't supply
> that information up-thread.

It got a bit lost in the scattering of bug IDs, but the following
was my M-x report-emacs-bug from the native-comp Emacs I compiled
initially:

https://lists.gnu.org/archive/html/bug-gnu-emacs/2021-04/msg00560.html

For the subsequent testing I just re-ran 'make install' after applying
each patch.

>   . how was Emacs configured? with what value of --prefix (or other
>     related directory variables, like libdir etc)?

Configured using:
 'configure --prefix=/home/phil/emacs/native-comp/usr/local
 --with-x-toolkit=lucid --without-sound --with-native-compilation'

>   . was Emacs installed, and if so, with what command?

make install

>   . what is the full absolute file name of the native-lisp/ directory
>     that this Emacs binary was supposed to find and load?

Seems like this:

/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/28.0.50-abd7aa58/

>   . what is the full absolute file name of the pdumper file for this
>     Emacs binary?

Seems like this:

/home/phil/emacs/native-comp/usr/local/libexec/emacs/28.0.50/x86_64-pc-linux-gnu/emacs.pdmp

(I'm making assumptions about the last two, but they're the only
possibilities that I can find under my --prefix dir.)

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Phil Sainty]

On 16/04/21 10:56 pm, Phil Sainty wrote:
> In https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44128#49 I'm
> seeing load_pdump_find_executable successfully figuring out the
> genuine path to the executable every time.

Clarification: It's only figuring it out on account of Eli's patch:
https://lists.gnu.org/archive/html/bug-gnu-emacs/2021-04/msg00678.html

So as Eli said subsequently: "It moves one step closer, AFAICT: the
real file name of the Emacs executable is now correctly identified."
-- https://lists.gnu.org/archive/html/bug-gnu-emacs/2021-04/msg00715.html

So IIUC, firstly we want that patch, and *then* that result can be
used by set_invocation_vars ?

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, psainty@orcon.net.nz, 47800@debbugs.gnu.org
> Date: Fri, 16 Apr 2021 09:27:51 +0000
> 
> > But I can at least confirm that the issue is that raw_name is not
> > resolved according to PATH prior to the call of real_filename in
> >
> >   char *filename = real_filename (SSDATA (raw_name));
> 
> I agree that's the issue.

I think the issue is slightly more complex.

Andrea, is native-compilation supported with unexec, or only with
pdumper?  If the former, where in the code and when do we load the
preloaded *.eln files? are they dumped into the Emacs executable?

If we don't support native-compilation with unexec, we should reject
that combination in configure, right?

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, psainty@orcon.net.nz, 47800@debbugs.gnu.org
>> Date: Fri, 16 Apr 2021 09:27:51 +0000
>> 
>> > But I can at least confirm that the issue is that raw_name is not
>> > resolved according to PATH prior to the call of real_filename in
>> >
>> >   char *filename = real_filename (SSDATA (raw_name));
>> 
>> I agree that's the issue.
>
> I think the issue is slightly more complex.

That's very possible today I had really no time to reproduce and
investigate this in details sorry.

> Andrea, is native-compilation supported with unexec, or only with
> pdumper?

Only pdumper.

> If the former, where in the code and when do we load the
> preloaded *.eln files? are they dumped into the Emacs executable?

We re-load the eln for each compilation unit around pdumper.c:5277.  The
position of the eln is already stored in CU object (formed in
loadup.el:467).

> If we don't support native-compilation with unexec, we should reject
> that combination in configure, right?

That's correct.

Thanks

  Andrea

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: dario.gjorgjevski@gmail.com, psainty@orcon.net.nz, 47800@debbugs.gnu.org
> Date: Fri, 16 Apr 2021 19:23:51 +0000
> 
> >> >   char *filename = real_filename (SSDATA (raw_name));
> >> 
> >> I agree that's the issue.
> >
> > I think the issue is slightly more complex.
> 
> That's very possible today I had really no time to reproduce and
> investigate this in details sorry.
> 
> > Andrea, is native-compilation supported with unexec, or only with
> > pdumper?
> 
> Only pdumper.

OK, that simplifies the solution.

The problem we need to solve is that we have two decisions that run in
parallel, but are not really synchronized: where to look for the
pdumper file and where to look for the *.eln files.  These two must
correspond to each other, but currently they have little in common,
which is a problem waiting to bite us.

The other issue is the reliance on Vinvocation_directory.  To rely on
it, we must change how and when it is computed, and that will most
probably change its value in some use cases.  So I think we need to
leave the Vinvocation_directory calculation as it was before, in
init_cmdargs, and use other variables to tell pdumper_load how to
resolve the native-lisp directory when restoring from the pdumper
file.

I'm currently working on making these changes, and hope to have a
solution which will solve all the known issues with symlinks etc.

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: dario.gjorgjevski@gmail.com, psainty@orcon.net.nz, 47800@debbugs.gnu.org
>> Date: Fri, 16 Apr 2021 19:23:51 +0000
>> 
>> >> >   char *filename = real_filename (SSDATA (raw_name));
>> >> 
>> >> I agree that's the issue.
>> >
>> > I think the issue is slightly more complex.
>> 
>> That's very possible today I had really no time to reproduce and
>> investigate this in details sorry.
>> 
>> > Andrea, is native-compilation supported with unexec, or only with
>> > pdumper?
>> 
>> Only pdumper.
>
> OK, that simplifies the solution.
>
> The problem we need to solve is that we have two decisions that run in
> parallel, but are not really synchronized: where to look for the
> pdumper file and where to look for the *.eln files.  These two must
> correspond to each other, but currently they have little in common,
> which is a problem waiting to bite us.
>
> The other issue is the reliance on Vinvocation_directory.  To rely on
> it, we must change how and when it is computed, and that will most
> probably change its value in some use cases.  So I think we need to
> leave the Vinvocation_directory calculation as it was before, in
> init_cmdargs, and use other variables to tell pdumper_load how to
> resolve the native-lisp directory when restoring from the pdumper
> file.
>
> I'm currently working on making these changes, and hope to have a
> solution which will solve all the known issues with symlinks etc.

Thanks, I'm sorry in the last day I could not help more with this issue.

  Andrea

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: dario.gjorgjevski@gmail.com, psainty@orcon.net.nz, 47800@debbugs.gnu.org
> Date: Fri, 16 Apr 2021 20:07:01 +0000
> 
> > I'm currently working on making these changes, and hope to have a
> > solution which will solve all the known issues with symlinks etc.
> 
> Thanks, I'm sorry in the last day I could not help more with this issue.

That's okay, we should have done this earlier.  It does mean we will
postpone the merge of the branch for a few days, as I'd like to see
that at least most of the issues with symlinks and other "unusual"
installation methods work.

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: dario.gjorgjevski@gmail.com, psainty@orcon.net.nz, 47800@debbugs.gnu.org
>> Date: Fri, 16 Apr 2021 20:07:01 +0000
>> 
>> > I'm currently working on making these changes, and hope to have a
>> > solution which will solve all the known issues with symlinks etc.
>> 
>> Thanks, I'm sorry in the last day I could not help more with this issue.
>
> That's okay, we should have done this earlier.  It does mean we will
> postpone the merge of the branch for a few days, as I'd like to see
> that at least most of the issues with symlinks and other "unusual"
> installation methods work.

That's sensible, thanks

  Andrea

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Eli Zaretskii]

> From: wilde@sha-bang.de
> Date: Fri, 16 Apr 2021 11:53:58 +0200
> 
> FWIW: when /proc/self/exe exists it is a symlink to the currently
> running executable, that should make the implementation of
> find_executable() in these cases straight forward.
> 
> Basically:
>   realpath ("/proc/self/exe", NULL);
> should do the job.

This was discussed, but AFAIR people felt uneasy with using it.

bug#47800: [native-comp] could not resolve realpath of "emacs"

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: dario.gjorgjevski@gmail.com, psainty@orcon.net.nz, 47800@debbugs.gnu.org
> Date: Fri, 16 Apr 2021 19:23:51 +0000
> 
> > If we don't support native-compilation with unexec, we should reject
> > that combination in configure, right?
> 
> That's correct.

Oh, I see we already do that.  I guess that's what you were trying to
say, but I misunderstood...

Thanks.

bug#47800: [native-comp] could not resolve realpath of "emacs"

[wilde]

Eli Zaretskii <eliz@gnu.org> wrote:
>> From: wilde@sha-bang.de
>> Date: Fri, 16 Apr 2021 11:53:58 +0200
>> 
>> FWIW: when /proc/self/exe exists it is a symlink to the currently
>> running executable, that should make the implementation of
>> find_executable() in these cases straight forward.
>> 
>> Basically:
>>   realpath ("/proc/self/exe", NULL);
>> should do the job.
>
> This was discussed, but AFAIR people felt uneasy with using it.

I wonder why -- but thats not too important as long as we have a
reliable solution anyway.  :)