bug#44128: [feature/native-comp]

bug#44128: [feature/native-comp]

[Jonas Bernoulli]

Hello Andrea

[We talked about this briefly on Twitter;
 https://twitter.com/magit_emacs/status/1313534891506757635.]

When running gccemacs from the git repository without installing but by
using a symlink like e.g. /usr/local/bin/emacs -> ~/git/emacs/src/emacs,
then that results in an error like:

  emacs: /usr/local/bin/../native-lisp/<snip>.eln: cannot open shared
  object file: No such file or directory

You mentioned that this happens because code in pdump[er].c just relies
on invocation-directory and that you are wonder whether symlinks should
be followed to address this.

  I think they should. :D
  Please have a look.
  Thanks!
  Jonas

bug#44128: [feature/native-comp]

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Hello Andrea

Hi Jonas!

> [We talked about this briefly on Twitter;
>  https://twitter.com/magit_emacs/status/1313534891506757635.]
>
> When running gccemacs from the git repository without installing but by
> using a symlink like e.g. /usr/local/bin/emacs -> ~/git/emacs/src/emacs,
> then that results in an error like:
>
>   emacs: /usr/local/bin/../native-lisp/<snip>.eln: cannot open shared
>   object file: No such file or directory
>
> You mentioned that this happens because code in pdump[er].c just relies
> on invocation-directory and that you are wonder whether symlinks should
> be followed to address this.
>
>   I think they should. :D

Well I agree :)

I believe the only question is if we want to change
`invocation-directory' to have the link followed or to do that in the
load mechanism.  I guess the second has less impact.

  Andrea

bug#44128: [feature/native-comp]

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo <akrl@sdf.org> writes:

> Jonas Bernoulli <jonas@bernoul.li> writes:
>
>> Hello Andrea
>
> Hi Jonas!
>
>> [We talked about this briefly on Twitter;
>>  https://twitter.com/magit_emacs/status/1313534891506757635.]
>>
>> When running gccemacs from the git repository without installing but by
>> using a symlink like e.g. /usr/local/bin/emacs -> ~/git/emacs/src/emacs,
>> then that results in an error like:
>>
>>   emacs: /usr/local/bin/../native-lisp/<snip>.eln: cannot open shared
>>   object file: No such file or directory
>>
>> You mentioned that this happens because code in pdump[er].c just relies
>> on invocation-directory and that you are wonder whether symlinks should
>> be followed to address this.
>>
>>   I think they should. :D
>
> Well I agree :)
>
> I believe the only question is if we want to change
> `invocation-directory' to have the link followed or to do that in the
> load mechanism.  I guess the second has less impact.

Hi all,

I think this question got answered by Eli's comment on bug#46790 :)

I've pushed 0c1fc9d581 that seams to work for me, please have a try.

Thanks!

  Andrea

bug#44128: [feature/native-comp]

[Johannes Grødem]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

>>> You mentioned that this happens because code in pdump[er].c just relies
>>> on invocation-directory and that you are wonder whether symlinks should
>>> be followed to address this.
>>>
>>>   I think they should. :D
>>
>> Well I agree :)
>>
>> I believe the only question is if we want to change
>> `invocation-directory' to have the link followed or to do that in the
>> load mechanism.  I guess the second has less impact.
>
> Hi all,
>
> I think this question got answered by Eli's comment on bug#46790 :)
>
> I've pushed 0c1fc9d581 that seams to work for me, please have a try.

This fixed it for me. Thanks!

-- 
Sent from my Emacs

bug#44128: [feature/native-comp]

[Phil Sainty]

On 15/04/21 1:48 am, Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors wrote:
> I think this question got answered by Eli's comment on bug#46790 :)
> 
> I've pushed 0c1fc9d581 that seams to work for me, please have a try.

I now get this behaviour:

$ emacs-native-comp -Q
emacs: could not resolve realpath of "emacs-native-comp": No such file or directory

$ ls -l `which emacs-native-comp`
lrwxrwxrwx 1 phil phil 48 Apr 14 00:01 /home/phil/bin/emacs-native-comp -> /home/phil/emacs/native-comp/usr/local/bin/emacs

$ /home/phil/emacs/native-comp/usr/local/bin/emacs --version
GNU Emacs 28.0.50
Copyright (C) 2021 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

$ emacs-native-comp --version
emacs: could not resolve realpath of "emacs-native-comp": No such file or directory

bug#44128: [feature/native-comp]

[Eli Zaretskii]

> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Thu, 15 Apr 2021 10:22:29 +1200
> Cc: Jonas Bernoulli <jonas@bernoul.li>, 44128@debbugs.gnu.org, eli@gnu.org
> 
> On 15/04/21 1:48 am, Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors wrote:
> > I think this question got answered by Eli's comment on bug#46790 :)
> > 
> > I've pushed 0c1fc9d581 that seams to work for me, please have a try.
> 
> I now get this behaviour:
> 
> $ emacs-native-comp -Q
> emacs: could not resolve realpath of "emacs-native-comp": No such file or directory
> 
> $ ls -l `which emacs-native-comp`
> lrwxrwxrwx 1 phil phil 48 Apr 14 00:01 /home/phil/bin/emacs-native-comp -> /home/phil/emacs/native-comp/usr/local/bin/emacs

Where in the Emacs sources does this message come from, please?  Bonus
points for explaining the reason(s) for that failure.

What happens if you call the symlink 'emacs' and not
'emacs-native-comp'?  What happens if you invoke it via a full
absolute file name, not relying on PATH?

Thanks.

bug#44128: symlink problem after applying commit 0c1fc9d

[Kent Engström]

Hi,

I use a setup for testing native-comp, where I install to prefix
/opt/emacs/native-comp and then symlink as below

/opt/emacs ▶ ls -l current
lrwxrwxrwx. 1 kent kent 11 Apr 15 11:08 current -> native-comp

as I have /opt/emacs/current/bin on my $PATH.

After updating to 0c1fc9d, "* Fix native-comp startup for symliked
binary (bug#44128)" my emacs startup fails with

emacs: could not resolve realpath of "emacs": No such file or directory

Checking out the previous commit b064ddd, "Merge remote-tracking branch
'savannah/master' into native-comp" and doing a "make install"
gives me a working emacs binary again.

bug#44128: [feature/native-comp]

[Phil Sainty]

On 15/04/21 6:52 pm, Eli Zaretskii wrote:>> $ emacs-native-comp -Q
>> emacs: could not resolve realpath of "emacs-native-comp": No such file or directory
>
> Where in the Emacs sources does this message come from, please?
Looks like real_filename() in emacs.c:

/* Return the real filename following symlinks in case.
   The caller should deallocate the returned buffer.  */

static char *
real_filename (char *filename)
{
  char *real_name;
#ifdef WINDOWSNT
  /* w32_my_exename resolves symlinks internally, so no need to
     call realpath.  */
  real_name = xstrdup (filename);
#else
  real_name = realpath (filename, NULL);
  if (!real_name)
    fatal ("could not resolve realpath of \"%s\": %s",
	   filename, strerror (errno));
#endif
  return real_name;
}



> What happens if you invoke it via a full absolute file name,
> not relying on PATH?

That still works.


> What happens if you call the symlink 'emacs' and not
> 'emacs-native-comp'?

Curious things.  It doesn't work, and it apparently gets confused
by the 'emacs' directory in my HOME directory too.


$ hash emacs

$ alias emacs
bash: alias: emacs: not found

$ command -v emacs
/home/phil/bin/emacs

$ ls -lad `command -v emacs`
lrwxrwxrwx 1 phil phil 48 Apr 15 23:51 /home/phil/bin/emacs -> /home/phil/emacs/native-comp/usr/local/bin/emacs

$ pwd
/home/phil

$ ls -lad emacs
drwxrwxr-x 25 phil phil 4096 Mar 28 23:30 emacs

$ emacs --version
emacs: /home/phil/../native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln: cannot open shared object file: No such file or directory

$ cd ..

$ pwd
/home

$ emacs --version
emacs: could not resolve realpath of "emacs": No such file or directory

$ /home/phil/emacs/native-comp/usr/local/bin/emacs --version
GNU Emacs 28.0.50
Copyright (C) 2021 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

$ cd /home/phil/emacs/native-comp/usr/local/bin/

$ emacs --version
GNU Emacs 28.0.50
Copyright (C) 2021 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

$ /home/phil/emacs/native-comp/usr/local/bin/emacs -Q --batch --eval "(message \"%s %s\" emacs-repository-branch emacs-repository-version)"
feature/native-comp bfaa6df492c85d7de007cf69316cbdeea653d703

$ git log -3 bfaa6df492c85d7de007cf69316cbdeea653d703
commit bfaa6df492c85d7de007cf69316cbdeea653d703 (HEAD -> feature/native-comp, origin/feature/native-comp)
Author: Andrea Corallo <akrl@sdf.org>
Date:   Wed Apr 14 20:00:04 2021 +0200

    * configure.ac: Fix native-comp FreeBSD build.

commit 95dd6bb08038e31515568943dcfae13afac8ff70
Author: Eli Zaretskii <eliz@gnu.org>
Date:   Wed Apr 14 17:28:19 2021 +0300

    Fix MS-Windows build following last change

    * src/emacs.c (real_filename) [WINDOWSNT]: Fix off-by-one error
    when allocating storage for a file name.

commit 0c1fc9d581ad64efc96c1efccbb4d057796ef807
Author: Andrea Corallo <akrl@sdf.org>
Date:   Wed Apr 14 15:04:19 2021 +0200

    * Fix native-comp startup for symliked binary (bug#44128)

            * src/emacs.c (real_filename): New function.
            (set_invocation_vars, load_pdump): Make use of.

bug#44128: [feature/native-comp]

[Eli Zaretskii]

> Cc: akrl@sdf.org, jonas@bernoul.li, 44128@debbugs.gnu.org, eli@gnu.org
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Fri, 16 Apr 2021 00:12:57 +1200
> 
> static char *
> real_filename (char *filename)
> {
>   char *real_name;
> #ifdef WINDOWSNT
>   /* w32_my_exename resolves symlinks internally, so no need to
>      call realpath.  */
>   real_name = xstrdup (filename);
> #else
>   real_name = realpath (filename, NULL);
>   if (!real_name)
>     fatal ("could not resolve realpath of \"%s\": %s",
> 	   filename, strerror (errno));
> #endif
>   return real_name;
> }
> 
> 
> 
> > What happens if you invoke it via a full absolute file name,
> > not relying on PATH?
> 
> That still works.

So the problem seems to be that real_filename is passed the literal
"emacs-native-comp", without the leading directories?  Why does that
happen? is it because the PATH search in load_pdump_find_executable
rejects symlinks?

bug#44128: [feature/native-comp]

[Phil Sainty]

I'll add that the following two results suggest that the code
which (this version of) Emacs tries to load or run might vary
depending on the files which happen to be in the CWD at the time.

I think at best this may result in failures or inconsistencies
as I've encountered, and at worst it's probably exploitable.

Surely the start-up process shouldn't be looking in the CWD
for anything?


> it apparently gets confused by the 'emacs' directory in my HOME
> 
> $ ls -lad emacs
> drwxrwxr-x 25 phil phil 4096 Mar 28 23:30 emacs
> 
> $ emacs --version
> emacs: /home/phil/../native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln: cannot open shared object file: No such file or directory
> 
> $ cd /home/phil/emacs/native-comp/usr/local/bin/
> 
> $ emacs --version
> GNU Emacs 28.0.50
> Copyright (C) 2021 Free Software Foundation, Inc.
> GNU Emacs comes with ABSOLUTELY NO WARRANTY.
> You may redistribute copies of GNU Emacs
> under the terms of the GNU General Public License.
> For more information about these matters, see the file named COPYING.

bug#44128: [feature/native-comp]

[Eli Zaretskii]

> From: Phil Sainty <psainty@orcon.net.nz>
> Cc: akrl@sdf.org, jonas@bernoul.li, 44128@debbugs.gnu.org, eli@gnu.org
> Date: Fri, 16 Apr 2021 01:01:08 +1200
> 
> I'll add that the following two results suggest that the code
> which (this version of) Emacs tries to load or run might vary
> depending on the files which happen to be in the CWD at the time.
> 
> I think at best this may result in failures or inconsistencies
> as I've encountered, and at worst it's probably exploitable.
> 
> Surely the start-up process shouldn't be looking in the CWD
> for anything?

I'm not sure I see how you reach that conclusion.  Isn't
/home/phil/emacs/native-comp/usr/local/bin/ on your PATH?  If not, how
come the Emacs executable in that directory gets invoked?  Or maybe
you have "." in your PATH?

bug#44128: [feature/native-comp]

[Phil Sainty]

On 16/04/21 1:52 am, Eli Zaretskii wrote:
> I'm not sure I see how you reach that conclusion.  Isn't
> /home/phil/emacs/native-comp/usr/local/bin/ on your PATH?

No, it isn't.

/home/phil/bin is in my PATH and the 'emacs' symlink in that
directory points to /home/phil/emacs/native-comp/usr/local/bin/

When I was running 'emacs' that symlink was being used; however
the *result* of running it varied, depending on which directory
I was in at the time.

> Or maybe you have "." in your PATH?

Yikes, no; definitely not.


$ echo $PATH
/home/phil/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

$ cd /tmp

$ ls emacs
ls: cannot access 'emacs': No such file or directory

$ command -v emacs
/home/phil/bin/emacs

$ emacs --version
emacs: could not resolve realpath of "emacs": No such file or directory

$ touch emacs

$ emacs --version
emacs: /tmp/../native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln: cannot open shared object file: No such file or directory

bug#44128: [feature/native-comp]

[Eli Zaretskii]

> Cc: akrl@sdf.org, jonas@bernoul.li, 44128@debbugs.gnu.org, eli@gnu.org
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Fri, 16 Apr 2021 02:05:42 +1200
> 
> $ cd /tmp
> 
> $ ls emacs
> ls: cannot access 'emacs': No such file or directory
> 
> $ command -v emacs
> /home/phil/bin/emacs
> 
> $ emacs --version
> emacs: could not resolve realpath of "emacs": No such file or directory
> 
> $ touch emacs
> 
> $ emacs --version
> emacs: /tmp/../native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln: cannot open shared object file: No such file or directory

That's Emacs trying to see if it is run uninstalled, so I see no
problem here.  What exactly do you think is wrong with this, again?

bug#44128: [feature/native-comp]

[Phil Sainty]

On 16/04/21 2:42 am, Eli Zaretskii wrote:
>> $ emacs --version
>> emacs: could not resolve realpath of "emacs": No such file or directory
>> $ touch emacs
>> $ emacs --version
>> emacs: /tmp/../native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln: cannot open shared object file: No such file or directory
>
> That's Emacs trying to see if it is run uninstalled, so I see no
> problem here.  What exactly do you think is wrong with this, again?

The first problem is that it's currently not possible to start
emacs if you're in a directory which contains a file or directory
called 'emacs'.  I think that's Bad, and it will undoubtedly be
reported as a bug by other people for as long as that persisted.
It certainly doesn't happen with non-native-comp builds.

The second problem is that this type of behaviour feels rather
like something you mentioned earlier: having "." in your PATH,
which is widely considered a bad idea.  I expect that exploits
are unlikely (e.g. arranging for a malicious
native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln
to exist is probably a lot of work), but I think Emacs should not
behave differently depending on where you are when you start it.

In future, once this feature is merged, many people will have
multiple local installs of various native-comp-enabled versions,
and might be moving between them and/or working on them.  If
Emacs then tried to run code for the version in the CWD even if
the executable that was invoked was for a different install, that
would be very surprising (and potentially very difficult for the
user to notice).

I do see the hashes in the filenames, so maybe that scenario is
already avoided, if the eln filenames are unique to the version
of Emacs?  This "looking in the CWD" behaviour still feels wrong
to me, though.  Are there other existing ways in which Emacs
changes its behaviour based on the CWD?


-Phil

bug#44128: [feature/native-comp]

[Eli Zaretskii]

> Cc: akrl@sdf.org, jonas@bernoul.li, 44128@debbugs.gnu.org, eli@gnu.org
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Fri, 16 Apr 2021 09:52:59 +1200
> 
> On 16/04/21 2:42 am, Eli Zaretskii wrote:
> >> $ emacs --version
> >> emacs: could not resolve realpath of "emacs": No such file or directory
> >> $ touch emacs
> >> $ emacs --version
> >> emacs: /tmp/../native-lisp/28.0.50-abd7aa58/preloaded/window-0d1b8b93-581f9fcd.eln: cannot open shared object file: No such file or directory
> >
> > That's Emacs trying to see if it is run uninstalled, so I see no
> > problem here.  What exactly do you think is wrong with this, again?
> 
> The first problem is that it's currently not possible to start
> emacs if you're in a directory which contains a file or directory
> called 'emacs'.

That's not relevant to the message above; the failure related to the
presence of a directory or file named 'emacs' is a bug that will be
solved.

> The second problem is that this type of behaviour feels rather
> like something you mentioned earlier: having "." in your PATH,
> which is widely considered a bad idea.

But that ship has sailed long, long, LONG ago: Emacs was always
looking for its Lisp files in the directory "../lisp" relative to
where it was invoked since about forever.  How else can we support
both installed and uninstalled invocations?  When Emacs is run
uninstalled, the Lisp files can be anywhere on the system.  The only
difference is that now we _also_ look for the *.eln files in a similar
fashion.

IOW, Emacs always tries the configured installation directory first;
if the files are not there, it tries relative to the invocation
directory on the assumption that we are running uninstalled.  And that
is why you see the error message above.  There's nothing wrong with
the message per se, and once the problem with finding the files in
your case is solved, the message will disappear.

> In future, once this feature is merged, many people will have
> multiple local installs of various native-comp-enabled versions,
> and might be moving between them and/or working on them.  If
> Emacs then tried to run code for the version in the CWD even if
> the executable that was invoked was for a different install, that
> would be very surprising (and potentially very difficult for the
> user to notice).

The *.eln files include various hashes in their file names that
prevent loading of a mismatched .eln file.  This should support having
several variants of a .eln file, corresponding to several Emacs
binaries, in the same directory.  So I don't quite see the danger that
you have in mind.

> I do see the hashes in the filenames, so maybe that scenario is
> already avoided, if the eln filenames are unique to the version
> of Emacs?

Yes.

> This "looking in the CWD" behaviour still feels wrong to me, though.
> Are there other existing ways in which Emacs changes its behaviour
> based on the CWD?

See above: we were doing that since day one, more or less.

bug#44128: [feature/native-comp]

[Phil Sainty]

On 16/04/21 6:50 pm, Eli Zaretskii wrote:
>> The first problem is that it's currently not possible to start
>> emacs if you're in a directory which contains a file or directory
>> called 'emacs'.
> 
> That's not relevant to the message above; the failure related to the
> presence of a directory or file named 'emacs' is a bug that will be
> solved.

Ok, I misinterpreted your question.


>> The second problem is that this type of behaviour feels rather
>> like something you mentioned earlier: having "." in your PATH,
>> which is widely considered a bad idea.
> 
> But that ship has sailed long, long, LONG ago: Emacs was always
> looking for its Lisp files in the directory "../lisp" relative to
> where it was invoked since about forever.  How else can we support
> both installed and uninstalled invocations?  When Emacs is run
> uninstalled, the Lisp files can be anywhere on the system.  The only
> difference is that now we _also_ look for the *.eln files in a similar
> fashion.

For clarity, by the directory "where it was invoked" do you mean
the arbitrary directory from which the user runs "emacs", or do
you mean the directory containing the (real) emacs executable?

If you mean the latter, then I don't see a problem.

If you mean the former, then...

If we are able to successfully establish where the (real) emacs
executable is (and your recent patch looked like it achieved this),
then surely that can account for the "uninstalled invocations"
cases as well?  (Because we know where to find all of the
uninstalled paths of interest relative to the uninstalled
executable.)

My only concern is that looking for particular filenames in the
user's arbitrary CWD will be prone to false-positives; so if this
*is* happening, and there's a better solution at hand, perhaps
there's no longer any need to do it.

bug#44128: [feature/native-comp]

[Eli Zaretskii]

> Cc: jonas@bernoul.li, 44128@debbugs.gnu.org, akrl@sdf.org
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Sat, 17 Apr 2021 00:04:12 +1200
> 
> >> The second problem is that this type of behaviour feels rather
> >> like something you mentioned earlier: having "." in your PATH,
> >> which is widely considered a bad idea.
> > 
> > But that ship has sailed long, long, LONG ago: Emacs was always
> > looking for its Lisp files in the directory "../lisp" relative to
> > where it was invoked since about forever.  How else can we support
> > both installed and uninstalled invocations?  When Emacs is run
> > uninstalled, the Lisp files can be anywhere on the system.  The only
> > difference is that now we _also_ look for the *.eln files in a similar
> > fashion.
> 
> For clarity, by the directory "where it was invoked" do you mean
> the arbitrary directory from which the user runs "emacs", or do
> you mean the directory containing the (real) emacs executable?

The latter, of course.

> If we are able to successfully establish where the (real) emacs
> executable is (and your recent patch looked like it achieved this),
> then surely that can account for the "uninstalled invocations"
> cases as well?

No, because if Emacs is invoked as just "emacs" (or, more generally,
any string without directory separators), then we don't know whether
Emacs was run installed or uninstalled.

> My only concern is that looking for particular filenames in the
> user's arbitrary CWD will be prone to false-positives; so if this
> *is* happening, and there's a better solution at hand, perhaps
> there's no longer any need to do it.

We never look in CWD, unless it happens to be invocation-directory.

bug#44128: [feature/native-comp]

[Phil Sainty]

On 17/04/21 12:20 am, Eli Zaretskii wrote:
>> For clarity, by the directory "where it was invoked" do you mean
>> the arbitrary directory from which the user runs "emacs", or do
>> you mean the directory containing the (real) emacs executable?
> 
> The latter, of course.
> 
> We never look in CWD, unless it happens to be invocation-directory.

My concerns were all due to a misunderstanding, then.

Plus a lack of familiarity with the `invocation-directory' variable,
which would have clued me in.  FWIW I think that mis-interpreting
"where it was invoked" and similar phrases as meaning "where the
user ran 'emacs' from" is a particularly easy mistake to make if you
don't already know better, so that could be a good thing to habitually
write as `invocation-directory' to steer people in the right direction.

Sorry for the noise.

bug#44128: [feature/native-comp]

[Jonas Bernoulli]

Andrea Corallo <akrl@sdf.org> writes:

> I've pushed 0c1fc9d581 that seams to work for me, please have a try.

Unfortunately this still doesn't work (as of f9c1008ced):

  $ emacs
  emacs: could not resolve realpath of "emacs": No such file or directory
  $ which emacs
  /usr/local/bin/emacs
  $ ls -l /usr/local/bin/emacs
  lrwxrwxrwx 1 root staff 55 Apr 16 14:51 /usr/local/bin/emacs -> /home/jonas/git/src/emacs/feature

But this works:

  $ /home/jonas/git/src/emacs/feature/native-comp/src/emacs

And so does:

  $ cat /usr/local/bin/emacs
  #!/bin/sh
  /home/jonas/git/src/emacs/feature/native-comp/src/emacs "$@"
  $ emacs

That worked in the past but until now I subsequently ran into errors due
to things being undefined including `comp-*' variables and/or completely
unrelated things that should not be problem because when using "master"
or a release, they are loaded when needed.

I cannot be more precise at the moment because back then these issues
caused me to give up; and I am just mentioning this because so far it
seems like this issues is gone.

     Jonas

bug#44128: [feature/native-comp]

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Date: Fri, 16 Apr 2021 15:21:47 +0200
> Cc: 44128@debbugs.gnu.org, eli@gnu.org
> 
> Andrea Corallo <akrl@sdf.org> writes:
> 
> > I've pushed 0c1fc9d581 that seams to work for me, please have a try.
> 
> Unfortunately this still doesn't work (as of f9c1008ced):
> 
>   $ emacs
>   emacs: could not resolve realpath of "emacs": No such file or directory
>   $ which emacs
>   /usr/local/bin/emacs
>   $ ls -l /usr/local/bin/emacs
>   lrwxrwxrwx 1 root staff 55 Apr 16 14:51 /usr/local/bin/emacs -> /home/jonas/git/src/emacs/feature
> 
> But this works:
> 
>   $ /home/jonas/git/src/emacs/feature/native-comp/src/emacs
> 
> And so does:
> 
>   $ cat /usr/local/bin/emacs
>   #!/bin/sh
>   /home/jonas/git/src/emacs/feature/native-comp/src/emacs "$@"
>   $ emacs

Thanks, I think I understand the issues, and I'm working on a fix.

bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> Date: Fri, 16 Apr 2021 18:08:00 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: psainty@orcon.net.nz, akrl@sdf.org, 44128@debbugs.gnu.org, eli@gnu.org
> 
> > From: Jonas Bernoulli <jonas@bernoul.li>
> > Date: Fri, 16 Apr 2021 15:21:47 +0200
> > Cc: 44128@debbugs.gnu.org, eli@gnu.org
> > 
> > Andrea Corallo <akrl@sdf.org> writes:
> > 
> > > I've pushed 0c1fc9d581 that seams to work for me, please have a try.
> > 
> > Unfortunately this still doesn't work (as of f9c1008ced):
> > 
> >   $ emacs
> >   emacs: could not resolve realpath of "emacs": No such file or directory
> >   $ which emacs
> >   /usr/local/bin/emacs
> >   $ ls -l /usr/local/bin/emacs
> >   lrwxrwxrwx 1 root staff 55 Apr 16 14:51 /usr/local/bin/emacs -> /home/jonas/git/src/emacs/feature
> > 
> > But this works:
> > 
> >   $ /home/jonas/git/src/emacs/feature/native-comp/src/emacs
> > 
> > And so does:
> > 
> >   $ cat /usr/local/bin/emacs
> >   #!/bin/sh
> >   /home/jonas/git/src/emacs/feature/native-comp/src/emacs "$@"
> >   $ emacs
> 
> Thanks, I think I understand the issues, and I'm working on a fix.

Please try the latest native-comp branch.  If it still doesn't solve
the problem with installing Emacs via symlinks, or if there are some
adverse side-effects of the changes I made, please report the details.

In a nutshell, Emacs should now decide where to look for its pdumper
file and where to look for the *.eln files in a synchronized manner.
I hope I got all the varieties of the symlinks involved correctly (but
I couldn't test all the possible variants, only some of them).

Please be sure to test both installed and uninstalled binaries, and
please verify that comp-eln-load-path has the right value after Emacs
loads in both cases (the last element of the list should in each case
reflect where the *.eln files will be looked for).

TIA

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Phil Sainty]

On 18/04/21 1:58 am, Eli Zaretskii wrote:
> Please try the latest native-comp branch.

Unfortunately that didn't compile for me:

  CC       emacs.o
emacs.c: In function ‘load_pdump’:
emacs.c:903:3: error: a label can only be part of a statement and a declaration is not a statement
  903 |   const char *argv0_base = "emacs";
      |   ^~~~~
Makefile:385: recipe for target 'emacs.o' failed
make[1]: *** [emacs.o] Error 1



$ gcc --version
gcc (Ubuntu 10.1.0-2ubuntu1~18.04) 10.1.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.



My 'configure' result was:

Configured for 'x86_64-pc-linux-gnu'.

  Where should the build process find the source code?    .
  What compiler should emacs be built with?               gcc -g3 -O2
  Should Emacs use the GNU version of malloc?             no
    (The GNU allocators don't work with this system configuration.)
  Should Emacs use a relocating allocator for buffers?    no
  Should Emacs use mmap(2) for buffer allocation?         no
  What window system should Emacs use?                    x11
  What toolkit should Emacs use?                          LUCID
  Where do we find X Windows header files?                Standard dirs
  Where do we find X Windows libraries?                   Standard dirs
  Does Emacs use -lXaw3d?                                 yes
  Does Emacs use -lXpm?                                   yes
  Does Emacs use -ljpeg?                                  yes
  Does Emacs use -ltiff?                                  yes
  Does Emacs use a gif library?                           yes -lgif
  Does Emacs use a png library?                           yes -lpng16 -lz
  Does Emacs use -lrsvg-2?                                yes
  Does Emacs use cairo?                                   yes
  Does Emacs use -llcms2?                                 yes
  Does Emacs use imagemagick?                             no
  Does Emacs use native APIs for images?                  no
  Does Emacs support sound?                               no
  Does Emacs use -lgpm?                                   no
  Does Emacs use -ldbus?                                  yes
  Does Emacs use -lgconf?                                 no
  Does Emacs use GSettings?                               yes
  Does Emacs use a file notification library?             yes -lglibc (inotify)
  Does Emacs use access control lists?                    no
  Does Emacs use -lselinux?                               no
  Does Emacs use -lgnutls?                                yes
  Does Emacs use -lxml2?                                  yes
  Does Emacs use -lfreetype?                              yes
  Does Emacs use HarfBuzz?                                yes
  Does Emacs use -lm17n-flt?                              no
  Does Emacs use -lotf?                                   no
  Does Emacs use -lxft?                                   no
  Does Emacs use -lsystemd?                               no
  Does Emacs use -ljansson?                               yes
  Does Emacs use the GMP library?                         yes
  Does Emacs directly use zlib?                           yes
  Does Emacs have dynamic modules support?                yes
  Does Emacs use toolkit scroll bars?                     yes
  Does Emacs support Xwidgets?                            no
  Does Emacs have threading support in lisp?              yes
  Does Emacs support the portable dumper?                 yes
  Does Emacs support legacy unexec dumping?               no
  Which dumping strategy does Emacs use?                  pdumper
  Does Emacs have native lisp compiler?                   yes

bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> Cc: jonas@bernoul.li, wilde@sha-bang.de,
>  Dario Gjorgjevski <dario.gjorgjevski@gmail.com>, 44128@debbugs.gnu.org,
>  47800@debbugs.gnu.org, akrl@sdf.org
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Sun, 18 Apr 2021 02:13:19 +1200
> 
>   CC       emacs.o
> emacs.c: In function ‘load_pdump’:
> emacs.c:903:3: error: a label can only be part of a statement and a declaration is not a statement
>   903 |   const char *argv0_base = "emacs";
>       |   ^~~~~
> Makefile:385: recipe for target 'emacs.o' failed
> make[1]: *** [emacs.o] Error 1

How about now?

bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Phil Sainty]

On 18/04/21 2:29 am, Eli Zaretskii wrote:
> How about now?

This time it compiled; but with the following warnings, and
when I run it from the installed location (whether using the
absolute path or a symlink) I get a seg fault / core dump:

$ /home/phil/emacs/native-comp/usr/local/bin/emacs --version
Segmentation fault (core dumped)

Running the uninstalled version works:

$ ./src/emacs --version
GNU Emacs 28.0.50
Copyright (C) 2021 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.


-Phil (finished for the night, but can test more tomorrow)


  CC       emacs.o
In function ‘load_pdump’,
    inlined from ‘main’ at emacs.c:1289:5:
emacs.c:920:13: warning: argument 1 null where non-null expected [-Wnonnull]
  920 |   needed += strlen (strip_suffix) - strlen (suffix) + strlen (go_up);
      |             ^~~~~~~~~~~~~~~~~~~~~
In file included from ../lib/string.h:41,
                 from lisp.h:29,
                 from emacs.c:33:
emacs.c: In function ‘main’:
/usr/include/string.h:384:15: note: in a call to function ‘strlen’ declared here
  384 | extern size_t strlen (const char *__s)
      |               ^~~~~~
In file included from /usr/include/stdio.h:862,
                 from ../lib/stdio.h:43,
                 from lisp.h:4731,
                 from emacs.c:33:
In function ‘sprintf’,
    inlined from ‘load_pdump’ at emacs.c:927:3,
    inlined from ‘main’ at emacs.c:1289:5:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:33:10: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
   33 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   34 |       __bos (__s), __fmt, __va_arg_pack ());
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> Cc: jonas@bernoul.li, wilde@sha-bang.de, 47800@debbugs.gnu.org,
>  dario.gjorgjevski@gmail.com, 44128@debbugs.gnu.org, akrl@sdf.org
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Sun, 18 Apr 2021 03:00:17 +1200
> 
> On 18/04/21 2:29 am, Eli Zaretskii wrote:
> > How about now?
> 
> This time it compiled; but with the following warnings, and
> when I run it from the installed location (whether using the
> absolute path or a symlink) I get a seg fault / core dump:

Sorry about that.  Next try, please.

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Dario Gjorgjevski]

> Sorry about that.  Next try, please.

Hi Eli,

The most recent commit, b8d3860, works fine here.  (I previously had the
same segfault as Phil.)

Many thanks for all your work!

Best regards,
Dario

-- 
$ keyserver=hkps://hkps.pool.sks-keyservers.net
$ keyid=744A4F0B4F1C9371
$ gpg --keyserver $keyserver --search-keys $keyid

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> From: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>
> Cc: Phil Sainty <psainty@orcon.net.nz>,  jonas@bernoul.li,
>   wilde@sha-bang.de,  47800@debbugs.gnu.org,  44128@debbugs.gnu.org,
>   akrl@sdf.org
> Date: Sat, 17 Apr 2021 17:39:28 +0200
> 
> The most recent commit, b8d3860, works fine here.  (I previously had the
> same segfault as Phil.)

Great, thanks for testing.

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[wilde]

Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>
>> Cc: Phil Sainty <psainty@orcon.net.nz>,  jonas@bernoul.li,
>>   wilde@sha-bang.de,  47800@debbugs.gnu.org,  44128@debbugs.gnu.org,
>>   akrl@sdf.org
>> Date: Sat, 17 Apr 2021 17:39:28 +0200
>> 
>> The most recent commit, b8d3860, works fine here.  (I previously had the
>> same segfault as Phil.)
>
> Great, thanks for testing.

I'm very happy to confirm that 75c898e (one past b8d3860) also works as
expected on NetBSD 9.1.

Thanks a  lot for fixing this!

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> From: wilde@sha-bang.de
> Cc: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>,  psainty@orcon.net.nz,
>   jonas@bernoul.li,  47800@debbugs.gnu.org,  44128@debbugs.gnu.org,
>   akrl@sdf.org
> Date: Sat, 17 Apr 2021 21:15:24 +0200
> 
> Eli Zaretskii <eliz@gnu.org> wrote:
> 
> >> From: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>
> >> Cc: Phil Sainty <psainty@orcon.net.nz>,  jonas@bernoul.li,
> >>   wilde@sha-bang.de,  47800@debbugs.gnu.org,  44128@debbugs.gnu.org,
> >>   akrl@sdf.org
> >> Date: Sat, 17 Apr 2021 17:39:28 +0200
> >> 
> >> The most recent commit, b8d3860, works fine here.  (I previously had the
> >> same segfault as Phil.)
> >
> > Great, thanks for testing.
> 
> I'm very happy to confirm that 75c898e (one past b8d3860) also works as
> expected on NetBSD 9.1.

Thanks for testing this.

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[wilde]

Eli Zaretskii <eliz@gnu.org> wrote:

>> From: wilde@sha-bang.de
>> Cc: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>,  psainty@orcon.net.nz,
>>   jonas@bernoul.li,  47800@debbugs.gnu.org,  44128@debbugs.gnu.org,
>>   akrl@sdf.org
>> Date: Sat, 17 Apr 2021 21:15:24 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>> >> From: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>
>> >> Cc: Phil Sainty <psainty@orcon.net.nz>,  jonas@bernoul.li,
>> >>   wilde@sha-bang.de,  47800@debbugs.gnu.org,  44128@debbugs.gnu.org,
>> >>   akrl@sdf.org
>> >> Date: Sat, 17 Apr 2021 17:39:28 +0200
>> >> 
>> >> The most recent commit, b8d3860, works fine here.  (I previously had the
>> >> same segfault as Phil.)
>> >
>> > Great, thanks for testing.
>> 
>> I'm very happy to confirm that 75c898e (one past b8d3860) also works as
>> expected on NetBSD 9.1.
>
> Thanks for testing this.

My pleasure.

However, on my test build (for this issue) on GNU/Linux unfortunately I
do still see issues:

# The situation after setup
# (build with --prefix=/home/wilde/apps/emacs-native-dev and installed
# with make install):

% ls -l `which emacs`
lrwxrwxrwx 1 wilde wilde 13 Apr 17 21:21 /home/wilde/apps/emacs-native-dev/bin/emacs -> emacs-28.0.50*
% ls -l `which emacs-28.0.50` 
-rwxr-xr-x 1 wilde wilde 25866768 Apr 17 21:21 /home/wilde/apps/emacs-native-dev/bin/emacs-28.0.50*

# The problem:

% emacs                      
emacs: /home/wilde/apps/emacs-native-dev/libexec/emacs/28.0.50/x86_64-p/home/wilde/apps/emacs-native-dev/libexec/emacs/28.0.50/x86_64-pc-linux-gnu/../native-lisp/28.0.50-f13b7cda/preloaded/frame-aa2cd9f8-88c2b85c.eln: cannot open shared object file: No such file or directory
1 wilde@marklar[~/src/stdsrc/emacs-native-comp_BUILD]

# What works:

% emacs-28.0.50
# → emacs starts fine.

FWIW, The file not found when invoked using the symlink "emasc" is at:
/home/wilde/apps/emacs-native-dev/lib/emacs/28.0.50/native-lisp/28.0.50-f13b7cda/preloaded/frame-aa2cd9f8-88c2b85c.eln

cheers
sascha

bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The version of master that I am running is from Dec 5.
I always run it through a symlink.

So I think the bug was introduced since then.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: jonas@bernoul.li, psainty@orcon.net.nz, wilde@sha-bang.de,
> 	dario.gjorgjevski@gmail.com, 44128@debbugs.gnu.org,
> 	47800@debbugs.gnu.org, akrl@sdf.org
> Date: Sat, 17 Apr 2021 23:40:43 -0400
> 
> The version of master that I am running is from Dec 5.
> I always run it through a symlink.
> 
> So I think the bug was introduced since then.

You are not running the native-comp branch, that's why you don't see
the problem.  The problem doesn't exist on the master branch, even not
today.

bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Phil Sainty]

Thank you Eli, this looks good to me too.

I tested with the following set of symlinks:

/home/phil/bin/emacs                         -> /home/phil/emacs/native-comp/usr/local/bin/emacs
/home/phil/bin/emacs-native-comp             -> /home/phil/emacs/native-comp/usr/local/bin/emacs
/home/phil/bin/emacs-28.0.50                 -> /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
/home/phil/bin/emacs-native-comp-28.0.50     -> /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
/home/phil/bin/emacs-native-comp-uninstalled -> /home/phil/emacs/native-comp/shallow-repository/src/emacs

For each of those symlinks, and also the absolute paths for the
link targets, I tested running them from multiple directories.
These included the directories of the real binaries, plus two
other directories containing a file or sub-directory named 'emacs':

drwxrwxr-x 25 phil phil 4096 Mar 28 23:30 /home/phil/emacs
-rw-rw-r--  1 phil phil    0 Apr 18 11:34 /tmp/emacs

In each test I ran --batch --eval "(prin1 comp-eln-load-path)"

$ (for dir in / /home /home/phil /home/phil/bin /home/phil/emacs /home/phil/emacs/native-comp/usr/local/bin /home/phil/emacs/native-comp/shallow-repository/src /tmp; do echo; echo $dir; cd $dir; for E in emacs emacs-native-comp emacs-28.0.50 emacs-native-comp-28.0.50 /home/phil/emacs/native-comp/usr/local/bin/emacs /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50 emacs-native-comp-uninstalled /home/phil/emacs/native-comp/shallow-repository/src/emacs; do $E --batch --eval "(prin1 comp-eln-load-path)"; echo " -- $E"; done; done)

Nothing failed, and for every directory the output was the same:

("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- emacs
("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- emacs-native-comp
("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- emacs-28.0.50
("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- emacs-native-comp-28.0.50
("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- /home/phil/emacs/native-comp/usr/local/bin/emacs
("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/shallow-repository/native-lisp/") -- emacs-native-comp-uninstalled
("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/shallow-repository/native-lisp/") -- /home/phil/emacs/native-comp/shallow-repository/src/emacs


-Phil

bug#44128: symlink problem after applying commit 0c1fc9d

[Kent Engström]

Kent Engström <kent@lysator.liu.se> writes:
> I use a setup for testing native-comp, where I install to prefix
> /opt/emacs/native-comp and then symlink as below
>
> /opt/emacs ▶ ls -l current
> lrwxrwxrwx. 1 kent kent 11 Apr 15 11:08 current -> native-comp
>
> as I have /opt/emacs/current/bin on my $PATH.
>
> After updating to 0c1fc9d, "* Fix native-comp startup for symliked
> binary (bug#44128)" my emacs startup fails with
>
> emacs: could not resolve realpath of "emacs": No such file or directory
>
> Checking out the previous commit b064ddd, "Merge remote-tracking branch
> 'savannah/master' into native-comp" and doing a "make install"
> gives me a working emacs binary again.

Just a quick followup: after updating to commit 75c898e, starting emacs
works again for my setup above.

/ kent

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> From: Phil Sainty <psainty@orcon.net.nz>
> Cc: jonas@bernoul.li, wilde@sha-bang.de, 47800@debbugs.gnu.org,
>  dario.gjorgjevski@gmail.com, 44128@debbugs.gnu.org, akrl@sdf.org
> Date: Sun, 18 Apr 2021 19:40:20 +1200
> 
> $ (for dir in / /home /home/phil /home/phil/bin /home/phil/emacs /home/phil/emacs/native-comp/usr/local/bin /home/phil/emacs/native-comp/shallow-repository/src /tmp; do echo; echo $dir; cd $dir; for E in emacs emacs-native-comp emacs-28.0.50 emacs-native-comp-28.0.50 /home/phil/emacs/native-comp/usr/local/bin/emacs /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50 emacs-native-comp-uninstalled /home/phil/emacs/native-comp/shallow-repository/src/emacs; do $E --batch --eval "(prin1 comp-eln-load-path)"; echo " -- $E"; done; done)
> 
> Nothing failed, and for every directory the output was the same:
> 
> ("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- emacs
> ("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- emacs-native-comp
> ("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- emacs-28.0.50
> ("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- emacs-native-comp-28.0.50
> ("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- /home/phil/emacs/native-comp/usr/local/bin/emacs
> ("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/usr/local/lib/emacs/28.0.50/native-lisp/") -- /home/phil/emacs/native-comp/usr/local/bin/emacs-28.0.50
> ("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/shallow-repository/native-lisp/") -- emacs-native-comp-uninstalled
> ("/home/phil/.emacs.d/eln-cache/" "/home/phil/emacs/native-comp/shallow-repository/native-lisp/") -- /home/phil/emacs/native-comp/shallow-repository/src/emacs

Thanks for testing, this looks very promising.

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> From: wilde@sha-bang.de
> Cc: dario.gjorgjevski@gmail.com,  psainty@orcon.net.nz,  jonas@bernoul.li,
>   47800@debbugs.gnu.org,  44128@debbugs.gnu.org,  akrl@sdf.org
> Date: Sat, 17 Apr 2021 21:32:23 +0200
> 
> % ls -l `which emacs`
> lrwxrwxrwx 1 wilde wilde 13 Apr 17 21:21 /home/wilde/apps/emacs-native-dev/bin/emacs -> emacs-28.0.50*
> % ls -l `which emacs-28.0.50` 
> -rwxr-xr-x 1 wilde wilde 25866768 Apr 17 21:21 /home/wilde/apps/emacs-native-dev/bin/emacs-28.0.50*
> 
> # The problem:
> 
> % emacs                      
> emacs: /home/wilde/apps/emacs-native-dev/libexec/emacs/28.0.50/x86_64-p/home/wilde/apps/emacs-native-dev/libexec/emacs/28.0.50/x86_64-pc-linux-gnu/../native-lisp/28.0.50-f13b7cda/preloaded/frame-aa2cd9f8-88c2b85c.eln: cannot open shared object file: No such file or directory

There's something here that I'm missing.  The file name it tries is
clearly a result of some invalid concatenation of string, but I don't
quite see how that could happen.

Can you step through the code with GDB?  If yes, I will ask to show
values of some variables, and that will hopefully pinpoint the buggy
code.

Thanks.

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> Date: Sun, 18 Apr 2021 11:42:32 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: jonas@bernoul.li, psainty@orcon.net.nz, 47800@debbugs.gnu.org,
>  dario.gjorgjevski@gmail.com, 44128@debbugs.gnu.org, akrl@sdf.org
> 
> Can you step through the code with GDB?  If yes, I will ask to show
> values of some variables, and that will hopefully pinpoint the buggy
> code.

But before you do anything else, please try the latest branch, where I
fixed some thinko.

Thanks.

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[wilde]

Eli Zaretskii <eliz@gnu.org> wrote:

>> Date: Sun, 18 Apr 2021 11:42:32 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: jonas@bernoul.li, psainty@orcon.net.nz, 47800@debbugs.gnu.org,
>>  dario.gjorgjevski@gmail.com, 44128@debbugs.gnu.org, akrl@sdf.org
>> 
>> Can you step through the code with GDB?  If yes, I will ask to show
>> values of some variables, and that will hopefully pinpoint the buggy
>> code.
>
> But before you do anything else, please try the latest branch, where I
> fixed some thinko.

That latest change fixed it for me.

Once again, may thanks for your work!

cheers
sascha

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> From: wilde@sha-bang.de
> Cc: jonas@bernoul.li,  psainty@orcon.net.nz,  47800@debbugs.gnu.org,
>   dario.gjorgjevski@gmail.com,  44128@debbugs.gnu.org,  akrl@sdf.org
> Date: Sun, 18 Apr 2021 14:24:15 +0200
> 
> > But before you do anything else, please try the latest branch, where I
> > fixed some thinko.
> 
> That latest change fixed it for me.

Great, then I think we are good.  I just hope Jonas will be able to
test in his configuration as well.

bug#44128: symlink problem after applying commit 0c1fc9d

[Eli Zaretskii]

> From: Kent Engström <kent@lysator.liu.se>
> Date: Sun, 18 Apr 2021 09:41:55 +0200
> 
> Kent Engström <kent@lysator.liu.se> writes:
> > I use a setup for testing native-comp, where I install to prefix
> > /opt/emacs/native-comp and then symlink as below
> >
> > /opt/emacs ▶ ls -l current
> > lrwxrwxrwx. 1 kent kent 11 Apr 15 11:08 current -> native-comp
> >
> > as I have /opt/emacs/current/bin on my $PATH.
> >
> > After updating to 0c1fc9d, "* Fix native-comp startup for symliked
> > binary (bug#44128)" my emacs startup fails with
> >
> > emacs: could not resolve realpath of "emacs": No such file or directory
> >
> > Checking out the previous commit b064ddd, "Merge remote-tracking branch
> > 'savannah/master' into native-comp" and doing a "make install"
> > gives me a working emacs binary again.
> 
> Just a quick followup: after updating to commit 75c898e, starting emacs
> works again for my setup above.

Great, thanks for testing.

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: jonas@bernoul.li, wilde@sha-bang.de, 47800@debbugs.gnu.org,
>>  dario.gjorgjevski@gmail.com, 44128@debbugs.gnu.org, akrl@sdf.org
>> From: Phil Sainty <psainty@orcon.net.nz>
>> Date: Sun, 18 Apr 2021 03:00:17 +1200
>> 
>> On 18/04/21 2:29 am, Eli Zaretskii wrote:
>> > How about now?
>> 
>> This time it compiled; but with the following warnings, and
>> when I run it from the installed location (whether using the
>> absolute path or a symlink) I get a seg fault / core dump:
>
> Sorry about that.  Next try, please.

Now it works for me too.
Thanks!

bug#44128: bug#47800: bug#44128: [feature/native-comp] When invoking a symlink to the 'emacs' binary Emacs fails to start

[Eli Zaretskii]

> >> This time it compiled; but with the following warnings, and
> >> when I run it from the installed location (whether using the
> >> absolute path or a symlink) I get a seg fault / core dump:
> >
> > Sorry about that.  Next try, please.
> 
> Now it works for me too.
> Thanks!

Thanks, so I think we can close this issue now.  Thanks to everybody
who reported their results and tried my buggy code.