bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

On a build directory:

src/emacs -Q

C-h N

M-: (buffer-file-name)

"/usr/local/share/emacs/27.0.50/etc/NEWS"

Maybe this is caused because, time ago, I installed Emacs from the build
directory, but since I pulled new changes and rebuilt Emacs. So now
emacs instances launched from the build directory are showing stalled
documentation.


In GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, X toolkit)
 of 2019-07-28 built on sky
Repository revision: 77ee23d1ede9eece3eab4cc67d7f2e72d30a1117
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux bullseye/sid

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 28 Jul 2019 17:07:20 +0200
> 
> src/emacs -Q
> 
> C-h N
> 
> M-: (buffer-file-name)
> 
> "/usr/local/share/emacs/27.0.50/etc/NEWS"
> 
> Maybe this is caused because, time ago, I installed Emacs from the build
> directory, but since I pulled new changes and rebuilt Emacs. So now
> emacs instances launched from the build directory are showing stalled
> documentation.

Emacs tries the installation directory first, and only falls back to
the build directory if the installation directory doesn't exist.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> Emacs tries the installation directory first, and only falls back to
> the build directory if the installation directory doesn't exist.

Does this happen if Emacs was never installed from that build directory?

And is there a good reason for doing that? When Emacs is executed from
the build directory, I expect that it works on the contents of that
directory (and the correspondent source directory, for out-of-tree
builds.)

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: 36828@debbugs.gnu.org
> Date: Sun, 28 Jul 2019 17:56:28 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Emacs tries the installation directory first, and only falls back to
> > the build directory if the installation directory doesn't exist.
> 
> Does this happen if Emacs was never installed from that build directory?

AFAIR, Emacs doesn't know.  It just looks in several directories
according to some order, and that order makes sense in the usual use
cases.  Yours is an unusual one.

> And is there a good reason for doing that? When Emacs is executed from
> the build directory, I expect that it works on the contents of that
> directory (and the correspondent source directory, for out-of-tree
> builds.)

We are talking about data-directory.  The convoluted logic for its
calculation is in callproc.c, near the end.  You will see that there
are quite a few use cases supported by that code.  There could be
EMACS_DATA in the environment; there could be a list of directories to
look in, not just a single directory; the build directory may or may
not be the same as the source directory.  The value you see is probed
early on during the startup, when Emacs didn't yet figure out the
value of installation-directory, and so cannot yet employ the logic
you want it to.  For an uninstalled Emacs that directory usually
doesn't exist, so later Emacs tries again after figuring out the
installation-directory.

Bottom line: I'm not sure we should rock that particular boat for an
unusual use case such as yours.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

Thanks for the explanation.

> Bottom line: I'm not sure we should rock that particular boat for an
> unusual use case such as yours.

I don't think this case is so unusual: updating, building and running
the new Emacs (for minutes, hours or weeks) to check that it is ok
before installing. (My use case is a bit different, but equivalent.)

Anyways, this is not very important for me either. Closing.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Štěpán Němec]

On Mon, 29 Jul 2019 02:36:05 +0200
Óscar Fuentes wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Bottom line: I'm not sure we should rock that particular boat for an
>> unusual use case such as yours.
>
> I don't think this case is so unusual: updating, building and running
> the new Emacs (for minutes, hours or weeks) to check that it is ok
> before installing. (My use case is a bit different, but equivalent.)
>
> Anyways, this is not very important for me either. Closing.

FWIW, it find your use case totally legit and not unusual at all. I can
understand if nobody feels like attempting a fix right now, but at least
keeping the report open would help anyone searching for the issue (and
I'm glad I have learned about it from your report as I can imagine being
confused by this behaviour myself).

-- 
Štěpán

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Štěpán Němec <stepnem@gmail.com> writes:

> FWIW, it find your use case totally legit and not unusual at all. I can
> understand if nobody feels like attempting a fix right now, but at least
> keeping the report open would help anyone searching for the issue (and
> I'm glad I have learned about it from your report as I can imagine being
> confused by this behaviour myself).

Feel free to reopen the bug. I only know how to create and close them
and I'm utterly uninterested on learning further about debbugs :-)

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Štěpán Němec
>  <stepnem@gmail.com>
> Date: Mon, 29 Jul 2019 14:13:24 +0200
> Cc: ofv@wanadoo.es
> 
> >> Bottom line: I'm not sure we should rock that particular boat for an
> >> unusual use case such as yours.
> >
> > I don't think this case is so unusual: updating, building and running
> > the new Emacs (for minutes, hours or weeks) to check that it is ok
> > before installing. (My use case is a bit different, but equivalent.)
> >
> > Anyways, this is not very important for me either. Closing.
> 
> FWIW, it find your use case totally legit and not unusual at all.

"Unusual" doesn't mean it isn't legitimate, it just means it's
infrequent.

> I can understand if nobody feels like attempting a fix right now

By saying "fix" you imply this is a bug.  But that's not necessarily
so, because it is also legitimate to want Emacs behave like it does
now, i.e. prefer the installed version to the one in the source tree.
Any fix will have to deal with the dichotomy, and let users request
the non-default variant.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Štěpán Němec]

reopen 36828
thanks

On Mon, 29 Jul 2019 16:25:30 +0200
Óscar Fuentes wrote:

> Feel free to reopen the bug. I only know how to create and close them
> and I'm utterly uninterested on learning further about debbugs :-)

OK. :-D

-- 
Štěpán

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Štěpán Němec]

On Mon, 29 Jul 2019 17:38:11 +0300
Eli Zaretskii wrote:

>> FWIW, it find your use case totally legit and not unusual at all.
>
> "Unusual" doesn't mean it isn't legitimate, it just means it's
> infrequent.

I'm sorry, I should have thought better about the wording. I didn't mean
to imply you didn't consider the use case legitimate.

>> I can understand if nobody feels like attempting a fix right now
>
> By saying "fix" you imply this is a bug.  But that's not necessarily
> so, because it is also legitimate to want Emacs behave like it does
> now, i.e. prefer the installed version to the one in the source tree.
> Any fix will have to deal with the dichotomy, and let users request
> the non-default variant.

Indeed I do consider this a bug, and quite explicitely so. That emacs
should always default to using the data most relevant for the running
executable seems obvious to me. What am I missing?

-- 
Štěpán

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Štěpán Němec <stepnem@gmail.com>
> Cc: 36828@debbugs.gnu.org, ofv@wanadoo.es
> Date: Mon, 29 Jul 2019 16:47:49 +0200
> 
> Indeed I do consider this a bug, and quite explicitely so. That emacs
> should always default to using the data most relevant for the running
> executable seems obvious to me. What am I missing?

I think you are missing the perspective of someone who considers the
current behavior to be TRT.

IOW, why should we assume that the version of NEWS in the source tree
is necessarily more up-to-date/useful than the one in the installation
directory.  It could be, but then it could not be.

And please also read the code, where you will see that this is a
kind-of bootstrap problem, whereby we only figure out where some
directory is based on some other directory.  At some point near the
beginning of this bootstrap we need to make some assumption more or
less out of thin air, so we make it based on the most frequent use
case.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> Date: Mon, 29 Jul 2019 18:11:45 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 36828@debbugs.gnu.org, ofv@wanadoo.es
> 
> > From: Štěpán Němec <stepnem@gmail.com>
> > Cc: 36828@debbugs.gnu.org, ofv@wanadoo.es
> > Date: Mon, 29 Jul 2019 16:47:49 +0200
> > 
> > Indeed I do consider this a bug, and quite explicitely so. That emacs
> > should always default to using the data most relevant for the running
> > executable seems obvious to me. What am I missing?
> 
> I think you are missing the perspective of someone who considers the
> current behavior to be TRT.

And, of course, in the "usual" case Emacs does pick up the NEWS file
in the source tree.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Štěpán Němec]

On Mon, 29 Jul 2019 18:11:45 +0300
Eli Zaretskii wrote:

> I think you are missing the perspective of someone who considers the
> current behavior to be TRT.

Certainly. So much that I can't even imagine why anybody would consider
that TRT (although I do try, and will appreciate if you try to explain).

> IOW, why should we assume that the version of NEWS in the source tree
> is necessarily more up-to-date/useful than the one in the installation
> directory.  It could be, but then it could not be.

Hm, it seems the misunderstanding is mutual.

Whether some version is more up-to-date or useful (whatever that means)
is besides the point. What I meant by "most relevant for the running
executable" and "obvious" above was simply what seems to me a case of
the principle of least surprise: if I run /usr/bin/emacs, I expect it to
use the same PREFIX for other things, i.e. /usr/share/emacs.... If I run
/usr/local/bin/emacs, I expect it to use /usr/local/share/emacs.... And
if I run src/emacs, I expect it to use the source tree.

Does that really seem less reasonable than your "TRT"?

I can imagine the current behaviour being hard to change, but that still
wouldn't make it the right thing IMO.

-- 
Štěpán

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Štěpán Němec <stepnem@gmail.com>
> Cc: 36828@debbugs.gnu.org, ofv@wanadoo.es
> Date: Mon, 29 Jul 2019 18:08:33 +0200
> 
> On Mon, 29 Jul 2019 18:11:45 +0300
> Eli Zaretskii wrote:
> 
> > I think you are missing the perspective of someone who considers the
> > current behavior to be TRT.
> 
> Certainly. So much that I can't even imagine why anybody would consider
> that TRT (although I do try, and will appreciate if you try to explain).

Once again, we are talking about the situation where there's both an
installed NEWS file and an uninstalled one for the same Emacs
version.  The situation where there's just one of them works exactly
as you want.  Right?

> Whether some version is more up-to-date or useful (whatever that means)
> is besides the point. What I meant by "most relevant for the running
> executable" and "obvious" above was simply what seems to me a case of
> the principle of least surprise: if I run /usr/bin/emacs, I expect it to
> use the same PREFIX for other things, i.e. /usr/share/emacs.... If I run
> /usr/local/bin/emacs, I expect it to use /usr/local/share/emacs.... And
> if I run src/emacs, I expect it to use the source tree.

But that's not how Emacs works.  Emacs looks for its data-directory
where it was configured to look for it.  At configure time, you can
specify that place via the --datadir=DIR option.  The result (or the
default, if you didn't specify --datadir=DIR) is recorded in
src/epaths.h, and that's where Emacs will look for the data files.

IOW, you are describing an Emacs that is very different from how the
real Emacs works, has been working for decades.  Emacs does NOT
determine all of the different directories relative to the directory
where its binary executable lives.  It does it according to how it was
configured.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Štěpán Němec]

On Mon, 29 Jul 2019 19:57:55 +0300
Eli Zaretskii wrote:

> Once again, we are talking about the situation where there's both an
> installed NEWS file and an uninstalled one for the same Emacs
> version.  The situation where there's just one of them works exactly
> as you want.  Right?

Yes. [Although I wouldn't use the formulation "the same Emacs version"
myself, or at least would need to clarify its definition, i.e. typically
the source tree build would be newer than the installed version, but
would end up in the same installation directory, yes, so that's probably
not relevant here.]

>> Whether some version is more up-to-date or useful (whatever that means)
>> is besides the point. What I meant by "most relevant for the running
>> executable" and "obvious" above was simply what seems to me a case of
>> the principle of least surprise: if I run /usr/bin/emacs, I expect it to
>> use the same PREFIX for other things, i.e. /usr/share/emacs.... If I run
>> /usr/local/bin/emacs, I expect it to use /usr/local/share/emacs.... And
>> if I run src/emacs, I expect it to use the source tree.
>
> But that's not how Emacs works.  Emacs looks for its data-directory
> where it was configured to look for it.  At configure time, you can
> specify that place via the --datadir=DIR option.  The result (or the
> default, if you didn't specify --datadir=DIR) is recorded in
> src/epaths.h, and that's where Emacs will look for the data files.
>
> IOW, you are describing an Emacs that is very different from how the
> real Emacs works, has been working for decades.  Emacs does NOT
> determine all of the different directories relative to the directory
> where its binary executable lives.  It does it according to how it was
> configured.

I think you are mixing up two things. One way to put it would be the
(veteran) developer view vs. the user view. You keep saying things like
"that's not how Emacs works" (well, that much we can all agree upon at
least) or "I'm not sure we should rock that particular boat for an
unusual use case such as yours." and I can appreciate that.

But Óscar wrote:

  And is there a good reason for doing that? When Emacs is executed from
  the build directory, I expect that it works on the contents of that
  directory (and the correspondent source directory, for out-of-tree
  builds.)

And I described the same expectation in slightly greater detail and
asked you

  Does that really seem less reasonable than your "TRT"?

And also asked you to explain your "right thing" and why you think it is
right.

And I still haven't got your answer to that, only more description of
the status quo (which is helpful, too, though, and thank you for that).

I can live with "this is how it works, the code is hairy, let's just
keep this as a wishlist item" or something to that effect, but you
somehow seem to insist on the current behaviour not being wrong at all
or at least seem to have some kind of "the right thing" notion
apparently quite different from the expectation of myself, the OP and I
suspect most users, and at the same time you fail to explain what that
TRT is and why it is right or more right than our expectation.

Failing all that, if there is no prospect of change at all, would it be
possible to at least warn the user in cases like this? Something like
"The NEWS file you are accessing is not the one you probably think it
is"?

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Štěpán Němec <stepnem@gmail.com>
> Cc: 36828@debbugs.gnu.org,  ofv@wanadoo.es
> Date: Mon, 29 Jul 2019 19:44:25 +0200
> 
> On Mon, 29 Jul 2019 19:57:55 +0300
> Eli Zaretskii wrote:
> 
> > Once again, we are talking about the situation where there's both an
> > installed NEWS file and an uninstalled one for the same Emacs
> > version.  The situation where there's just one of them works exactly
> > as you want.  Right?
> 
> Yes. [Although I wouldn't use the formulation "the same Emacs version"
> myself, or at least would need to clarify its definition, i.e. typically
> the source tree build would be newer than the installed version, but
> would end up in the same installation directory, yes, so that's probably
> not relevant here.]

data-directory is versioned.  E.g., for Emacs 26.2.90 I'm using to
type this, data-directory is "/usr/share/emacs/26.2.90/etc/".  So yes,
we are talking about having an installed and an uninstalled NEWS
_for_the_same_Emacs_version_.  I use "version" in the sense of
"version number".

> I can live with "this is how it works, the code is hairy, let's just
> keep this as a wishlist item" or something to that effect, but you
> somehow seem to insist on the current behaviour not being wrong at all
> or at least seem to have some kind of "the right thing" notion
> apparently quite different from the expectation of myself, the OP and I
> suspect most users, and at the same time you fail to explain what that
> TRT is and why it is right or more right than our expectation.

I wasn't aware that Óscar meant to change entirely how Emacs finds all
of its directories.  I still don't think that's what he meant.

You certainly do, though.  And my response to that was not to describe
the implementation, it was to describe the _design_.  Emacs looks for
its directories according to how it was configured _by_design_.  I
understand now that you want to change that design from the ground up,
but in that case this specific use case with data-directory in two
places is really not what's important.  What's important is that you
are asking for a radical change in how Emacs is configured, and
basically redesign and reimplement a large chunk of its startup code.
You are also asking it to start behaving unlike most other programs
out there, almost all of them looking for auxiliary files in
directories computed at configure time.  Programs that are
"relocatable", i.e. look for the directories relative to their binary,
are a minority; one reason is that this requires them to know where
the binary lives, something that not every program knows (Emacs does).
In any case, it's a very significant change, way more than just make
this particular use case behave as Óscar expected.

> Failing all that, if there is no prospect of change at all, would it be
> possible to at least warn the user in cases like this? Something like
> "The NEWS file you are accessing is not the one you probably think it
> is"?

You mean, instead of using the first NEWS we find, look in _all_ the
possible places and print a warning if we find more than one NEWS
file?

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Štěpán Němec]

On Mon, 29 Jul 2019 21:13:34 +0300
Eli Zaretskii wrote:

> I wasn't aware that Óscar meant to change entirely how Emacs finds all
> of its directories.  I still don't think that's what he meant.

I don't either, and I'm aware of how configure works (in general, I
wasn't aware of the --datadir option in particular).

> You certainly do, though.

Not really, I don't think so. And you still haven't replied to my
questions. So anyway, let's keep this focused and simple:

In the special case considered, do you agree that it would be better if
Emacs recognized that it was started from the source tree and showed the
corresponding NEWS file instead of the one installed?

Can you please answer that question?

>> Failing all that, if there is no prospect of change at all, would it be
>> possible to at least warn the user in cases like this? Something like
>> "The NEWS file you are accessing is not the one you probably think it
>> is"?
>
> You mean, instead of using the first NEWS we find, look in _all_ the
> possible places and print a warning if we find more than one NEWS
> file?

Again, "_all_ the possible places" seems vague, but please could I get
an answer to the one question above first? It seems crucial to me for
further discussion.

Thanks,

-- 
Štěpán

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> I wasn't aware that Óscar meant to change entirely how Emacs finds all
> of its directories.  I still don't think that's what he meant.

FWIW, what I was trying to say is that I consider confusing (in the
sense of "wrong") that when I type C-h N the resulting document does not
correspond to what I just built and executed.

M-x describe-function shows the docstrings of the Emacs I'm executing,
which is the Right Thing. Likewise when I ask for the source code. I
would expect the same from C-h N, C-h i, etc. and not depend on the
existence of an install with the same version number which could be
quite out of sync with what I'm actually using.

If you consider that the change is difficult and not worth the hassle,
ok, I trust your judgement. But I fail to see a reason why making such
change would negatively affect anyone.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Štěpán Němec <stepnem@gmail.com>
> Cc: 36828@debbugs.gnu.org,  ofv@wanadoo.es
> Date: Mon, 29 Jul 2019 20:33:46 +0200
> 
> In the special case considered, do you agree that it would be better if
> Emacs recognized that it was started from the source tree and showed the
> corresponding NEWS file instead of the one installed?

In the situation described by Óscar, where the "installed" version
wound up there by accident, sure.  But I don't think Emacs can
discover that by just looking at the filesystem.

In a more general use case, where there are two copies of NEWS in two
places where Emacs looks, not so sure.  It could be the other way
around: the "installed" version could be a better candidate.  E.g.,
when I have installed some version on my systems, I make changes in
the installation tree, because it's simpler: it doesn't require me to
say "make install".

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Štěpán Němec]

On Mon, 29 Jul 2019 21:48:08 +0300
Eli Zaretskii wrote:

> In the situation described by Óscar, where the "installed" version
> wound up there by accident, sure.  But I don't think Emacs can
> discover that by just looking at the filesystem.

AFAICT we're only talking about one special case: running from source. I
can't see why Emacs couldn't handle that one special case (and prefer
the source tree to `configure'd paths), but I guess the only way to
really know is to study the source code, indeed.

> In a more general use case, where there are two copies of NEWS in two
> places where Emacs looks, not so sure.  It could be the other way
> around: the "installed" version could be a better candidate.  E.g.,
> when I have installed some version on my systems, I make changes in
> the installation tree, because it's simpler: it doesn't require me to
> say "make install".

You mean you then copy the files back into VCS to track the changes?
That's simpler?

Anyway, in such a case I see no ambiguity: you're running the installed
emacs, accessing the installed data files as `configure'd.

-- 
Štěpán

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: Štěpán Němec <stepnem@gmail.com>,
>   36828@debbugs.gnu.org
> Date: Mon, 29 Jul 2019 20:42:06 +0200
> 
> FWIW, what I was trying to say is that I consider confusing (in the
> sense of "wrong") that when I type C-h N the resulting document does not
> correspond to what I just built and executed.
> 
> M-x describe-function shows the docstrings of the Emacs I'm executing,

How do you know?  What is the value of doc-directory?

> I fail to see a reason why making such change would negatively
> affect anyone.

You'll need to describe that change in more detail, then we could see
who or what it will affect.  I don't think we got to those details
yet.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Štěpán Němec <stepnem@gmail.com>
> Cc: 36828@debbugs.gnu.org, ofv@wanadoo.es
> Date: Mon, 29 Jul 2019 21:20:51 +0200
> 
> > In a more general use case, where there are two copies of NEWS in two
> > places where Emacs looks, not so sure.  It could be the other way
> > around: the "installed" version could be a better candidate.  E.g.,
> > when I have installed some version on my systems, I make changes in
> > the installation tree, because it's simpler: it doesn't require me to
> > say "make install".
> 
> You mean you then copy the files back into VCS to track the changes?

I didn't say anything about a VCS.  I was talking about building a
release tarball and then installing it.

> Anyway, in such a case I see no ambiguity: you're running the installed
> emacs, accessing the installed data files as `configure'd.

I can also run the same binary from the build directory. and I still
want to use the updated files in the installation tree.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Cc: Štěpán Němec <stepnem@gmail.com>,
>>   36828@debbugs.gnu.org
>> Date: Mon, 29 Jul 2019 20:42:06 +0200
>> 
>> FWIW, what I was trying to say is that I consider confusing (in the
>> sense of "wrong") that when I type C-h N the resulting document does not
>> correspond to what I just built and executed.
>> 
>> M-x describe-function shows the docstrings of the Emacs I'm executing,
>
> How do you know?  What is the value of doc-directory?

/home/oscar/dev/emacs/stable/build/etc/

The installed version is in /usr/local.

>> I fail to see a reason why making such change would negatively
>> affect anyone.
>
> You'll need to describe that change in more detail, then we could see
> who or what it will affect.  I don't think we got to those details
> yet.

Are you asking for the description in changes to the source code or to
the procedure? In any case I haven't studied the current procedure so I
don't know what must be changed, I just can describe, for this specific
case, what happens and what IMAO is the correct thing to do, which I've
already done.

A cursory look at view-emacs-news shows that it uses data-directory to
locate the file, and data-directory is

/usr/local/share/emacs/27.0.50/etc/

in my uninstalled build.

A quick look at info.el hints that data-directory is also used for
locating the info file.

The change would amount to determine if we are running from an
uninstalled build and, if positive, set data-directory et al. to the
values they would have if no installed instance existed.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Mon, 29 Jul 2019 23:09:09 +0200
> 
> >> M-x describe-function shows the docstrings of the Emacs I'm executing,
> >
> > How do you know?  What is the value of doc-directory?
> 
> /home/oscar/dev/emacs/stable/build/etc/
> 
> The installed version is in /usr/local.

Hmm... then this is not the use case I had in mind, and some other
factor is at work here.  Your use case is supposed to be handled by
this fragment from init_callproc:

  if (data_dir == 0)
    {
      Lisp_Object tem, tem1, srcdir;
      Lisp_Object lispdir = Fcar (decode_env_path (0, PATH_DUMPLOADSEARCH, 0));

      srcdir = Fexpand_file_name (build_string ("../src/"), lispdir);

      tem = Fexpand_file_name (build_string ("NEWS"), Vdata_directory);
      tem1 = Ffile_exists_p (tem);
      if (!NILP (Fequal (srcdir, Vinvocation_directory)) || NILP (tem1))
	{
	  Lisp_Object newdir;
	  newdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
	  tem = Fexpand_file_name (build_string ("NEWS"), newdir);
	  tem1 = Ffile_exists_p (tem);
	  if (!NILP (tem1))
	    Vdata_directory = newdir;
	}
    }

So somehow the call to Fequal returns nil in your case, when it
shouldn't.  Please step with a debugger into this code and see why it
fails to detect that you are invoking Emacs from the source directory.
My crystal ball says some symlinks are involved.

> A quick look at info.el hints that data-directory is also used for
> locating the info file.

Only for the NS build.

> The change would amount to determine if we are running from an
> uninstalled build and, if positive, set data-directory et al. to the
> values they would have if no installed instance existed.

That's the code above, but it somehow isn't working in your case.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

Ping!  Could we please make some progress in this matter?  Can you
please answer my questions below?  TIA

> Date: Tue, 30 Jul 2019 18:08:31 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 36828@debbugs.gnu.org, stepnem@gmail.com
> 
> > From: Óscar Fuentes <ofv@wanadoo.es>
> > Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> > Date: Mon, 29 Jul 2019 23:09:09 +0200
> > 
> > >> M-x describe-function shows the docstrings of the Emacs I'm executing,
> > >
> > > How do you know?  What is the value of doc-directory?
> > 
> > /home/oscar/dev/emacs/stable/build/etc/
> > 
> > The installed version is in /usr/local.
> 
> Hmm... then this is not the use case I had in mind, and some other
> factor is at work here.  Your use case is supposed to be handled by
> this fragment from init_callproc:
> 
>   if (data_dir == 0)
>     {
>       Lisp_Object tem, tem1, srcdir;
>       Lisp_Object lispdir = Fcar (decode_env_path (0, PATH_DUMPLOADSEARCH, 0));
> 
>       srcdir = Fexpand_file_name (build_string ("../src/"), lispdir);
> 
>       tem = Fexpand_file_name (build_string ("NEWS"), Vdata_directory);
>       tem1 = Ffile_exists_p (tem);
>       if (!NILP (Fequal (srcdir, Vinvocation_directory)) || NILP (tem1))
> 	{
> 	  Lisp_Object newdir;
> 	  newdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
> 	  tem = Fexpand_file_name (build_string ("NEWS"), newdir);
> 	  tem1 = Ffile_exists_p (tem);
> 	  if (!NILP (tem1))
> 	    Vdata_directory = newdir;
> 	}
>     }
> 
> So somehow the call to Fequal returns nil in your case, when it
> shouldn't.  Please step with a debugger into this code and see why it
> fails to detect that you are invoking Emacs from the source directory.
> My crystal ball says some symlinks are involved.
> 
> > A quick look at info.el hints that data-directory is also used for
> > locating the info file.
> 
> Only for the NS build.
> 
> > The change would amount to determine if we are running from an
> > uninstalled build and, if positive, set data-directory et al. to the
> > values they would have if no installed instance existed.
> 
> That's the code above, but it somehow isn't working in your case.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> Ping!  Could we please make some progress in this matter?  Can you
> please answer my questions below?  TIA

It is in my to-do queue.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: 36828@debbugs.gnu.org,  stepnem@gmail.com
> Date: Wed, 07 Aug 2019 17:00:14 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Ping!  Could we please make some progress in this matter?  Can you
> > please answer my questions below?  TIA
> 
> It is in my to-do queue.

Thanks, will wait patiently.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Sorry for the long delay.

Eli Zaretskii <eliz@gnu.org> writes:

>> >> M-x describe-function shows the docstrings of the Emacs I'm executing,
>> >
>> > How do you know?  What is the value of doc-directory?
>> 
>> /home/oscar/dev/emacs/stable/build/etc/
>> 
>> The installed version is in /usr/local.
>
> Hmm... then this is not the use case I had in mind, and some other
> factor is at work here.  Your use case is supposed to be handled by
> this fragment from init_callproc:

What follows is the result of a debug session for the chunk of code you
indicated.

  if (data_dir == 0)
    {
      Lisp_Object tem, srcdir;
      Lisp_Object lispdir = Fcar (decode_env_path (0, PATH_DUMPLOADSEARCH, 0));

      srcdir = Fexpand_file_name (build_string ("../src/"), lispdir);

      tem = Fexpand_file_name (build_string ("NEWS"), Vdata_directory);

Vdata_directory is "/usr/local/share/emacs/27.0.50/etc/"

tem is "/usr/local/share/emacs/27.0.50/etc/NEWS"

srcdir is "/home/oscar/dev/emacs/emacs/src/"

Vinvocation_directory is "/home/oscar/dev/emacs/debug/src/"

      if (!NILP (Fequal (srcdir, Vinvocation_directory))
	  || NILP (Ffile_exists_p (tem)))
	{

Execution never enters this block, because srcdir !=
Vinvocation_directory (because this is an out-of-source build) and
"/usr/local/share/emacs/27.0.50/etc/NEWS" exists (because emacs was
installed on the past).

	  Lisp_Object newdir;
	  newdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
	  tem = Fexpand_file_name (build_string ("NEWS"), newdir);
	  if (!NILP (Ffile_exists_p (tem)))
	    Vdata_directory = newdir;
	}
    }

The code basically ignores the local, non-installed NEWS when there is
an installed NEWS and we are executing from an out-of-source build tree.

I propose the reverse: unconditionally test for the non-installed NEWS and
adjust Vdata_directory if it exists. Something like this (untested for
the installed emacs case):

modified   src/callproc.c
@@ -1560,21 +1560,13 @@ init_callproc (void)
      source directory.  */
   if (data_dir == 0)
     {
-      Lisp_Object tem, srcdir;
+      Lisp_Object tem, etcdir;
       Lisp_Object lispdir = Fcar (decode_env_path (0, PATH_DUMPLOADSEARCH, 0));
 
-      srcdir = Fexpand_file_name (build_string ("../src/"), lispdir);
-
-      tem = Fexpand_file_name (build_string ("NEWS"), Vdata_directory);
-      if (!NILP (Fequal (srcdir, Vinvocation_directory))
-	  || NILP (Ffile_exists_p (tem)))
-	{
-	  Lisp_Object newdir;
-	  newdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
-	  tem = Fexpand_file_name (build_string ("NEWS"), newdir);
-	  if (!NILP (Ffile_exists_p (tem)))
-	    Vdata_directory = newdir;
-	}
+      etcdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
+      tem = Fexpand_file_name (build_string ("NEWS"), etcdir);
+      if (!NILP (Ffile_exists_p (tem)))
+        Vdata_directory = newdir;
     }
 
   if (!will_dump_p ())

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

The previous patch contains an undefined variable result of an
incomplete renaming. Fixed here:

modified   src/callproc.c
@@ -1560,21 +1560,13 @@ init_callproc (void)
      source directory.  */
   if (data_dir == 0)
     {
-      Lisp_Object tem, srcdir;
+      Lisp_Object tem, etcdir;
       Lisp_Object lispdir = Fcar (decode_env_path (0, PATH_DUMPLOADSEARCH, 0));
 
-      srcdir = Fexpand_file_name (build_string ("../src/"), lispdir);
-
-      tem = Fexpand_file_name (build_string ("NEWS"), Vdata_directory);
-      if (!NILP (Fequal (srcdir, Vinvocation_directory))
-	  || NILP (Ffile_exists_p (tem)))
-	{
-	  Lisp_Object newdir;
-	  newdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
-	  tem = Fexpand_file_name (build_string ("NEWS"), newdir);
-	  if (!NILP (Ffile_exists_p (tem)))
-	    Vdata_directory = newdir;
-	}
+      etcdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
+      tem = Fexpand_file_name (build_string ("NEWS"), etcdir);
+      if (!NILP (Ffile_exists_p (tem)))
+        Vdata_directory = etcdir;
     }
 
   if (!will_dump_p ())

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Sat, 26 Oct 2019 05:16:01 +0200
> 
> Sorry for the long delay.

I no longer remember the details from what I discovered back there, so
expect some repeated questions, mistakes, and confusion.  Apologies in
advance.

> >> >> M-x describe-function shows the docstrings of the Emacs I'm executing,
> >> >
> >> > How do you know?  What is the value of doc-directory?
> >> 
> >> /home/oscar/dev/emacs/stable/build/etc/
> >> 
> >> The installed version is in /usr/local.
> >
> > Hmm... then this is not the use case I had in mind, and some other
> > factor is at work here.  Your use case is supposed to be handled by
> > this fragment from init_callproc:
> 
> What follows is the result of a debug session for the chunk of code you
> indicated.
> 
>   if (data_dir == 0)
>     {
>       Lisp_Object tem, srcdir;
>       Lisp_Object lispdir = Fcar (decode_env_path (0, PATH_DUMPLOADSEARCH, 0));
> 
>       srcdir = Fexpand_file_name (build_string ("../src/"), lispdir);
> 
>       tem = Fexpand_file_name (build_string ("NEWS"), Vdata_directory);
> 
> Vdata_directory is "/usr/local/share/emacs/27.0.50/etc/"
> 
> tem is "/usr/local/share/emacs/27.0.50/etc/NEWS"
> 
> srcdir is "/home/oscar/dev/emacs/emacs/src/"

I note that the directories you show now are different from what you
reported back then: more place for confusion.

> Execution never enters this block, because srcdir !=
> Vinvocation_directory (because this is an out-of-source build) and
> "/usr/local/share/emacs/27.0.50/etc/NEWS" exists (because emacs was
> installed on the past).
> 
> 	  Lisp_Object newdir;
> 	  newdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
> 	  tem = Fexpand_file_name (build_string ("NEWS"), newdir);
> 	  if (!NILP (Ffile_exists_p (tem)))
> 	    Vdata_directory = newdir;
> 	}
>     }
> 
> The code basically ignores the local, non-installed NEWS when there is
> an installed NEWS and we are executing from an out-of-source build tree.
> 
> I propose the reverse: unconditionally test for the non-installed NEWS and
> adjust Vdata_directory if it exists. Something like this (untested for
> the installed emacs case):

I'm not sure I understand the relevance of data-directory to the issue
at hand.  AFAIR you were talking about the Info documentation, but
that is looked up in configure-info-directory; see
Info-default-directory-list.  We only use data-directory for that in
the NS builds, which I believe is not your case.

If this all is due to the fact I pointed to this code, then either I
was mistaken (sorry!), or there's something I discovered back then,
but forgot in the meantime.

So I guess we need to back up a notch and see how all this is related
to Info manuals?  Or what else am I missing?

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> I no longer remember the details from what I discovered back there, so
> expect some repeated questions, mistakes, and confusion.  Apologies in
> advance.

It's all my fault for delaying this task for a long time.

> I note that the directories you show now are different from what you
> reported back then: more place for confusion.

I made a new build directory for a debug build. The only thing that
changes is `debug' instead of `build' on the paths.

>> I propose the reverse: unconditionally test for the non-installed NEWS and
>> adjust Vdata_directory if it exists. Something like this (untested for
>> the installed emacs case):
>
> I'm not sure I understand the relevance of data-directory to the issue
> at hand.

data-directory is used for locating the NEWS file and, IIRC, other
documentation.

> AFAIR you were talking about the Info documentation,

No, it was about NEWS. Although making it work for the Info
documentation would be nice too (my proposed change does not provide
that).

> but that is looked up in configure-info-directory; see
> Info-default-directory-list. We only use data-directory for that in
> the NS builds, which I believe is not your case.

It is not, indeed.

> If this all is due to the fact I pointed to this code, then either I
> was mistaken (sorry!), or there's something I discovered back then,
> but forgot in the meantime.
>
> So I guess we need to back up a notch and see how all this is related
> to Info manuals?  Or what else am I missing?

For the time being, I propose that we concentrate on NEWS, which
motivated this bug report. Then we can move on to the Info
documentation.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Sat, 26 Oct 2019 13:55:11 +0200
> 
> > I'm not sure I understand the relevance of data-directory to the issue
> > at hand.
> 
> data-directory is used for locating the NEWS file and, IIRC, other
> documentation.
> 
> > AFAIR you were talking about the Info documentation,
> 
> No, it was about NEWS. Although making it work for the Info
> documentation would be nice too (my proposed change does not provide
> that).

It sounds strange to solve only the issue with NEWS, but not with
other documentation.

> For the time being, I propose that we concentrate on NEWS, which
> motivated this bug report. Then we can move on to the Info
> documentation.

OK, but then the logic in your proposed change changes too radically
what we've been doing until now: it will cause Emacs to use the NEWS
in the source tree even if you run the installed binary.  That is not
TRT, IMO.

We need some way of detecting the situation whereby Emacs is being run
from a separate build directory, and using the NEWS from the source
tree only then, as fallback.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> It sounds strange to solve only the issue with NEWS, but not with
> other documentation.

Agreed.

>> For the time being, I propose that we concentrate on NEWS, which
>> motivated this bug report. Then we can move on to the Info
>> documentation.
>
> OK, but then the logic in your proposed change changes too radically
> what we've been doing until now: it will cause Emacs to use the NEWS
> in the source tree even if you run the installed binary.  That is not
> TRT, IMO.

If my change causes that, it is no acceptable, indeed.

> We need some way of detecting the situation whereby Emacs is being run
> from a separate build directory, and using the NEWS from the source
> tree only then, as fallback.

Right now I must leave so a quick question: what about beginning with
setting data-directory to the source tree in init_callproc_1 when we are
running from a non-installed Emacs? (Assuming that we can detect that
condition)

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Sat, 26 Oct 2019 15:27:00 +0200
> 
> > OK, but then the logic in your proposed change changes too radically
> > what we've been doing until now: it will cause Emacs to use the NEWS
> > in the source tree even if you run the installed binary.  That is not
> > TRT, IMO.
> 
> If my change causes that, it is no acceptable, indeed.

AFAICT, decode_env_path (0, PATH_DUMPLOADSEARCH, 0) returns the lisp
directory in the source tree, even in the installed binary, as I'd
expect.  Then ../etc relative to that will point to the source tree as
well.

> > We need some way of detecting the situation whereby Emacs is being run
> > from a separate build directory, and using the NEWS from the source
> > tree only then, as fallback.
> 
> Right now I must leave so a quick question: what about beginning with
> setting data-directory to the source tree in init_callproc_1 when we are
> running from a non-installed Emacs? (Assuming that we can detect that
> condition)

We currently detect that by the condition that doesn't work in your
case:

      if (!NILP (Fequal (srcdir, Vinvocation_directory))

If we solve that problem, we can solve your problem as side effect,
right?

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Right now I must leave so a quick question: what about beginning with
>> setting data-directory to the source tree in init_callproc_1 when we are
>> running from a non-installed Emacs? (Assuming that we can detect that
>> condition)
>
> We currently detect that by the condition that doesn't work in your
> case:
>
>       if (!NILP (Fequal (srcdir, Vinvocation_directory))
>
> If we solve that problem, we can solve your problem as side effect,
> right?

Yes, I checked that forcing that `if' to true works for NEWS, but I
wonder why we assign Vdata_directory on init_callproc_1 to the installed
Emacs directory ("/usr/local/share/emacs/27.0.50/etc/" on this case)
when we are running a non-installed Emacs. Having Vdata_directory around
pointing to the "wrong" directory looks like a potential problem. And
IMO it is also undesirable to assign it on two different functions, if
we can avoid that.

BTW, fixing the `if' mentioned above also causes C-h r, C-h t and
probably others to use the non-installed docs. Buth C-h i still shows
the directory of info files of the installed version. I guess that
without a `dir' file on the non-installed doc directory we can't solve
this, right?

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Sat, 26 Oct 2019 22:05:03 +0200
> 
> I wonder why we assign Vdata_directory on init_callproc_1 to the
> installed Emacs directory ("/usr/local/share/emacs/27.0.50/etc/" on
> this case) when we are running a non-installed Emacs.

I thought the comments there explained that?

> BTW, fixing the `if' mentioned above also causes C-h r, C-h t and
> probably others to use the non-installed docs. Buth C-h i still shows
> the directory of info files of the installed version. I guess that
> without a `dir' file on the non-installed doc directory we can't solve
> this, right?

But we do have 'dir' in the source tree, it's in info/.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> I wonder why we assign Vdata_directory on init_callproc_1 to the
>> installed Emacs directory ("/usr/local/share/emacs/27.0.50/etc/" on
>> this case) when we are running a non-installed Emacs.
>
> I thought the comments there explained that?

It is not clear to me why assigning Vdata_directory must wait until
init_callproc when we are running a non-installed emacs. It looks like
all the required information is present.

Furthermore, the logic should be "first check if this is an
non-installed emacs; if false, locate the installed stuff" instead of
"first locate the installed stuff, then check if we are a non-installed
emacs and fix the vars we assigned previously."

> But we do have 'dir' in the source tree, it's in info/.

Right, thanks. So I must check why C-h i fetches the `dir' in the
installed info directory.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Sat, 26 Oct 2019 22:40:59 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I wonder why we assign Vdata_directory on init_callproc_1 to the
> >> installed Emacs directory ("/usr/local/share/emacs/27.0.50/etc/" on
> >> this case) when we are running a non-installed Emacs.
> >
> > I thought the comments there explained that?
> 
> It is not clear to me why assigning Vdata_directory must wait until
> init_callproc when we are running a non-installed emacs. It looks like
> all the required information is present.

We currently decide about non-installed by looking at
Vinstallation_directory, and that is not set in init_callproc_1, per
the comments.

> Furthermore, the logic should be "first check if this is an
> non-installed emacs; if false, locate the installed stuff" instead of
> "first locate the installed stuff, then check if we are a non-installed
> emacs and fix the vars we assigned previously."

I prefer not to consider such radical changes in this logic, because
we've been using what's there now for ages.  Who knows how many
legitimate use cases this could potentially break?  Your use case is
quite specialized: having an older binary of the exact same version
installed is unusual, IME.  So radical changes due to this use case
are unjustified, IMO.

Let's try to find a way of detecting your situation without making too
many waves elsewhere, okay?

> > But we do have 'dir' in the source tree, it's in info/.
> 
> Right, thanks. So I must check why C-h i fetches the `dir' in the
> installed info directory.

I think it's just because configure-info-directory is set by epaths.h
to point to ${prefix}/share/info (via PATH_INFO).  We never change
that, even in a non-installed Emacs.  See Info-default-directory-list.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> I prefer not to consider such radical changes in this logic, because
> we've been using what's there now for ages.  Who knows how many
> legitimate use cases this could potentially break?  Your use case is
> quite specialized: having an older binary of the exact same version
> installed is unusual, IME.  So radical changes due to this use case
> are unjustified, IMO.
>
> Let's try to find a way of detecting your situation without making too
> many waves elsewhere, okay?

Once we know that we are running a non-installed emacs, detecting that
it is an out-of-source build seems quite easy, because the build
directory contains build artifacts without the sources. Checking for the
presence of Makefile and the absence of Makefile.in, for instance,
should be enough.

Said that, I'll like to clarify that this is nothing about me. This
issue caught me by surprise once and, now that I know about it, it is
not a problem anymore. In fact, because your past comments about its
small relevance I closed the bug and it was reopened by other
participant who thinks (as I do) that the use case is not as rare as you
think. The difference between that other participant and me seems to be
that I don't consider this issue worth of arguing about. If you close it
as wontfix, I'm fine. If you think it is worth fixing, I'll try to
provide an acceptable patch.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Mon, 28 Oct 2019 00:50:11 +0100
> 
> > Let's try to find a way of detecting your situation without making too
> > many waves elsewhere, okay?
> 
> Once we know that we are running a non-installed emacs, detecting that
> it is an out-of-source build seems quite easy, because the build
> directory contains build artifacts without the sources.

Yes.  But how to detect that we are running non-installed?

> Said that, I'll like to clarify that this is nothing about me. This
> issue caught me by surprise once and, now that I know about it, it is
> not a problem anymore. In fact, because your past comments about its
> small relevance I closed the bug and it was reopened by other
> participant who thinks (as I do) that the use case is not as rare as you
> think. The difference between that other participant and me seems to be
> that I don't consider this issue worth of arguing about. If you close it
> as wontfix, I'm fine. If you think it is worth fixing, I'll try to
> provide an acceptable patch.

I'm not arguing against finding a solution to this particular
situation.  So if you can work on a patch to detect it, please do.
I'm just saying that the change cannot unconditionally prefer the
files from the source tree, because an installed binary shouldn't do
that, it should use the files that are part of its installation.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Once we know that we are running a non-installed emacs, detecting that
>> it is an out-of-source build seems quite easy, because the build
>> directory contains build artifacts without the sources.
>
> Yes.  But how to detect that we are running non-installed?

Checking for the presence of the "Makefile" file using the same method
we use for "NEWS".

> I'm not arguing against finding a solution to this particular
> situation.  So if you can work on a patch to detect it, please do.
> I'm just saying that the change cannot unconditionally prefer the
> files from the source tree, because an installed binary shouldn't do
> that, it should use the files that are part of its installation.

Agreed.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Mon, 28 Oct 2019 20:40:03 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Once we know that we are running a non-installed emacs, detecting that
> >> it is an out-of-source build seems quite easy, because the build
> >> directory contains build artifacts without the sources.
> >
> > Yes.  But how to detect that we are running non-installed?
> 
> Checking for the presence of the "Makefile" file using the same method
> we use for "NEWS".

Presence where?

Anyway, Makefile is too common a file, I think we should look for
something more specific to Emacs.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Checking for the presence of the "Makefile" file using the same method
>> we use for "NEWS".
>
> Presence where?

In installation-directory (provided that it is not nil).

> Anyway, Makefile is too common a file, I think we should look for
> something more specific to Emacs.

We have plenty to choose (lib-src/emacsclient, for instance) but IMO if
Makefile exists in installation-directory that's a sure sign.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Mon, 28 Oct 2019 21:21:21 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Checking for the presence of the "Makefile" file using the same method
> >> we use for "NEWS".
> >
> > Presence where?
> 
> In installation-directory (provided that it is not nil).
> 
> > Anyway, Makefile is too common a file, I think we should look for
> > something more specific to Emacs.
> 
> We have plenty to choose (lib-src/emacsclient, for instance) but IMO if
> Makefile exists in installation-directory that's a sure sign.

If it is not nil, and the test of equality between srcdir and
invocation-directory fails, then you already know that you are running
uninstalled, but outside of the src directory.  Right?  Then you don't
need looking for Makefile or anything else, right?

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> We have plenty to choose (lib-src/emacsclient, for instance) but IMO if
>> Makefile exists in installation-directory that's a sure sign.
>
> If it is not nil, and the test of equality between srcdir and
> invocation-directory fails, then you already know that you are running
> uninstalled, but outside of the src directory.  Right?  Then you don't
> need looking for Makefile or anything else, right?

Indeed, this simple patch does the trick:

modified   src/callproc.c
@@ -1567,7 +1567,7 @@ init_callproc (void)
 
       tem = Fexpand_file_name (build_string ("NEWS"), Vdata_directory);
       if (!NILP (Fequal (srcdir, Vinvocation_directory))
-	  || NILP (Ffile_exists_p (tem)))
+	  || NILP (Ffile_exists_p (tem)) || !NILP (Vinstallation_directory))
 	{
 	  Lisp_Object newdir;
 	  newdir = Fexpand_file_name (build_string ("../etc/"), lispdir);

But then we can also get rid of

!NILP (Fequal (srcdir, Vinvocation_directory))

because that condition should always be true when Vinvocation_directory
is non-nil, right?

Actually, we can remove the `if' altogether because if we enter

  if (data_dir == 0)
    {

then we know that this is a non-installed emacs and even if we end
assigning a non-existent directory to Vdata_directory within that `if'
(for reasons I can't imagine) it is not a regression: Vdata_directory
would contain a wrong directory anyways.

All that function looks unnecessarily complex to me. It comes from a
long time ago and it smells like it didn't adapt to the availability of
new variables. I'm a bit reluctant to adding yet another condition and
cause future hackers to scratch their heads trying to figure out whas is
about with the redundant stuff.

Said that, if you don't want me to touch other parts of the function and
just add the new condition, ok. Later I'll try to adapt C-h i to show
the non-installed `dir' file.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Once we know that we are running a non-installed emacs, detecting that
  > > it is an out-of-source build seems quite easy, because the build
  > > directory contains build artifacts without the sources.

  > Yes.  But how to detect that we are running non-installed?

I think the variable 'installation-directory' should give the answer.

-- 
Dr Richard Stallman
Founder, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Cc: stepnem@gmail.com,  36828@debbugs.gnu.org
> Date: Mon, 28 Oct 2019 22:45:44 +0100
> 
> Indeed, this simple patch does the trick:
> 
> modified   src/callproc.c
> @@ -1567,7 +1567,7 @@ init_callproc (void)
>  
>        tem = Fexpand_file_name (build_string ("NEWS"), Vdata_directory);
>        if (!NILP (Fequal (srcdir, Vinvocation_directory))
> -	  || NILP (Ffile_exists_p (tem)))
> +	  || NILP (Ffile_exists_p (tem)) || !NILP (Vinstallation_directory))
>  	{
>  	  Lisp_Object newdir;
>  	  newdir = Fexpand_file_name (build_string ("../etc/"), lispdir);
> 
> But then we can also get rid of
> 
> !NILP (Fequal (srcdir, Vinvocation_directory))
> 
> because that condition should always be true when Vinvocation_directory
> is non-nil, right?

We could, but I'd like to leave it for now, with a FIXME comment
saying whether this condition is necessary.  The reason is that I'm
not sure whether there aren't any additional use cases which this
condition allows.

> Actually, we can remove the `if' altogether because if we enter
> 
>   if (data_dir == 0)
>     {
> 
> then we know that this is a non-installed emacs and even if we end
> assigning a non-existent directory to Vdata_directory within that `if'
> (for reasons I can't imagine) it is not a regression: Vdata_directory
> would contain a wrong directory anyways.

Sorry, I don't follow: data_dir being NULL just means EMACSDATA is not
defined in the environment.  If EMACSDATA is defined, its value tramps
everything else, and we still want to support that.  How is EMACSDATA
related to the issue at hand? it's a completely separate use case.

> All that function looks unnecessarily complex to me. It comes from a
> long time ago and it smells like it didn't adapt to the availability of
> new variables. I'm a bit reluctant to adding yet another condition and
> cause future hackers to scratch their heads trying to figure out whas is
> about with the redundant stuff.

I don't think I agree.  That function calculates values of several
variables:

  exec-path
  exec-directory
  doc-directory
  data-directory
  shell-file-name
  shared-game-score-directory

It also emits warnings if exec-directory or data-directory are
inaccessible.  Each of the above variables takes just a few lines of
code to compute, so I don't think I see the complexity, let alone an
unnecessary one.  Can you elaborate on what looks redundant there, and
how would you suggest to use the available new variables?

> Said that, if you don't want me to touch other parts of the function and
> just add the new condition, ok. Later I'll try to adapt C-h i to show
> the non-installed `dir' file.

Yes, let's for now make a simple and safe change.

Thanks.

bug#36828: 27.0.50; Uninstalled emacs shows installed documentation

[Óscar Fuentes]

Fixed in master e6b806f29e06e472a44763def6b4d904b5f1ba54.

C-h i still does not show the non-installed info directory.