bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Drew Adams]

C-h r
g Glossary
Go to entry "Cut and Paste".

There you see this:

Cut and Paste
     *Note Glossary---Killing::, and *note Glossary---Yanking::.

But clicking those links does not take you to the Glossary entries for
Killing and Yanking, respectively.

Instead, the first one takes you to node Acknowledgements, and the
second one takes you to node `Key Index'.

This regression was introduced in Emacs 24.5.  Prior to that, in Emacs
24 the links worked fine.



In GNU Emacs 26.1 (build 1, x86_64-w64-mingw32)
 of 2018-04-10
Repository revision: c267421647510319d2a70554e42f0d1c394dba0a
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install 'CFLAGS=-O2 -static -g3''

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Robert Pluim]

Drew Adams <drew.adams@oracle.com> writes:

> C-h r
> g Glossary
> Go to entry "Cut and Paste".
>
> There you see this:
>
> Cut and Paste
>      *Note Glossary---Killing::, and *note Glossary---Yanking::.
>
> But clicking those links does not take you to the Glossary entries for
> Killing and Yanking, respectively.
>
> Instead, the first one takes you to node Acknowledgements, and the
> second one takes you to node `Key Index'.
>
> This regression was introduced in Emacs 24.5.  Prior to that, in Emacs
> 24 the links worked fine.

This works fine for me in emacs-26 -Q, admittedly on GNU/Linux, not
Windows.

Robert

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Thu, 12 Apr 2018 10:21:43 +0200
> Cc: 31130@debbugs.gnu.org
> 
> Drew Adams <drew.adams@oracle.com> writes:
> 
> > C-h r
> > g Glossary
> > Go to entry "Cut and Paste".
> >
> > There you see this:
> >
> > Cut and Paste
> >      *Note Glossary---Killing::, and *note Glossary---Yanking::.
> >
> > But clicking those links does not take you to the Glossary entries for
> > Killing and Yanking, respectively.
> >
> > Instead, the first one takes you to node Acknowledgements, and the
> > second one takes you to node `Key Index'.
> >
> > This regression was introduced in Emacs 24.5.  Prior to that, in Emacs
> > 24 the links worked fine.
> 
> This works fine for me in emacs-26 -Q, admittedly on GNU/Linux, not
> Windows.

It shouldn't matter, because the Info files come from the source
tarball in this case.

Anyway, I cannot reproduce this on Windows, either, neither with Emacs
26.1 which I built myself, not with the binary from the GNU site.
Drew, are you sure you are reading the Info files that came with the
Emacs binary whose details you included in the report?  ISTR that you
reported in the past similar problems, and each time the reason turned
out to be old Info files and/or old Emacs versions.  Both Texinfo and
Emacs changed a few years ago how offsets of index entries are
calculated and used, and the change matters when the Info file has DOS
CR-LF line endings, so now old Info files and new Emacs might be
incompatible.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Drew Adams]

> > 1. Trying again now, I can repro the problem only in Emacs 24.5,
> >    not in the more recent releases/builds.
> 
> That could be, but I guess it means you reported the bug from a binary
> other than the one where the problem happened?

No.  I reported it from the Emacs 26 RC1, and the problem
happened in that version.  And I tested it with `emacs -Q'
in that version.  I likely reported it (M-x report-emacs-bug)
using my setup, but with the same binary that I tested using
emacs -Q.  And I tested using emacs -Q also in Emacs 25.3.1,
24.5, 24.4, 23.4, and even 22.3 and 20.

> > 2. I'm sure I tested those multiple builds, using `emacs -Q',
> >    going back to see whether it ever worked and, if so, in
> >    what release it became broken.  IIRC, found that prior to
> >    Emacs 24 those terms were not links; in Emacs 24.4 the
> >    links worked correctly; and starting with Emacs 24.5,
> >    through 27 (3rd snapshot) the links were broken.
> >
> > 3. I don't know how or why I saw different behavior then
> >    than now, when I retest.
> 
> We've been through that before.

Have we?  Please tell me how what I saw is possible, in
that case.

> May I suggest that next time this happens you keep notes
> about the exact steps you took while reproducing the problem?
>
> >    I don't know what you mean,
> >    about somehow reading Info files that didn't come with
> >    the same binary.  How would that even happen?
> 
> It depends on how your system is configured wrt multiple Emacs
> versions installed on it, and in particular how you invoke Info.  If
> you just type "C-h i" or "C-h r", the Info manual you get depends on
> whether you keep the share/info directories of different versions
> separate or not.

They are all separate.  Each Emacs version/release is in
a separate directory/folder, and all of it is there, in
the original subdirs (bin, etc, include,lib, libexec,
share, ssl).  All I do is extract the zip file
obtained from GNU Emacs (e.g. P. Lord's snapshots).  I
make no changes to the directories or their files.

(Even with my own setup I make no such changes - no
changes to the distributed files and their directories.
But with my own setup I do make changes to some Info
functions etc.)

So I repeat the question, how could it happen that I
would read an Info file that didn't come with the
current binary, when using emacs -Q?

Unless there is some kind of cache file that Emacs
uses for Info that is outside that directory, I don't
see how different versions could show the same Info
manual version when each is started with emacs -Q.

> Also, "C-h i" and "C-h r" go to different places by
> default.  This is why I always use "C-u C-h i", and then specify the
> Info file that corresponds to the Emacs version I'm running.

I used `C-h r' to simplify the repro recipe.  If invoked
from emacs -Q, how can that go elsewhere than the Emacs
manual for that binary?

> > Sorry for any confusion.  I don't really understand what's
> > causing the mixup in behavior, but I'm sure that I tested
> > each of the releases 24.5, 25.3.1, and 26 using `emacs -Q'.
> > But I confirm that doing that again now I don't see the
> > problem except in 24.5.
> 
> Then I guess we can close this bug.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Eli Zaretskii]

> Date: Thu, 12 Apr 2018 08:53:49 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: rpluim@gmail.com, 31130-done@debbugs.gnu.org
> 
> > > 1. Trying again now, I can repro the problem only in Emacs 24.5,
> > >    not in the more recent releases/builds.
> > 
> > That could be, but I guess it means you reported the bug from a binary
> > other than the one where the problem happened?
> 
> No.  I reported it from the Emacs 26 RC1, and the problem
> happened in that version.  And I tested it with `emacs -Q'
> in that version.  I likely reported it (M-x report-emacs-bug)
> using my setup, but with the same binary that I tested using
> emacs -Q.

So you are saying that either (a) something in your setup causes this
problem to happen, perhaps after some time the session is alive, or
(b) something outside of Emacs causes the problem, and that something
is not always happening, is that right?

> And I tested using emacs -Q also in Emacs 25.3.1,
> 24.5, 24.4, 23.4, and even 22.3 and 20.

Which means the (b) part above is more likely the culprit?

Are you reasonably sure that your reproduction steps are the same in
all of these attempts, whether successful or not?  Or is that yet
another potential reason for the problem to happen only sometimes?

> > > 3. I don't know how or why I saw different behavior then
> > >    than now, when I retest.
> > 
> > We've been through that before.
> 
> Have we?  Please tell me how what I saw is possible, in
> that case.

AFAIR, we ended up giving up on understanding the exact reasons back
then as well, because you were unable to reliably reproduce the
problem, except in an old version of Emacs or an old version of Info
files.  Exactly like now.

> > It depends on how your system is configured wrt multiple Emacs
> > versions installed on it, and in particular how you invoke Info.  If
> > you just type "C-h i" or "C-h r", the Info manual you get depends on
> > whether you keep the share/info directories of different versions
> > separate or not.
> 
> They are all separate.  Each Emacs version/release is in
> a separate directory/folder, and all of it is there, in
> the original subdirs (bin, etc, include,lib, libexec,
> share, ssl).  All I do is extract the zip file
> obtained from GNU Emacs (e.g. P. Lord's snapshots).  I
> make no changes to the directories or their files.
> 
> (Even with my own setup I make no such changes - no
> changes to the distributed files and their directories.
> But with my own setup I do make changes to some Info
> functions etc.)
> 
> So I repeat the question, how could it happen that I
> would read an Info file that didn't come with the
> current binary, when using emacs -Q?

Which Info files Emacs finds is determined by the code which computes
Info-default-directory-list, and on code in info-initialize (and its
subroutine Info-default-dirs).  In a nutshell, that makes Emacs
consider several alternative places until it finds one that "works".
So the answer to your question depends on the directories you have or
don't have on your system, out of those Emacs considers, something I
cannot know.  It also depends on whether you have INFOPATH defined in
the environment, and if you do, what is its value.

> Unless there is some kind of cache file that Emacs
> uses for Info that is outside that directory

There is no cache.

> I don't see how different versions could show the same Info manual
> version when each is started with emacs -Q.

If you read the code in the functions I pointed to, you will see that
it doesn't only consider directories under the directory where you
unzipped the archive.  It also looks in other places.

> > Also, "C-h i" and "C-h r" go to different places by
> > default.  This is why I always use "C-u C-h i", and then specify the
> > Info file that corresponds to the Emacs version I'm running.
> 
> I used `C-h r' to simplify the repro recipe.  If invoked
> from emacs -Q, how can that go elsewhere than the Emacs
> manual for that binary?

Again, I don't know enough to answer that.  But the only way to be in
full control of which manual file is loaded is to use "C-u C-h i".

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Drew Adams]

FYI -

I still don't understand why I could be accessing the wrong
Info file when using emacs -Q.

But I've figured out why I see the problem with releases
after 24.5 when using my setup: I need to update my version
of `Info-find-node-2'.  I followed the execution in the
debugger in both emacs -Q and my setup, both Emacs 26 RC1,
and came across the problem.

My version iks missing the code that was added to use
`filepos-to-bufferpos' to convert the target position from
the position in the tags file data to a buffer position.
I'll need to update that.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Drew Adams]

> > > ISTR that you reported in the past similar problems, and
> > > each time the reason turned out to be old Info files
> > > and/or old Emacs versions.
> >
> > I don't think so.  Please point me to a thread about that.
> 
> See the discussion in bug#29839.

Yes, thanks for that xref.  That sounds like exactly
the same problem.  Still no clue just what was going on.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Eli Zaretskii]

> Date: Thu, 12 Apr 2018 10:31:29 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: rpluim@gmail.com, 31130@debbugs.gnu.org
> 
> > > > ISTR that you reported in the past similar problems, and
> > > > each time the reason turned out to be old Info files
> > > > and/or old Emacs versions.
> > >
> > > I don't think so.  Please point me to a thread about that.
> > 
> > See the discussion in bug#29839.
> 
> Yes, thanks for that xref.  That sounds like exactly
> the same problem.  Still no clue just what was going on.

I think the fact that you have a private version of Info-find-node-2,
which is different from the one in stock info.el, explains the
behavior you see.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Drew Adams]

> > > This regression was introduced in Emacs 24.5.  Prior to that, in
> > > Emacs 24 the links worked fine.
> >
> > This works fine for me in emacs-26 -Q, admittedly on GNU/Linux, not
> > Windows.
> 
> It shouldn't matter, because the Info files come from the source
> tarball in this case.
> 
> Anyway, I cannot reproduce this on Windows, either, neither with Emacs
> 26.1 which I built myself, not with the binary from the GNU site.
>
> Drew, are you sure you are reading the Info files that came with the
> Emacs binary whose details you included in the report?  ISTR that you
> reported in the past similar problems, and each time the reason turned
> out to be old Info files and/or old Emacs versions.
>
> Both Texinfo and
> Emacs changed a few years ago how offsets of index entries are
> calculated and used, and the change matters when the Info file has DOS
> CR-LF line endings, so now old Info files and new Emacs might be
> incompatible.

1. Trying again now, I can repro the problem only in Emacs 24.5,
   not in the more recent releases/builds.

2. I'm sure I tested those multiple builds, using `emacs -Q',
   going back to see whether it ever worked and, if so, in
   what release it became broken.  IIRC, found that prior to
   Emacs 24 those terms were not links; in Emacs 24.4 the
   links worked correctly; and starting with Emacs 24.5,
   through 27 (3rd snapshot) the links were broken.

3. I don't know how or why I saw different behavior then
   than now, when I retest.  I don't know what you mean,
   about somehow reading Info files that didn't come with
   the same binary.  How would that even happen?

4. But if it can happen then yes, perhaps that is the cause.
   I often have Emacs 20, 24.5, and something recent (e.g.
   27) open at the outset, using my setup.  And it's likely
   that I've already used Info in one or more of those,
   before I notice a problem and then retest using `emacs -Q'.

   In this case, I know that I discovered this problem first
   in a build using my setup.  I don't recall whether it was
   Emacs 24.5 or 26, but I'm pretty sure it was 26.  However,
   I may have already used Info with Emacs 24.5 at some point.

Sorry for any confusion.  I don't really understand what's
causing the mixup in behavior, but I'm sure that I tested
each of the releases 24.5, 25.3.1, and 26 using `emacs -Q'.
But I confirm that doing that again now I don't see the
problem except in 24.5.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Eli Zaretskii]

> Date: Thu, 12 Apr 2018 07:42:47 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: drew.adams@oracle.com, 31130@debbugs.gnu.org
> 
> 1. Trying again now, I can repro the problem only in Emacs 24.5,
>    not in the more recent releases/builds.

That could be, but I guess it means you reported the bug from a binary
other than the one where the problem happened?

> 2. I'm sure I tested those multiple builds, using `emacs -Q',
>    going back to see whether it ever worked and, if so, in
>    what release it became broken.  IIRC, found that prior to
>    Emacs 24 those terms were not links; in Emacs 24.4 the
>    links worked correctly; and starting with Emacs 24.5,
>    through 27 (3rd snapshot) the links were broken.
> 
> 3. I don't know how or why I saw different behavior then
>    than now, when I retest.

We've been through that before.  May I suggest that next time this
happens you keep notes about the exact steps you took while
reproducing the problem?

>    I don't know what you mean,
>    about somehow reading Info files that didn't come with
>    the same binary.  How would that even happen?

It depends on how your system is configured wrt multiple Emacs
versions installed on it, and in particular how you invoke Info.  If
you just type "C-h i" or "C-h r", the Info manual you get depends on
whether you keep the share/info directories of different versions
separate or not.  Also, "C-h i" and "C-h r" go to different places by
default.  This is why I always use "C-u C-h i", and then specify the
Info file that corresponds to the Emacs version I'm running.

> Sorry for any confusion.  I don't really understand what's
> causing the mixup in behavior, but I'm sure that I tested
> each of the releases 24.5, 25.3.1, and 26 using `emacs -Q'.
> But I confirm that doing that again now I don't see the
> problem except in 24.5.

Then I guess we can close this bug.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Drew Adams]

> Drew, are you sure you are reading the Info files that came with the
> Emacs binary whose details you included in the report?

I thought I was.  I used emacs -Q, and all of the files
for each build are in the same directory.

> ISTR that you reported in the past similar problems, and
> each time the reason turned out to be old Info files
> and/or old Emacs versions.

I don't think so.  Please point me to a thread about that.

> Both Texinfo and Emacs changed a few years ago how offsets
> of index entries are calculated and used, and the change
> matters when the Info file has DOS CR-LF line endings, so
> now old Info files and new Emacs might be incompatible.

The only thing I recall that is similar to what you are
saying is that the Emacs snapshots/binaries made available
to us did not use the latest texinfo or whatever.  So what
I reported wrt curly quotes did not correspond to what you
thought should have been the case.  That was a problem
caused not by my setup but by the Emacs build (I don't
build Emacs).

Does that sound familiar?  Is it what you really meant?
If not, please do point out a thread where this was
discussed and the cause of the problem was found.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Eli Zaretskii]

> Date: Thu, 12 Apr 2018 09:16:05 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: drew.adams@oracle.com, 31130@debbugs.gnu.org
> 
> > ISTR that you reported in the past similar problems, and
> > each time the reason turned out to be old Info files
> > and/or old Emacs versions.
> 
> I don't think so.  Please point me to a thread about that.

See the discussion in bug#29839.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Drew Adams]

> > > > 1. Trying again now, I can repro the problem only in Emacs 24.5,
> > > >    not in the more recent releases/builds.
> > >
> > > That could be, but I guess it means you reported the bug from a
> > > binary other than the one where the problem happened?
> >
> > No.  I reported it from the Emacs 26 RC1, and the problem
> > happened in that version.  And I tested it with `emacs -Q'
> > in that version.  I likely reported it (M-x report-emacs-bug)
> > using my setup, but with the same binary that I tested using
> > emacs -Q.
> 
> So you are saying that either (a) something in your setup causes this
> problem to happen, perhaps after some time the session is alive, or
> (b) something outside of Emacs causes the problem, and that something
> is not always happening, is that right?

Dunno what happened.

> > And I tested using emacs -Q also in Emacs 25.3.1,
> > 24.5, 24.4, 23.4, and even 22.3 and 20.
> 
> Which means the (b) part above is more likely the culprit?

It suggests that, if (a) and (b) are the possibilities.

> Are you reasonably sure that your reproduction steps are the same in
> all of these attempts, whether successful or not?  Or is that yet
> another potential reason for the problem to happen only sometimes?

Reasonably sure, yes.  But I will try to pay more attention
to this.

I believe that I:

1. Discovered the problem with my setup, probably in E26.
(See my other msg about why my setup led to the problem,
which I just now figured out.)

2. Used emacs -Q in various releases, from 26 (and 27?)
back through to 20.  I was both (a) checking it is still
a problem and (b) trying to find out whether it ever was
OK and if so where the regression was introduced.
I found that the problem was there starting with Emacs 24.5,
that it worked in 24.4, and that before 24.4 the text was
not linked.

> AFAIR, we ended up giving up on understanding the exact reasons back
> then as well, because you were unable to reliably reproduce the
> problem, except in an old version of Emacs or an old version of Info
> files.  Exactly like now.

That's possible.  I don't recall.
 
> Which Info files Emacs finds is determined by the code which computes
> Info-default-directory-list, and on code in info-initialize (and its
> subroutine Info-default-dirs).  In a nutshell, that makes Emacs
> consider several alternative places until it finds one that "works".
> So the answer to your question depends on the directories you have or
> don't have on your system, out of those Emacs considers, something I
> cannot know.  It also depends on whether you have INFOPATH defined in
> the environment, and if you do, what is its value.

INFOPATH is nil.
Each Emacs build is in a separate directory, and the dir contents
are what is extracted from a Windows snapshot.  I don't modify
`Info-default-directory-list' or `info-initialize' when I use
emacs -Q.  (When I use my setup, I don't modify `info-initialize',
but Cygwin does modify `Info-default-directory-list' to add
"c:/cygwin/usr/info".)

> > I don't see how different versions could show the same Info manual
> > version when each is started with emacs -Q.
> 
> If you read the code in the functions I pointed to, you will see that
> it doesn't only consider directories under the directory where you
> unzipped the archive.  It also looks in other places.

Yes, but I don't do anything (when I use emacs -Q) that affects
any of those places.

bug#31130: 26; Regression: intra-glossary links broken in Emacs manual

[Drew Adams]

> > > > > ISTR that you reported in the past similar problems, and
> > > > > each time the reason turned out to be old Info files
> > > > > and/or old Emacs versions.
> > > >
> > > > I don't think so.  Please point me to a thread about that.
> > >
> > > See the discussion in bug#29839.
> >
> > Yes, thanks for that xref.  That sounds like exactly
> > the same problem.  Still no clue just what was going on.
> 
> I think the fact that you have a private version of Info-find-node-2,
> which is different from the one in stock info.el, explains the
> behavior you see.

That explains why I saw this using my setup.
But I don't see how it explains why I saw it using
emacs -Q, even after having used my setup in a separate
session, and even if that separate session is still open.

That's still a mystery to me.  And the fact that I
reported the same thing twice, years apart (having
forgotten the first report) supports the description
that I gave (both times): I saw the problem with emacs
-Q (in multiple versions even), but later couldn't
repro it.

Anyway, I'm fixing my version of I`nfo-find-node-2'...