Re: Info-index fails on node-name "~/.cvsrc"

Re: Info-index fails on node-name "~/.cvsrc"

[Juri Linkov]

Glenn Morris <gmorris+emacs@ast.cam.ac.uk> writes on emacs-pretest-bug:
> emacs -q --no-site-file
> C-h i
> m cvs
> i .cvsrc RET
>
> Result: "No such node or anchor: ~/"
>
> The relevant index entry reads:
>
> * .cvsrc file:                           ~/.cvsrc.
>
> The standalone info reader [info (GNU texinfo) 4.1] has no problems
> with this.

The standalone info reader has no problems with this because its
skip_node_characters function contains "quite a bit of hair" to
ignore periods without whitespace after them.  But it still fails
on node names containing periods with following whitespace.

One possible solution is to extract node names up to the final period
ignoring all intermediate periods, e.g.

* J. Random Hacker:        J. Random Hacker.      (line  2)
                           ----------------

with the assumption that no index entries have more text after the node
name (except the line number in the new Info format).

I wouldn't want to make incompatible changes, so please tell, are there
such cases where it will not work for index entries?

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Info-index fails on node-name "~/.cvsrc"

[Karl Berry]

Hi Juri,

    One possible solution is to extract node names up to the final period
    ignoring all intermediate periods, e.g.

    * J. Random Hacker:        J. Random Hacker.      (line  2)
                               ----------------

    with the assumption that no index entries have more text after the node
    name (except the line number in the new Info format).

That seems to me like it will work for the specific case of index
entries.  Since we control index entry generation, we know for a fact
there is no descriptive text on the index line that contains a period.

So if you'd like to implement that change, it seems beneficial.
Is it feasible to change just index entry parsing, without changing menu
entry parsing?

Thanks,
karl

Re: Info-index fails on node-name "~/.cvsrc"

[Juri Linkov]

>     One possible solution is to extract node names up to the final period
>     ignoring all intermediate periods, e.g.
>
>     * J. Random Hacker:        J. Random Hacker.      (line  2)
>                                ----------------
>
>     with the assumption that no index entries have more text after the node
>     name (except the line number in the new Info format).
>
> That seems to me like it will work for the specific case of index
> entries.  Since we control index entry generation, we know for a fact
> there is no descriptive text on the index line that contains a period.
>
> So if you'd like to implement that change, it seems beneficial.
> Is it feasible to change just index entry parsing, without changing menu
> entry parsing?

Yes, it is feasible as well as desirable.  The following patch does it.

By the way, I implemented a related feature introduced by the latest
version of Texinfo: using the line numbers (line NNN) for locating
index entries (but it is not included in the following patch).

Index: lisp/info.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/info.el,v
retrieving revision 1.390
diff -u -r1.390 info.el
--- lisp/info.el	22 Apr 2004 19:49:11 -0000	1.390
+++ lisp/info.el	25 Apr 2004 04:06:44 -0000
@@ -1858,11 +1872,27 @@
 Because of ambiguities, this should be concatenated with something like
 `:' and `Info-following-node-name-re'.")
 
-(defun Info-extract-menu-node-name (&optional multi-line)
+(defun Info-extract-menu-node-name (&optional multi-line index-node)
   (skip-chars-forward " \t\n")
   (when (looking-at (concat Info-menu-entry-name-re ":\\(:\\|"
 			    (Info-following-node-name-re
-			     (if multi-line "^.,\t" "^.,\t\n")) "\\)"))
+                             (cond
+                              (index-node "^,\t\n")
+                              (multi-line "^.,\t")
+                              (t          "^.,\t\n")))
+                            "\\)"
+                            (if index-node "\\." "")))
     (replace-regexp-in-string
      "[ \n]+" " "
      (or (match-string 2)
@@ -2327,7 +2357,7 @@
   (if (equal Info-current-file "dir")
       (error "The Info directory node has no index; use m to select a manual"))
   (let ((orignode Info-current-node)
-	(pattern (format "\n\\* +\\([^\n]*%s[^\n]*\\):[ \t]+\\([^.\n]*\\)\\.[ \t]*\\([0-9]*\\)"
+	(pattern (format "\n\\* +\\([^\n]*%s[^\n]*\\):[ \t]+\\([^\n]*\\)\\.\\(?:[ \t\n]*(line +\\([0-9]+\\))\\)?"
 			 (regexp-quote topic)))
 	node
 	(case-fold-search t))
@@ -2418,7 +2448,7 @@
 Build a menu of the possible matches."
   (interactive "sIndex apropos: ")
   (unless (string= string "")
-    (let ((pattern (format "\n\\* +\\([^\n]*%s[^\n]*\\):[ \t]+\\([^.]+\\)."
+    (let ((pattern (format "\n\\* +\\([^\n]*%s[^\n]*\\):[ \t]+\\([^\n]+\\)\\."
 			   (regexp-quote string)))
 	  (ohist Info-history)
 	  (ohist-list Info-history-list)
@@ -2583,21 +2616,16 @@
      ;; menu item: node name
      ((setq node (Info-get-token (point) "\\* +" "\\* +\\([^:]*\\)::"))
       (Info-goto-node node fork))
-     ;; menu item: index entry
+     ;; menu item: node name or index entry
      ((Info-get-token (point) "\\* +" "\\* +\\(.*\\): ")
       (beginning-of-line)
       (forward-char 2)
-      (setq node (Info-extract-menu-node-name))
+      (setq node (Info-extract-menu-node-name
+                  nil (string-match "\\<index\\>" Info-current-node)))
       (Info-goto-node node fork))
      ((setq node (Info-get-token (point) "Up: " "Up: \\([^,\n\t]*\\)"))
       (Info-goto-node node fork))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Info-index fails on node-name "~/.cvsrc"

[Karl Berry]

    Yes, it is feasible as well as desirable.  The following patch does it.

Great, thanks!

    By the way, I implemented a related feature introduced by the latest
    version of Texinfo: using the line numbers (line NNN) for locating
    index entries

Excellent.

Re: Info-index fails on node-name "~/.cvsrc"

[Glenn Morris]

I noticed that Info-index now fails in the VM info pages (it works OK
in 21.3). The index entries all take the format:

* vm-move-messages-physically:  Sorting Messages.       5.

and a typical error message is:

i sort -> "No such node or anchor: Sorting Messages.       5"

Re: Info-index fails on node-name "~/.cvsrc"

[Juri Linkov]

Glenn Morris <gmorris+emacs@ast.cam.ac.uk> writes:
> I noticed that Info-index now fails in the VM info pages (it works OK
> in 21.3). The index entries all take the format:
>
> * vm-move-messages-physically:  Sorting Messages.       5.
>
> and a typical error message is:
>
> i sort -> "No such node or anchor: Sorting Messages.       5"

How did you produce this Info index in such non-standard format?

I tried to regenerate the VM Info file with an old version of
makeinfo and got:

* vm-move-messages-physically:           Sorting Messages.

Then I tried to generate it with the latest version of makeinfo
and the result was:

* vm-move-messages-physically:           Sorting Messages.    (line   6)

But OK.  Since you have Info pages in such format, we should find a
solution to support it.

Recently info.el was fixed to include periods into Info node names from
the Info index (like "~/.cvsrc").  But in your file the period is not
a part of the node name.

The stand-alone Info reader interprets the period with the following
whitespace as a terminating period.  But it don't work with node names
where period with the following whitespace is a part of the node name,
like in abbreviations (e.g. "J. Random Hacker" from the Jargon file).

So I propose the following solution: if an Info node is not found and
its name contains the period, then try to find a node with a part
of the name after period removed.

Index: emacs/lisp/info.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/info.el,v
retrieving revision 1.394
diff -c -r1.394 info.el
*** emacs/lisp/info.el	27 Apr 2004 14:43:03 -0000	1.394
--- emacs/lisp/info.el	18 May 2004 04:47:27 -0000
***************
*** 863,871 ****
  	      (let ((pos (Info-find-node-in-buffer regexp)))
  		(when pos
  		  (goto-char pos)
! 		  (throw 'foo t))
!                 ;; No such anchor in tag table or node in tag table or file
! 		(error "No such node or anchor: %s" nodename)))
  
  	    (Info-select-node)
  	    (goto-char (point-min))
--- 863,879 ----
  	      (let ((pos (Info-find-node-in-buffer regexp)))
  		(when pos
  		  (goto-char pos)
! 		  (throw 'foo t)))
! 
!               (when (string-match "\\([^.]+\\)\\." nodename)
!                 (let (Info-point-loc)
!                   (Info-find-node-2
!                    filename (match-string 1 nodename) no-going-back))
!                 (widen)
!                 (throw 'foo t))
! 
!               ;; No such anchor in tag table or node in tag table or file
!               (error "No such node or anchor: %s" nodename))
  
  	    (Info-select-node)
  	    (goto-char (point-min))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Info-index fails on node-name "~/.cvsrc"

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Date: Tue, 18 May 2004 09:18:53 +0300
> 
> Glenn Morris <gmorris+emacs@ast.cam.ac.uk> writes:
> > I noticed that Info-index now fails in the VM info pages (it works OK
> > in 21.3). The index entries all take the format:
> >
> > * vm-move-messages-physically:  Sorting Messages.       5.
> >
> > and a typical error message is:
> >
> > i sort -> "No such node or anchor: Sorting Messages.       5"
> 
> How did you produce this Info index in such non-standard format?

I think it's a new feature of Texinfo 4.7, which was released
recently.

> So I propose the following solution: if an Info node is not found and
> its name contains the period, then try to find a node with a part
> of the name after period removed.

I think we should try to make the stand-alone Info and the Emacs Info
reader support the same features in identical or at least equivalent
ways.  It is very confusing to have two different behaviors in two GNU
packages that both are of such a central importance for the GNU
project.  Such differences add to maintenance burden (as, for example,
makeinfo needs to consider Emacs features and misfeatures when it is
developed), and make the job of documenting Info in info.texi harder.

So what I suggest is for the Emacs Info reader to be amended to
support the index format introduced in Texinfo 4.7, in a manner that
is as close as possible to what the stand-alone reader does.

Re: Info-index fails on node-name "~/.cvsrc"

[Juri Linkov]

"Eli Zaretskii" <eliz@gnu.org> writes:
> I think we should try to make the stand-alone Info and the Emacs Info
> reader support the same features in identical or at least equivalent
> ways.  It is very confusing to have two different behaviors in two GNU
> packages that both are of such a central importance for the GNU
> project.  Such differences add to maintenance burden (as, for example,
> makeinfo needs to consider Emacs features and misfeatures when it is
> developed), and make the job of documenting Info in info.texi harder.

I completely agree on this general principle.

> So what I suggest is for the Emacs Info reader to be amended to
> support the index format introduced in Texinfo 4.7, in a manner that
> is as close as possible to what the stand-alone reader does.

I already implemented the new Texinfo 4.7 index format in the Emacs
Info reader a month ago.  It uses the (line NNN) format for locating
index entries:

* vm-move-messages-physically:           Sorting Messages.    (line   6)

But an error was caused by some unknown format which is not supported
neither by the Emacs Info reader nor by the stand-alone reader:

* vm-move-messages-physically:  Sorting Messages.       5.

It seems that 5 is a line number.  But the stand-alone reader
don't interpret it as a line number and simply ignores it.

I proposed to ignore it in the Emacs reader as well.  But my fix tries
both variants "Sorting Messages" and "Sorting Messages. 5".  to allow
possible Info node names with periods in them.  And I suggest to fix
the stand-alone reader to do the same since currently it fails on such
node names.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Info-index fails on node-name "~/.cvsrc"

[Glenn Morris]

Juri Linkov wrote:

> How did you produce this Info index in such non-standard format?

I ran `make vm.info' in the vm build directory. That runs:

emacs -batch -q -no-site-file -insert vm.texinfo \
    -l texinfmt -f texinfo-format-buffer -f save-buffer

So if the resulting index format is not correct, perhaps that is
another Emacs problem. Emacs-21.3 gives the same result as the
current CVS.

Re: Info-index fails on node-name "~/.cvsrc"

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Date: Tue, 18 May 2004 14:28:55 +0300
> 
> I already implemented the new Texinfo 4.7 index format in the Emacs
> Info reader a month ago.

Sorry, I didn't know that.  Thank you for your work.

> It uses the (line NNN) format for locating
> index entries:
> 
> * vm-move-messages-physically:           Sorting Messages.    (line   6)
> 
> But an error was caused by some unknown format which is not supported
> neither by the Emacs Info reader nor by the stand-alone reader:
> 
> * vm-move-messages-physically:  Sorting Messages.       5.

Karl, is that maybe some interim version of Texinfo pretest?  If so,
we don't really need to support it in the readers, do we?

> It seems that 5 is a line number.  But the stand-alone reader
> don't interpret it as a line number and simply ignores it.
> 
> I proposed to ignore it in the Emacs reader as well.  But my fix tries
> both variants "Sorting Messages" and "Sorting Messages. 5".  to allow
> possible Info node names with periods in them.  And I suggest to fix
> the stand-alone reader to do the same since currently it fails on such
> node names.

If the stand-alone reader can be fixed to not fail for node names with
periods, I agree.  But if it cannot, I'd rather see the same behavior
in Emacs.

Re: Info-index fails on node-name "~/.cvsrc"

[Karl Berry]

    Karl, is that maybe some interim version of Texinfo pretest?  If so,
    we don't really need to support it in the readers, do we?

As you saw, apparently it is the output from texinfo-format-buffer,
probably Bob implemented an interim version we talked about.  (I didn't
know this.)  t-f-b should be fixed.

I don't think the readers need to support it, people who use t-f-b
deserve what they get, it hasn't been recommended for years.  (We should
report it to Kyle ...)

Re: Info-index fails on node-name "~/.cvsrc"

[Eli Zaretskii]

> From: Glenn Morris <gmorris+emacs@ast.cam.ac.uk>
> Date: Tue, 18 May 2004 13:03:06 +0100
> 
> Juri Linkov wrote:
> 
> > How did you produce this Info index in such non-standard format?
> 
> I ran `make vm.info' in the vm build directory. That runs:
> 
> emacs -batch -q -no-site-file -insert vm.texinfo \
>     -l texinfmt -f texinfo-format-buffer -f save-buffer

I suggest to report that as a bug to the VM maintainer(s).  I think VM
should use `makeinfo', not texinfmt.el, to produce Info files.  That
is because texinfmt.el is unmaintained and is generally out of sync
with the development of Texinfo and the Emacs Info reader.

> So if the resulting index format is not correct, perhaps that is
> another Emacs problem.

If someone would like to step forward, take over the texinfmt
maintenance, and update it to produce the same results as the current
`makeinfo', I'm sure the patches will be welcome.  Otherwise, texinfmt
should IMHO be considered deprecated and not used by GNU projects.

Re: Info-index fails on node-name "~/.cvsrc"

[Glenn Morris]

Eli Zaretskii wrote:

> I suggest to report that as a bug to the VM maintainer(s). I think
> VM should use `makeinfo', not texinfmt.el, to produce Info files.

Ok, I did that.

> Otherwise, texinfmt should IMHO be considered deprecated and not
> used by GNU projects.

Should it be marked as obsolete in some way then?

Re: Info-index fails on node-name "~/.cvsrc"

[Robert J. Chassell]

   > emacs -batch -q -no-site-file -insert vm.texinfo \
   >     -l texinfmt -f texinfo-format-buffer -f save-buffer

   I suggest to report that as a bug to the VM maintainer(s).  I think VM
   should use `makeinfo', not texinfmt.el, to produce Info files.  

As the person who has mostly supported texinfmt.el these past few
years, I agree.  Indeed, for making Info files, I now use

    makeinfo --no-split --paragraph-indent=0 --verbose foo.texi

which works fine.  (It did not initially.)

   texinfmt should IMHO be considered deprecated and not used by GNU
   projects.

I agree.  Its time is past.

-- 
    Robert J. Chassell                         Rattlesnake Enterprises
    As I slowly update it,                     bob@rattlesnake.com
        I rewrite a "What's New" segment for   http://www.rattlesnake.com

Re: Info-index fails on node-name "~/.cvsrc"

[Stefan Monnier]

> should use `makeinfo', not texinfmt.el, to produce Info files.  That
> is because texinfmt.el is unmaintained and is generally out of sync
> with the development of Texinfo and the Emacs Info reader.

But note that the entries it generated have a valid format, so we better
accept it.


        Stefan

Re: Info-index fails on node-name "~/.cvsrc"

[Karl Berry]

    But note that the entries it generated have a valid format, so we better
    accept it.

Is that trailing "5." really a valid format?  Not in my mind.

It would probably be trivial to change it to generate "(line  5)" instead
of "5.".  No harm in that, regardless of the deprecation, which I agree
with.

Re: Info-index fails on node-name "~/.cvsrc"

[Stefan Monnier]

>     But note that the entries it generated have a valid format, so we better
>     accept it.

> Is that trailing "5." really a valid format?  Not in my mind.

> It would probably be trivial to change it to generate "(line  5)" instead
> of "5.".  No harm in that, regardless of the deprecation, which I agree
> with.

AFAIK, index nodes are just normal nodes with a big menu.
Thus index entries have the same format as menu entries:

     * NAME: LOCATION.  DESCRIPTION

It's just that traditionally index nodes use an empty DESCRIPTION
and align all the LOCATIONs so it looks like

     * NAME1:                     LOCATION1.
     * NAME2:                     LOCATION2.

Adding a DESCRIPTION of the form "5." is just as valid as "(line 5)".
Now if the node has a new special index-tag (can't remember what it looks
like), maybe things are different, but with old-style index nodes, it looks
perfectly valid to me.


        Stefan

Re: Info-index fails on node-name "~/.cvsrc"

[Eli Zaretskii]

> From: Glenn Morris <gmorris+emacs@ast.cam.ac.uk>
> Date: Wed, 19 May 2004 13:13:59 +0100
> 
> > Otherwise, texinfmt should IMHO be considered deprecated and not
> > used by GNU projects.
> 
> Should it be marked as obsolete in some way then?

Yes, I think so.

Re: Info-index fails on node-name "~/.cvsrc"

[Juri Linkov]

>> Is that trailing "5." really a valid format?  Not in my mind.
>
>> It would probably be trivial to change it to generate "(line  5)" instead
>> of "5.".  No harm in that, regardless of the deprecation, which I agree
>> with.

Even if it is considered deprecated, I will install a fix.

> AFAIK, index nodes are just normal nodes with a big menu.
> Thus index entries have the same format as menu entries:
>
>      * NAME: LOCATION.  DESCRIPTION
>
> It's just that traditionally index nodes use an empty DESCRIPTION
> and align all the LOCATIONs so it looks like
>
>      * NAME1:                     LOCATION1.
>      * NAME2:                     LOCATION2.
>
> Adding a DESCRIPTION of the form "5." is just as valid as "(line 5)".

Two distinctions between index and menu entries were added recently.
One is getting line numbers from "(line NNN)".  Currently it works
only in Info indexes.  I don't know if it makes sense for menu items
in non-index nodes as well.

Another difference is special handling of periods in index entries,
which now includes all periods to the node name:

     * .cvsrc file: ~/.cvsrc.
                    ========
     * I see no X here.: I see no X here..
                         ================
     * J. Random Hacker: J. Random Hacker.
                         ================

This may not work for menu items.  Currently, most menu items are
safely limited by ::

     * ~/.cvsrc::     Default options with the ~/.cvsrc file
     * I see no X here.::
     * J. Random Hacker::

so there is no problem.  But we might get into trouble with menu
entries with periods like:

     * .cvsrc file: ~/.cvsrc.     Default options with the ~/.cvsrc file.

Since such menu items are very rare (if exist at all), so perhaps we
shouldn't care for them.

> Now if the node has a new special index-tag (can't remember what it looks
> like), maybe things are different, but with old-style index nodes, it looks
> perfectly valid to me.

If I understand correctly, a new index tag ^@^H[index^@^H] is for
determining if an Info node is an index node, _additionally_ to
current checking the node name for the "\\<Index\\>" regexp.

Karl, could you confirm, is a new index tag supposed to work this way:

1. To determine if the current node is an index node, the Emacs Info
reader should first try to match the node name for the "\\<Index\\>"
regexp.  If that fails, then to search for the "^@^H[index^@^H]" tag
in the current node.

2. To find all index nodes in the current Info file, the Emacs Info
reader should first try to find in the Top node's menu a menu item
whose node name matches the "\\<Index\\>" regexp, and look also in
subsequent nodes matching "\\<Index\\>" (this is as it currently
works).  If that fails, then to scan the whole Info file from the top
for the "^@^H[index^@^H]" tag, and look also in subsequent nodes which
contain the "^@^H[index^@^H]" tag.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Info-index fails on node-name "~/.cvsrc"

[Karl Berry]

    I don't know if it makes sense for menu items
    in non-index nodes as well.

We don't need to support (line NNN) in non-index menus. 

    If I understand correctly, a new index tag ^@^H[index^@^H] is for
    determining if an Info node is an index node, _additionally_ to
    current checking the node name for the "\\<Index\\>" regexp.

It doesn't have to be additional.  The index cookie is output whenever
there's a printindex, so you can safely rely on it if it's there,
whatever the name of the node is.

    Karl, could you confirm, is a new index tag supposed to work this way:

    1. To determine if the current node is an index node, the Emacs Info
    reader should first try to match the node name for the "\\<Index\\>"
    regexp.  If that fails, then to search for the "^@^H[index^@^H]" tag
    in the current node.

In principle, I would say, if the cookie is present, it's an index node,
and only look at the node name if the cookie is not present (for info
files generated before 4.7).  That way, a node whose name contains
Index but isn't an index node won't be found.  

But in practice, I don't expect it really matters, since we've been
living with the Index heuristic forever.

    2. To find all index nodes in the current Info file, the Emacs Info

I didn't realize/remember that's one of the jobs.  Again, the algorithm
you outline seems sensible, but you could also use the index cookie
first if you like.  (I imagine it's implemented as finding the first
potential node, then apply the "is the current node an index node" test
for that node and all subsequent nodes, so you'd get it for free, anyway.)

Thanks,
k

Re: Info-index fails on node-name "~/.cvsrc"

[Juri Linkov]

karl@freefriends.org (Karl Berry) writes:
>     1. To determine if the current node is an index node, the Emacs Info
>     reader should first try to match the node name for the "\\<Index\\>"
>     regexp.  If that fails, then to search for the "^@^H[index^@^H]" tag
>     in the current node.
>
> In principle, I would say, if the cookie is present, it's an index node,
> and only look at the node name if the cookie is not present (for info
> files generated before 4.7).  That way, a node whose name contains
> Index but isn't an index node won't be found.  

The main problem is backward compatibility.  We should ensure that
the Info reader will still find index nodes in old Info files.
To do this there should be a reliable method to distinguish them
from Info files with index cookies, because if a node don't contain
the cookie and its name contains "Index", then it's not known whether
it is a non-index node in the 4.7 format or an index node in the pre-4.7
format.

Perhaps the most reliable way is to scan the whole Info file and its
subfiles for index cookies.  If there is at least one such cookie,
then to assume that it is in the 4.7 format.  Otherwise, to use the old
heuristic by determining the index node by its name.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Info-index fails on node-name "~/.cvsrc"

[Karl Berry]

    The main problem is backward compatibility.  

Of course.

    Perhaps the most reliable way is to scan the whole Info file and its
    subfiles for index cookies.  If there is at least one such cookie,
    then to assume that it is in the 4.7 format.  Otherwise, to use the old
    heuristic by determining the index node by its name.

That sounds good to me.  I can't think of anything better, and in
practice, I think it will be just fine.

I guess this is a case where we need to have global information about
the Info version being written, for 100% perfect accuracy.