Invisible colons in Emacs Info.

Invisible colons in Emacs Info.

[Luc Teirlinck]

I believe that the colons after menu subtopic names should be made
visible again.

The reasons are:

1.  Subtopic names "ending in a colon".  To be concrete, say "$Id:",
    well known from RCS.  One can not really use the name "$Id:",
    because the extra colon that is automatically added will make it
    "$Id::" and obvious trouble ensues.  So what one would like to do
    is name it "$Id" and rely on the extra colon to make things "look
    right".  Unfortunately, the extra colon is hidden by default and
    things do not look right.  The trick works in indices, because
    there the extra colon is not hidden.

2.  Consistency with indices.  The colon is shown in indices, but not
    in other menus.  Why the difference?

3.  Avoid confusion when people try commands such as M-x man and the
    like on menu subtopic names.

Does that colon really look that hideous?

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

>From my previous message:

   I believe that the colons after menu subtopic names should be made
   visible again.

I meant the "colon".  It might be OK to hide one of the two colons in
names like:

* Nifty Topic::              Stuff for real experts.

because the double colon might look weird, but certainly one single
colon does not look weird at all.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Stefan Monnier]

> 2.  Consistency with indices.  The colon is shown in indices, but not
>     in other menus.  Why the difference?

In nornal menus, things look like:

	* menu:entry.		description.

whereas in indices, the description is missing and things look like:

	* menu:			entry.

note the clever use of he fact that whitespaces between `menu:'
and `entry.' is ignored.


	Stefan

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Stefan Monnier wrote:

   In nornal menus, things look like:

	   * menu:entry.		description.

   whereas in indices, the description is missing and things look like:

	   * menu:			entry.

   note the clever use of he fact that whitespaces between `menu:'
   and `entry.' is ignored.

But the question is: how does that difference have anything to do with
whether or not colons should be hidden?

On another note, the whitespace is ignored by makeinfo and the
stand-alone Info reader, but not by Emacs.  In Emacs

* menu: entry                description. 

will visit entry, but

* menu:entry.                description.

will yield an error.  It does not matter if one replaces "menu" by
some less ambiguous name.  The stand-alone Info visits entry.

Emacs insists on the space after the colon, because it wants to
support colons in the middle of menu subtopic names.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Stefan Monnier]

> Stefan Monnier wrote:
> 
>    In nornal menus, things look like:
> 
> 	   * menu:entry.		description.
> 
>    whereas in indices, the description is missing and things look like:
> 
> 	   * menu:			entry.
> 
>    note the clever use of he fact that whitespaces between `menu:'
>    and `entry.' is ignored.
> 
> But the question is: how does that difference have anything to do with
> whether or not colons should be hidden?

Eh?  It's not just the colon.
In normal menus, Emacs hides ":entry", but in indices that would end
up hiding ":		entry." which is not what we want, so the
hiding code refrains from hiding menu entries if the node name
is the name of an index.

> On another note, the whitespace is ignored by makeinfo and the
> stand-alone Info reader, but not by Emacs.  In Emacs
> 
> * menu: entry                description. 
> 
> will visit entry,

Huh?  That's a bug, it should visit "entry              description".

> but
> 
> * menu:entry.                description.
> 
> will yield an error.

I've used such entries, so I know they've worked at some point.  Please
report a precise test case for this bug.


	Stefan

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Stefan Monnier wrote:

   Huh?  That's a bug, it should visit "entry              description".

Sorry, that was a typo, I forgot the period after entry.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

   > but
   > 
   > * menu:entry.                description.
   > 
   > will yield an error.

   I've used such entries, so I know they've worked at some point.  Please
   report a precise test case for this bug.

===File ~/experiment.texi===================================
\input texinfo    @c -*-texinfo-*-
@comment %**start of header
@setfilename experiment.info
@settitle An experiment.
@comment %**end of header

@node Top
@top Top

@menu
* name:entry.           description.
@end menu

@node entry
@chapter Entry

This node is called @samp{entry}

@bye
============================================================

===File ~/experiment.info===================================
This is experiment.info, produced by makeinfo version 4.6 from
/home/teirllm/info/experiment.texi.

\x1f
File: experiment.info,  Node: Top,  Next: entry,  Up: (dir)

Top
***

* Menu:

* name:entry.           description.

\x1f
File: experiment.info,  Node: entry,  Prev: Top,  Up: Top

Entry
*****

This node is called `entry'


\x1f
Tag Table:
Node: Top\x7f100
Node: entry\x7f219
\x1f
End Tag Table
============================================================

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Stefan Monnier wrote:

   I've used such entries, so I know they've worked at some point.  Please
   report a precise test case for this bug.

Before you "fix" this bug, let me point out that other people (do not
blame me if you disagree) very much want to support entries like:

* menu:entry:  entry.       description.

The stand-alone Info does not support such entries.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Stefan Monnier wrote:

   > But the question is: how does that difference have anything to do with
   > whether or not colons should be hidden?

   Eh?  It's not just the colon.

Yes, but my proposal concerns just the colon.  It even just concerns
one of the two colons in entries like:

* stuff::

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Stefan Monnier wrote:

   > but
   > 
   > * menu:entry.                description.
   > 
   > will yield an error.

   I've used such entries, so I know they've worked at some point.

Note also that

* menu:entry.

might look bad for people who use the stand-alone Info or have
Info-hide-note-references set to nil, so it is probably better to
insert a space there regardless.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Karl Berry]

    In nornal menus, things look like:
            * menu:entry.		description.

    whereas in indices, the description is missing and things look like:
            * menu:			entry.

Indexes are completely normal menus.  There is nothing to stop a user
from doing something like
@menu
* Foo:          bar.
@end menu

    * menu:entry.                description.
    will yield an error.

That's a bug.  There can be any number of spaces after the colon
in the menu:entry form, including zero.

    support colons in the middle of menu subtopic names.

That just can't be done right now.  A new syntax for quoting node names
has to be introduced to fix it, kludging with whitespace won't solve the
problem.  As we've discussed before.  This won't be happening for Emacs 21.4.

    blame me if you disagree) very much want to support entries like:
    * menu:entry:  entry.       description.

To what end?  I mean, what are the real examples that are hoped to be
supported?  This is new Info syntax, as far as I know.  If makeinfo
doesn't complain about it, it probably should.

I don't understand even this skeletal example.  Surely the point here is
not to repeat the node name `entry', that wouldn't be useful.  

As far as I know, there are exactly two valid menu formats.  From the
Menu Example node in the Texinfo manual:

    @menu
    * menu entry name: Node name.   A short description.
    * Node name::                   This form is preferred.
    @end menu

  This produces:

    * menu:

    * menu entry name: Node name.   A short description.
    * Node name::                   This form is preferred.

That's it.  It covers regular menus, indexes, and dir entries.  All have
the same syntax.  You can't just randomly decide to do new stuff with
colons, periods, parens, etc. in menu entries without breaking many
existing documents/installations.

Re: Invisible colons in Emacs Info.

[Karl Berry]

    Note also that

    * menu:entry.

    might look bad for people 

What looks bad is one question.  What's a valid Info file is quite a
different question, and as far as I know zero spaces has always been
valid in that context, and we can't just declare it invalid all of a
sudden.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Karl Berry wrote:

       blame me if you disagree) very much want to support entries like:
       * menu:entry:  entry.       description.

   To what end?  I mean, what are the real examples that are hoped to be
   supported?  This is new Info syntax, as far as I know.  If makeinfo
   doesn't complain about it, it probably should.

   I don't understand even this skeletal example.  Surely the point here is
   not to repeat the node name `entry', that wouldn't be useful.  

The concrete names are irrelevant:

* text1:text2:   text3.   text4.

>From info.el:

;; Note that nowadays we expect info files to be made using makeinfo.
;; In particular we make these assumptions:
;;  - a menu item MAY contain colons but not colon-space ": "
;;  - a menu item ending with ": " (but not ":: ") is an index entry
;;  - a node name MAY NOT contain a colon
;; This distinction is to support indexing of computer programming
;; language terms that may contain ":" but not ": ".

This text clearly wants to support menu items like:

* text1:text2:     text3.   text4.

(Unless I misunderstood, sorry for the confusion in that case.)

I thought I understood from previous messages that you "agreed to
disagree" with people who held the views expressed above.  (Or am I
completely misunderstanding things?)

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Karl Berry]

    ;;  - a menu item MAY contain colons but not colon-space ": "
    ;;  - a menu item ending with ": " (but not ":: ") is an index entry
    ;;  - a node name MAY NOT contain a colon

Sigh, this is so confusing.

What does `menu item' mean?  Because the Emacs manual, for example,
contains many menu items like this:
basic.texi:24:* Files: Basic Files.   Visiting, creating, and saving files.

That's a menu item with colon-space in it, and we certainly can't
declare it invalid.  Does it still work with the current code?

I assume it does, and what `menu item' means is `menu entry name', as in
the canonical example:
* menu entry name: Node name.   A short description.

    ;; This distinction is to support indexing of computer programming
    ;; language terms that may contain ":" but not ": ".

Obviously we want to support this, but it seems to me the best way to do
it is by introducing a real quoting syntax instead of putting more
restrictions on menu item syntax.  (Which will then allow us
support lots of other good things, like : and . in node names after
all.)  But see below ...

    I thought I understood from previous messages that you "agreed to
    disagree" with people who held the views expressed above.  (Or am I
    completely misunderstanding things?)

No, you're not misunderstanding, that's basically what I said before,
but then it sounded like a lot more things were becoming invalid than I
thought before.

I'm still not completely clear on it, but it seems now that the only
thing that is becoming invalid is your example of

* menu entry name:node name.
(with zero spaces after the menu entry name:).

Upon reflection, I guess that seems ok to me. I just checked all the
Texinfo source files on my system and the only ones which would fail are
all part of XEmacs (lookup-guide.tex, lookup.texi, and sml-mode.texi),
which have entries like this:
/usr/share/xemacs/mule-packages/man/lookup/lookup.texi:* ndeb+/cdrom:WAEI    ^[$BOB1Q<-E5^[(B             =<>
/usr/share/xemacs/xemacs-packages/man/sml-mode/sml-mode.texi:* sml:(sml-mode).  Emacs mode for editing SML

I guess I should implement a warning (at least) in makeinfo for this,
and document it in the manual.  It would be good to implement it in
standalone Info too, of course, but that is not as important to me.

The other thing I agreed to was that makeinfo will always output at
least one space between `menu entry name:' and `node name.' in indexes.
That's no problem and is already the case.

Re: Invisible colons in Emacs Info.

[Stefan Monnier]

So it seems that the problem was introduced by revision 1.333 where
we're trying to allow menu entry names with colons in them.
The problem is that we skip `:' that could be part of the name without
checking whether that is the last colon and is thus obviously not part
of the name.

Others will say the problem is simply that we allow `:', but assuming we
still want to allow colon despite the obvious ambiguity, I think the patch
below should help.

It uses regexp matching rather than combinations of skip-chars-forward
and stuff.  The reason for the change is because, by using regexp-matching,
we can take advantage of its backtracking to resolve the ambiguity.

Problem is that it still fails on:

   * name1:foo:node.           des:cription.

because it thinks the menu entry name is "name1:foo:node.           des"
and the node name is "cription".  I'm sure we can tune the disambiguation
by saying that menu entries can have `:' but cannot have `.' after the `:'
or some other such heuristic.  If anybody wants to spice up the
Info-menu-entry-name-re regexp accordingly...
We really need a better solution.



	Stefan


Index: info.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/info.el,v
retrieving revision 1.358
diff -u -b -r1.358 info.el
--- info.el	19 Jun 2003 20:53:06 -0000	1.358
+++ info.el	25 Jun 2003 20:56:33 -0000
@@ -902,7 +902,9 @@
 		    nodename end)
 		(re-search-backward "^\^_")
 		(search-forward "Node: ")
-		(setq nodename (Info-following-node-name))
+		(setq nodename
+		      (and (looking-at (Info-following-node-name-re))
+			   (match-string 1)))
 		(search-forward "\n\^_" nil 'move)
 		(beginning-of-line)
 		(setq end (point))
@@ -1201,8 +1203,8 @@
 		  nodename)
     (setq filename (if (= (match-beginning 1) (match-end 1))
 		       ""
-		     (substring nodename (match-beginning 2) (match-end 2)))
-	  nodename (substring nodename (match-beginning 3) (match-end 3)))
+		     (match-string 2 nodename))
+	  nodename (match-string 3 nodename))
     (let ((trim (string-match "\\s *\\'" filename)))
       (if trim (setq filename (substring filename 0 trim))))
     (let ((trim (string-match "\\s *\\'" nodename)))
@@ -1385,35 +1387,33 @@
 (defun Info-extract-pointer (name &optional errorname)
   "Extract the value of the node-pointer named NAME.
 If there is none, use ERRORNAME in the error message;
-if ERRORNAME is nil, just return nil.
-Bind this in case the user sets it to nil."
+if ERRORNAME is nil, just return nil."
+  ;; Bind this in case the user sets it to nil.
   (let ((case-fold-search t))
     (save-excursion
-      (save-restriction
 	(goto-char (point-min))
 	(let ((bound (point)))
 	  (forward-line 1)
-	  (cond ((re-search-backward (concat name ":") bound t)
-		 (goto-char (match-end 0))
-		 (Info-following-node-name))
+	(cond ((re-search-backward
+		(concat name ":" (Info-following-node-name-re)) bound t)
+	       (match-string 1))
 		((not (eq errorname t))
 		 (error "Node has no %s"
-			(capitalize (or errorname name))))))))))
+		      (capitalize (or errorname name)))))))))
 
-(defun Info-following-node-name (&optional allowedchars)
-  "Return the node name in the buffer following point.
+(defun Info-following-node-name-re (&optional allowedchars)
+  "Return a regexp matching a node name.
 ALLOWEDCHARS, if non-nil, goes within [...] to make a regexp
-saying which chars may appear in the node name."
-  (skip-chars-forward " \t")
-  (buffer-substring-no-properties
-   (point)
-   (progn
-     (while (looking-at (concat "[" (or allowedchars "^,\t\n") "]"))
-       (skip-chars-forward (concat (or allowedchars "^,\t\n") "("))
-       (if (looking-at "(")
-	   (skip-chars-forward "^)")))
-     (skip-chars-backward " ")
-     (point))))
+saying which chars may appear in the node name.
+Submatch 1 is the complete node name.
+Submatch 2 if non-nil is the parenthesized file name part of the node name.
+Submatch 3 is the local part of the node name.
+End of submatch 0, 1, and 3 are the same, so you can safely concat."
+  (concat "[ \t]*"			;Skip leading space.
+	  "\\(\\(([^)]+)\\)?"	;Node name can start with a file name.
+	  "\\([" (or allowedchars "^,\t\n") "]*" ;Any number of allowed chars.
+	  "[" (or allowedchars "^,\t\n") " ]" ;The last char can't be a space.
+	  "\\|\\)\\)"))			      ;Allow empty node names.
 
 (defun Info-next ()
   "Go to the next node of this node."
@@ -1472,9 +1472,7 @@
 
        (goto-char (point-min))
        (while (re-search-forward "\\*note[ \n\t]*\\([^:]*\\):" nil t)
-	 (setq str (buffer-substring-no-properties
-		    (match-beginning 1)
-		    (1- (point))))
+	 (setq str (match-string-no-properties 1))
 	 ;; See if this one should be the default.
 	 (and (null default)
 	      (<= (match-beginning 0) start-point)
@@ -1494,23 +1492,14 @@
 	 (if (eq default t) (setq default str))
 	 (if (eq alt-default t) (setq alt-default str))
 	 ;; Don't add this string if it's a duplicate.
-	 ;; We use a loop instead of "(assoc str completions)" because
-	 ;; we want to do a case-insensitive compare.
-	 (let ((tail completions)
-	       (tem (downcase str)))
-	   (while (and tail
-		       (not (string-equal tem (downcase (car (car tail))))))
-	     (setq tail (cdr tail)))
-	   (or tail
-	       (setq completions
-		     (cons (cons str nil)
-			   completions))))))
+	 (or (assoc-string str completions t)
+	     (push str completions))))
      ;; If no good default was found, try an alternate.
      (or default
 	 (setq default alt-default))
      ;; If only one cross-reference found, then make it default.
      (if (eq (length completions) 1)
-         (setq default (car (car completions))))
+         (setq default (car completions)))
      (if completions
 	 (let ((input (completing-read (if default
 					   (concat
@@ -1543,19 +1532,21 @@
       (setq i (+ i 1)))
     (Info-goto-node target)))
 
+(defconst Info-menu-entry-name-re "\\(?:[^:\n]\\|:[^,.;() \t\n]\\)*"
+  "Regexp that matches a menu entry name upto but not including the colon.
+Because of ambiguities, this should be concatenated with something like
+`:' and `Info-following-node-name-re'.")
+
 (defun Info-extract-menu-node-name (&optional multi-line)
   (skip-chars-forward " \t\n")
-  (let ((beg (point))
-	str)
-    (while (not (looking-at ":*[,.;() \t\n]"))
-      (skip-chars-forward "^:")
-      (forward-char 1))
-    (setq str
-	  (if (looking-at ":")
-	      (buffer-substring-no-properties beg (1- (point)))
-	    (skip-chars-forward " \t\n")
-	    (Info-following-node-name (if multi-line "^.,\t" "^.,\t\n"))))
-    (replace-regexp-in-string "[ \n]+" " " str)))
+  (let (str)
+    (when (looking-at (concat Info-menu-entry-name-re ":\\(:\\|"
+			      (Info-following-node-name-re
+			       (if multi-line "^.,\t" "^.,\t\n")) "\\)"))
+      (setq str (or (match-string 2)
+		    (buffer-substring (match-beginning 0)
+				      (1- (match-beginning 1)))))
+      (replace-regexp-in-string "[ \n]+" " " str))))
 
 ;; No one calls this.
 ;;(defun Info-menu-item-sequence (list)
@@ -1567,7 +1558,8 @@
 (defvar Info-complete-next-re nil)
 (defvar Info-complete-cache nil)
 
-(defconst Info-node-spec-re "[^.,:(]*\\(([^)]*)[^.,:]*\\)?[,:.]"
+(defconst Info-node-spec-re
+  (concat (Info-following-node-name-re "^.,:") "[,:.]")
   "Regexp to match the text after a : until the terminating `.'.")
 
 (defun Info-complete-menu-item (string predicate action)
@@ -1594,7 +1586,7 @@
 	   (concat "\n\\* +" (regexp-quote string) ":") nil t)
 	(let ((pattern (concat "\n\\* +\\("
 			       (regexp-quote string)
-			       "[^\t\n]*?\\):" Info-node-spec-re))
+			       Info-menu-entry-name-re "\\):" Info-node-spec-re))
 	      completions)
 	  ;; Check the cache.
 	  (if (and (equal (nth 0 Info-complete-cache) Info-current-file)
@@ -2373,9 +2365,7 @@
 	  (save-excursion
 	    (goto-char (point-min))
 	    (while (re-search-forward "\\*note[ \n\t]*\\([^:]*\\):" nil t)
-	      (setq str (buffer-substring
-			 (match-beginning 1)
-			 (1- (point))))
+	      (setq str (match-string 1))
 	      (setq i 0)
 	      (while (setq i (string-match "[ \n\t]+" str i))
 		(setq str (concat (substring str 0 i) " "
@@ -2798,7 +2788,7 @@
 	    (let* ((nbeg (match-beginning 2))
 		   (nend (match-end 2))
 		   (tbeg (match-beginning 1))
-		   (tag (buffer-substring tbeg (match-end 1))))
+		   (tag (match-string 1)))
 	      (if (string-equal tag "Node")
 		  (put-text-property nbeg nend 'font-lock-face 'info-header-node)
 		(put-text-property nbeg nend 'font-lock-face 'info-header-xref)
@@ -2817,7 +2807,7 @@
 				    ((equal tag "Up") Info-up-link-keymap))))))
 	  (when Info-use-header-line
 	    (goto-char (point-min))
-	    (let ((header-end (save-excursion (end-of-line) (point)))
+	    (let ((header-end (line-end-position))
 		  header)
 	      ;; If we find neither Next: nor Prev: link, show the entire
 	      ;; node header.  Otherwise, don't show the File: and Node:
@@ -2829,7 +2819,7 @@
 		  (progn
 		    (goto-char (match-beginning 1))
 		    (setq header (buffer-substring (point) header-end)))
-		(if (re-search-forward "node:[ \t]*[^ \t]+[ \t]*" nil t)
+		(if (re-search-forward "node:[ \t]*[^ \t]+[ \t]*" header-end t)
 		    (setq header
 			  (concat "No next, prev or up links  --  "
 				  (buffer-substring (point) header-end)))
@@ -2933,9 +2923,9 @@
 		   (< (- (point-max) (point)) Info-fontify-maximum-menu-size))
 	  (let ((n 0)
 		cont)
-	    (while (re-search-forward (concat "^\\* +\\([^:\t\n]*\\)\\(:"
-					      Info-node-spec-re
-					      "\\([ \t]*\\)\\)")
+	    (while (re-search-forward
+		    (concat "^\\* +\\(" Info-menu-entry-name-re "\\)\\(:"
+			    Info-node-spec-re "\\([ \t]*\\)\\)")
 				      nil t)
 	      (setq n (1+ n))
 	      (if (and (<= n 9) (zerop (% n 3))) ; visual aids to help with 1-9 keys
@@ -2952,11 +2942,11 @@
 		      '(font-lock-face info-xref
 			mouse-face highlight))))
 	      (when (eq Info-hide-note-references t)
-		(put-text-property (match-beginning 2) (1- (match-end 4))
+		(put-text-property (match-beginning 2) (1- (match-end 6))
 				   'invisible t)
 		;; We need a stretchable space like :align-to but with
 		;; a minimum value.
-		(put-text-property (1- (match-end 4)) (match-end 4) 'display
+		(put-text-property (1- (match-end 6)) (match-end 6) 'display
 				   (if (>= 22 (- (match-end 1)
 						 (match-beginning 0)))
 				       '(space :align-to 24)

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Stefan Monnier wrote:

   Others will say the problem is simply that we allow `:', but assuming we
   still want to allow colon despite the obvious ambiguity, I think the patch
   below should help.

   It uses regexp matching rather than combinations of skip-chars-forward
   and stuff.  The reason for the change is because, by using regexp-matching,
   we can take advantage of its backtracking to resolve the ambiguity.

   Problem is that it still fails on:

      * name1:foo:node.           des:cription.

I believe we should wait to install any patches until we can agree on
the specs.  In particular, I will wait with my proposed patch to
Info-extract-menu-node-name until such specs are agreed upon.  I
personally do not have any strong opinions in this matter, but it
certainly is important to come up with specs that both Richard and
Karl can agree on.  I was not a part of previous discussions on the
topic and have not the slightest idea what Richard's views are.  These
could have changed, due to recent examples and problems.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

We probably need a decision on the following issue:

It seems that the current de facto situation for menu subtopic names
is the following:

1.  They can contain colons, but not colons followed by a space.
2.  There must be at least one space separating the subtopic name and
    the node name.

If this is considered OK, then nothing needs to be done.  

Some, like Karl, would prefer not to allow colons at all until a new
syntax for quoting node names is introduced in Texinfo.  That would
also be easy, one would just have to revert earlier changes.

Stefan and Karl pointed out that condition 2 used not to be
necessary.  There seems to be some doubt as to whether its
introduction was deliberate or accidental.

If we want 1.  but not 2. then things become much more complicated and
precise specs would need to be agreed upon.

Quoting Stefan, who proposed an (if I understand correctly) admittedly
non-ideal patch:

   Problem is that it still fails on:

      * name1:foo:node.           des:cription.

   because it thinks the menu entry name is "name1:foo:node.           des"
   and the node name is "cription".  I'm sure we can tune the disambiguation
   by saying that menu entries can have `:' but cannot have `.' after the `:'
   or some other such heuristic.  If anybody wants to spice up the
   Info-menu-entry-name-re regexp accordingly...
   We really need a better solution.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Karl Berry]

    ... situation for menu subtopic names is the following:
    1.  They can contain colons, but not colons followed by a space.
    2.  There must be at least one space separating the subtopic name and
        the node name.

I think this is ok, because there are few manuals which lack a space
between a subtopic and node name.  As I said, the only ones I could find
were a few of the xemacs subsidiary manuals.  We should inform their
authors.

    Some, like Karl, would prefer not to allow colons at all until a new

Well, I think the rule above is ok.  The benefit, of allowing colons in
subtopic names, is worthwhile enough that I don't mind the additional
complexity, since the real fix is not on the horizon.

    Stefan and Karl pointed out that condition 2 used not to be necessary

And condition 1 did not obtain; colons were not allowed.

    If we want 1.  but not 2. then things become much more complicated and
    precise specs would need to be agreed upon.

1-but-not-2 requires the "real fix" or "better solution" or whatever you
want to call it, as far as I can tell.  (Quoting syntax.)

   Problem is that it still fails on:

      * name1:foo:node.           des:cription.

   because it thinks the menu entry name is "name1:foo:node.           des"
   and the node name is "cription".  I'm sure we can tune the disambiguation
   by saying that menu entries can have `:' but cannot have `.' after the `:'
   or some other such heuristic.  

Huh?  According to the rule above, the above is invalid because it lacks
a space, right?  It would have to be written as:

Name1:foo: node.  des:description.

I don't like the idea of trying to allow subtopic names to be followed
by :<nonspace> based on periods.  If that's what you were saying.
It's simpler both to understand and to implement to just require :<space>
as a separator.  Or :<tab>, I suppose.

Ok?

k

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Karl Berry wrote:

   Huh?  According to the rule above, the above is invalid because it lacks
   a space, right?  It would have to be written as:

   Name1:foo: node.  des:description.

Yes, but I was quoting Stefan who seemed to be trying to implement
1-but-not-2.

   I don't like the idea of trying to allow subtopic names to be followed
   by :<nonspace> based on periods.  If that's what you were saying.

It was what I was mentioning as a theoretical possibility, because
Stefan seemed to be trying to implement it.  It was not something I
was arguing for.

   It's simpler both to understand and to implement to just require :<space>
   as a separator.  Or :<tab>, I suppose.

I agree, since 1-but-not-2 would be unnecessarily complex.  Without a
space or tab, things also _look_ bad when note references are not
hidden.  There does not seem to be any reason to desperately want to
avoid the space in question.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Stefan Monnier]

>     Stefan and Karl pointed out that condition 2 used not to be necessary
> 
> And condition 1 did not obtain; colons were not allowed.

I thought the idea was to allow colons such that previously invalid
entries (with colons) are now allowed.  I think previously valid
entries should stay valid.

>     If we want 1.  but not 2. then things become much more complicated and
>     precise specs would need to be agreed upon.
> 
> 1-but-not-2 requires the "real fix" or "better solution" or whatever you
> want to call it, as far as I can tell.  (Quoting syntax.)
> 
>    Problem is that it still fails on:
> 
>       * name1:foo:node.           des:cription.
> 
>    because it thinks the menu entry name is "name1:foo:node.           des"
>    and the node name is "cription".  I'm sure we can tune the disambiguation
>    by saying that menu entries can have `:' but cannot have `.' after the `:'
>    or some other such heuristic.  
> 
> Huh?  According to the rule above, the above is invalid because it lacks
> a space, right?

Indeed, it's not a big deal: it was not valid before and it's not
valid now either.  It's thus a non-issue.  Sorry for not noticing it.

> If that's what you were saying.
> It's simpler both to understand and to implement to just require :<space>
> as a separator.  Or :<tab>, I suppose.

Well, actually my regexp implementation is simpler than the hand-parsing code
that was used before, AFAICT.


	Stefan

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

After your recent change to info.el, or maybe related changes in the C
code, Info rather easily seems to go into infinite loops, with C-g
being of no help.  I used Info without problems after Kim's recent
changes, so it would seem that the problems are due to your changes.

Example:

emacs-21.3.50 -q --eval "(blink-cursor-mode 0)" &

C-h i m Info RETURN.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Even if you can fix the infinite loop problems, some questions might
remain.  How is your code supposed to handle:

* Mtools:(mtools).        Mtools: utilities to access DOS disks in Unix.

which is an actual entry with one critical space removed.  The actual
entry starts:

* Mtools: (mtools).

What are your exact rules for which subtopic names are valid and which
ones are not?  Is:

Mtools:(mtools).        Mtools:

a valid subtopic name?

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Robert J. Chassell]

Luc Teirlinck <teirllm@dms.auburn.edu> wrote:

    ... Info rather easily seems to go into infinite loops, with C-g
    being of no help. ...

    Example:

    emacs-21.3.50 -q --eval "(blink-cursor-mode 0)" &

I can confirm this using

    Today's CVS snapshot, Sat, 2003 Jul  5  18:18 UTC
    GNU Emacs 21.3.50.117 (i686-pc-linux-gnu, X toolkit)

Here is a backtrace:


(gdb) bt
#0  0x081186be in re_match_2_internal (bufp=0x82aec9c, string1=0x0, size1=0, 
    string2=0x40411582 "File: info,  Node: Top,  Next: Getting Started,  Up: (dir)\n\nInfo: An Introduction\n", '*' <repeats 21 times>, "\n\nThe GNU Project distributes most of its on-line manuals in the \"Info\nformat\", which you read us"..., size2=857, pos=586, regs=0x81c3fa4, stop=234) at regex.c:5559
#1  0x08116864 in re_search_2 (bufp=0x82aec9c, 
    str1=0x40411582 "File: info,  Node: Top,  Next: Getting Started,  Up: (dir)\n\nInfo: An Introduction\n", '*' <repeats 21 times>, "\n\nThe GNU Project distributes most of its on-line manuals in the \"Info\nformat\", which you read us"..., size1=857, str2=0x40420ce2 "", size2=0, startpos=586, range=271, 
    regs=0x81c3fa4, stop=857) at regex.c:4262
#2  0x0810d7b3 in search_buffer (string=-58471, pos=1987, pos_byte=584, 
    lim=2260, lim_byte=2260, n=1, RE=1, trt=1212420280, 
    inverse_trt=1212441200, posix=0) at search.c:1103
#3  0x0810d0ad in search_command (string=946626620, bound=405492404, 
    noerror=405492452, count=140045003, direction=1, RE=1, posix=0)
    at search.c:904
#4  0x0810f3a1 in Fre_search_forward (regexp=946626620, bound=405492404, 
    noerror=405492452, count=405492404) at search.c:2125
#5  0x08131556 in Ffuncall (nargs=4, args=0xbfffeba0) at eval.c:2724
#6  0x08158e91 in Fbyte_code (bytestr=-1073746928, vector=3, 
    maxdepth=-1073746928) at bytecode.c:710
#7  0x08131864 in funcall_lambda (fun=1213612088, nargs=0, 
    arg_vector=0xbfffed34) at eval.c:2911
#8  0x0813142f in Ffuncall (nargs=1, args=0xbfffed30) at eval.c:2781
#9  0x08158e91 in Fbyte_code (bytestr=408418124, vector=0, 
    maxdepth=-1073746640) at bytecode.c:710
#10 0x08131864 in funcall_lambda (fun=1213280688, nargs=0, 
    arg_vector=0xbfffee54) at eval.c:2911
#11 0x0813142f in Ffuncall (nargs=1, args=0xbfffee50) at eval.c:2781
#12 0x08158e91 in Fbyte_code (bytestr=407516436, vector=0, 
    maxdepth=-1073746352) at bytecode.c:710
#13 0x08131864 in funcall_lambda (fun=1212903736, nargs=3, 
    arg_vector=0xbfffef74) at eval.c:2911
#14 0x0813142f in Ffuncall (nargs=4, args=0xbfffef70) at eval.c:2781
#15 0x08158e91 in Fbyte_code (bytestr=407516460, vector=3, 
    maxdepth=-1073746064) at bytecode.c:710
#16 0x08131864 in funcall_lambda (fun=1212256064, nargs=2, 
    arg_vector=0xbffff094) at eval.c:2911
#17 0x0813142f in Ffuncall (nargs=3, args=0xbffff090) at eval.c:2781
#18 0x08158e91 in Fbyte_code (bytestr=407516436, vector=2, 
    maxdepth=-1073745776) at bytecode.c:710
#19 0x08131864 in funcall_lambda (fun=1212933496, nargs=2, 
    arg_vector=0xbffff1b4) at eval.c:2911
#20 0x0813142f in Ffuncall (nargs=3, args=0xbffff1b0) at eval.c:2781
#21 0x08158e91 in Fbyte_code (bytestr=408321860, vector=2, 
    maxdepth=-1073745488) at bytecode.c:710
#22 0x08131864 in funcall_lambda (fun=1212876272, nargs=2, 
    arg_vector=0xbffff2c4) at eval.c:2911
#23 0x0813142f in Ffuncall (nargs=3, args=0xbffff2c0) at eval.c:2781
#24 0x08130e5d in Fapply (nargs=2, args=0xbffff340) at eval.c:2231
#25 0x08131225 in apply1 (fn=408211020, arg=1483482500) at eval.c:2487
#26 0x0812df34 in Fcall_interactively (function=408211020, 
    record_flag=405492404, keys=1210874048) at callint.c:399
#27 0x080e2976 in Fcommand_execute (cmd=408211020, record_flag=405492404, 
    keys=405492404, special=405492404) at keyboard.c:9724
#28 0x080d7fb3 in command_loop_1 () at keyboard.c:1755
#29 0x0812fa6a in internal_condition_case (bfun=0x80d7c5c <command_loop_1>, 
    handlers=405604540, hfun=0x80d7850 <cmd_error>) at eval.c:1333
#30 0x080d7b32 in command_loop_2 () at keyboard.c:1292
#31 0x0812f609 in internal_catch (tag=86, func=0x80d7b14 <command_loop_2>, 
    arg=405492404) at eval.c:1094
#32 0x080d7ae8 in command_loop () at keyboard.c:1271
#33 0x080d762c in recursive_edit_1 () at keyboard.c:987
#34 0x080d773c in Frecursive_edit () at keyboard.c:1043
#35 0x080d609d in main (argc=4, argv=0xbffffae4) at emacs.c:1666

Re: Invisible colons in Emacs Info.

[Robert J. Chassell]

       > Luc Teirlinck <teirllm@dms.auburn.edu> wrote:
       > 
       >     ... Info rather easily seems to go into infinite loops, with C-g
       >     being of no help. ...
       > 
       >     Example:
       > 
       >     emacs-21.3.50 -q --eval "(blink-cursor-mode 0)" &

    storm@cua.dk (Kim F. Storm) writes

       I have just installed a fix for this.  

Success!  Thank you.  I can confirm that CVS GNU Emacs works again
using

    Today's CVS snapshot, Sun, 2003 Jul  6  12:47 UTC
    GNU Emacs 21.3.50.118 (i686-pc-linux-gnu, X toolkit)

-- 
    Robert J. Chassell                         Rattlesnake Enterprises
    http://www.rattlesnake.com                  GnuPG Key ID: 004B4AC8
    http://www.teak.cc                             bob@rattlesnake.com

Re: Invisible colons in Emacs Info.

[Kim F. Storm]

"Robert J. Chassell" <bob@rattlesnake.com> writes:

> Luc Teirlinck <teirllm@dms.auburn.edu> wrote:
> 
>     ... Info rather easily seems to go into infinite loops, with C-g
>     being of no help. ...
> 
>     Example:
> 
>     emacs-21.3.50 -q --eval "(blink-cursor-mode 0)" &
> 
> I can confirm this using

I have just installed a fix for this.  

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Kim Storm wrote:

   "Robert J. Chassell" <bob@rattlesnake.com> writes:

   > Luc Teirlinck <teirllm@dms.auburn.edu> wrote:
   > 
   >     ... Info rather easily seems to go into infinite loops, with C-g
   >     being of no help. ...
   > 
   >     Example:
   > 
   >     emacs-21.3.50 -q --eval "(blink-cursor-mode 0)" &
   > 
   > I can confirm this using

   I have just installed a fix for this.  

There are still ptoblems with the very case Stefan tried to address.
The next two files are the exact same files I sent to Stefan earlier:

===File ~/experiment.texi===================================
\input texinfo    @c -*-texinfo-*-
@comment %**start of header
@setfilename experiment.info
@settitle An experiment.
@comment %**end of header

@node Top
@top Top

@menu
* name:entry.           description.
@end menu

@node entry
@chapter Entry

This node is called @samp{entry}

@bye
============================================================

===File ~/experiment.info===================================
This is experiment.info, produced by makeinfo version 4.6 from
/home/teirllm/info/experiment.texi.

^_
File: experiment.info,  Node: Top,  Next: entry,  Up: (dir)

Top
***

* Menu:

* name:entry.           description.

^_
File: experiment.info,  Node: entry,  Prev: Top,  Up: Top

Entry
*****

This node is called `entry'


^_
Tag Table:
Node: Top\x7f100
Node: entry\x7f219
^_
End Tag Table
============================================================

Put:

* experiment: (experiment).               Just an experiment

in an appropriate dir file.

Trying to visit `experiment' still produces an infinite loop.

Actually, so does trying to run C-c C-m C-b on experiment.texi.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Kim Storm wrote:

   I have just installed a fix for this.

As I pointed out in an earlier message there are still problems after
that fix.  The problems only occur in a "manufactured" example.
However, these kind of examples are the only reason behind Stefan's
recent changes, with which there are other problems as well, as I
pointed out in an earlier message.

Are the infinite loops due to Stefan's changes?  They could only be
due to your changes if I would have accidentally forgotten to run
`make install' after (believing to) have installed your changes.

Stefan, what about just reverting your changes?

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

>From my earlier message:

   Kim Storm wrote:

      I have just installed a fix for this.

   As I pointed out in an earlier message there are still problems after
   that fix.

Sorry, false alarm.  The bug I reported was fixed by Stefan a couple
of hours ago.  I should have updated one more time before filing the
report.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Karl Berry]

    Mtools:(mtools).        Mtools:

    a valid subtopic name?

According to what we've discussed, that is invalid because the space
between the : and the ( is missing.  I thought we had all agreed on this?

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Karl Berry wrote:

       Mtools:(mtools).        Mtools:

       a valid subtopic name?

   According to what we've discussed, that is invalid because the space
   between the : and the ( is missing.  I thought we had all agreed on this?

You and me agreed, but I do not believe Stefan ever did.  Otherwise
there would have been no reason for his recent changes.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Stefan Monnier]

> Karl Berry wrote:
> 
>        Mtools:(mtools).        Mtools:
> 
>        a valid subtopic name?
> 
>    According to what we've discussed, that is invalid because the space
>    between the : and the ( is missing.  I thought we had all agreed on this?
> 
> You and me agreed, but I do not believe Stefan ever did.  Otherwise
> there would have been no reason for his recent changes.

My recent change was done for the following reasons:

1 - centralize the definition of what is an acceptable menu entry name
2 - centralize the definition of what is an acceptable node name
3 - get rid of ad-hoc parsing code and use a more declarative style
    using regexps.
4 - accept some old menu entries (that are now considered invalid) as long
    as we can unambiguously recognize them.

I'm fine with labelling menu entries that lack the space after the colon
as invalid, but I see no reason not to tweak the regexp such that we can
still recognize some of those old entries (as long as such tweaking does
not impact valid cases).
My change makes the code accept more cases than what we consider as "valid",
but that seems like a feature rather than a bug since such entries exist
and were valid until Emacs-21.3.  Of course, the priority should be given
to the correct treatment of valid entries.


	Stefan

Re: Invisible colons in Emacs Info.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Kim Storm wrote:
> 
>    I have just installed a fix for this.
> 
> Are the infinite loops due to Stefan's changes?

Yes, it was an error in a regexp.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

In as far as the concrete example is concerned, I might have responded
a little bit too quickly.  Karl copied only part of it.  Here is the
full line:

* Mtools:(mtools).        Mtools: utilities to access DOS disks in Unix.

The point of my original message of several days ago was that one
can consider 

Mtools:(mtools).          Mtools 

to be a valid subtopic name according to the rules we discussed,
because there *is* a space after the *second* Mtools and not after the
first.  Because the lack of a space after the first Mtools,

Mtools:(mtools).          Mtools

would really seem to be the actual subtopic name.  If it is not, there
must be some reason why it is not, other than that it just looks silly.

So, in other words, the importance of putting a space after the first
Mtools is really to make clear that this is where the real subtopic
name ends.  The reason why Karl and me would like to insist on the
space after the final colon is not: "Because we tell so!", but to make
clear that the final colon is really the final colon and to avoid
confusion in cases like this one.

Actually, let us take a look at the actual menu entry which does have
the space we want:

* Mtools: (mtools).        Mtools: utilities to access DOS disks in Unix.

I do not know whether you have this entry and hence can check the
following:

Put the cursor on that line and do `m'.

Look at the echo area:

Menu item (default Mtools: (mtools).        Mtools):

Even in this case `m' still seems to consider

Mtools: (mtools).        Mtools

as the subtopic name, as least in as far as the echo area message is
concerned.  This definitely seems to make no sense, given the presence
of the required space.  Pressing RETURN does visit `Mtools' however.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Karl Berry]

    still recognize some of those old entries (as long as such tweaking does
    not impact valid cases).

Of course that's fine and desirable.

I guess that if there are no other colons or periods in the menu entry
after the first :, it can be unambiguously parsed. (Not counting the ::
form, of course.)  But if there are following colons or periods, then it
seems the space is needed.

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Karl Berry wrote:

       still recognize some of those old entries (as long as such tweaking does
       not impact valid cases).

   Of course that's fine and desirable.

   I guess that if there are no other colons or periods in the menu entry
   after the first :, it can be unambiguously parsed. (Not counting the ::
   form, of course.)  But if there are following colons or periods, then it
   seems the space is needed.

Which means that Stefan's recent changes do not reduce the importance
of making people aware that the rules have changed.  The example I
gave shows that it is easy for colons to appear in descriptions.  So
it is important that Stefan's partial and unreliable support for the
old syntax does not give people the misleading impression that the old
syntax is still reliably supported.  Any form of incomplete support
always carries the danger of misleading people into assuming full
support.

I guess that we could have makeinfo print warnings in case of subtopic
names that contain a colon or end in a colon without following
whitespace, pointing out that:

1.  Subtopic names containing colons, while supported by Emacs Info, are
    not currently supported by the stand-alone Info.

2.  Only a colon followed by a space (or tab), or two colons, still
    reliably terminate a subtopic name.  Other subtopic names are
    deprecated and may or may not still be recognized.

So 

* Mtools:(mtools).        Mtools: utilities to access DOS disks in Unix.

would consider:

Mtools:(mtools).        Mtools

to be the subtopic name and:

utilities to access DOS disks in Unix

to be the node name, but makeinfo would warn the user of that fact and
also of the fact that this syntax is not currently recognized by the
stand-alone Info.

On the other hand:

* Mtools: (mtools).        Mtools: utilities to access DOS disks in Unix.

would consider Mtools to be the subtopic name without warnings of any
kind (of course).

I believe that Emacs' current echo area behavior for `m' in this case
should be changed, as I pointed out before.

Sincerely,

Luc.

Re: Invisible colons in Emacs Info.

[Stefan Monnier]

> So, in other words, the importance of putting a space after the first
> Mtools is really to make clear that this is where the real subtopic
> name ends.  The reason why Karl and me would like to insist on the
> space after the final colon is not: "Because we tell so!", but to make
> clear that the final colon is really the final colon and to avoid
> confusion in cases like this one.

I don't know what makes you think that this hasn't been clear, but
just in case: yes we all agree about that (including the regexp, AFAIK).

> Actually, let us take a look at the actual menu entry which does have
> the space we want:
> 
> * Mtools: (mtools).        Mtools: utilities to access DOS disks in Unix.
> 
> I do not know whether you have this entry and hence can check the
> following:
> 
> Put the cursor on that line and do `m'.
> 
> Look at the echo area:

Nitpick: this is not the echo area but the minibuffer.
The reason why I'm nitpicking here is because you also referred to the echo
area in the other email and I thought you were referring the help-echo
property (which is the only thing appearing in the echo area).

> Menu item (default Mtools: (mtools).        Mtools):

It looks like a bug due to the fact that this part of the code had its
own idea of what is a valid menu entry name.  The bug was
introduced several months ago along with the patch that allowed `:'
in menu entries.  See patch below.


	Stefan


--- info.el.~1.364.~	Sun Jul  6 13:01:11 2003
+++ info.el	Mon Jul  7 10:44:54 2003
@@ -1652,7 +1652,9 @@
 	    (save-excursion
 	      (goto-char p)
 	      (end-of-line)
-	      (if (re-search-backward "\n\\* +\\([^\t\n]*\\):" beg t)
+	      (if (re-search-backward (concat "\n\\* +\\("
+					      Info-menu-entry-name-re
+					      "\\):") beg t)
 		  (setq default (match-string-no-properties 1))))))
      (let ((item nil))
        (while (null item)

Re: Invisible colons in Emacs Info.

[Luc Teirlinck]

Is the following problem due to your recent changes?

In (emacs)Faces, do f transient mark RETURN

Result:

Info-follow-reference: Wrong type argument: stringp, nil

This seems to happen with all *multi-line* cross-references.

Sincerely,

Luc.