bug#74844: 29.4; m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

bug#74844: 29.4; m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Suhail Singh]

The Texinfo entry in the "dir" and "m-buffer-doc.info" files are
malformed.  As a consequence the font-locking as well as the completion
for the Info-menu command are broken in the Top directory.  CC-ing
Stefan given his familiarity with documentation generation for the ELPA
package in question (based on git commits).

The @direntry for Texinfo is of the form ‘* DIRNAME: NODE.’ where ‘NODE’
is usually just ‘(FILENAME)’.  In the case of m-buffer it ought to be
something like "* m-buffer: (m-buffer-doc)."  However, the @direntry as
noted in the "dir" and "m-buffer-doc.info" files is "* m-buffer.doc."
As a result, `Info-follow-nearest-node' (bound to RET in Info-mode-map)
is unable to determine the appropriate node when point is on the
m-buffer-doc entry.  Digging deeper, the failure is due to the following
code resulting in nil:

#+begin_src emacs-lisp
  (Info-get-token (point) "\\* +" "\\* +\\(.*\\): ")
#+end_src

Fixing the direntry in the "dir" and "m-buffer-doc-info" files resolves
the issue.

Given that the documentation is generated from m-buffer-doc.org via
ox-texinfo, presumably the issue lies in the incorrect usage of the
obsolete TEXINFO_DIR_TITLE keyword.  Presumably the fix would be to use
the TEXINFO_FILENAME and, possibly, the TEXINFO_DIR_NAME keywords.
However, for some reason I am unable to generate the documentation
locally.  As such I am not submitting a patch.  Hope this helps.

In GNU Emacs 29.4 (build 2, x86_64-suse-linux-gnu, GTK+ Version 3.24.43,
cairo version 1.18.2)
System Description: openSUSE Tumbleweed

Configured using:
 'configure --disable-build-details --without-pop --with-mailutils
 --with-native-compilation --without-hesiod --with-gameuser=:games
 --with-kerberos --with-kerberos5 --with-file-notification=inotify
 --with-modules --enable-autodepend --prefix=/usr
 --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share
 --localstatedir=/var --sharedstatedir=/var/lib
 --libexecdir=/usr/libexec --with-file-notification=yes
 --libdir=/usr/lib64
 --enable-locallisppath=/usr/share/emacs/29.4/site-lisp:/usr/share/emacs/site-lisp
 --with-x --with-xim --with-sound --with-xpm --with-jpeg --with-tiff
 --with-gif --with-png --with-rsvg --with-dbus --with-webp --with-xft
 --with-imagemagick --without-gpm --with-x-toolkit=gtk3 --with-pgtk
 --with-toolkit-scroll-bars --x-includes=/usr/include
 --x-libraries=/usr/lib64 --with-libotf --with-m17n-flt --with-cairo
 --with-xwidgets --build=x86_64-suse-linux --with-dumping=pdumper
 'CFLAGS=-O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3
 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables
 -fstack-clash-protection -Werror=return-type -flto=auto -g
 -D_GNU_SOURCE -DGDK_DISABLE_DEPRECATION_WARNINGS
 -DGLIB_DISABLE_DEPRECATION_WARNINGS -pipe -Wno-pointer-sign
 -Wno-unused-variable -Wno-unused-label -fno-optimize-sibling-calls
 -DPDMP_BASE='\''"emacs-wayland"'\''' LDFLAGS=-Wl,-O2'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM XWIDGETS GTK3 ZLIB

Important settings:
  value of $LC_NUMERIC: POSIX
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Info

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  isearch-fold-quotes-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr info emacsbug message mailcap yank-media puny
dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg
rfc6068 epg-config gnus-util text-property-search time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils comp
comp-cstr warnings icons subr-x rx cl-seq cl-macs gv cl-extra help-mode
cl-loaddefs cl-lib bytecomp byte-compile ispell rmc delsel lpr
easy-mmode pcase iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win
pgtk-win term/common-win pgtk-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode
register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads xwidget-internal dbusbind inotify dynamic-setting
system-font-setting font-render-setting cairo gtk pgtk lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 101816 9593)
 (symbols 48 7600 0)
 (strings 32 23316 1856)
 (string-bytes 1 792835)
 (vectors 16 16145)
 (vector-slots 8 336083 17636)
 (floats 8 31 20)
 (intervals 56 4892 0)
 (buffers 984 12))

bug#74844: m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> However, the @direntry as
> noted in the "dir" and "m-buffer-doc.info" files is "* m-buffer.doc."

Hmm... yup.
I pushed to Emacs a fix for the generation of that Texinfo element in
`ox-texinfo.el` and to `m-buffer` I pushed another change which tries to
workaround that bug (for when it's used with an older Emacs).

The patch for `ox-texinfo.el` is below.  It's "obviously safe", but
I suspect it's too late for Emacs-30.  Eli?


        Stefan


diff --git a/lisp/org/ox-texinfo.el b/lisp/org/ox-texinfo.el
index 6adee9fca3f..deceeca8947 100644
--- a/lisp/org/ox-texinfo.el
+++ b/lisp/org/ox-texinfo.el
@@ -826,9 +826,7 @@ org-texinfo-template
               ;; `dn' is presumed to be just the DIRNAME part, so generate
               ;; either `* DIRNAME: (FILENAME).' or `* FILENAME.', whichever
               ;; is shortest.
-              ((and dn (not (equal dn file)))
-               (format "* %s: (%s)." dn (or file dn)))
-              (t (format "* %s." file)))))
+              (t (format "* %s: (%s)." (or dn file) (or file dn))))))
        (concat "@dircategory " dircat "\n"
 	       "@direntry\n"
 	       (let ((dirdesc

bug#74844: m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Eli Zaretskii]

> Cc: 74844@debbugs.gnu.org
> Date: Fri, 13 Dec 2024 16:50:02 -0500
> From:  Stefan Monnier via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> > However, the @direntry as
> > noted in the "dir" and "m-buffer-doc.info" files is "* m-buffer.doc."
> 
> Hmm... yup.
> I pushed to Emacs a fix for the generation of that Texinfo element in
> `ox-texinfo.el` and to `m-buffer` I pushed another change which tries to
> workaround that bug (for when it's used with an older Emacs).
> 
> The patch for `ox-texinfo.el` is below.  It's "obviously safe", but
> I suspect it's too late for Emacs-30.  Eli?

Maybe.  For now, I don't think I understand the fix.  What is 'dn' in
this snippet?  The command above says something (which need to be
fixed to follow the code change, btw), but I'm not sure it is accurate
or complete.

An entry in DIR can be either

     * TITLE: (FILE).           DESCRIPTION
or
     * TITLE: (FILE)NODE.       DESCRIPTION

What is 'dn' in the above scheme?  And what is 'file'?

P.S. Should we CC the Org folks on this discussion?

bug#74844: m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > However, the @direntry as
>> > noted in the "dir" and "m-buffer-doc.info" files is "* m-buffer.doc."
>> 
>> Hmm... yup.
>> I pushed to Emacs a fix for the generation of that Texinfo element in
>> `ox-texinfo.el` and to `m-buffer` I pushed another change which tries to
>> workaround that bug (for when it's used with an older Emacs).
>> 
>> The patch for `ox-texinfo.el` is below.  It's "obviously safe", but
>> I suspect it's too late for Emacs-30.  Eli?
>
> Maybe.  For now, I don't think I understand the fix.  What is 'dn' in
> this snippet?  The command above says something (which need to be
> fixed to follow the code change, btw), but I'm not sure it is accurate
> or complete.
>
> An entry in DIR can be either
>
>      * TITLE: (FILE).           DESCRIPTION
> or
>      * TITLE: (FILE)NODE.       DESCRIPTION

[ Side question: Where is this documented?  ]

> What is 'dn' in the above scheme?  And what is 'file'?

`dn` is your TITLE and `file` is your FILE.

> P.S. Should we CC the Org folks on this discussion?

Duh, forgot about that, I just added Ihor, thanks,


        Stefan

bug#74844: m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: suhailsingh247@gmail.com,  74844@debbugs.gnu.org, Ihor Radchenko
>  <yantar92@posteo.net>
> Date: Sat, 14 Dec 2024 08:53:04 -0500
> 
> >> > However, the @direntry as
> >> > noted in the "dir" and "m-buffer-doc.info" files is "* m-buffer.doc."
> >> 
> >> Hmm... yup.
> >> I pushed to Emacs a fix for the generation of that Texinfo element in
> >> `ox-texinfo.el` and to `m-buffer` I pushed another change which tries to
> >> workaround that bug (for when it's used with an older Emacs).
> >> 
> >> The patch for `ox-texinfo.el` is below.  It's "obviously safe", but
> >> I suspect it's too late for Emacs-30.  Eli?
> >
> > Maybe.  For now, I don't think I understand the fix.  What is 'dn' in
> > this snippet?  The command above says something (which need to be
> > fixed to follow the code change, btw), but I'm not sure it is accurate
> > or complete.
> >
> > An entry in DIR can be either
> >
> >      * TITLE: (FILE).           DESCRIPTION
> > or
> >      * TITLE: (FILE)NODE.       DESCRIPTION
> 
> [ Side question: Where is this documented?  ]

In the Texinfo manual, in the node "Menu Parts" (since the DIR file is
just a giant menu).

> > What is 'dn' in the above scheme?  And what is 'file'?
> 
> `dn` is your TITLE and `file` is your FILE.

Then I'm not sure this is correct:

+              (t (format "* %s: (%s)." (or dn file) (or file dn))))))

What if FILE is nil?  Can it be nil at this point?  The part in the
parentheses _must_ identify an Info file, with or without a node.  It
cannot be the TITLE, because that one can be arbitrary text.

The change is simple, so once we agree that it cannot do the wrong
thing, installing on emacs-30 is fine by me, assuming Ihor doesn't
object.

bug#74844: m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > An entry in DIR can be either
>> >
>> >      * TITLE: (FILE).           DESCRIPTION
>> > or
>> >      * TITLE: (FILE)NODE.       DESCRIPTION
>> 
>> [ Side question: Where is this documented?  ]
>
> In the Texinfo manual, in the node "Menu Parts" (since the DIR file is
> just a giant menu).

That says:

    A menu entry has three parts, only the second of which is required:
    
        The menu entry name (optional).
        The name of the node (required).
        A description of the item (optional). 

It's not clear what "optional" means for the menu entry name: how do we
write without it?  Does it refer to the

    * NAME::

syntax described in the next node?
BTW, that next node says that `* NAME::` is equivalent to:

    * NAME: NAME.

but shouldn't that be

    * NAME: (NAME).

?

The manual reads like a "user manual" and I think I need something more
like a "reference manual", or a specification, with a precise grammar.

>> > What is 'dn' in the above scheme?  And what is 'file'?
>> 
>> `dn` is your TITLE and `file` is your FILE.
>
> Then I'm not sure this is correct:
>
> +              (t (format "* %s: (%s)." (or dn file) (or file dn))))))
>
> What if FILE is nil?

My reading of the code says it *may* potentially be nil in some corner
case, but I have no idea how that could happen.  A nil value for `dn` is
normal, in contrast.

> Can it be nil at this point?  The part in the
> parentheses _must_ identify an Info file, with or without a node.  It
> cannot be the TITLE, because that one can be arbitrary text.

If FILE is nil, we're in trouble.  IIRC the `(or file dn)` in the code
just tried to preserve the previous behavior in the unlikely case that
FILE is nil, for lack of understanding about when (or even if) that
can happen.


        Stefan

bug#74844: 29.4; m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Suhail Singh]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> and to `m-buffer` I pushed another change which tries to workaround
> that bug (for when it's used with an older Emacs).

> modified    m-buffer-doc.org
> @@ -5,7 +5,7 @@
> 
> # FIXME: Shouldn't `ox-texinfo` use sane defaults like the file's name
> #        for TEXINFO_DIR_TITLE and the TITLE for TEXINFO_DIR_DESC?

I believe TEXINFO_DIR_TITLE is obsolete (based on a comment in
ox-texinfo).

> -#+TEXINFO_DIR_TITLE: m-buffer-doc
> +#+TEXINFO_DIR_TITLE: * m-buffer-do: (m-buffer-doc).
                          ^^^^^^^^^^^

Is that a typo, or intentional?  Either m-buffer or m-buffer-doc in that
position would've been more expected.

Btw, based on my interpretation of the Org manual
(info "(org) Texinfo specific export settings"), I would've expected the
below to work:

#+begin_src diff
  -#+TEXINFO_DIR_TITLE: m-buffer-doc
  -#+TEXINFO_DIR_DESC: Manipulate the Contents of Emacs Buffers
  +#+TEXINFO_FILENAME: m-buffer-doc.texi
  +#+TEXINFO_DIR_NAME: m-buffer
#+end_src

With the TEXINFO_DIR_NAME setting being optional, i.e., if omitted the
title of the menu-item would default to m-buffer-doc.  If this doesn't
happen, then there's a bug in ox-texinfo (either in the documentation or
the code).  I am unable to confirm this locally, since it's not clear
what code is run on ELPA to generate the info manual.

-- 
Suhail

bug#74844: m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: suhailsingh247@gmail.com,  74844@debbugs.gnu.org,  yantar92@posteo.net
> Date: Sat, 14 Dec 2024 16:05:34 -0500
> 
> >> > An entry in DIR can be either
> >> >
> >> >      * TITLE: (FILE).           DESCRIPTION
> >> > or
> >> >      * TITLE: (FILE)NODE.       DESCRIPTION
> >> 
> >> [ Side question: Where is this documented?  ]
> >
> > In the Texinfo manual, in the node "Menu Parts" (since the DIR file is
> > just a giant menu).
> 
> That says:
> 
>     A menu entry has three parts, only the second of which is required:
>     
>         The menu entry name (optional).
>         The name of the node (required).
>         A description of the item (optional). 
> 
> It's not clear what "optional" means for the menu entry name: how do we
> write without it?

Like this:

 * (FILE)NODE.

> Does it refer to the
> 
>     * NAME::
> 
> syntax described in the next node?

That stands for

  * NAME: NAME.

So you could also write

  * (FILE)NODE::

> BTW, that next node says that `* NAME::` is equivalent to:
> 
>     * NAME: NAME.
> 
> but shouldn't that be
> 
>     * NAME: (NAME).
> 
> ?

No, because NAME here is the name of a node, and in DIR all the nodes
are in other files, so NAME should look like "(FILE)NODE" or "(FILE)"
(which stands for "(FILE)Top").  If NAME is a node in the same file as
the menu, the (FILE) part can be omitted.

> The manual reads like a "user manual" and I think I need something more
> like a "reference manual", or a specification, with a precise grammar.

That's because arbitrary programs are not supposed to generate Info
formats, they are supposed to invoke Texinfo programs to do that.  For
example, to update DIR, you are supposed to invoke install-info, which
knows about all these rules (and more).

> > Then I'm not sure this is correct:
> >
> > +              (t (format "* %s: (%s)." (or dn file) (or file dn))))))
> >
> > What if FILE is nil?
> 
> My reading of the code says it *may* potentially be nil in some corner
> case, but I have no idea how that could happen.  A nil value for `dn` is
> normal, in contrast.

A nil value for 'dn' is okay, it produces a correct entry.

> > Can it be nil at this point?  The part in the
> > parentheses _must_ identify an Info file, with or without a node.  It
> > cannot be the TITLE, because that one can be arbitrary text.
> 
> If FILE is nil, we're in trouble.  IIRC the `(or file dn)` in the code
> just tried to preserve the previous behavior in the unlikely case that
> FILE is nil, for lack of understanding about when (or even if) that
> can happen.

In that case, I suggest the following code:

   (dn
     (format "* %s: (%s)." dn (or file dn)))
   (t (format "* (%s)." file))

That is, the bug in the original code is that they failed to put FILE
in parentheses (and also wanted a premature optimization of producing
the "shortest" entry).  This assumes that FILE does not include
parentheses and does not specify a node, i.e. it is NOT in the form
"(FILE)NODE".  My reading of ox-texinfo.el is that if that could
happen, the code in org-texinfo-template is already in trouble,
because it is obviously not ready for that.

WDYT?

bug#74844: 29.4; m-buffer: Broken Top Directory node in Info manual due to malformed Texinfo direntry

[Eli Zaretskii]

> Cc: Ihor Radchenko <yantar92@posteo.net>, 74844@debbugs.gnu.org,
>  Suhail Singh <suhailsingh247@gmail.com>
> From: Suhail Singh <suhailsingh247@gmail.com>
> Date: Sat, 14 Dec 2024 16:48:01 -0500
> 
> Btw, based on my interpretation of the Org manual
> (info "(org) Texinfo specific export settings"), I would've expected the
> below to work:
> 
> #+begin_src diff
>   -#+TEXINFO_DIR_TITLE: m-buffer-doc
>   -#+TEXINFO_DIR_DESC: Manipulate the Contents of Emacs Buffers
>   +#+TEXINFO_FILENAME: m-buffer-doc.texi
>   +#+TEXINFO_DIR_NAME: m-buffer
> #+end_src

Why do you suggest removing TEXINFO_DIR_TITLE and TEXINFO_DIR_DESC?
And why is TEXINFO_FILENAME important?