Indentation conventions for Info manuals; recognizing code

Indentation conventions for Info manuals; recognizing code

[Drew Adams]

I don't know what conventions/styles are defined for our Info manuals, if there are any.  Is that documented somewhere?

Given only an Info buffer, i.e., not Texinfo source code but its output that's used by Emacs users, is there a convention wrt indentation of, say code blocks?  Or is there some other way (e.g. a text property) to recognize code or similar text that really deserves a fixed-pitch font?

Code blocks, ASCII diagrams, and the like seem to be indented 10 chars in the Elisp manual.

That's a start.  Based on that, I added an experimental option to Info+ to fontify text that's indented at least 10 chars (by default).  The face used is fixed-pitch by default.

(There's also a minor mode that lets you use variable-pitch for the main text body - but that's independent.  Being able to fontify code blocks is useful even when all of the text is fixed-pitch.)

For the Elisp manual this fontifying of indented text (>= 10 chars) works quite well. 

My code also optionally highlights quoted sexps (`...', "...") with fixed-pitch faces (by default), so code is fixed-pitch pretty much everywhere in the Elisp manual, even if you choose to make the main text variable-pitch.  Likewise, ASCII-art diagrams (they seem to be indented 10 chars also).

But other manuals don't seem to use the same indentation.  Several (org, eintr, ccmode, efaq(-w32),...) indent code (including Elisp) 5 spaces - which is the same amount that other, non-code text is indented.

(There are also lines in the Emacs manual and ccmode represent user input or program output.  Those too are indented only 5 chars, so they won't appear in fixed-pitch font.)

And if a code block is indented only 5 chars, but some of its lines are indented at least 10 chars, then this optional fontifying kicks in for those latter lines - not great, obviously.

The CC-Mode manual is somewhat like the Elisp manual - it too indents many code blocks 10 chars, but others are indented 5 chars.

Ccmode also has some bullets that are, themselves, indented under indented paragraphs.  (Why is the bullet itself indented wrt the preceding paragraph?  Same thing for numbered lists - why indent the number itself?)

And the continuation lines of such bullets start at ... 10 chars, so they get fontified (incorrectly).  Some bullets are even indented relative to indented paragraphs, in which case even the bullet line gets fontified (incorrectly).  E.g. (ccmode) `Indentation Commands':

‘<TAB>’ (‘c-indent-command’)
     This command indents the current line.  That is all you need to
     know about it for normal use...
...
      -- User Option: c-tab-always-indent
          This variable modifies how <TAB> operates.
             • When it is ‘t’ (the default), <TAB> simply indents the
               current line.
             • When it is ‘nil’, <TAB> (re)indents the line only if
               point is to the left of the first non-whitespace...

(5 levels of indentation.)

Maybe such a `User Option:' entry could/should really be at the left margin (1-char indent)?  (That would also enable the highlighting I do for `User Option:' lines.)  But this seems to be intentional - it's done in several places.

Ccmode also has subheadings of a sort, with the text that follows being indented 5 chars.  And there can be nested text that's indented still further.

Using “Hungry Delete Mode” with ‘<DEL>’ and ‘C-d’
     Here you toggle Hungry Delete minor mode with ‘M-x
     c-toggle-hungry-state’(1) (*note Minor Modes::.)  This makes
     ‘<DEL>’ and ‘C-d’ do backwards and forward hungry deletion.

     ‘<DEL>’ (‘c-electric-backspace’)
          This command is run by default when you hit the ‘<DEL>’ key.
          When hungry delete mode is enabled, it deletes any amount of

That last text is indented ... 10 chars.  Similarly, these indented subheadings:

     The display of the guessed style contains these elements:

     Placeholder Name
          You should replace this with a style name of your own.
     Parent Style
          The style current when the guessing began, from which the
          guessed style inherits (*note Config Basics::) the settings
          which weren’t guessed.
___

So given only an Info buffer, I know of no good way to distinguish code blocks and such meaningfully.  Is there an indentation style/convention that has some kind of semantics in this regard?  I'm guessing no.

My code that does this works only in an Info buffer - it can't see any Texinfo source.  It just tweaks `Info-fontify-node' to optionally fontify some things.  So like the rest of `Info-fontify-node', it can only distinguish what's distinct in the Info buffer.  If displayed code isn't declared/recognized as such at the buffer level then we're out of luck I guess.

I know that the use of Texinfo output (what ends up as Info buffer text) is wider than just Emacs Info.  But is there, at the Texinfo level at least, some semantic markup that distinguishes something like code?  If there is, could that info be maintained - transferred to Info buffers in some way, or does the translation just have to be lossy in this way?

It would be handy to be able to somehow distinguish lines of code from other indented text.
___

To see what I mean, take a look at the Elisp manual, after loading info+.el.  If you want to also see the main text with variable-pitch, then first turn on minor mode `Info-variable-pitch-text-mode'.  (The fixed-pitch face used by default is ugly, unless you happen to have font Lucida Console.  But you can of course customize it.)  Then, to see the problems mentioned, try a manual such as Ccmode or Org.

https://www.emacswiki.org/emacs/download/info%2b.el

Re: Indentation conventions for Info manuals; recognizing code

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> Date: Sun, 7 Mar 2021 01:58:43 +0000
> 
> I don't know what conventions/styles are defined for our Info manuals, if there are any.  Is that documented somewhere?

It's documented in the Texinfo manual.

> Given only an Info buffer, i.e., not Texinfo source code but its output that's used by Emacs users, is there a convention wrt indentation of, say code blocks?  Or is there some other way (e.g. a text property) to recognize code or similar text that really deserves a fixed-pitch font?

No, I don't think so.  Not from examining the indentation, anyway.
You could try some heuristics to detect code from the contents,
perhaps.

> But other manuals don't seem to use the same indentation.  Several (org, eintr, ccmode, efaq(-w32),...) indent code (including Elisp) 5 spaces - which is the same amount that other, non-code text is indented.
> 
> (There are also lines in the Emacs manual and ccmode represent user input or program output.  Those too are indented only 5 chars, so they won't appear in fixed-pitch font.)
> 
> And if a code block is indented only 5 chars, but some of its lines are indented at least 10 chars, then this optional fontifying kicks in for those latter lines - not great, obviously.
> 
> The CC-Mode manual is somewhat like the Elisp manual - it too indents many code blocks 10 chars, but others are indented 5 chars.
> 
> Ccmode also has some bullets that are, themselves, indented under indented paragraphs.  (Why is the bullet itself indented wrt the preceding paragraph?  Same thing for numbered lists - why indent the number itself?)

I think it's hopeless to rely on indentation for such a feature.  As
you discovered, the indentation can vary if the code block is inside
some other "environment", like a table (this is what causes
multi-level indentation in CC Mode manual).  Moreover, there's a
(rarely used) Texinfo directive that changes the default indentation
of a code block, even if it is not nested inside some other indented
text.  If, for example, such a directive is used in the ELisp manual,
it will immediately ruin the results for your feature.

> I know that the use of Texinfo output (what ends up as Info buffer text) is wider than just Emacs Info.  But is there, at the Texinfo level at least, some semantic markup that distinguishes something like code?

Yes, the @example, @smallexample, @lisp, and @smallisp are such
markup in Texinfo.

> If there is, could that info be maintained - transferred to Info buffers in some way, or does the translation just have to be lossy in this way?

The conversion to Info is performed by 'makeinfo', a program that is
part of the Texinfo package.  Its output must be mostly plain text,
and the format of that file must be understood by the various Info
readers out there.  So if some kind of "code" marker would be retained
in the Info output, it would need to be implemented in the 2 or 3
other Info readers out there, not just in Emacs.  The chances of such
a change in the Info output format, just for the sake of some fancy
display in a single Emacs-related package sound small -- but you need
to talk about this with the Texinfo developers, not with us.

RE: [External] : Re: Indentation conventions for Info manuals; recognizing code

[Drew Adams]

[-- Attachment #1: Type: text/plain, Size: 3896 bytes --]

> > Given only an Info buffer, i.e., not Texinfo source code but its output
> that's used by Emacs users, is there a convention wrt indentation of, say
> code blocks?  Or is there some other way (e.g. a text property) to recognize
> code or similar text that really deserves a fixed-pitch font?
> 
> No, I don't think so.  Not from examining the indentation, anyway.
> You could try some heuristics to detect code from the contents,
> perhaps.

I don't see any possible heuristics.  It could be
any kind of code.  And not just code.  Shell input
and output, ASCII art,...  Really anything that it
makes sense to put in a fixed-pitch font.  The kind
of thing that usually gets something like an HTML
<code> tag.

> I think it's hopeless to rely on indentation for such a feature.  As
> you discovered, the indentation can vary if the code block is inside
> some other "environment", like a table (this is what causes
> multi-level indentation in CC Mode manual).  Moreover, there's a
> (rarely used) Texinfo directive that changes the default indentation
> of a code block, even if it is not nested inside some other indented
> text.  If, for example, such a directive is used in the ELisp manual,
> it will immediately ruin the results for your feature.

Yeah, I've discovered as much.

> > I know that the use of Texinfo output (what ends up as Info buffer text) is
> wider than just Emacs Info.  But is there, at the Texinfo level at least,
> some semantic markup that distinguishes something like code?
> 
> Yes, the @example, @smallexample, @lisp, and @smallisp are such
> markup in Texinfo.

Is what `makeinfo' produces for them for Info
hard-coded, or can it be modified in some ways?

Or even if hard-coded, is it possible for the
resulting Info text to be somehow distinguished
as such - e.g., as we get '...' in the Info text,
resulting from some Texinfo @ construct?  (I
don't have a Texinfo or `makeinfo' manual.)

I don't see any way to recognize this in the Info
output we currently get, but is there perhaps
some way to get some indication there, e.g., to
define what Info output `makeinfo' produces for
something like @example or @lisp?

> > If there is, could that info be maintained - transferred to Info buffers in
> some way, or does the translation just have to be lossy in this way?
> 
> The conversion to Info is performed by 'makeinfo', a program that is
> part of the Texinfo package.  Its output must be mostly plain text,
> and the format of that file must be understood by the various Info
> readers out there.  So if some kind of "code" marker would be retained
> in the Info output, it would need to be implemented in the 2 or 3
> other Info readers out there, not just in Emacs.  The chances of such
> a change in the Info output format, just for the sake of some fancy
> display in a single Emacs-related package sound small -- but you need
> to talk about this with the Texinfo developers, not with us.

Too bad the translation has to be lossy.  The
effect of showing <code>-like stuff is helpful
(and is common in technical doc).

And something will be needed if we ever hope to
allow for variable-pitch text for most regular
text while using fixed-pitch for places it makes
sense (e.g. code).

With the `...' quoting there's no problem
recognizing (most) code (and key) bits embedded
in ordinary text, and using a fixed-pitch font
on the former.

I don't suppose it's feasible to have another
such construct to indicate the equivalent of
<code>...</code>?  We already have some Texinfo
markup or whatever that gets converted to '...'.
Is there some existing Texinfo construct that
could be leveraged for @lisp etc.?

See attached screenshots, for an idea of the
usefulness.  They're the same node - the only
difference is toggling minor mode
`Info-variable-pitch-text-mode'.

[-- Attachment #2: throw-emacs-Info-code-block.png --]
[-- Type: image/png, Size: 95468 bytes --]

[-- Attachment #3: throw-emacs-Info-var-pitch+code-block.png --]
[-- Type: image/png, Size: 95836 bytes --]

Re: [External] : Re: Indentation conventions for Info manuals; recognizing code

[Yuri Khan]

On Mon, 8 Mar 2021 at 10:49, Drew Adams <drew.adams@oracle.com> wrote:

> Too bad the translation has to be lossy.  The
> effect of showing <code>-like stuff is helpful
> (and is common in technical doc).
>
> And something will be needed if we ever hope to
> allow for variable-pitch text for most regular
> text while using fixed-pitch for places it makes
> sense (e.g. code).

You *want* nice variable-pitch text in Info, so you *ask for* a way to
distinguish code from body text in the Info output format, but what
you really *need* is the HTML output and a convenient browser for
that.

Re: [External] : Re: Indentation conventions for Info manuals; recognizing code

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Mon, 8 Mar 2021 12:49:32 +0700
> Cc: Eli Zaretskii <eliz@gnu.org>, "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> 
> You *want* nice variable-pitch text in Info, so you *ask for* a way to
> distinguish code from body text in the Info output format, but what
> you really *need* is the HTML output and a convenient browser for
> that.

Yes, I believe the right direction to having a better styled display
of manuals is to begin with the HTML output.  The Texinfo developers
are actually moving in that direction: to make the HTML output more
functional, so that it doesn't miss the important advantages of Info.

RE: [External] : Re: Indentation conventions for Info manuals; recognizing code

[Drew Adams]

> > Too bad the translation has to be lossy.  The
> > effect of showing <code>-like stuff is helpful
> > (and is common in technical doc).
> >
> > And something will be needed if we ever hope to
> > allow for variable-pitch text for most regular
> > text while using fixed-pitch for places it makes
> > sense (e.g. code).
> 
> You *want* nice variable-pitch text in Info, so you *ask for* a way to
> distinguish code from body text in the Info output format, but what
> you really *need* is the HTML output and a convenient browser for
> that.

Actually, I do *want* (in addition to Info in Emacs)
HTML manuals - and we already have that.

We *need*, I think, an ability to (optionally) use
variable-pitch for most text in Info (in Emacs) and
fixed-pitch for some text, such as code, examples
of user input/output, shell use, ASCII art etc.

All that I find (so far) missing for this is an
ability to recognize the latter, apart from `...',
<...>, function/macro/etc. header lines, etc.

Re: Indentation conventions for Info manuals; recognizing code

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The structure of an Info file is documented somewhere, but style
conventions are not.  The style conventions for Info files are defined
by what properly written Texinfo source converts into.

I did not write separate documentation for Info format because (1) it
would be more or less duplicative and (2) Info files are normally
generated from Texinfo files.

  > But other manuals don't seem to use the same indentation.  Several
  > (org, eintr, ccmode, efaq(-w32),...) indent code (including Elisp)
  > 5 spaces - which is the same amount that other, non-code text is
  > indented.

Aside from eintr, I don't know anything specific about those manuals.

The eintr manual was written by Bob Chassell; I presume it is still
generated from Texinfo, right?  So I find it puzzling that it produces
a different format in the Info file.  

Why does it do that?  Is the source formatted in an unusual way?  Does
it somehow specify different parameters for generating the Info file?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Indentation conventions for Info manuals; recognizing code

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Mon, 08 Mar 2021 00:37:21 -0500
> Cc: emacs-devel@gnu.org
> 
>   > But other manuals don't seem to use the same indentation.  Several
>   > (org, eintr, ccmode, efaq(-w32),...) indent code (including Elisp)
>   > 5 spaces - which is the same amount that other, non-code text is
>   > indented.
> 
> Aside from eintr, I don't know anything specific about those manuals.
> 
> The eintr manual was written by Bob Chassell; I presume it is still
> generated from Texinfo, right?  So I find it puzzling that it produces
> a different format in the Info file.  
> 
> Why does it do that?  Is the source formatted in an unusual way?  Does
> it somehow specify different parameters for generating the Info file?

Neither eintr nor any other manual do anything special with @code.
It's an illusion.  The actual indentation of a code snippet in the
Info output is determined not only by the immediate Texinfo source,
but also by the surrounding context.  The Introduction manual is
written as a book, so @code blocks there are inserted in regular text,
and the indentation of @code there is the original 5-space
indentation.  By contrast, most other manuals are written as reference
manuals, and thus are full of @table's, @defun's, etc., and the @code
blocks there are more often than not part of those environments.  So
the indentation of @code in those other manuals starts at the
indentation of the innermost environment and not at the left margin.
That's all.

RE: [External] : Re: Indentation conventions for Info manuals; recognizing code

[Drew Adams]

>   > But other manuals don't seem to use the same indentation.  Several
>   > (org, eintr, ccmode, efaq(-w32),...) indent code (including Elisp)
>   > 5 spaces - which is the same amount that other, non-code text is
>   > indented.
> 
> Aside from eintr, I don't know anything specific about those manuals.
> 
> The eintr manual was written by Bob Chassell; I presume it is still
> generated from Texinfo, right?  So I find it puzzling that it produces
> a different format in the Info file.
> 
> Why does it do that?  Is the source formatted in an unusual way?  Does
> it somehow specify different parameters for generating the Info file?

Sorry, this was my misunderstanding, from a too-cursory
look at the Elisp manual.

All manuals (including Elisp) have varying indentation,
for different things.  It's not possible, in any general
way, to rely on indentation level to determine whether
some content is code-like, i.e., deserves a fixed-pitch font.

Re: Indentation conventions for Info manuals; recognizing code

[tydrdn]

> So if some kind of "code" marker would be retained
> in the Info output, it would need to be implemented in the 2 or 3
> other Info readers out there, not just in Emacs.

makeinfo could have a new option to add separators for the generated
code examples, something like:

--- example begin

....

--- example end


This is just text, so others info readers could display it as is,
while emacs could use the separators to recognize code sections.

RE: [External] : Re: Indentation conventions for Info manuals; recognizing code

[Drew Adams]

> > So if some kind of "code" marker would be retained
> > in the Info output, it would need to be implemented in the 2 or 3
> > other Info readers out there, not just in Emacs.
> 
> makeinfo could have a new option to add separators for the generated
> code examples, something like:
> 
> --- example begin
> ....
> --- example end
> 
> This is just text, so others info readers could display it as is,
> while emacs could use the separators to recognize code sections.

Yes, just the kind of thing I was suggesting
(hoping for).

Emacs handling of that can hide that "markup" and
DTRT wrt the font type.

There are likely several kinds of text "elements"
that might be candidates for this.  Eli mentioned
Texinfo @example, @smallexample, @lisp, and
@smallisp.

Given that (different purposes/semantics) ideally
we would have more than just one resulting kind of
"markup" as well, i.e., not only -- example begin/end.

RE: [External] : Re: Indentation conventions for Info manuals; recognizing code

[scame]

On Monday, March 8, 2021 4:33 PM, Drew Adams <drew.adams@oracle.com> wrote:

> > > So if some kind of "code" marker would be retained
> > > in the Info output, it would need to be implemented in the 2 or 3
> > > other Info readers out there, not just in Emacs.
> >
> > makeinfo could have a new option to add separators for the generated
> > code examples, something like:
> > --- example begin
> > ....
> > --- example end
> > This is just text, so others info readers could display it as is,
> > while emacs could use the separators to recognize code sections.
>
> Yes, just the kind of thing I was suggesting
> (hoping for).
>


It occurred to me makeinfo doesn't need a new option at all if
a preprocessing step is added to info generation.

For example this text from the current info:

@smallexample
;; Avoid this pattern.
(let ((beg ...) (end ...) (my-completions (my-make-completions)))
  (list beg end my-completions))
;; Use this instead.
(let ((beg ...) (end ...))
  (list beg
        end
        (completion-table-dynamic
          (lambda (_)
            (my-make-completions)))))
@end smallexample


could be converted automatically by an emacs build script before
info generation to:


--- example begin -----------------------

@smallexample
;; Avoid this pattern.
(let ((beg ...) (end ...) (my-completions (my-make-completions)))
  (list beg end my-completions))
;; Use this instead.
(let ((beg ...) (end ...))
  (list beg
        end
        (completion-table-dynamic
          (lambda (_)
            (my-make-completions)))))
@end smallexample

--- example end -----------------------


so that makeinfo converts the example as usual and leave the
added section markers in place for the final output, so that
emacs can use it to detect example sections to font lock them
and hide the markers.

And other info readers can simply display the text as is, showing
the added section separators around the example.