Colorizing source code in Info manuals

Colorizing source code in Info manuals

[Cecilio Pardo]

Hello,

As texinfo is not going to add features like color to the Info format,
I think maybe emacs could add some itself.

For example, the Emacs Lisp Intro uses @smallexample blocks for
presenting lisp code. I modified makeinfo more or less successfully to
generate a file with the line numbers where these blocks start. With
this kind of information, emacs could fontify the source code.

Does this sound good?

Re: Colorizing source code in Info manuals

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 799 bytes --]

Cecilio Pardo <cpardo@imayhem.com> writes:

> As texinfo is not going to add features like color to the Info format,

This could also be added in HTML Exports via Javascript-Libraries.

> I think maybe emacs could add some itself.
>
> For example, the Emacs Lisp Intro uses @smallexample blocks for
> presenting lisp code. I modified makeinfo more or less successfully to
> generate a file with the line numbers where these blocks start. With
> this kind of information, emacs could fontify the source code.
>
> Does this sound good?

That definitely sounds interesting!

Could it also detect source blocks in *info* on the fly and then
highlight the block like orgmode does it?

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Colorizing source code in Info manuals

[Arsen Arsenović]

[-- Attachment #1: Type: text/plain, Size: 1139 bytes --]

"Dr. Arne Babenhauserheide" <arne_bab@web.de> writes:

> Cecilio Pardo <cpardo@imayhem.com> writes:
>
>> As texinfo is not going to add features like color to the Info format,
>
> This could also be added in HTML Exports via Javascript-Libraries.

And without: (texinfo) Syntax Highlighting

>> I think maybe emacs could add some itself.
>>
>> For example, the Emacs Lisp Intro uses @smallexample blocks for
>> presenting lisp code. I modified makeinfo more or less successfully to
>> generate a file with the line numbers where these blocks start. With
>> this kind of information, emacs could fontify the source code.
>>
>> Does this sound good?
>
> That definitely sounds interesting!
>
> Could it also detect source blocks in *info* on the fly and then
> highlight the block like orgmode does it?

Instead of guesswork, the info format should be extended to contain the
language ID @example can take as an argument.

Really, the entire info format can be significantly better.  It is
currently a marked-up catfile.

IMO, it should be replaced with something that has a proper DOM.
-- 
Arsen Arsenović

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 381 bytes --]

Re: Colorizing source code in Info manuals

[Eli Zaretskii]

> Date: Thu, 5 Dec 2024 23:06:37 +0100
> From: Cecilio Pardo <cpardo@imayhem.com>
> 
> As texinfo is not going to add features like color to the Info format,
> I think maybe emacs could add some itself.

I'm not sure "is not going to add features like color to the Info
format" is correct.  Texinfo does support syntax-highlighting of code
snippets in HTML output (for now as an experimental feature), so it is
not outlandish to assume something similar can be done for the Info
format as well.

> For example, the Emacs Lisp Intro uses @smallexample blocks for
> presenting lisp code. I modified makeinfo more or less successfully to
> generate a file with the line numbers where these blocks start. With
> this kind of information, emacs could fontify the source code.
> 
> Does this sound good?

I think this should be discussed with the Texinfo developers, not
here.  I can think about at least two possible ways of implementing
this in Texinfo:

 . makeinfo could mark the code blocks with special markers, similar
   to what it already does with image specs and indices
 . makeinfo could produce ANSI color escape sequences in the Info
   output, driven by the same syntax-highlighting machinery already
   used for HTML

Re: Colorizing source code in Info manuals

[Cecilio Pardo]

On 05/12/2024 23:33, Dr. Arne Babenhauserheide wrote:
> Could it also detect source blocks in *info* on the fly and then
> highlight the block like orgmode does it?

Org mode has markup to identify the code sections. Info does not, so we 
need some more info.

Re: Colorizing source code in Info manuals

[Cecilio Pardo]

On 06/12/2024 8:38, Eli Zaretskii wrote:

> I'm not sure "is not going to add features like color to the Info
> format" is correct.  Texinfo does support syntax-highlighting of code
> snippets in HTML output (for now as an experimental feature), so it is
> not outlandish to assume something similar can be done for the Info
> format as well.

It says so in the texinfo manual:

      Can the Info format be extended to support fonts, colors or
      reflowable text?

      [...]

      Info's core purpose is to display documentation on text terminals.
      If you want more, you are recommended to use the HTML output from
      ‘texi2any’ instead.

Maybe they can add a 'subformat'.

> I think this should be discussed with the Texinfo developers, not
> here.  I can think about at least two possible ways of implementing
> this in Texinfo:
> 
>   . makeinfo could mark the code blocks with special markers, similar
>     to what it already does with image specs and indices
>   . makeinfo could produce ANSI color escape sequences in the Info
>     output, driven by the same syntax-highlighting machinery already
>     used for HTML

I'll take a look at what it does with images and take it to them. Will 
try to include a patch, but I can't really perl.

Re: Colorizing source code in Info manuals

[Eli Zaretskii]

> Date: Fri, 6 Dec 2024 09:52:41 +0100
> Cc: emacs-devel@gnu.org
> From: Cecilio Pardo <cpardo@imayhem.com>
> 
> On 06/12/2024 8:38, Eli Zaretskii wrote:
> 
> > I'm not sure "is not going to add features like color to the Info
> > format" is correct.  Texinfo does support syntax-highlighting of code
> > snippets in HTML output (for now as an experimental feature), so it is
> > not outlandish to assume something similar can be done for the Info
> > format as well.
> 
> It says so in the texinfo manual:
> 
>       Can the Info format be extended to support fonts, colors or
>       reflowable text?
> 
>       [...]
> 
>       Info's core purpose is to display documentation on text terminals.
>       If you want more, you are recommended to use the HTML output from
>       ‘texi2any’ instead.
> 
> Maybe they can add a 'subformat'.

The "^@^H[NAME CONTENTS ^@^H]" cookies are already in the Info format,
so I'm not necessarily talking about extensions.

> > I think this should be discussed with the Texinfo developers, not
> > here.  I can think about at least two possible ways of implementing
> > this in Texinfo:
> > 
> >   . makeinfo could mark the code blocks with special markers, similar
> >     to what it already does with image specs and indices
> >   . makeinfo could produce ANSI color escape sequences in the Info
> >     output, driven by the same syntax-highlighting machinery already
> >     used for HTML
> 
> I'll take a look at what it does with images and take it to them. Will 
> try to include a patch, but I can't really perl.

Thanks.

Re: Colorizing source code in Info manuals

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Info format lacks ways to express many sorts of stype information
that would be desirable to have on a graphics display.  I am not sure
if it makes sense tp try to extend Info forat, or use HTML instead,
or what?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Colorizing source code in Info manuals

[Cecilio Pardo]

On 08/12/2024 6:17, Richard Stallman wrote:

> Info format lacks ways to express many sorts of stype information
> that would be desirable to have on a graphics display.  I am not sure
> if it makes sense tp try to extend Info forat, or use HTML instead,
> or what?

Texinfo's HTML uses CSS, which I think emacs does not support.  Also, 
emacs' Info machinery would need some work to adapt it for HTML.

I think some 'cheap' changes in info, such as the presentation of source 
code and emacs key sequences, in my unqualified opinion, are worth the 
effort at this point.

Re: Colorizing source code in Info manuals

[Björn Bidar]

Cecilio Pardo <cpardo@imayhem.com> writes:

> On 08/12/2024 6:17, Richard Stallman wrote:
>
>> Info format lacks ways to express many sorts of stype information
>> that would be desirable to have on a graphics display.  I am not sure
>> if it makes sense tp try to extend Info forat, or use HTML instead,
>> or what?
>
> Texinfo's HTML uses CSS, which I think emacs does not support.  Also,
> emacs' Info machinery would need some work to adapt it for HTML.
>
> I think some 'cheap' changes in info, such as the presentation of
> source code and emacs key sequences, in my unqualified opinion, are
> worth the effort at this point.

There are examples to handle color information in this context e.g.
like org-mode to reuse existing styling documentation based on the
property of faces defined for other modes.

Another option would be to do something like the like the minted LaTeX
package. It should be a good example to transform the styling elements
into the right elements for the info readers or HTML.

Re: Colorizing source code in Info manuals

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

We need to keep Info and Makeinfo consisent.  To have multiple
versions of them in different GNU packages would be chaos.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Colorizing source code in Info manuals

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Texinfo's HTML uses CSS, which I think emacs does not support.

Indeeed, handling CSS would be a lot of additional work.

Does HTML itself have any extensibility, any way to define (in effect)
macros, so that `<dfn>...</dfn>' could translate into `*<i>...</i>*'?

                                                                    Also, 
  > emacs' Info machinery would need some work to adapt it for HTML.

If that approach is basically workable, I am sure we could do this
programming job.  What I can't tell is whether that approach is
workable at all.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Colorizing source code in Info manuals

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I think some 'cheap' changes in info, such as the presentation of source 
  > code and emacs key sequences, in my unqualified opinion, are worth the 
  > effort at this point.

Would you please say specifically what extensions you're suggesting?
How would one use them?  What jobs would they do?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Colorizing source code in Info manuals

[Cecilio Pardo]

On 11/12/2024 5:33, Richard Stallman wrote:

>> Texinfo's HTML uses CSS, which I think emacs does not support.
> 
> Indeeed, handling CSS would be a lot of additional work.
> 
> Does HTML itself have any extensibility, any way to define (in effect)
> macros, so that `<dfn>...</dfn>' could translate into `*<i>...</i>*'?

No. We could use javascript for the browsers, and preprocess with lisp 
for emacs if we go that way, but it doesn't sound natural.

Re: Colorizing source code in Info manuals

[Cecilio Pardo]

On 11/12/2024 5:33, Richard Stallman wrote:

>    > I think some 'cheap' changes in info, such as the presentation of source
>    > code and emacs key sequences, in my unqualified opinion, are worth the
>    > effort at this point.
> 
> Would you please say specifically what extensions you're suggesting?
> How would one use them?  What jobs would they do?


I suggest modifying makinfo to generate markers around certain text in 
its Info output, so that we can find them (with lisp) and change text 
properties.

Those markers can be implemented using the construct that is used for 
images and indices, as Eli suggested.
They are ignored by the standalone Info viewer when they are not 
recognized, so it will not be affected.

We would put them around some commands, like:

@kbd @key @code @example

For @code, @kbd and @key, Emacs Info reader keeps the standard font, and 
ads quotes around it, making them in my opinion even harder to read.

If you look at "A note for novices" on the Emacs lisp intro, which show 
many keys in the text, I think it its obvious that readability can be 
improved just by applying a different face to them.

This is the starting line from 1.1 Lisp lists, on the Lisp intro:

   In Lisp, a list looks like this: ‘'(rose violet daisy buttercup)’.

Would much easier without the quotes, and with a different face.

And for the @example command, we can colorize the code according to the 
language. Also add some tricks like a button to copy the example.

RE: [External] : Re: Colorizing source code in Info manuals

[Drew Adams]

[-- Attachment #1: Type: text/plain, Size: 2023 bytes --]

> > > I think some 'cheap' changes in info, such as the presentation of
> > > source code and emacs key sequences, in my unqualified opinion,
> > > are worth the effort at this point.
> >
> > Would you please say specifically what extensions you're suggesting?
> > How would one use them?  What jobs would they do?
> 
> I suggest modifying makinfo to generate markers around certain text in
> its Info output, so that we can find them (with lisp) and change text
> properties.
> 
> Those markers can be implemented using the construct that is used for
> images and indices, as Eli suggested.
> They are ignored by the standalone Info viewer when they are not
> recognized, so it will not be affected.

I'd welcome some such way to detect things in displayed
Info manuals.  A priori, it doesn't matter to me what's
used, (text properties, markers,...), as long as it works.

Picking up Lisp code examples is currently problematic.

I try doing that as an optional behavior in info+.el
(option `Info-fontify-indented-text-chars'), but because
indented contexts are different/various it's flaky.

Having code stand out, in a fixed-width font, is
especially useful if the main font for regular text is
variable-width.

> If you look at "A note for novices" on the Emacs lisp intro, which show
> many keys in the text, I think it its obvious that readability can be
> improved just by applying a different face to them.

FWIW, info+.el does that with no problem.  See attached.
(The "sunken" face is used for links to glossary terms,
so the word "region" in `indent-region' gets that face
instead of the face for code/keys.  Putting a face and
link on such glossary-term first occurrences is optional.)

Yes, fontifying such code/key occurrences uses hairy
regexps that look for different kinds of quotes and angle
brackets etc.  If Info included actual markup - whether
visible before final rendering or with text properties or
different kinds of markers, that would be preferable.

[-- Attachment #2: throw-info+-key-highlighting.png --]
[-- Type: image/png, Size: 98761 bytes --]

Re: Colorizing source code in Info manuals

[chad]

[-- Attachment #1: Type: text/plain, Size: 1492 bytes --]

On Wed, Dec 11, 2024 at 5:41 AM Cecilio Pardo <cpardo@imayhem.com> wrote:

> On 11/12/2024 5:33, Richard Stallman wrote:
>
> >> Texinfo's HTML uses CSS, which I think emacs does not support.
> >
> > Indeeed, handling CSS would be a lot of additional work.
> >
> > Does HTML itself have any extensibility, any way to define (in effect)
> > macros, so that `<dfn>...</dfn>' could translate into `*<i>...</i>*'?
>
> No. We could use javascript for the browsers, and preprocess with lisp
> for emacs if we go that way, but it doesn't sound natural.
>

This is not quite correct; HTML has DIV and SPAN elements that can
attach arbitrary ID's to sections or spans of text in an html
document. This is _usually_ used for styling, via CSS, but it would be
easy to include CSS in the HTML output that makes <span
id="definition">code here</span> or <span className="definition">code
here</span> style the text in italics. This usage is not currently
common, but it was part of the original intended usage way-back-when,
and I don't know why it wouldn't still work.

It would also be pretty easy to add support for such "custom tags" to
eww's HTML renderer (shr.el, which seems to be not thoroughly
documented, at least in my current install). I admit that I'm not sure
why one might want to look at info manuals in HTML format inside
emacs, but if it's wanted, it shouldn't be difficult.

Apologies if I've added noise due to missing something in the thread.

~Chad

[-- Attachment #2: Type: text/html, Size: 2101 bytes --]

Re: Colorizing source code in Info manuals

[Cecilio Pardo]

[-- Attachment #1: Type: text/plain, Size: 660 bytes --]

With a modification in Texinfo to add the location of @example commands 
to the "Tags Table" of Info files, and some lisp to search that and 
propertize text, I'm getting the results in the attached images (with 
and without the code colors).

To take advantage of this, Texinfo files need to indicate the language 
used on the example so we know how to fontify.  Also, uses of 
@smallexample should change to @example.  The Lisp Intro uses 
@smallexample a lot. If no language is specified, we still get the gray 
box around the code.

Now I need to take this change (which will take some work) to the 
Texinfo developers, if it looks worth the effort.







[-- Attachment #2: emacs_6vCjiX38X6.png --]
[-- Type: image/png, Size: 96433 bytes --]

[-- Attachment #3: emacs_cs4rqmLikg.png --]
[-- Type: image/png, Size: 93438 bytes --]

Re: Re: Colorizing source code in Info manuals

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The feature is good, but some of those colors are too light in color
to be easy to see.  For instance, the color used for comments
and the color used for `defun' are too light, so that text in the PDF is hard
to read on the screen.

I suggest playing with different default colors for use in the Texinfo
PDFs.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Colorizing source code in Info manuals

[Cecilio Pardo]

On 26/12/2024 5:30, Richard Stallman wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
> The feature is good, but some of those colors are too light in color
> to be easy to see.  For instance, the color used for comments
> and the color used for `defun' are too light, so that text in the PDF is hard
> to read on the screen.
> 
> I suggest playing with different default colors for use in the Texinfo
> PDFs.

This works on emacs' Info viewer, not on the PDF output, and uses the 
same colors as font-lock do.

Re: Colorizing source code in Info manuals

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This works on emacs' Info viewer, not on the PDF output, and uses the 
  > same colors as font-lock do.

I understand now.  Thanks.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)