[BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Xiyue Deng]

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.  You don't know how to make a good report?  See

     https://orgmode.org/manual/Feedback.html#Feedback

Your bug report will be posted to the Org mailing list.
------------------------------------------------------------------------

(This was first reported to Emacs at
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=69483)

"mu4e"[1] (a popular Emacs mail client) uses Org to generate its
manpages.  However, the generated output contains macros that are not
understood by groff.  After some debugging, Jeremy traced this back to
the macro "\fC" used in ox-man.el[2].  Git history shows that this may
have been there since the beginning.  We tried to find a documentation
for the "\fC" macro but has not been able to find one.  Jeremy suggests
that "C" may be an old alias for Courier, and if that's the case it
should be changed to "\f[CR]".  Would be great if Org people can
confirm.

Emacs  : GNU Emacs 29.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, cairo version 1.16.0)
 of 2024-01-20, modified by Debian
Package: Org mode version 9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

Xiyue Deng <manphiz@gmail.com> writes:

> (This was first reported to Emacs at
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=69483)
>
> "mu4e"[1] (a popular Emacs mail client) uses Org to generate its
> manpages.  However, the generated output contains macros that are not
> understood by groff.  After some debugging, Jeremy traced this back to
> the macro "\fC" used in ox-man.el[2].  Git history shows that this may
> have been there since the beginning.  We tried to find a documentation
> for the "\fC" macro but has not been able to find one.  Jeremy suggests
> that "C" may be an old alias for Courier, and if that's the case it
> should be changed to "\f[CR]".  Would be great if Org people can
> confirm.

This is not an unknown problem. AFAIU, the \fC macro is widely used for
troff, although it is not supported by groff. Check out the ongoing
discussion at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1049968#15

They suggest the following instead of \fC:

   The best solution known to me is to use an extension to the man(7)
   language.  It first appeared in Ninth Edition Unix (1986) and was
   adopted by a groff release in 2009.  That is the `EX`/`EE` macro pair,
   which sets a monospaced display.  (In other words, filling is disabled
   and a monospaced font selected if necessary.)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Xiyue Deng]

Hi Ihor,

Ihor Radchenko <yantar92@posteo.net> writes:

> Xiyue Deng <manphiz@gmail.com> writes:
>
>> (This was first reported to Emacs at
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=69483)
>>
>> "mu4e"[1] (a popular Emacs mail client) uses Org to generate its
>> manpages.  However, the generated output contains macros that are not
>> understood by groff.  After some debugging, Jeremy traced this back to
>> the macro "\fC" used in ox-man.el[2].  Git history shows that this may
>> have been there since the beginning.  We tried to find a documentation
>> for the "\fC" macro but has not been able to find one.  Jeremy suggests
>> that "C" may be an old alias for Courier, and if that's the case it
>> should be changed to "\f[CR]".  Would be great if Org people can
>> confirm.
>
> This is not an unknown problem. AFAIU, the \fC macro is widely used for
> troff, although it is not supported by groff. Check out the ongoing
> discussion at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1049968#15
>
> They suggest the following instead of \fC:
>
>    The best solution known to me is to use an extension to the man(7)
>    language.  It first appeared in Ninth Edition Unix (1986) and was
>    adopted by a groff release in 2009.  That is the `EX`/`EE` macro pair,
>    which sets a monospaced display.  (In other words, filling is disabled
>    and a monospaced font selected if necessary.)

I'm not very familiar with roff so my understanding may be off.
According to the `Safe subset' section in man(7), they mentioned the
following:

,----
| Font changes (ft and the \f escape sequence) should only have the
| values 1, 2, 3, 4, R, I, B, P, or CW (the ft command may also have no
| parameters).
`----

Does it mean `\fC' should be replaced by `\f[CW]'?

Also CCing Jeremy who may have a better idea on how this should be
handled.

-- 
Xiyue Deng

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

Xiyue Deng <manphiz@gmail.com> writes:

> I'm not very familiar with roff so my understanding may be off.
> According to the `Safe subset' section in man(7), they mentioned the
> following:
>
> ,----
> | Font changes (ft and the \f escape sequence) should only have the
> | values 1, 2, 3, 4, R, I, B, P, or CW (the ft command may also have no
> | parameters).
> `----
>
> Does it mean `\fC' should be replaced by `\f[CW]'?

I am not sure

man 7 groff has

      Fonts often have trademarked names, and even Free Software fonts
      can require renaming upon modification. groff maintains a
      convention that a de‐ vice’s serif font family is given the name
      T (“Times”), its sans-serif family H (“Helvetica”), and its
      monospaced family C (“Courier”). Histori‐ cal inertia has driven
      groff’s font identifiers to short uppercase abbreviations of font
      names, as with TR, TB, TI, TBI, and a special font S.

So, \fC refers to "Courier".

I did not find any text description of CW font, but my groff
installation has usr/share/groff/1.23.0/font/devdvi/CW font spec:

    name CW
    special
    ...

for comparison, /usr/share/groff/1.23.0/font/devps/CR has

    name CR
    internalname Courier
    ...

which looks more suitable. But CR is not listed in "safe" subset
(man 7 man)

Also, neither CW nor CR work with html output:

with \fC

    .TH "" "1" 
    .PP
    \fBThis is test\fP 
    \fCcode a+b\fP here a+b.

yields (groff -Thtml test.man)

<p><b>This is test</b> <tt>code a+b</tt> here a+b.</p>

Note <tt> tag.

but with \f[CW]

    .TH "" "1" 
    .PP
    \fBThis is test\fP 
    \f[CW]code a+b\fP here a+b.

<p><b>This is test</b> code a+b here a+b.</p>

No special markup is applied to the code.

Same for \f[CR].

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Jeremy Sowden]

[-- Attachment #1: Type: text/plain, Size: 4864 bytes --]

On 2024-03-13, at 11:25:23 +0000, Ihor Radchenko wrote:
> On 2024-03-11, at 17:06:17 -0700, Xiyue Deng wrote:
> > Ihor Radchenko writes:
> > > Xiyue Deng writes:
> > > > (This was first reported to Emacs at
> > > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=69483)
> > > >
> > > > "mu4e"[1] (a popular Emacs mail client) uses Org to generate its
> > > > manpages.  However, the generated output contains macros that
> > > > are not understood by groff.  After some debugging, Jeremy
> > > > traced this back to the macro "\fC" used in ox-man.el[2].  Git
> > > > history shows that this may have been there since the beginning.
> > > > We tried to find a documentation for the "\fC" macro but has not
> > > > been able to find one.  Jeremy suggests that "C" may be an old
> > > > alias for Courier, and if that's the case it should be changed
> > > > to "\f[CR]".  Would be great if Org people can confirm.
> > >
> > > This is not an unknown problem. AFAIU, the \fC macro is widely
> > > used for troff, although it is not supported by groff. Check out
> > > the ongoing discussion at
> > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1049968#15
> > >
> > > They suggest the following instead of \fC:
> > >
> > >    The best solution known to me is to use an extension to the
> > >    man(7) language.  It first appeared in Ninth Edition Unix
> > >    (1986) and was adopted by a groff release in 2009.  That is the
> > >    `EX`/`EE` macro pair, which sets a monospaced display.  (In
> > >    other words, filling is disabled and a monospaced font selected
> > >    if necessary.)

Thanks for the pointer.  Mildly embarrassed that I didn't find this when
I was doing the initial triage in Debian. :)

> > I'm not very familiar with roff so my understanding may be off.
> > According to the `Safe subset' section in man(7), they mentioned the
> > following:
> >
> > ,----
> > | Font changes (ft and the \f escape sequence) should only have the
> > | values 1, 2, 3, 4, R, I, B, P, or CW (the ft command may also have no
> > | parameters).
> > `----
> >
> > Does it mean `\fC' should be replaced by `\f[CW]'?
>
> I am not sure
>
> man 7 groff has
>
>       Fonts often have trademarked names, and even Free Software fonts
>       can require renaming upon modification. groff maintains a
>       convention that a device’s serif font family is given the name T
>       (“Times”), its sans-serif family H (“Helvetica”), and its
>       monospaced family C (“Courier”). Historical inertia has driven
>       groff’s font identifiers to short uppercase abbreviations of
>       font names, as with TR, TB, TI, TBI, and a special font S.
>
> So, \fC refers to "Courier".
>
> I did not find any text description of CW font, but my groff
> installation has usr/share/groff/1.23.0/font/devdvi/CW font spec:
>
>     name CW
>     special
>     ...
>
> for comparison, /usr/share/groff/1.23.0/font/devps/CR has
>
>     name CR
>     internalname Courier
>     ...
>
> which looks more suitable. But CR is not listed in "safe" subset (man
> 7 man)
>
> Also, neither CW nor CR work with html output:
>
> with \fC
>
>     .TH "" "1"
>     .PP
>     \fBThis is test\fP
>     \fCcode a+b\fP here a+b.
>
> yields (groff -Thtml test.man)
>
> <p><b>This is test</b> <tt>code a+b</tt> here a+b.</p>
>
> Note <tt> tag.
>
> but with \f[CW]
>
>     .TH "" "1"
>     .PP
>     \fBThis is test\fP
>     \f[CW]code a+b\fP here a+b.
>
> <p><b>This is test</b> code a+b here a+b.</p>
>
> No special markup is applied to the code.
>
> Same for \f[CR].
>
> > Also CCing Jeremy who may have a better idea on how this should be
> > handled.

What I did for the mu4e man-pages was to patch them to alias font C to B:

    .ftr C B

My initial assumption when I first looked into this is that the font to
use would be `CR`, not `CW`.  Doing this with `CR` does seem to work:

    $ cat /space/azazel/tmp/test.man 
    .ftr C CR

    .TH "" "1"
    .PP
    \fBThis is test\fP
    \fCcode a+b\fP here a+b.
    $ groff -Thtml /space/azazel/tmp/test.man | tail -5 | head -2
    <p style="margin-top: 1em"><b>This is test</b> <tt>code
    a+b</tt> here a+b.</p>

However, as you observe, `\f[CR]` doesn't (nor does `\f(CR`).  I note
that groff's HTML support is stated in the grohtml(1) man-page to be in
beta.  Haven't checked the source to determine whether that is what's
going on here.

In any case, my understanding from reading the conversation in the
Debian bug-report is that this issue affects multiple roff generators in
Debian.  Therefore, it probably makes sense to consult within Debian
before asking the maintainers of those generators to make changes.  I
need to go over that conversation again and think about this more.

J.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

Jeremy Sowden <azazel@debian.org> writes:

> In any case, my understanding from reading the conversation in the
> Debian bug-report is that this issue affects multiple roff generators in
> Debian.  Therefore, it probably makes sense to consult within Debian
> before asking the maintainers of those generators to make changes.  I
> need to go over that conversation again and think about this more.

It looks like the final fix they arrived to is
https://github.com/cpuguy83/go-md2man/commit/d85280db9b54b5747fe8cf9380a9f278e1a352d3
- use \fB and drop \fC completely.
For Org mode, it would imply rendering things like fixed-width text in
bold... I doubt that it is a good idea at all.

That said, I still have not the slightest clue what we can do to get rid
of the warning.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 12791 bytes --]

[looping in groff list; please reply to it as well, as I am not
subscribed to emacs-orgmode]

Hi Xiyue, Ihor, and Jeremy,

I stumbled across this thread while researching problems have
encountered with groff, which I maintain for the GNU Project.

At 2024-02-29T23:51:29-0800, Xiyue Deng wrote:
> "mu4e"[1] (a popular Emacs mail client) uses Org to generate its
> manpages.  However, the generated output contains macros that are not
> understood by groff.  After some debugging, Jeremy traced this back to
> the macro "\fC" used in ox-man.el[2].

Strictly speaking, that's an escape sequence, not a macro.

> Git history shows that this may have been there since the beginning.
> We tried to find a documentation for the "\fC" macro but has not been
> able to find one.

Here's some historical background.  Some may want to skip it.

--- begin background ---

`C` is a non-portable font name.  Few font identifiers _are_ portable
across the history of troff.  When Brian Kernighan refactored Joe
Ossanna's original troff for device-independence circa 1980, digital
fonts were in their infancy and not generally portable across hardware
devices as they are now.  The original device target for Unix troff, the
Graphic Systems, Inc. (later acquired by Wang Laboratories, Inc.) C/A/T
machine, did not have digital fonts.  It was a phototypesetter: its
fonts were glyphs etched on glass plates mounted in a rotating
mechanical carousel and magnified with an optical lens to the desired
type size, then flashed to photosensitive paper.

Kernighan perhaps did not foresee that people targeting multiple output
device from a single master document might be able to avail themselves
of the same typefaces, or at least metrically compatible ones, across
devices, and so made no provision for font aliasing or renaming.  (You
could wangle it, perhaps, by restricting oneself to numerical "mounting
positions", but this seems to have been a seldom-taken recourse in
surviving *roff documents of the 1980s that have come to my attention.)

--- end background --

> Jeremy suggests that "C" may be an old alias for Courier, and if
> that's the case it should be changed to "\f[CR]".  Would be great if
> Org people can confirm.

That is good advice and it is what I recommend if you're writing in
"raw" roff.  The context of the discussion is not ultra-clear to me; is
ox-man.el a replacement for the old GNU Emacs man pager, "woman"?

If what you're dealing with is man pages, I (and mandoc(1) maintainer
Ingo Schwarze) discourage the use of formatter features for font
selection.  Use the man(7) package's macros instead.  I'll return to
this point below.

At 2024-03-03T13:30:59+0000, Ihor Radchenko wrote:
> This is not an unknown problem. AFAIU, the \fC macro is widely used
> for troff, although it is not supported by groff. Check out the
> ongoing discussion at
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1049968#15

It _is_ supported, however there are some confounding factors.

Historically, groff has quietly remapped the font name "C" to "CR"
(Courier roman, see below).  However this was done only for the "ps"
(PostScript) output device (and PDF via file inclusion).

https://git.savannah.gnu.org/cgit/groff.git/tree/tmac/ps.tmac?h=1.23.0#n8

This means that the font name did not exist, and was not supported, on
terminal devices, which is what I suppose ox-man.el is using to stage
the rendered man page before doing whatever it does to make it look nice
in an Emacs window.  So, when you try to switch to font "C" or "CR" on a
terminal device, nothing happens.

Here comes the next complication.  Historically, groff also did not
throw warnings when a nonexistent font was selected.  In groff 1.23.0,
it started emitting these warnings.  People mistook this for withdrawal
of "support" for font names "C" and "CW" (and occasionally others).
What's actually happening is that, when rendering for a device other
than "ps" or "pdf", they are being warned of a no-op that they weren't
warned about before.  Nothing about the rendering of the document
actually changed.

https://lists.gnu.org/archive/html/groff-commit/2022-06/msg00114.html

This experience convinced me that these legacy font names are more
trouble than they are worth, so for groff 1.24 (for which, now that I
have a fencepost account, I hope to make a release candidate available
soon), we're starting a deprecation cycle for them.

https://git.savannah.gnu.org/cgit/groff.git/tree/NEWS?id=db49abe9ee9035f248f0893b4a1beb4ce3db8e38#n245

>>   The best solution known to me is to use an extension to the man(7)
>>   language.  It first appeared in Ninth Edition Unix (1986) and was
>>   adopted by a groff release in 2009.  That is the `EX`/`EE` macro
>>   pair, which sets a monospaced display.  (In other words, filling is
>>   disabled and a monospaced font selected if necessary.)

Yes.

At 2024-03-11T17:06:17-0700, Xiyue Deng wrote:
> I'm not very familiar with roff so my understanding may be off.
> According to the `Safe subset' section in man(7), they mentioned the
> following:
> 
> ,----
> | Font changes (ft and the \f escape sequence) should only have the
> | values 1, 2, 3, 4, R, I, B, P, or CW (the ft command may also have
> | no parameters).
> `----
> 
> Does it mean `\fC' should be replaced by `\f[CW]'?

No.  The Linux man-pages version of man(7) is going to be (or already
has been, in a recent release) withdrawn in favor of groff_man(7), which
along with a new page for groff 1.23.0, groff_man_style(7), has been
intensely worked on for the past seven years to make man page
composition (and, to an extent, reading) a less mysterious process.

https://lore.kernel.org/linux-man/7f7f2644-d408-969b-6916-ee9cae0962b9@kernel.org/

At 2024-03-13T11:25:23+0000, Ihor Radchenko wrote:
> man 7 groff has
> 
>       Fonts often have trademarked names, and even Free Software fonts
>       can require renaming upon modification. groff maintains a
>       convention that a de‐ vice’s serif font family is given the name
>       T (“Times”), its sans-serif family H (“Helvetica”), and its
>       monospaced family C (“Courier”). Histori‐ cal inertia has driven
>       groff’s font identifiers to short uppercase abbreviations of
>       font names, as with TR, TB, TI, TBI, and a special font S.
> 
> So, \fC refers to "Courier".

Yes, but.  The "C" here, in groff, refers to a font _family_, not a
fully resolved font name.  In groff, following the precedent of Adobe
PostScript, four styles of Courier are available: Courier roman, Courier
italic, Courier bold, and Courier bold-italic: CR, CB, CI, and CBI.

https://www.gnu.org/software/groff/manual/groff.html.node/Using-Fonts.html#Using-Fonts

Because of the aforementioned aliasing, you could "get away with" saying
`\fC`, because, when rendering for "ps" or "pdf", `C` would be remapped
to `CR` for you, but that was no help for terminal output devices.  They
never were able to select such a font, and in groff 1.23.0, the
formatter started warning about this.

> I did not find any text description of CW font, but my groff
> installation has usr/share/groff/1.23.0/font/devdvi/CW font spec:
> 
>     name CW
>     special
>     ...

Yes.  This is a tangentially related issue.  As grodvi(1) says:

   Typefaces
     grodvi supports the standard four styles: R (roman), I (italic), B
     (bold), and BI (bold‐italic).  Fonts are grouped into families T
     and H having members in each style.  “CM” abbreviates “Computer
     Modern”.
...
     The following fonts are not members of a family.

            CW     CM Typewriter Text (cmtt10)
            CWI    CM Italic Typewriter Text (cmitt10)

TeX DVI's Computer Modern Typewriter faces don't offer a full "family"
of four styles as its ordinary and sans serif faces do.

The choice of "CW" and "CWI" as abbreviations for these is unfortunate.
It apes a convention occasionally used in AT&T troff, specifically in
Unix System III (1980) where some special contrivances were made for
typesetting "constant-width" (thus "CW") faces.

Unix hackers are enamored of their terse identifiers, and never repent
of the confusing ambiguity they introduce in pursuit of them.  And if
such practice shrouds their development activity in mystique, so much
the better, it is thought.

If I were an even more aggressive reformer than I am, I'd revise all of
the font names used by grodvi to match those used by Knuth for them, but
in caps and chopping off the "10".  This would acquire the virtue of
_familiarity_ for experienced TeX users.  (I'd retain "T" and "H" family
remappings to maintain groff's attempt at typeface ecumenicism and
document portability, of course.  And people trying to select "CB" and
"CBI" would be warned as they should be, since those faces would be
unavailable.)

> which looks more suitable. But CR is not listed in "safe" subset
> (man 7 man)

The Linux man-pages man(7) document was stale or inaccurate in this
respect.  However, man pages generally should not attempt to access
specific fonts by name anyway.  They should use the macro facilities
afforded by the _man_ package to style their text.

groff_man_style(7):
   Portability
...
     The two major features that control formatting in the roff language
     are requests and escape sequences.  Since the man macros are
     implemented in terms of these, one can, in principle, supplement
     the functionality of man with these lower‐level elements where
     necessary.

     However, use of roff requests (apart from the empty request “.”)
     risks poor rendering when your page is processed by non‐roff
     formatters that attempt to interpret page sources.  (Historically,
     this was commonly attempted for HTML conversion.)  Requests may
     make assumptions that do not hold in an HTML environment.  Many of
     these programs don’t interpret the full roff language (let alone
     extensions): they may be incapable of handling numeric expressions,
     control structures, or register, string, and macro definitions.
     Such limitations can lead to portions of a document being presented
     incomprehensibly or omitted altogether.
...
     Exercise restraint with escape sequences as with requests.

> Also, neither CW nor CR work with html output:
> 
> with \fC
> 
>     .TH "" "1" 
>     .PP
>     \fBThis is test\fP 
>     \fCcode a+b\fP here a+b.
> 
> yields (groff -Thtml test.man)
> 
> <p><b>This is test</b> <tt>code a+b</tt> here a+b.</p>
> 
> Note <tt> tag.
>
> but with \f[CW]
> 
>     .TH "" "1" 
>     .PP
>     \fBThis is test\fP 
>     \f[CW]code a+b\fP here a+b.
> 
> <p><b>This is test</b> code a+b here a+b.</p>
> 
> No special markup is applied to the code.
> 
> Same for \f[CR].

The "html" output device has its own significant pile of problems and I
won't make this long mail even longer by going into them here.  I hope
to improve the situation in the coming years.

> What I did for the mu4e man-pages was to patch them to alias font C to
> B:
> 
>     .ftr C B

Can you share a link to the mu4e man pages for me?  I'd like to have a
look at them so I can make suggestions.  If you'd be open to them.

As a rule, I think doing the foregoing in a man page document should be
unnecessary.

> My initial assumption when I first looked into this is that the font
> to use would be `CR`, not `CW`.  Doing this with `CR` does seem to
> work:
> 
>     $ cat /space/azazel/tmp/test.man 
>     .ftr C CR
> 
>     .TH "" "1"
>     .PP
>     \fBThis is test\fP
>     \fCcode a+b\fP here a+b.
>     $ groff -Thtml /space/azazel/tmp/test.man | tail -5 | head -2
>     <p style="margin-top: 1em"><b>This is test</b> <tt>code
>     a+b</tt> here a+b.</p>
> 
> However, as you observe, `\f[CR]` doesn't (nor does `\f(CR`).  I note
> that groff's HTML support is stated in the grohtml(1) man-page to be
> in beta.  Haven't checked the source to determine whether that is
> what's going on here.

It's a mess. :(

https://savannah.gnu.org/bugs/index.php?61915

That's the tip of a large iceberg.

> In any case, my understanding from reading the conversation in the
> Debian bug-report is that this issue affects multiple roff generators
> in Debian.  Therefore, it probably makes sense to consult within
> Debian before asking the maintainers of those generators to make
> changes.  I need to go over that conversation again and think about
> this more.

Please consider me, and the groff mailing list, a resource.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

"G. Branden Robinson" <g.branden.robinson@gmail.com> writes:

>> Jeremy suggests that "C" may be an old alias for Courier, and if
>> that's the case it should be changed to "\f[CR]".  Would be great if
>> Org people can confirm.
>
> That is good advice and it is what I recommend if you're writing in
> "raw" roff.  The context of the discussion is not ultra-clear to me; is
> ox-man.el a replacement for the old GNU Emacs man pager, "woman"?

Nope. ox-man is a converter between Org markup and Man page sources:

Given the following Org markup:

  *This is test*
  ~code a+b~ here a+b.

 [*...* is bold markup. ~...~ is code markup.]

we aim to produce a valid man file that approximates the initial Org
markup as much as possible:

    .TH "" "1" 
    .PP
    \fBThis is test\fP
    \fCcode a+b\fP here a+b.

And this discussion was about using \fC to represent "code" (also,
"fixed width" and tables). We use \fC historically, and it is not very
clear what could be a replacement that does not break Man export
compared to previously produced results.

>>>   The best solution known to me is to use an extension to the man(7)
>>>   language.  It first appeared in Ninth Edition Unix (1986) and was
>>>   adopted by a groff release in 2009.  That is the `EX`/`EE` macro
>>>   pair, which sets a monospaced display.  (In other words, filling is
>>>   disabled and a monospaced font selected if necessary.)
>
> Yes.

Ok. But will it work inline?
From my reading of man 7 man, .EX/.EE are more suitable for paragraph
markup:

       .EX
       .EE    Begin and end example.  After .EX, filling is disabled and a
              constant-width (monospaced) font is selected.  Calling .EE
              enables filling and restores the previous font.

              These macros are extensions introduced in Ninth Edition
              Research Unix.  Systems running that troff, or those from
              Documenter’s Workbench, Heirloom Doctools, or Plan 9 troff
              support them.  To be certain your page will be portable to
              systems that do not, copy their definitions from the
              an-ext.tmac file of a groff installation.

>> However, as you observe, `\f[CR]` doesn't (nor does `\f(CR`).  I note
>> that groff's HTML support is stated in the grohtml(1) man-page to be
>> in beta.  Haven't checked the source to determine whether that is
>> what's going on here.
>
> It's a mess. :(
>
> https://savannah.gnu.org/bugs/index.php?61915
>
> That's the tip of a large iceberg.

Ok. So much for testing via man->HTML exporter.
What about PDF?

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 6991 bytes --]

[Ingo: see bottom of message for a possible mandoc(1) bug in footnote 2]

Hi Ihor,

At 2024-12-22T15:41:57+0000, Ihor Radchenko wrote:
> "G. Branden Robinson" <g.branden.robinson@gmail.com> writes:
> 
> >> Jeremy suggests that "C" may be an old alias for Courier, and if
> >> that's the case it should be changed to "\f[CR]".  Would be great
> >> if Org people can confirm.
> >
> > That is good advice and it is what I recommend if you're writing in
> > "raw" roff.  The context of the discussion is not ultra-clear to me;
> > is ox-man.el a replacement for the old GNU Emacs man pager, "woman"?
> 
> Nope. ox-man is a converter between Org markup and Man page sources:
> 
> Given the following Org markup:
> 
>   *This is test*
>   ~code a+b~ here a+b.
> 
>  [*...* is bold markup. ~...~ is code markup.]
> 
> we aim to produce a valid man file that approximates the initial Org
> markup as much as possible:
> 
>     .TH "" "1" 
>     .PP
>     \fBThis is test\fP
>     \fCcode a+b\fP here a+b.

I understand now.  I've taken some time to learn a bit more about Org
mode, and also that org-man.el seems to be something close to the
bleeding edge; in a Debian unstable chroot it was not provided by the
version of GNU Emacs (29.4) nor the "elpa-org" package (9.7.16).

So I'll make some recommendations unfortunately without the benefit of
having field-tested them.

> And this discussion was about using \fC to represent "code" (also,
> "fixed width" and tables). We use \fC historically, and it is not very
> clear what could be a replacement that does not break Man export
> compared to previously produced results.

The answer depends on how portable you want your man(7) output to be.  A
major reason is that font names are not portable across output devices;
this was not thought to be a need when Brian Kernighan refactored the
troff of Seventh Edition Unix for device-independence circa 1980.  When
James Clark wrote groff in about 1989, he tried to standardize font
names across devices.  But he also accommodated users of AT&T troff,
providing aliases for many of the font names used by its dpost(1) output
driver, and also didn't write all the diagnostic messages he could have,
with the consequence that `\fC` sometimes worked, but also often
silently failed, and few people noticed the difference.  The last factor
especially applies to users of terminals, where changing the font family
in this manner is impossible anyway.

`\f(CR` will work with groff, Heirloom Doctools troff, and neatroff.

It won't work with mandoc(1) 1.14.6 (or earlier, I suspect), but then
neither does `\fC`.[1][2]  It won't work with Solaris 10 troff (and
therefore Illumos troff) or Plan 9 from User Space troff.

Given that, you might as well spell the escape sequence as `\f[CR]`,
using GNU troff syntax for arbitrary-length escape sequence arguments.
You can say either `\fP` or `\f[]` to restore the previous typeface.

> Ok. But will it work inline?

There is no portable way to achieve that across all known man(7) (or
troff(1)) implementations.  But if you care only about the formatters I
mentioned as groff-compatible above, then `\f[CR]whatever\f[]` should
work fine.

> From my reading of man 7 man, .EX/.EE are more suitable for paragraph
> markup:
> 
>        .EX
>        .EE    Begin and end example.  After .EX, filling is disabled
>               and a constant-width (monospaced) font is selected.
>               Calling .EE enables filling and restores the previous
>               font.
> 
>               These macros are extensions introduced in Ninth Edition
>               Research Unix.  Systems running that troff, or those
>               from Documenter’s Workbench, Heirloom Doctools, or Plan
>               9 troff support them.  To be certain your page will be
>               portable to systems that do not, copy their definitions
>               from the an-ext.tmac file of a groff installation.

Yes.  `EX`/`EE` works only with "displays".

But it is worth noting that `EX`/`EE` is _more_ portable than `\fC`.

> Ok. So much for testing via man->HTML exporter.
>
> What about PDF?

groff's PDF support is pretty good.  I use it all the time myself.

Here's an exhibit of dogfooding:

https://www.gnu.org/software/groff/manual/groff-man-pages.pdf

In groff 1.24, that document will be about 75% smaller in file size (due
to subsetting of embedded fonts), and will feature hyperlinks for all
man page cross references, internally where applicable, and as "man:"
URIs for external references.

So.  To my recommendations:

Going by
<https://github.com/emacs-mirror/emacs/blob/master/lisp/org/ox-man.el>
and ignoring indentation,

I would change line 369 to:

(format "\\f[CR]%s\\f[]"

and line 438 to:

(format "\\f[CR]\n%s\n\\f[]"

and line 538 to:

(format ".RS\n.EX\n\\m[black]%s\\m[]\n.EE\n.RE\n"

and lines 543-544 to:

(concat ".RS\n.EX\n" (org-man--protect-example code) "\n"
        ".EE\n.RE\n")))))

and line 758 to:

(format ".RS\n.EX\n%s\n.EE\n.RE\n\n"

(I would also delete that trailing "\n".  Avoid blank lines in man(7)
input.[3])

and line 782 to:

(format ".RS\n.EX\n\\m[black]%s\\m[]\n.EE\n.RE" (org-man--protect-example code))))))

and finally line 844 to:

(format ".EX\n%s\n.EE"

(As another aside, the stroke color selection escape sequences `\m` are
(a) GNU troff extensions, albeit supported by the same formatters that
`\f[CR]` is, but (b) probably unnecessary because nothing else in this
entire file seems to use any other color for anything.)

You can test a generated man(7) document by asking groff(1) and man(7)
to lint it.

$ groff -ww -rCHECKSTYLE=3 -man my-org-doc.man

Please let me know if/how I can be of further assistance.

Regards,
Branden

[1] If you "lint" your document, mandoc(1) will even tell you so.

    $ mandoc -T lint blah.man
    mandoc: blah.man:5:12: WARNING: invalid escape sequence: \fC

[2] Actually, it looks like `\f(CR` (or `\f[CR]`) _should_ work with
    HTML output in mandoc(1).  It brackets the word with a "span"
    element using the "Li" class (a reference to the mdoc(7) `Li`
    macro--mandoc(1)'s authors and maintainers are big-time mdoc(7)
    partisans).  But, nothing in the `head` element bothers to define
    the `Li` class when formatting a man(7) document.  This seems like a
    bug in mandoc(1) to me.

    mandoc(1) uses an external program to produce PDF from HTML, so
    inspecting its PDF output tells you nothing that its HTML output
    doesn't.

[3] groff_man_style(7):

     •  The empty request (.), which does nothing, vertically spaces the
        input file for readability by the document maintainer.  Do not
        put blank (empty) lines in a man page source document.  Some
        man(1) programs “squeeze” multiple blank output lines into one.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

"G. Branden Robinson" <g.branden.robinson@gmail.com> writes:

> I understand now.  I've taken some time to learn a bit more about Org
> mode, and also that org-man.el seems to be something close to the
> bleeding edge; in a Debian unstable chroot it was not provided by the
> version of GNU Emacs (29.4) nor the "elpa-org" package (9.7.16).

You looked up wrong file.
Not org-man.el, but ox-man.el.
It is a part of Org mode since forever (introduced before Org 4.12a -
the first tagged release in our git repo - Jan 31, 2008; 16 years ago).

> So I'll make some recommendations unfortunately without the benefit of
> having field-tested them.

You can do the following to try exporting to man from Emacs 29.4:

1. emacs -Q
2. M-: (require 'ox-man) RET
3. Open a sample .org file
4. C-c C-e M m

(4) will produce a man file named after the .org file, in the same directory.

>> And this discussion was about using \fC to represent "code" (also,
>> "fixed width" and tables). We use \fC historically, and it is not very
>> clear what could be a replacement that does not break Man export
>> compared to previously produced results.
>
> The answer depends on how portable you want your man(7) output to be.  A
> major reason is that font names are not portable across output
> devices;...

1. We want to avoid breaking cases when \fC did work (it is a minimum)
2. Ideally, we want things to work in _more_ cases
3. Even more ideally, we want things to work in all the cases

If (1) is not possible, compromises will have to be made. I hope that we
do not have to make compromises.

Also, what if we leave \fC and add \f[CR]/.EX on top?
AFAIU, the worst case scenario for \fC is that it does nothing. By
leaving it there, we thus retain old working exports working while also
adding appropriate format when \f[CR] is supported.

>> From my reading of man 7 man, .EX/.EE are more suitable for paragraph
>> markup:
> ...
> Yes.  `EX`/`EE` works only with "displays".

May you elaborate about the meaning of "displays"?

> But it is worth noting that `EX`/`EE` is _more_ portable than `\fC`.

In non-inclusive manner though. (cases when \fC works are not a subset
of cases where EX/EE works). Correct me if I am wrong.

> (I would also delete that trailing "\n".  Avoid blank lines in man(7)
> input.[3])

Do I understand correctly that blank line is sometimes interpreted as
vertical spacing and sometimes ignored?

> (As another aside, the stroke color selection escape sequences `\m` are
> (a) GNU troff extensions, albeit supported by the same formatters that
> `\f[CR]` is, but (b) probably unnecessary because nothing else in this
> entire file seems to use any other color for anything.)

May black sometimes be different from the default color?

> You can test a generated man(7) document by asking groff(1) and man(7)
> to lint it.
>
> $ groff -ww -rCHECKSTYLE=3 -man my-org-doc.man

Thanks! This might be a good thing to add to our tests.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[onf]

On Tue Dec 31, 2024 at 7:15 PM CET, Ihor Radchenko wrote:
> >> And this discussion was about using \fC to represent "code" (also,
> >> "fixed width" and tables). We use \fC historically, and it is not very
> >> clear what could be a replacement that does not break Man export
> >> compared to previously produced results.
> >
> > The answer depends on how portable you want your man(7) output to be.  A
> > major reason is that font names are not portable across output
> > devices;...
>
> 1. We want to avoid breaking cases when \fC did work (it is a minimum)
> 2. Ideally, we want things to work in _more_ cases
> 3. Even more ideally, we want things to work in all the cases
>
> If (1) is not possible, compromises will have to be made. I hope that we
> do not have to make compromises.
>
> Also, what if we leave \fC and add \f[CR]/.EX on top?
> AFAIU, the worst case scenario for \fC is that it does nothing. By
> leaving it there, we thus retain old working exports working while also
> adding appropriate format when \f[CR] is supported.

The \f escape sets current font. \fP restores previous font.
What this means is that doing \fC\f[CR]Lorem\fP:
 (1) sets the text Lorem in Courier if font name C or CR exists
 (2) restores previous font if neither font C nor CR exists
 (3) sets font to Courier if both font name C and CR exist

> >> From my reading of man 7 man, .EX/.EE are more suitable for paragraph
> >> markup:
> > ...
> > Yes.  `EX`/`EE` works only with "displays".
>
> May you elaborate about the meaning of "displays"?

Branden will likely provide you with a more detailed description, but
in short, displays are text that is set off from the rest. Example from
groff(7):
  Long words may then look intimidating in the input; a clarifying
  approach might be to use the input line continuation escape sequence
  \newline to place each underlined character on its own input line.
  Thus,
         .nf
         \&\fB: ${\fIvar\fR\c
         \zo\(ul\
         \zp\(ul\c
         \&\fIvalue\fB}
         .fi
  produces
         : ${varopvalue}
  as output.

Checking the manpage's source shows that this is done precisely using
the EX/EE macros:
  Thus,
  .
  .RS
  .EX
  \&.nf
  \[rs]&\[rs]fB: ${\[rs]fIvar\[rs]fR\[rs]c
  \[rs]zo\[rs](ul\[rs]
  \[rs]zp\[rs](ul\[rs]c
  \[rs]&\[rs]fIvalue\[rs]fB}
  \&.fi
  .EE
  .RE
  .
  produces
  .
  .RS
  .EX
  .BI ": ${" var \c
  \zo\(ul\
  \zp\(ul\c
  .IB value }
  .EE
  .RE
  .
  as output.

> Do I understand correctly that blank line is sometimes interpreted as
> vertical spacing and sometimes ignored?

A blank input line is equivalent to .sp unless the .blm request was called
to change this behavior. .blm is groff's invention (see groff_diff(7)).

~ onf

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[onf]

On Tue Dec 31, 2024 at 7:42 PM CET, onf wrote:
> Branden will likely provide you with a more detailed description, but
> in short, displays are text that is set off from the rest. Example from
> groff(7):
> [...]
> Checking the manpage's source shows that this is done precisely using
> the EX/EE macros:
> [...]

Small correction: in that example, the increased indent is done via RS/RE.
However, EX/EE break the line, format its content in monospaced font, and
break line again, so this really cannot be used inline.

~ onf

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

"onf" <onf@disroot.org> writes:

>> Also, what if we leave \fC and add \f[CR]/.EX on top?
>> AFAIU, the worst case scenario for \fC is that it does nothing. By
>> leaving it there, we thus retain old working exports working while also
>> adding appropriate format when \f[CR] is supported.
>
> The \f escape sets current font. \fP restores previous font.
> What this means is that doing \fC\f[CR]Lorem\fP:
>  (1) sets the text Lorem in Courier if font name C or CR exists
>  (2) restores previous font if neither font C nor CR exists
>  (3) sets font to Courier if both font name C and CR exist

What about \fC\f[CR]Lorem\fP\fP?

>> Do I understand correctly that blank line is sometimes interpreted as
>> vertical spacing and sometimes ignored?
>
> A blank input line is equivalent to .sp unless the .blm request was called
> to change this behavior. .blm is groff's invention (see groff_diff(7)).

Then maybe we can put .sp explicitly instead of a blank line.
In our code, we add blank after blocks (or displays) of code. Extra
vertical space after seems to be reasonable.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[onf]

On Wed Jan 1, 2025 at 10:38 AM CET, Ihor Radchenko wrote:
> >> Also, what if we leave \fC and add \f[CR]/.EX on top?
> >> AFAIU, the worst case scenario for \fC is that it does nothing. By
> >> leaving it there, we thus retain old working exports working while also
> >> adding appropriate format when \f[CR] is supported.
> >
> > The \f escape sets current font. \fP restores previous font.
> > What this means is that doing \fC\f[CR]Lorem\fP:
> >  (1) sets the text Lorem in Courier if font name C or CR exists
> >  (2) restores previous font if neither font C nor CR exists
> >  (3) sets font to Courier if both font name C and CR exist
>
> What about \fC\f[CR]Lorem\fP\fP?

(1) if both C and CR exist, sets Lorem in Courier
(2) if only C or CR exists, sets font to Courier
(3) if neither exists, does nothing

To understand #2, let's assume that C is defined and CR is not. Then:
  \fC -> set font to C
  \f[CR] -> ignored
  \fP -> restore previous font, set previous font to C
  \fP -> restore previous font (back to C)

It's important to understand that troff commands don't really use
nesting or a stack as one might be used to from HTML and similar
markup languages. That's why it's also not possible to e.g. nest
bold and italic like this:
  Normal \fBbold \fIbold-italic\fP bold\fP normal

...which actually gives you:
  Normal bold italic bold italic

One has to do one of these instead:
  Normal \fBbold \f[BI]bold-italic\fP bold\fR normal
  Normal \fBbold \f[BI]bold-italic\fB bold\fR normal
  Normal \fBbold\fP \f[BI]bold-italic\fP \fBbold\fP normal

> >> Do I understand correctly that blank line is sometimes interpreted as
> >> vertical spacing and sometimes ignored?
> >
> > A blank input line is equivalent to .sp unless the .blm request was called
> > to change this behavior. .blm is groff's invention (see groff_diff(7)).
>
> Then maybe we can put .sp explicitly instead of a blank line.
> In our code, we add blank after blocks (or displays) of code. Extra
> vertical space after seems to be reasonable.

They are equivalent as far as troff formatters are concerned. The question
is whether they are equivalent to the various other programs that attempt
to interpret man source code directly. So I guess this depends on whether
you're targetting only troff (+ BSD mandoc) or not. Branden will likely
tell you more about this topic.

~ onf

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[onf]

Hi Ihor,

On Wed Jan 1, 2025 at 1:30 PM CET, onf wrote:
> On Wed Jan 1, 2025 at 10:38 AM CET, Ihor Radchenko wrote:
> > >> Also, what if we leave \fC and add \f[CR]/.EX on top?
> > >> AFAIU, the worst case scenario for \fC is that it does nothing. By
> > >> leaving it there, we thus retain old working exports working while also
> > >> adding appropriate format when \f[CR] is supported.
> > >
> > > The \f escape sets current font. \fP restores previous font.
> > > What this means is that doing \fC\f[CR]Lorem\fP:
> > >  (1) sets the text Lorem in Courier if font name C or CR exists
> > >  (2) restores previous font if neither font C nor CR exists
> > >  (3) sets font to Courier if both font name C and CR exist
> >
> > What about \fC\f[CR]Lorem\fP\fP?
>
> (1) if both C and CR exist, sets Lorem in Courier
> (2) if only C or CR exists, sets font to Courier
> (3) if neither exists, does nothing
>
> [...]
>
> It's important to understand that troff commands don't really use
> nesting or a stack as one might be used to from HTML and similar
> markup languages. That's why it's also not possible to e.g. nest
> bold and italic like this:
>   Normal \fBbold \fIbold-italic\fP bold\fP normal
> [...]
> One has to do one of these instead:
>   Normal \fBbold \f[BI]bold-italic\fP bold\fR normal
>   Normal \fBbold \f[BI]bold-italic\fB bold\fR normal
>   Normal \fBbold\fP \f[BI]bold-italic\fP \fBbold\fP normal

I don't know why it took me so long, but I think what you're looking
for is this:
  \fC\f(CRLorem\fR

\fR sets regular / "roman" font, i.e. what's usually the default.
Just as shown in the examples with nested italic within bold, you
will likely have to track the style changes desired by Org-mode
and map them into the appropriate fonts, since troff knows only
fonts, not "text styles", and so these changes cannot be nested.

Note that I've used the syntax \f(, not \f[], because older
implementations of troff (such as Plan9 troff iirc) don't support it.
If you used \f[CR] on those platforms, it would be interpretted like:
  \f[ -> set font [ (non-existent => ignored)
  CR] -> text CR]

meaning you would get CR] in the text.

~ onf

[BUG] ox-man: Nested markup is broken (was: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)])

[Ihor Radchenko]

"onf" <onf@disroot.org> writes:

>> What about \fC\f[CR]Lorem\fP\fP?
>
> (1) if both C and CR exist, sets Lorem in Courier
> (2) if only C or CR exists, sets font to Courier
> (3) if neither exists, does nothing
>
> To understand #2, let's assume that C is defined and CR is not. Then:
>   \fC -> set font to C
>   \f[CR] -> ignored
>   \fP -> restore previous font, set previous font to C
>   \fP -> restore previous font (back to C)
>
> It's important to understand that troff commands don't really use
> nesting or a stack as one might be used to from HTML and similar
> markup languages. That's why it's also not possible to e.g. nest
> bold and italic like this:
>   Normal \fBbold \fIbold-italic\fP bold\fP normal
>
> ...which actually gives you:
>   Normal bold italic bold italic

Yikes!

We currently implicitly rely on the assumption that things are working
differently:

    normal *bold /italic/ bold* normal

is exported as

    .TH "" "1" 
    .PP
    normal \fBbold \fIitalic\fP bold\fP normal

and the last "normal" is indeed rendered as italic.

> One has to do one of these instead:
>   Normal \fBbold \f[BI]bold-italic\fP bold\fR normal
>   Normal \fBbold \f[BI]bold-italic\fB bold\fR normal
>   Normal \fBbold\fP \f[BI]bold-italic\fP \fBbold\fP normal

I'm afraid that I'll need to rethink that export algo for ox-man to
account for this nuance.

Probably, the simplest way implementation-wise will be applying faces to
each "plain text" segment individually:

   \fRnormal \fR\fBbold \fR\f[BI]italic\fR\fB bold\fR\fR normal\fR

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] ox-man: Nested markup is broken (was: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)])

[onf]

On Thu Jan 2, 2025 at 6:47 PM CET, Ihor Radchenko wrote:
> "onf" <onf@disroot.org> writes:
> [...]
> > It's important to understand that troff commands don't really use
> > nesting or a stack as one might be used to from HTML and similar
> > markup languages. That's why it's also not possible to e.g. nest
> > bold and italic like this:
> >   Normal \fBbold \fIbold-italic\fP bold\fP normal
> >
> > ...which actually gives you:
> >   Normal bold italic bold italic
>
> Yikes!
>
> We currently implicitly rely on the assumption that things are working
> differently:
>
>     normal *bold /italic/ bold* normal
>
> is exported as
>
>     .TH "" "1" 
>     .PP
>     normal \fBbold \fIitalic\fP bold\fP normal
>
> and the last "normal" is indeed rendered as italic.
>
> > One has to do one of these instead:
> >   Normal \fBbold \f[BI]bold-italic\fP bold\fR normal
> >   Normal \fBbold \f[BI]bold-italic\fB bold\fR normal
> >   Normal \fBbold\fP \f[BI]bold-italic\fP \fBbold\fP normal
>
> I'm afraid that I'll need to rethink that export algo for ox-man to
> account for this nuance.
>
> Probably, the simplest way implementation-wise will be applying faces to
> each "plain text" segment individually:
>
>    \fRnormal \fR\fBbold \fR\f[BI]italic\fR\fB bold\fR\fR normal\fR

I assume you meant:
  \fRnormal \fP\fBbold \fP\f[BI]italic\fP\fB bold\fP\fR normal\fP

You might find it easier to omit the \fP escapes, since they have
little effect here anyway. Situation might be different if you're
targetting non-troff man processors too, though; no idea how
robust their handling of font changes is.

But I agree; I would track each style change (bold -> not bold,
italic -> not italic) and emit an appropriate \f sequence that
would match the currently active styles after each such change,
like so:
  code & bold & italic  \f[CBI]
  code & bold           \fC\f(CB
  code & italic         \fC\f(CI
  code                  \fC\f(CR
  bold & italic         \f(BI
  bold                  \fB
  italic                \fI
  none                  \fR

Note that \f[CBI] will break in old troffs due to the "new" \f[]
syntax, so a warning should probably be emitted. And I have no
idea how good support for {bold,italic} Courier is in older troffs
either. (That is assuming Org-mode markup allows code to be styled
like that.)

Also note that if a change involving italic happens between two
non-whitespace characters, like so:
  groff(/7/)

...an italic correction should be emitted to prevent the two glyphs
colliding into each other:
  groff(\,\fI7\fR\/)
where:
  \,  is left italic correction
  \/  is right italic correction

~ onf

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 7650 bytes --]

Hi onf,

[replying to 2 messages]

At 2025-01-02T15:29:59+0100, onf wrote:
> I don't know why it took me so long, but I think what you're looking
> for is this:
>   \fC\f(CRLorem\fR

Not a bad idea, but while you're there, given Ihor's ambitious
portability objectives, you might as well scoop up the population of
troffs that named the font `CW`.[1]

So you'd emit `\fC\fCW\fCRLorem\fR`.

> Note that I've used the syntax \f(, not \f[], because older
> implementations of troff (such as Plan9 troff iirc) don't support it.

Right.  Plan 9 troff (whether original or "from User Space"), Solaris 10
troff (also used by Illumos)[2], and DWB 3.3 troff, which as far as I
know is not used by anyone for real typesetting purposes, but is a
valuable specimen for exploring Kernighan troff behavior.

At 2025-01-02T22:51:47+0100, onf wrote:
> You might find it easier to omit the \fP escapes, since they have
> little effect here anyway. Situation might be different if you're
> targetting non-troff man processors too, though; no idea how
> robust their handling of font changes is.

The only one I'd think of trusting is mandoc(1), and since that project
consciously limits its own scope, I wouldn't necessarily expect support
for a lot of fancy font tricks.

> But I agree; I would track each style change (bold -> not bold,
> italic -> not italic) and emit an appropriate \f sequence that
> would match the currently active styles after each such change,
> like so:
>   code & bold & italic  \f[CBI]
>   code & bold           \fC\f(CB
>   code & italic         \fC\f(CI
>   code                  \fC\f(CR
>   bold & italic         \f(BI
>   bold                  \fB
>   italic                \fI
>   none                  \fR
> 
> Note that \f[CBI] will break in old troffs due to the "new" \f[]
> syntax, so a warning should probably be emitted. And I have no
> idea how good support for {bold,italic} Courier is in older troffs
> either.

Poor, in general.  Where Courier bold-italic existed, it was sometimes
called "CX".

> (That is assuming Org-mode markup allows code to be styled like that.)
> 
> Also note that if a change involving italic happens between two
> non-whitespace characters, like so:
>   groff(/7/)
> 
> ...an italic correction should be emitted to prevent the two glyphs
> colliding into each other:
>   groff(\,\fI7\fR\/)
> where:
>   \,  is left italic correction
>   \/  is right italic correction

So another thing to know here is that these italic correction escape
sequences are, yet again, GNU troff extensions.  A legacy formatter is
likely to render them as if the backslash were absent, which is very
much not what you want in your output.

I reiterate that Ihor's objective of "everything working everywhere" is
a tall challenge.  My advice is to the org-mode project is to limit its
compatibility horizon to groff, Heirloom Doctools troff, neatroff, and
mandoc.  I say that not for self-serving reasons[4] but because other
implementations are moribund or not aimed at general-purpose use.[5]

If org-mode did that, it could ignore the `CW` font whose history I just
went to the trouble to explore, and the problem with `\,` and `\/`
escape sequences as well.  With a little more research we might find
that org-mode could disregard `\fC` as well.

Having now seen org-mode's man(7) output in the flesh from my own input,
I see plenty of room for easy wins and improvements.  I'll venture those
in a separate discussion as I make time, or as someone throws me
questions to answer or problems to solve.

Regards,
Branden

[1] `CW` was the name used for a long time by Kernighan troff as
    distributed in Unix System V and therefore overlapped with, or was
    identical to DWB 2.0 troff.  (I'm a little unclear on the precise
    history.)

    Anyway, that name was supported on some, but not all, devices.

$ git grep -w CW | grep DESC
SysVr2.0_32000/SysVr2.0_32000/dwb/troff.d/devX97/X97.tim10p/DESC:fonts 5 R I B S CW
SysVr2.0_32000/SysVr2.0_32000/dwb/troff.d/devX97/X97.tim12p/DESC:fonts 5 R I B S CW
SysVr2.0_32000/SysVr2.0_32000/dwb/troff.d/devaps/DESC:fonts 8 R I B H CW S S1 GR
SysVr2.0_32000/SysVr2.0_32000/dwb/troff.d/devi10/DESC:fonts 8 R I B H HI HK CW S
SysVr2.0_32000/SysVr2.0_32000/dwb/troff.d/devi10/rasti10/aps-i10/DESC:fonts 55 R I B H CW S S1 GR R.S R.bld I.S I.bld B.S B.bld H.S H.bld S.R GR.R PA.S PA.bld PB.S PB.bld PI.S PI.bld CE.S CE.bld CI.S CI.bld HI.S HI.bld HB.S HB.bld G.S G.bld GI.S GI.bld BI.S BI.bld SM.S CT.S C.S C.bld SC.S TB.S TB.bld GS.S GB.S MR.S MR.bld MI.S MI.bld MB.S MB.bld MX.S MX.bld
sysvr4/svr4/cmd/lp/filter/postscript/buildtables/devpost/DESC:fonts 10 R I B BI CW H HB HX S1 S
sysvr4/svr4/cmd/lp/filter/postscript/font/devpost/DESC:fonts 10 R I B BI CW H HB HX S1 S
sysvr4/svr4/cmd/lp/filter/postscript/font/devpost/DESC.big:fonts 10 R I B BI CW H HB HX S1 S
sysvr4/svr4/cmd/lp/filter/postscript/font/devpost/DESC.small:# space. biggestfont is set to the size of the CW font, and will accomodate all
sysvr4/svr4/cmd/lp/filter/postscript/font/devpost/DESC.small:fonts 10 R I B BI CW H HB HX S1 S
sysvr4/svr4/cmd/lp/filter/postscript/font/devpostaps/DESC:fonts 8 R I B H CW S S1 GR
sysvr4/svr4/ucbcmd/troff/troff.d/devaps/DESC:fonts 8 R I B H CW S S1 GR

    In this System V archive, I get hits on `C` in "DESC" files, but not
    as font names.  (The special character name `*C`, capital Greek
    gamma, is a common hit.)

    I get no hits on `CR` (in "DESC" files) at all in this archive.

    The `CW` font name seems to have originated in Unix System III
    (1980) but my copy of it doesn't have any "DESC" files.  Yet it
    "does" ship the "Addendum to the NROFF/TROFF User's Manual" (April
    1980) document that seems to reflect some of Kernighan's changes
    (like "compacted macros", a feature later dropped).  It's very
    strange to me to see this accompany a "DESC"-less troff
    installation.  I guess I need to consult an AT&T commercial Unix
    historian.

    Another note is that DWB 3.3 retained "CW".  Oddly, the dev202
    support is stripped out from the Free Software version of DWB 3.3.
    I don't know why that is.  Maybe at the time it was released,
    Mergenthaler Linotron (or its successor-in-interest) was still
    unhappy with Bell Labs (or Lucent Technologies, its then
    successor-in-interest) for reverse engineering their fonts.[3]

$ git grep -w CW | grep DESC
postscript/devLatin1/DESC:fonts 10 R I B BI CW H HI HB S1 S
postscript/devopost/DESC:fonts 10 R I B BI CW H HB HX S1 S
postscript/devpost/DESC:fonts 10 R I B BI CW H HI HB S1 S
text/devnroff-12/DESC:fonts 5 R I B CW S
text/troff/makefile:DFILES=dev202/DESC dev202/R dev202/I dev202/B dev202/S dev202/CW

[2] Solaris 11 uses groff for its troff.
[3] https://princeton-staging.elsevierpure.com/en/publications/revisiting-a-summer-vacation-digital-restoration-and-typesetter-f

[4] It is true that Heirloom Doctools troff, neatroff, and mandoc
    support the most important/disruptive syntactical extensions of GNU
    troff.  But that situation eventuated before I joined groff
    development--by several years in most or all instances.  groff's
    dialect of *roff is as close to a lingua franca for this application
    domain ("traditional Unix typesetting") as we have.

[5] I admit I'm a little fuzzy on Plan 9 from User Space's objectives
    for its troff.  I can say that the pace of its development is not
    swift.  I don't think Plan 9 advocates pick up the system to use its
    typesetter.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[onf]

Hi Branden,

On Fri Jan 3, 2025 at 9:38 AM CET, G. Branden Robinson wrote:
> At 2025-01-02T15:29:59+0100, onf wrote:
> > I don't know why it took me so long, but I think what you're looking
> > for is this:
> >   \fC\f(CRLorem\fR
>
> Not a bad idea, but while you're there, given Ihor's ambitious
> portability objectives, you might as well scoop up the population of
> troffs that named the font `CW`.[1]
>
> So you'd emit `\fC\fCW\fCRLorem\fR`.

Fair point.

> [...]
> At 2025-01-02T22:51:47+0100, onf wrote:
> [...]
> > But I agree; I would track each style change (bold -> not bold,
> > italic -> not italic) and emit an appropriate \f sequence that
> > would match the currently active styles after each such change,
> > like so:
> >   code & bold & italic  \f[CBI]
> >   code & bold           \fC\f(CB
> >   code & italic         \fC\f(CI
> >   code                  \fC\f(CR
> >   bold & italic         \f(BI
> >   bold                  \fB
> >   italic                \fI
> >   none                  \fR
> > 
> > Note that \f[CBI] will break in old troffs due to the "new" \f[]
> > syntax, so a warning should probably be emitted. And I have no
> > idea how good support for {bold,italic} Courier is in older troffs
> > either.
>
> Poor, in general.  Where Courier bold-italic existed, it was sometimes
> called "CX".

I meant support for CB and CI, not CBI.

> > (That is assuming Org-mode markup allows code to be styled like that.)
> > 
> > Also note that if a change involving italic happens between two
> > non-whitespace characters, like so:
> >   groff(/7/)
> > 
> > ...an italic correction should be emitted to prevent the two glyphs
> > colliding into each other:
> >   groff(\,\fI7\fR\/)
> > where:
> >   \,  is left italic correction
> >   \/  is right italic correction
>
> So another thing to know here is that these italic correction escape
> sequences are, yet again, GNU troff extensions.  A legacy formatter is
> likely to render them as if the backslash were absent, which is very
> much not what you want in your output.

Oh. Thanks for pointing that out, I had no idea.

> [...]
> If org-mode did that, it could ignore the `CW` font whose history I just
> went to the trouble to explore, and the problem with `\,` and `\/`
> escape sequences as well.  With a little more research we might find
> that org-mode could disregard `\fC` as well.

Speaking of the left italic correction (\,), could you please give me
some example(s) where it's actually useful? I have yet to see a single
instance where it changes anything... (I don't rule out the possibility
that I may be using it incorrectly or that my fonts might be
misconfigured, though.)

> [...]
> [5] I admit I'm a little fuzzy on Plan 9 from User Space's objectives
>     for its troff.  I can say that the pace of its development is not
>     swift.  I don't think Plan 9 advocates pick up the system to use its
>     typesetter.

My understanding is that Plan 9 from User Space is merely providing a
port of the Plan 9 program. My impression of Plan 9's troff is that
it's a poorly maintained descendant of DWB troff that's used only to
format manpages.

~ onf

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[G. Branden Robinson]

[-- Attachment #1.1: Type: text/plain, Size: 2358 bytes --]

Hi onf,

At 2025-01-04T01:23:02+0100, onf wrote:
> > > Note that \f[CBI] will break in old troffs due to the "new" \f[]
> > > syntax, so a warning should probably be emitted. And I have no
> > > idea how good support for {bold,italic} Courier is in older troffs
> > > either.
> >
> > Poor, in general.  Where Courier bold-italic existed, it was
> > sometimes called "CX".
> 
> I meant support for CB and CI, not CBI.

Also poor, outside of the horizon I previously articulated.  :(

System V Unix supported these names _only_ on the "aps" device
(Autologic APS-5).  Plan 9 doesn't.  DWB 3.3 doesn't.

The "aps" device on Solaris 10 troff supports CB but not CI.

> Speaking of the left italic correction (\,), could you please give me
> some example(s) where it's actually useful? I have yet to see a single
> instance where it changes anything... (I don't rule out the
> possibility that I may be using it incorrectly or that my fonts might
> be misconfigured, though.)

Sure.  I'm attaching an extract of groff_diff(7) in PDF, which not only
explains the issue but illustrates the problem it solves.

Our Texinfo manual has the same textual content but doesn't deliberately
exhibit the problem.

> > [5] I admit I'm a little fuzzy on Plan 9 from User Space's
> >     objectives for its troff.  I can say that the pace of its
> >     development is not swift.  I don't think Plan 9 advocates pick
> >     up the system to use its typesetter.
> 
> My understanding is that Plan 9 from User Space is merely providing a
> port of the Plan 9 program.

It's a whole OS!  Give it credit... ;-)

> My impression of Plan 9's troff is that it's a poorly maintained
> descendant of DWB troff that's used only to format manpages.

I wouldn't say _poorly_ maintained; its commit rate seems to match or
even exceed its defect rate.  This could illustrate (1) low adoption or
(2) adequacy for the (limited?) purposes to which people apply it.

I'd like to see an attempt at reconstructing DWB 3.4 from the existing
DWB 3.3 code base (made portable) and an old snapshot of Plan 9, maybe
circa 2000.  It'd be good to have a plausible exhibit of the end of the
line for AT&T Unix troff--what the body looked like at the murder scene.

https://lists.gnu.org/archive/html/groff/2022-12/msg00097.html

Regards,
Branden

[-- Attachment #1.2: italic.pdf --]
[-- Type: application/pdf, Size: 431369 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[onf]

Hi Branden,

On Sat Jan 4, 2025 at 7:37 AM CET, G. Branden Robinson wrote:
> At 2025-01-04T01:23:02+0100, onf wrote:
> [...]
> > Speaking of the left italic correction (\,), could you please give me
> > some example(s) where it's actually useful? I have yet to see a single
> > instance where it changes anything... (I don't rule out the
> > possibility that I may be using it incorrectly or that my fonts might
> > be misconfigured, though.)
>
> Sure.  I'm attaching an extract of groff_diff(7) in PDF, which not only
> explains the issue but illustrates the problem it solves. [...]

Thanks!

> > [...]
> > My understanding is that Plan 9 from User Space is merely providing a
> > port of the Plan 9 program.
>
> It's a whole OS!  Give it credit... ;-)

You misunderstood; I meant:
  Plan 9 from User Space is merely providing a port of the Plan 9
  [troff] program.

I know that Plan 9 is an OS.

> > My impression of Plan 9's troff is that it's a poorly maintained
> > descendant of DWB troff that's used only to format manpages.
>
> I wouldn't say _poorly_ maintained; its commit rate seems to match or
> even exceed its defect rate.  This could illustrate (1) low adoption or
> (2) adequacy for the (limited?) purposes to which people apply it. [...]

Which commit rate are you referring to? I thought the official sources
are at https://plan9.io/sources/plan9/sys/src/cmd/troff/ ... or maybe
you meant the 9front version?

~ onf

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Lennart Jablonka]

[replying to two messages]

Quoth onf:
>On Sat Jan 4, 2025 at 7:37 AM CET, G. Branden Robinson wrote:
>> At 2025-01-04T01:23:02+0100, onf wrote:
>[…]
>> > My impression of Plan 9's troff is that it's a poorly maintained
>> > descendant of DWB troff that's used only to format manpages.

It’s not only used to format man pages: It’s also used to format 
volume 2 of the manual; i.e., the papers in /sys/doc are formatted 
by troff.  The papers at the state of the fourth edition’s 
abandonment (2015) is at https://9p.io/sys/doc/index.html ; see, 
for example, the https://9p.io/sys/doc/troff.pdf linked thence.

Beside /sys/doc and /sys/man, the International Workshop on
Plan 9¹ accepts papers written in -ms.  And I know of at least one 
person who wrote some sort of thesis for a degree using troff on 
9front.  (Said thesis was on Plan 9.)

Troff on 9front is not used very much to format man pages not 
written for Plan 9.  It sometimes is, but not very often.  There 
was no need to import -mdoc yet, for example.

>> I wouldn't say _poorly_ maintained; its commit rate seems to match or
>> even exceed its defect rate.  This could illustrate (1) low adoption or
>> (2) adequacy for the (limited?) purposes to which people apply it. [...]
>
>Which commit rate are you referring to? I thought the official sources
>are at https://plan9.io/sources/plan9/sys/src/cmd/troff/ ... or maybe
>you meant the 9front version?

Yes, the sources there are the last sources of Plan 9 as 
distributed by Bell Labs.  It was abandoned in 2015 and is now 
dead.  Beside that, there is the commit rate of plan9port and that 
of 9front.

I don’t care for plan9port.  I do care about 9front.

Per my understanding, troff on Plan 9 (and thus 9front and 
plan9port) is just bwk’s continuation of troff.  The Unix system 
would have been replaced by Plan 9 with bwk continuing developing 
troff like nothing happened, only now on Plan 9 instead of on v10.

Yes, there are few changes to troff on 9front.  I remember fixing 
a little defect.  If anything, it might get replaced by Neatroff 
at some point.

Quoth G. Branden Robinson:
>I'd like to see an attempt at reconstructing DWB 3.4 from the existing
>DWB 3.3 code base (made portable) and an old snapshot of Plan 9, maybe
>circa 2000.  It'd be good to have a plausible exhibit of the end of the
>line for AT&T Unix troff--what the body looked like at the murder scene.

You can find old versions of Plan 9 at the website of the 
“Plan 9 Foundation”: https://p9f.org/dl/index.html .  The troff 
sources are in /sys/src/cmd/troff.  I doubt they changed very much 
in the last 30 years.

>> > Poor, in general.  Where Courier bold-italic existed, it was
>> > sometimes called "CX".
>>
>> I meant support for CB and CI, not CBI.
>
>Also poor, outside of the horizon I previously articulated.  :(
>
>System V Unix supported these names _only_ on the "aps" device
>(Autologic APS-5).  Plan 9 doesn't.  DWB 3.3 doesn't.
>
>The "aps" device on Solaris 10 troff supports CB but not CI.

Plan 9 does have those names.  It has all of C, CO, and CW for 
Courier and CI, CB, and CX for Courier-{Oblique,Bold,BoldOblique}.

¹ The website is at https://iwp9.org/ , the last proceedings at 
https://10e.iwp9.org/10iwp9proceedings.pdf .  The papers with the 
usual, traditional, ugly -ms extra half-inch at the right were set 
with troff.

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

"G. Branden Robinson" <g.branden.robinson@gmail.com> writes:

>> Also note that if a change involving italic happens between two
>> non-whitespace characters, like so:
>>   groff(/7/)
>> 
>> ...an italic correction should be emitted to prevent the two glyphs
>> colliding into each other:
>>   groff(\,\fI7\fR\/)
>> where:
>>   \,  is left italic correction
>>   \/  is right italic correction
>
> So another thing to know here is that these italic correction escape
> sequences are, yet again, GNU troff extensions.  A legacy formatter is
> likely to render them as if the backslash were absent, which is very
> much not what you want in your output.

Hmm. I am wondering why GNU troff is not making these adjustments
automatically when rendering things.

> I reiterate that Ihor's objective of "everything working everywhere" is
> a tall challenge.  My advice is to the org-mode project is to limit its
> compatibility horizon to groff, Heirloom Doctools troff, neatroff, and
> mandoc.  I say that not for self-serving reasons[4] but because other
> implementations are moribund or not aimed at general-purpose use.[5]

I do not oppose limiting scope somewhat.
At the end, our official policy is simply supporting the latest version
of third-party tools we need to cooperate with.
However, yet another policy is to not break Org documents that worked in
the past. That's way I prefer to be over-cautions and support more
things, especially if such support is not too difficult.

The idea with adding multiple legacy escapes seems simple enough to do
it once and forget.

> If org-mode did that, it could ignore the `CW` font whose history I just
> went to the trouble to explore, and the problem with `\,` and `\/`
> escape sequences as well.  With a little more research we might find
> that org-mode could disregard `\fC` as well.

There is no problem with \, and \/ sequences in Org - we do not use
them. Whether we should is another story.

What would be the downside of adding \fCW (in addition to other things)?

> Having now seen org-mode's man(7) output in the flesh from my own input,
> I see plenty of room for easy wins and improvements.  I'll venture those
> in a separate discussion as I make time, or as someone throws me
> questions to answer or problems to solve.

That would indeed be welcome.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Dave Kemper]

On Sat, Jan 4, 2025 at 7:24 AM Ihor Radchenko <yantar92@posteo.net> wrote:
> "G. Branden Robinson" <g.branden.robinson@gmail.com> writes:
> > So another thing to know here is that these italic correction escape
> > sequences are, yet again, GNU troff extensions.  A legacy formatter is
> > likely to render them as if the backslash were absent, which is very
> > much not what you want in your output.
>
> Hmm. I am wondering why GNU troff is not making these adjustments
> automatically when rendering things.

You can click through the thread to see the numerous replies I
received when I asked this question on the groff list 11 years ago.

http://lists.gnu.org/r/groff/2013-11/msg00018.html

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

Dave Kemper <saint.snit@gmail.com> writes:

> On Sat, Jan 4, 2025 at 7:24 AM Ihor Radchenko <yantar92@posteo.net> wrote:
>> "G. Branden Robinson" <g.branden.robinson@gmail.com> writes:
>> > So another thing to know here is that these italic correction escape
>> > sequences are, yet again, GNU troff extensions.  A legacy formatter is
>> > likely to render them as if the backslash were absent, which is very
>> > much not what you want in your output.
>>
>> Hmm. I am wondering why GNU troff is not making these adjustments
>> automatically when rendering things.
>
> You can click through the thread to see the numerous replies I
> received when I asked this question on the groff list 11 years ago.
>
> http://lists.gnu.org/r/groff/2013-11/msg00018.html

My reading of that thread is that doing the adjustments automatically
requires precise knowledge of the fonts involved - something that would
be even harder to do in Org (Org does automatic org markup -> groff
markup conversion).

So, it looks like these fine details are out of interest for ox-man.el
library. I see no good way to do it automatically.

Users can always use direct markup injection like @@man:\,@@ if they
need to fine-tune the man output.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 2546 bytes --]

Hi Ihor,

At 2025-01-01T09:38:37+0000, Ihor Radchenko wrote:
> "onf" <onf@disroot.org> writes:
> 
> >> Also, what if we leave \fC and add \f[CR]/.EX on top?
> >> AFAIU, the worst case scenario for \fC is that it does nothing. By
> >> leaving it there, we thus retain old working exports working while also
> >> adding appropriate format when \f[CR] is supported.
> >
> > The \f escape sets current font. \fP restores previous font.
> > What this means is that doing \fC\f[CR]Lorem\fP:
> >  (1) sets the text Lorem in Courier if font name C or CR exists
> >  (2) restores previous font if neither font C nor CR exists
> >  (3) sets font to Courier if both font name C and CR exist
> 
> What about \fC\f[CR]Lorem\fP\fP?

Unreliable.  As onf pointed out, not only is the font selection not
backed up by a stack, but it the identity of "the previous font" is not
well-defined if you attempt to select a non-existent font (as either or
both of `C` and `CR` may be).

https://lists.gnu.org/archive/html/groff/2022-11/msg00162.html

There simply is no portable way to change the font family (serif, sans
serif, monospaced) inline in the man(7) macro language.

And even if there were, such a change would not show up on terminals.

> >> Do I understand correctly that blank line is sometimes interpreted
> >> as vertical spacing and sometimes ignored?
> >
> > A blank input line is equivalent to .sp unless the .blm request was
> > called to change this behavior. .blm is groff's invention (see
> > groff_diff(7)).
> 
> Then maybe we can put .sp explicitly instead of a blank line.

I advise against it.  If what follows is a paragraph or a (sub)section
heading, then by calling the applicable macro, you will get any
necessary vertical space automatically.

Also, on typesetters, the vertical space between paragraphs and between
paragraphs and (sub)section headings, is not 1v, but 0.4v.  This has
been the case since McIlroy's original man(7) 45 years ago.

https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/lib/tmac/tmac.an

See the comment '# paragraph distance'.

> In our code, we add blank after blocks (or displays) of code.

As noted, the macro package will take care of this for you, while
leaving you the flexibility to set a code block _without_ immediately
subsequent vertical space.

> Extra vertical space after seems to be reasonable.

Also not reliable.  Some man(1) implementations, including man-db
man(1), "squeeze" multiple blank lines into one.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

"G. Branden Robinson" <g.branden.robinson@gmail.com> writes:

>> Then maybe we can put .sp explicitly instead of a blank line.
>
> I advise against it.  If what follows is a paragraph or a (sub)section
> heading, then by calling the applicable macro, you will get any
> necessary vertical space automatically.
>
> Also, on typesetters, the vertical space between paragraphs and between
> paragraphs and (sub)section headings, is not 1v, but 0.4v.  This has
> been the case since McIlroy's original man(7) 45 years ago.
>
> https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/lib/tmac/tmac.an
>
> See the comment '# paragraph distance'.

Ok.
I got rid of the blank lines in the output except if explicitly put
there by users (via export snippets or similar).
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=bc84ac7b26

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 6596 bytes --]

Hi Ihor,

At 2024-12-31T18:15:57+0000, Ihor Radchenko wrote:
> "G. Branden Robinson" <g.branden.robinson@gmail.com> writes:
> 
> > I understand now.  I've taken some time to learn a bit more about
> > Org mode, and also that org-man.el seems to be something close to
> > the bleeding edge; in a Debian unstable chroot it was not provided
> > by the version of GNU Emacs (29.4) nor the "elpa-org" package
> > (9.7.16).
> 
> You looked up wrong file.  Not org-man.el, but ox-man.el.

D'oh!  Thanks.

> It is a part of Org mode since forever (introduced before Org 4.12a -
> the first tagged release in our git repo - Jan 31, 2008; 16 years
> ago).
> 
> > So I'll make some recommendations unfortunately without the benefit
> > of having field-tested them.
> 
> You can do the following to try exporting to man from Emacs 29.4:
> 
> 1. emacs -Q
> 2. M-: (require 'ox-man) RET
> 3. Open a sample .org file
> 4. C-c C-e M m
> 
> (4) will produce a man file named after the .org file, in the same
> directory.

Thank you very much.  I'll test out my proposed changes locally.

> 1. We want to avoid breaking cases when \fC did work (it is a minimum)
> 2. Ideally, we want things to work in _more_ cases
> 3. Even more ideally, we want things to work in all the cases
> 
> If (1) is not possible, compromises will have to be made.

As noted in the other subthread, that is the case.  Terminal emulators
generally don't support rendering fonts from multiple families
simultaneously.  (3) is an unreachable goal with present terminal
emulator technology and for the foreseeable future.

> > Yes.  `EX`/`EE` works only with "displays".
> 
> May you elaborate about the meaning of "displays"?

Sure.

https://www.gnu.org/software/groff/manual/groff.html.node/Displays-and-Keeps.html

> > But it is worth noting that `EX`/`EE` is _more_ portable than `\fC`.
> 
> In non-inclusive manner though. (cases when \fC works are not a subset
> of cases where EX/EE works). Correct me if I am wrong.

It's an incomplete overlap, yes.  But it's easier to get `EX`/`EE`
support on systems that lack support for a font `C` than vice versa.

A document can define its own `EX` and `EE` macros if necessary.  We
have portable and permissively licensed implementations that we
encourage anyone to use.

https://git.savannah.gnu.org/cgit/groff.git/tree/tmac/an-ext.tmac?h=1.23.0

So ox-man.el might add the macro definitions in lines 162-172 and
178-186 to the preamble of the generated document (that is, immediately
after the `TH` call) to be _sure_ `EX` and `EE` will be available.

> > (I would also delete that trailing "\n".  Avoid blank lines in man(7)
> > input.[3])
> 
> Do I understand correctly that blank line is sometimes interpreted as
> vertical spacing and sometimes ignored?

Yes.  A blank line is ignored when the formatter's "no-space mode" is
enabled.

https://www.gnu.org/software/groff/manual/groff.html.node/Manipulating-Spacing.html#index-mode_002c-no_002dspace-_0028ns_0029

That's mainly for information.  Generally we discourage the use of *roff
requests in man(7) documents.  As groff_man_style(7) says:

   Portability
[...]
     The two major features that control formatting in the roff language
     are requests and escape sequences.  Since the man macros are
     implemented in terms of these, one can, in principle, supplement
     the functionality of man with these lower‐level elements where
     necessary.

     However, use of roff requests (apart from the empty request “.”)
     risks poor rendering when your page is processed by non‐roff
     formatters that attempt to interpret page sources.  (Historically,
     this was commonly attempted for HTML conversion.)  Requests may
     make assumptions that do not hold in an HTML environment.  Many of
     these programs don’t interpret the full roff language (let alone
     extensions): they may be incapable of handling numeric expressions,
     control structures, or register, string, and macro definitions.
     Such limitations can lead to portions of a document being presented
     incomprehensibly or omitted altogether.

     The wise man author quotes multi‐word section and subsection
     headings; the .SH and .SS macros of man(7) implementations
     descended from Seventh Edition Unix supported six arguments at
     most.  This restriction also applied to the .B, .I, .SM, and font
     style alternation macros.

> > (As another aside, the stroke color selection escape sequences `\m`
> > are (a) GNU troff extensions, albeit supported by the same
> > formatters that `\f[CR]` is, but (b) probably unnecessary because
> > nothing else in this entire file seems to use any other color for
> > anything.)
> 
> May black sometimes be different from the default color?

Yes, but it's impossible to know what the default color is; there is
interface for GNU troff to obtain this information from the output
device (and even if there were, due to device-independent troff's
multi-stage architecture, which groff also uses, there is no way[1] to
use that information to make decisions in the *roff(1) language altering
the document's formatting in response to it.

Also, explicitly selecting `\m[black]` as you are is going to backfire
on terminals using a black background--the text will become invisible!

(If you/your users haven't seen that often, or ever, it's likely because
some environments, including man(1) programs, if aware of groff, arrange
to disable the output of SGR escape sequences from groff's output driver
for terminals, grotty(1).  So the good news is, on those systems, the
problem I just described won't happen.  The bad news is, _all_ of the
`\m` and `\M` escape sequences you produce will be completely ignored.)

> > You can test a generated man(7) document by asking groff(1) and
> > man(7) to lint it.
> >
> > $ groff -ww -rCHECKSTYLE=3 -man my-org-doc.man
> 
> Thanks! This might be a good thing to add to our tests.

I forgot to mention the `-z` option.  You'll want that, too.  That will
suppress ordinary output so you get nothing _but_ diagnostic messages.

Regards,
Branden

[1] No _straightforward_ way.  In light of recent discussions I've had
    with Deri James, I'd be remiss if I didn't mention device extension
    commands.  But using these to get the formatter's idea of the stroke
    color deliberately out of sync with the output device's does _not_
    seem like a promising line of development to me.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[onf]

On Thu Jan 2, 2025 at 1:38 PM CET, G. Branden Robinson wrote:
> > > (I would also delete that trailing "\n".  Avoid blank lines in man(7)
> > > input.[3])
> > 
> > Do I understand correctly that blank line is sometimes interpreted as
> > vertical spacing and sometimes ignored?
>
> Yes.  A blank line is ignored when the formatter's "no-space mode" is
> enabled.
>
> [...]

Just to clarify, .sp is ignored in such cases as well.

~ onf

Re: [BUG] "\fC" macro in ox-man.el [9.6.15 (release_9.6.15 @ /usr/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

"G. Branden Robinson" <g.branden.robinson@gmail.com> writes:

>>  ...
>> (4) will produce a man file named after the .org file, in the same
>> directory.
>
> Thank you very much.  I'll test out my proposed changes locally.

Note that the official Org mode repository is on savannah.
See https://orgmode.org/worg/org-contribute.html

>> > (As another aside, the stroke color selection escape sequences `\m`
>> > are (a) GNU troff extensions, albeit supported by the same
>> > formatters that `\f[CR]` is, but (b) probably unnecessary because
>> > nothing else in this entire file seems to use any other color for
>> > anything.)
>> 
>> May black sometimes be different from the default color?
>
> Yes, but it's impossible to know what the default color is; there is
> interface for GNU troff to obtain this information from the output
> device (and even if there were, due to device-independent troff's
> multi-stage architecture, which groff also uses, there is no way[1] to
> use that information to make decisions in the *roff(1) language altering
> the document's formatting in response to it.
>
> Also, explicitly selecting `\m[black]` as you are is going to backfire
> on terminals using a black background--the text will become invisible!
> ...

I dropped force-setting black color now.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=db995c8003

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>