bug#5566: 23.1.92; man page header ugly on narrow terminals

bug#5566: 23.1.92; man page header ugly on narrow terminals

[jidanni]

X-debbugs-cc: man-db@packages.debian.org, libwww-facebook-api-perl@packages.debian.org

Here is a complicated bug that involves several packages, and produces
"WWW::Facebook::API::^HUI^Hsn^Het^Hr..." (^H's visible too)
on one's emacs screen.

# su - nobody #(LC_ALL=C etc. too.)
$ HOME=/tmp emacs-snapshot -nw
M-x man WWW::Facebook::API::Intl (from libwww-facebook-api-perl package)
on a wide enough terminal, gives a nice
WWW::Facebook::API::Intl(3pm)        User Contributed Perl Documentation       WWW::Facebook::API::Intl(3pm)
however, if the screen is only as wide as the below, this is what one sees:
WWW::Facebook::API::^HUI^Hsn^Het^Hrl(^HC3^Hop^Hnm^Ht)^Hributed Perl Docu^H
Similarly under X-windows. If one just uses xterm and no emacs, one gets
WWW::Facebook::API:UserlContributed Perl DoWWW::Facebook::API::Intl(3pm)
which at least doesn't have the visible ^H stuff.
$ man -w WWW::Facebook::API::Friends
/usr/share/man/man3/WWW::Facebook::API::Friends.3pm.gz
where we see the line at question...
.TH WWW::Facebook::API::Friends 3pm "2009-11-26" "perl v5.10.1" "User Contributed Perl Documentation"

In GNU Emacs 23.1.92.1 (i486-pc-linux-gnu, GTK+ Version 2.18.6)
 of 2010-02-10 on elegiac, modified by Debian
 (emacs-snapshot package, version 1:20100209-1)

bug#5566: 23.1.92; man page header ugly on narrow terminals

[Colin Watson]

On Thu, Feb 11, 2010 at 11:26:48PM +0100, gregor herrmann wrote:
> On Fri, 12 Feb 2010 02:49:00 +0800, jidanni@jidanni.org wrote:
> > Here is a complicated bug that involves several packages, and produces
> > "WWW::Facebook::API::^HUI^Hsn^Het^Hr..." (^H's visible too)
> > on one's emacs screen.
> 
> Looks indeed ugly, but I fail to see what the
> libwww-facebook-api-perl package could do about it?!

Likewise, I agree that it's ugly, but I'm not convinced that man-db (or
groff, which is what's really responsible here) can do anything about
it.  If the header line just plain doesn't fit, it doesn't fit.

No doubt it would be possible to fix this by extending the .TH macro to
be able to declare some kind of fallback for use when the page width is
too short for the normal display, but, aside from the fact that I have
no idea where to start (this would be a groff upstream kind of thing),
I'm not sure that the substantial effort involved is worth it.  I'm sure
it doesn't actually cause any significant confusion.

Regarding emacs' odd ^H display, which I think is the meat of Dan's bug
report, this would appear to be essentially a bug in emacs.  grotty is
rendering the just-won't-fit text by way of backspacing over the first
part of the header and overstriking the middle part; it's within its
rights to do that.  I assume that whatever (lack of) terminal emulation
is used by M-x man isn't smart enough to handle this; perhaps it
special-cases the overstriking used for bold and underlining, and can't
cope with overstriking one character over a completely different
character?

-- 
Colin Watson                                       [cjwatson@debian.org]

bug#5566: 23.1.92; man page header ugly on narrow terminals

[Juri Linkov]

>> Looks indeed ugly, but I fail to see what the
>> libwww-facebook-api-perl package could do about it?!
>
> Likewise, I agree that it's ugly, but I'm not convinced that man-db (or
> groff, which is what's really responsible here) can do anything about
> it.  If the header line just plain doesn't fit, it doesn't fit.
>
> No doubt it would be possible to fix this by extending the .TH macro to
> be able to declare some kind of fallback for use when the page width is
> too short for the normal display, but, aside from the fact that I have
> no idea where to start (this would be a groff upstream kind of thing),
> I'm not sure that the substantial effort involved is worth it.  I'm sure
> it doesn't actually cause any significant confusion.
>
> Regarding emacs' odd ^H display, which I think is the meat of Dan's bug
> report, this would appear to be essentially a bug in emacs.  grotty is
> rendering the just-won't-fit text by way of backspacing over the first
> part of the header and overstriking the middle part; it's within its
> rights to do that.  I assume that whatever (lack of) terminal emulation
> is used by M-x man isn't smart enough to handle this; perhaps it
> special-cases the overstriking used for bold and underlining, and can't
> cope with overstriking one character over a completely different
> character?

I think this is not an Emacs bug.  I can reproduce it in xterm.

When xterm is narrowed to the 50-character width, running the shell
command `man HTML::TokeParser::Simple::Token::ProcessInstruction`
displays garbage like

^H^H^H^H^H^H^H^HHTML::TokeParser::Simple::Token::ProcessInstruction(3pm)ion(3pm)

where some random letters are highlighted in bold (where the character
before ^H coincide with the character after ^H).

What we can do in Emacs is simply to remove all ^H but we can't do the
broken header line more readable.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#5566: 23.1.92; man page header ugly on narrow terminals

[Juri Linkov]

> Here is a complicated bug that involves several packages, and produces
> "WWW::Facebook::API::^HUI^Hsn^Het^Hr..." (^H's visible too)
> on one's emacs screen.
>
> # su - nobody #(LC_ALL=C etc. too.)
> $ HOME=/tmp emacs-snapshot -nw
> M-x man WWW::Facebook::API::Intl (from libwww-facebook-api-perl package)
> on a wide enough terminal, gives a nice
> WWW::Facebook::API::Intl(3pm)        User Contributed Perl Documentation       WWW::Facebook::API::Intl(3pm)
> however, if the screen is only as wide as the below, this is what one sees:
> WWW::Facebook::API::^HUI^Hsn^Het^Hrl(^HC3^Hop^Hnm^Ht)^Hributed Perl Docu^H
> Similarly under X-windows. If one just uses xterm and no emacs, one gets
> WWW::Facebook::API:UserlContributed Perl DoWWW::Facebook::API::Intl(3pm)
> which at least doesn't have the visible ^H stuff.
> $ man -w WWW::Facebook::API::Friends
> /usr/share/man/man3/WWW::Facebook::API::Friends.3pm.gz
> where we see the line at question...

This looks like a bug in `man' from the package `man-db'.  When the
header line is longer than the preferred output width (defined by
`MANWIDTH' or `COLUMNS'), the header line becomes a mess where letters
from original header are interleaved with ^H characters.

This is reproducible with any sufficiently long manpage name
and a shell command like `COLUMNS=50 MAN_KEEP_FORMATTING=1
man HTML::TokeParser::Simple::Token::ProcessInstruction | cat -v`.

Emacs doesn't process such `man' output.  `Man-fontify-manpage' removes
^H's only when the character before ^H is the same as the character after ^H:

    (while (re-search-forward "\\(.\\)\\(\b+\\1\\)+" nil t)
      (replace-match "\\1")
      (put-text-property (1- (point)) (point) 'face Man-overstrike-face))

But in this case the characters before ^H and after ^H are different.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#5566: 23.1.92; man page header ugly on narrow terminals

[Colin Watson]

On Fri, Feb 12, 2010 at 09:20:18PM +0200, Juri Linkov wrote:
> This looks like a bug in `man' from the package `man-db'.

Er, no.  groff is responsible for this output; man merely calls it.
Dan, the reporter of this bug, has a habit of attributing just about
every problem related to manual pages to man-db, leaving me to spend
maintainer time redirecting him to the right place; but that doesn't
make it so, and it would be great if other people didn't pick up the
same bad habit. :-)

> When the header line is longer than the preferred output width
> (defined by `MANWIDTH' or `COLUMNS'), the header line becomes a mess
> where letters from original header are interleaved with ^H characters.

That's the overstriking I mentioned in my earlier comment on this bug
report, to which I refer you for more detail.  It's rather unlikely to
change; this is, I suppose, a legacy of *roff's history as a physical
typesetter.

The only thing I could usefully do in man would be to pass the -P -o
option to groff, which would then pass the -o option to grotty and thus
disable some of the overstriking, but this would still leave
apparently-stray characters lying around where spaces are overstruck
with non-spaces.  I don't think it's worth it given that such lines end
up being unintelligible to all intents and purposes anyway.  As I said
in my previous comment, which I encourage you to read:

  No doubt it would be possible to fix this by extending the .TH macro
  to be able to declare some kind of fallback for use when the page
  width is too short for the normal display, but, aside from the fact
  that I have no idea where to start (this would be a groff upstream
  kind of thing), I'm not sure that the substantial effort involved is
  worth it.  I'm sure it doesn't actually cause any significant
  confusion.

> Emacs doesn't process such `man' output.  `Man-fontify-manpage' removes
> ^H's only when the character before ^H is the same as the character after ^H:
> 
>     (while (re-search-forward "\\(.\\)\\(\b+\\1\\)+" nil t)
>       (replace-match "\\1")
>       (put-text-property (1- (point)) (point) 'face Man-overstrike-face))
> 
> But in this case the characters before ^H and after ^H are different.

As I mentioned before, emacs could improve its output and make it more
consistent with typical terminal emulators by displaying X^HY as Y when
X != Y.  But I don't think it's worth much in the way of argument.

Regards,

-- 
Colin Watson                                       [cjwatson@debian.org]

bug#5566: 23.1.92; man page header ugly on narrow terminals

[Juri Linkov]

>> Emacs doesn't process such `man' output.  `Man-fontify-manpage' removes
>> ^H's only when the character before ^H is the same as the character after ^H:
>>
>>     (while (re-search-forward "\\(.\\)\\(\b+\\1\\)+" nil t)
>>       (replace-match "\\1")
>>       (put-text-property (1- (point)) (point) 'face Man-overstrike-face))
>>
>> But in this case the characters before ^H and after ^H are different.
>
> As I mentioned before, emacs could improve its output and make it more
> consistent with typical terminal emulators by displaying X^HY as Y when
> X != Y.

From your explanation I gather that for legacy reasons it's unlikely
that the groff's output will be fixed any time soon.  So below is
a patch for man.el that removes these ^H:

=== modified file 'lisp/man.el'
--- lisp/man.el	2010-02-11 20:57:10 +0000
+++ lisp/man.el	2010-02-12 21:56:49 +0000
@@ -1105,6 +1105,11 @@ (defun Man-fontify-manpage ()
     (while (re-search-forward "[-|]\\(\b[-|]\\)+" nil t)
       (replace-match "+")
       (put-text-property (1- (point)) (point) 'face 'bold))
+    ;; When the header is longer than the manpage name, groff tries to
+    ;; condense it to a shorter line interspered with ^H.  Remove ^H with
+    ;; their preceding chars (but don't put Man-overstrike-face).  (Bug#5566)
+    (goto-char (point-min))
+    (while (re-search-forward ".\b" nil t) (backward-delete-char 2))
     (goto-char (point-min))
     ;; Try to recognize common forms of cross references.
     (Man-highlight-references)
@@ -1192,6 +1197,11 @@ (defun Man-cleanup-manpage (&optional in
 	))
   (goto-char (point-min))
   (while (re-search-forward "[-|]\\(\b[-|]\\)+" nil t) (replace-match "+"))
+  ;; When the header is longer than the manpage name, groff tries to
+  ;; condense it to a shorter line interspered with ^H.  Remove ^H with
+  ;; their preceding chars (but don't put Man-overstrike-face).  (Bug#5566)
+  (goto-char (point-min))
+  (while (re-search-forward ".\b" nil t) (backward-delete-char 2))
   (Man-softhyphen-to-minus)
   (message "%s man page cleaned up" Man-arguments))

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#5566: 23.1.92; man page header ugly on narrow terminals

[Chong Yidong]

Juri Linkov <juri@jurta.org> writes:

> From your explanation I gather that for legacy reasons it's unlikely
> that the groff's output will be fixed any time soon.  So below is
> a patch for man.el that removes these ^H:

Looks OK to me.

bug#5566: 23.1.92; man page header ugly on narrow terminals

[Juri Linkov]

>> From your explanation I gather that for legacy reasons it's unlikely
>> that the groff's output will be fixed any time soon.  So below is
>> a patch for man.el that removes these ^H:
>
> Looks OK to me.

Done.

-- 
Juri Linkov
http://www.jurta.org/emacs/