Man-mode showing bytes at line end positions

Man-mode showing bytes at line end positions

[Narendra Joshi]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

Hi,

`Man-mode' is showing bytes in man-pages. I checked with `emacs -Q` and I
see that the text for a man page isn't wrapped and this problem isn't
there. I don't have any special configuration for `Man-mode`. Can
someone point in the right direction for debugging what might be causing
this?

I have attached a screenshot of a man-page in `Man-mode` that
demonstrates the problem.


[-- Attachment #2: Man-mode showing bytes --]
[-- Type: image/png, Size: 115991 bytes --]

[-- Attachment #3: Type: text/plain, Size: 716 bytes --]


Enabled minor modes: Abbrev Async-Bytecomp-Package Auto-Composition
Auto-Compression Auto-Encryption Beginend-Global Blink-Cursor Column-Number
Company-Statistics Delete-Selection Diff-Auto-Refine Discover Display-Battery
Display-Time Electric-Indent Electric-Pair Eval-Sexp-Fu-Flash Eyebrowse
File-Name-Shadow Font-Lock Global-Auto-Revert Global-Company Global-Discover
Global-Eldoc Global-Flycheck Global-Font-Lock Global-Git-Commit
Global-Git-Gutter Ivy Keyfreq Keyfreq-Autosave Line-Number Magit-Auto-Revert
Menu-Bar Midnight Minibuffer-Depth-Indicate Mouse-Wheel Override-Global Recentf
Save-Place Savehist Shell-Dirtrack Show-Paren Tooltip Transient-Mark Which-Key
Yas Yas-Global


Thanks,
-- 
Narendra Joshi

Re: Man-mode showing bytes at line end positions

[Yuri Khan]

On Sun, Dec 17, 2017 at 6:48 PM, Narendra Joshi <narendraj9@gmail.com> wrote:

> `Man-mode' is showing bytes in man-pages. […]
>
> I have attached a screenshot of a man-page in `Man-mode` that
> demonstrates the problem.

Looks like a case of UTF-8 text erroneously decoded as if it were iso8859-1.

Re: Man-mode showing bytes at line end positions

[Narendra Joshi]

Yes, you are right I tried re-encoding the region and it was shown as the
dash character. Do you know how does Emacs pick the coding system for this
specific case? My default coding-system is set to utf-8.

Narendra Joshi
On 17 Dec 2017 17:31, "Yuri Khan" <yuri.v.khan@gmail.com> wrote:

> On Sun, Dec 17, 2017 at 6:48 PM, Narendra Joshi <narendraj9@gmail.com>
> wrote:
>
> > `Man-mode' is showing bytes in man-pages. […]
> >
> > I have attached a screenshot of a man-page in `Man-mode` that
> > demonstrates the problem.
>
> Looks like a case of UTF-8 text erroneously decoded as if it were
> iso8859-1.
>

Re: Man-mode showing bytes at line end positions

[Eli Zaretskii]

> From: Narendra Joshi <narendraj9@gmail.com>
> Date: Sun, 17 Dec 2017 18:25:01 +0530
> Cc: Help Gnu Emacs mailing list <help-gnu-emacs@gnu.org>
> 
> Yes, you are right I tried re-encoding the region and it was shown as the
> dash character. Do you know how does Emacs pick the coding system for this
> specific case? My default coding-system is set to utf-8.

Does it happen in "emacs -Q"?

Re: Man-mode showing bytes at line end positions

[Narendra Joshi]

Eli Zaretskii <eliz@gnu.org> writes:

>
> Does it happen in "emacs -Q"?
>
No, it doesn't. The text isn't wrapped to 80 columns with `emacs
-Q`. It's not clear to me what might be causing it to be limited to 80
columns. The character that's added to denote word continuations is
causing problems.

-- 
Narendra Joshi

Re: Man-mode showing bytes at line end positions

[Eli Zaretskii]

> From: Narendra Joshi <narendraj9@gmail.com>
> Cc: help-gnu-emacs@gnu.org
> Date: Mon, 18 Dec 2017 00:23:58 +0530
> 
> > Does it happen in "emacs -Q"?
> >
> No, it doesn't. The text isn't wrapped to 80 columns with `emacs
> -Q`.

Is that relevant to the issue at hand?  Are you saying that because of
a different wrapping, you don't see the non-ASCII hyphen characters at
all?  If so, set COLUMNS=80 in the environment, or play with the width
of your window/frame, to get the wrapping.

> It's not clear to me what might be causing it to be limited to 80
> columns.

The width of the window in which the man page will be displayed.

Re: Man-mode showing bytes at line end positions

[Narendra Joshi]

Narendra Joshi <narendraj9@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>
>> Does it happen in "emacs -Q"?
Yes, it's happening in "emacs -Q" for the man page of `fseek(3)`. 
I am using GNU Emacs 26.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.11) of 2017-12-10

-- 
Narendra Joshi

Re: Man-mode showing bytes at line end positions

[Eli Zaretskii]

> From: Narendra Joshi <narendraj9@gmail.com>
> Cc: help-gnu-emacs@gnu.org
> Date: Fri, 22 Dec 2017 21:32:16 +0530
> 
> Narendra Joshi <narendraj9@gmail.com> writes:
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> >>
> >> Does it happen in "emacs -Q"?
> Yes, it's happening in "emacs -Q" for the man page of `fseek(3)`. 
> I am using GNU Emacs 26.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.11) of 2017-12-10

Very strange, I cannot reproduce this on GNU/Linux.

What is the value of default-process-coding-system inside "emacs -Q"?

Re: Man-mode showing bytes at line end positions

[Eli Zaretskii]

> Date: Fri, 22 Dec 2017 18:29:03 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> What is the value of default-process-coding-system inside "emacs -Q"?

Actually, I see that the value of locale-coding-system is more
relevant to this case.  What is it inside "emacs -Q"?

Re: Man-mode showing bytes at line end positions

[Narendra Joshi]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Narendra Joshi <narendraj9@gmail.com>
>> Cc: help-gnu-emacs@gnu.org
>> Date: Fri, 22 Dec 2017 21:32:16 +0530
>> 
>> Narendra Joshi <narendraj9@gmail.com> writes:
>> 
>> > Eli Zaretskii <eliz@gnu.org> writes:
>> >
>> >>
>> >> Does it happen in "emacs -Q"?
>> Yes, it's happening in "emacs -Q" for the man page of `fseek(3)`. 
>> I am using GNU Emacs 26.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.11) of 2017-12-10
>
> Very strange, I cannot reproduce this on GNU/Linux.
>
> What is the value of default-process-coding-system inside "emacs -Q"?
The values are as follows:

default-process-coding-system
(iso-latin-1-unix . iso-latin-1-unix)


locale-coding-system
iso-latin-1-unix

Please help me understand what exactly is happening here.

Thanks,
-- 
Narendra Joshi

Re: Man-mode showing bytes at line end positions

[Eli Zaretskii]

> From: Narendra Joshi <narendraj9@gmail.com>
> Cc: help-gnu-emacs@gnu.org
> Date: Fri, 22 Dec 2017 23:22:12 +0530
> 
> The values are as follows:
> 
> default-process-coding-system
> (iso-latin-1-unix . iso-latin-1-unix)
> 
> 
> locale-coding-system
> iso-latin-1-unix

That's your problem, right there: your locale is Latin-1, but the man
pages were generated for a UTF-8 locale.  This mismatch is the root
cause of the problem you are having.

The solution for you would be to change your locale to be a UTF-8
locale, i.e. a locale which uses UTF-8 as its codeset.  Then Emacs
will default to UTF-8 when it decodes the man page, and you will see
the hyphen characters instead of binary bytes.

Re: Man-mode showing bytes at line end positions

[Narendra Joshi]

Thanks, that fixed it. :)

Narendra Joshi
On 22 Dec 2017 23:59, "Eli Zaretskii" <eliz@gnu.org> wrote:

> > From: Narendra Joshi <narendraj9@gmail.com>
> > Cc: help-gnu-emacs@gnu.org
> > Date: Fri, 22 Dec 2017 23:22:12 +0530
> >
> > The values are as follows:
> >
> > default-process-coding-system
> > (iso-latin-1-unix . iso-latin-1-unix)
> >
> >
> > locale-coding-system
> > iso-latin-1-unix
>
> That's your problem, right there: your locale is Latin-1, but the man
> pages were generated for a UTF-8 locale.  This mismatch is the root
> cause of the problem you are having.
>
> The solution for you would be to change your locale to be a UTF-8
> locale, i.e. a locale which uses UTF-8 as its codeset.  Then Emacs
> will default to UTF-8 when it decodes the man page, and you will see
> the hyphen characters instead of binary bytes.
>
>