Problems in Displaying endash in Emacs-w3m

Problems in Displaying endash in Emacs-w3m

[Charles philip Chan]

Hello all:

In my attempt to have better integration with planner-mode, I have
switched to using the cvs version of emacs-w3m. However, I have problems
displaying endash (&#x2013) in xhtml documents (for example those on
http://www.emacswiki.org). My emacs-w3m coding system is set to
utf-8. The strange thing is that if I do a "C-uC-x=" on the misdisplayed
symbol, this is the output I get:
  
,----[ Output from Emacs ]
|   character: (102585, #o310271, #x190b9)
|     charset: chinese-cns11643-1
| 	     (CNS11643 Plane 1 Chinese traditional: ISO-IR-171.)
|  code point: #x21 #x39
|      syntax: w 	which means: word
|    category: C:Chinese (Han) characters of 2-byte character sets c:Chinese
| 	     |:While filling, we can break a line at this character.
| buffer code: #x95 #xA1 #xB9
|   file code: #xE2 #x80 #x93 (encoded by coding system utf-8)
|     display: no font available
`----

I am using emacs cvs. How would I go about solving this?

Thanks.

Charles

-- 
printk(KERN_WARNING "%s: Thanks, I feel much better now!\n", dev->name);
        linux-2.6.6/drivers/net/de620.c

Re: Problems in Displaying endash in Emacs-w3m

[Kevin Rodgers]

Charles philip Chan wrote:
> Hello all:
> 
> In my attempt to have better integration with planner-mode, I have
> switched to using the cvs version of emacs-w3m. However, I have problems
> displaying endash (&#x2013) in xhtml documents (for example those on
> http://www.emacswiki.org). My emacs-w3m coding system is set to
> utf-8. The strange thing is that if I do a "C-uC-x=" on the misdisplayed
> symbol, this is the output I get:
>   
> ,----[ Output from Emacs ]
> |   character: (102585, #o310271, #x190b9)
> |     charset: chinese-cns11643-1
> | 	     (CNS11643 Plane 1 Chinese traditional: ISO-IR-171.)
> |  code point: #x21 #x39
> |      syntax: w 	which means: word
> |    category: C:Chinese (Han) characters of 2-byte character sets c:Chinese
> | 	     |:While filling, we can break a line at this character.
> | buffer code: #x95 #xA1 #xB9
> |   file code: #xE2 #x80 #x93 (encoded by coding system utf-8)
> |     display: no font available
> `----
> 
> I am using emacs cvs. How would I go about solving this?

That's strange, CVS Emacs usually displays the Unicode code point as
well, e.g.

   character: é (2281, #o4351, #x8e9, U+00E9)
     charset: latin-iso8859-1
	     (Right-Hand Part of Latin Alphabet 1 (ISO/IEC 8859-1): ISO-IR-100.)
  code point: #x69
      syntax: w 	which means: word
    category: l:Latin
buffer code: #x81 #xE9
   file code: #xC3 #xA9 (encoded by coding system mule-utf-8-dos)
     display: by this font (glyph code)
      -outline-Courier 
New-normal-r-normal-normal-13-97-96-96-c-*-iso8859-1 (#xE9)

I suggest asking for help on emacs-w3m@namazu.org

-- 
Kevin

Re: Problems in Displaying endash in Emacs-w3m

[Charles philip Chan]

On  3 Oct 2006, ihs_4664@yahoo.com wrote:

> That's strange, CVS Emacs usually displays the Unicode code point as
> well, e.g.

It does on other characters that I have tried, but not the endash. I
forget to mention that I am using an iso10646-1 font as my default
font. Now it look more and more like a bug in CVS Emacs.

> I suggest asking for help on emacs-w3m@namazu.org

I did, but I got no answer.

Thanks.

Charles

-- 
LILO, you've got me on my knees!
(from David Black, dblack@pilot.njin.net, with apologies to Derek and the
Dominos, and Werner Almsberger)

Re: Problems in Displaying endash in Emacs-w3m

[Peter Dyballa]

Am 03.10.2006 um 20:12 schrieb Charles philip Chan:

> It does on other characters that I have tried, but not the endash. I
> forget to mention that I am using an iso10646-1 font as my default
> font. Now it look more and more like a bug in CVS Emacs.

No. When I launch GNU Emacs 22.0.50 with -Q and look into an UTF-8  
file from Kermit I get:

   character: – (342387, #o1234563, #x53973, U+2013)
     charset: mule-unicode-0100-24ff (Unicode characters of the range  
U+0100..U+24FF.)
code point: #x72 #x73
      syntax: _ 	which means: symbol
buffer code: #x9C #xF4 #xF2 #xF3
   file code: #xE2 #x80 #x93 (encoded by coding system mule-utf-8-unix)
     display: by this font (glyph code)
      -B&H-LucidaTypewriter-Medium-R-Normal-Sans-10-100-75-75-M-60- 
ISO10646-1 (#x2013)

which looks OK. How is the output when you launch GNU Emacs 22.0.50  
with -Q?


Check particularly the "buffer code!" Yours is different. I am using  
a simple text buffer, mode-line starts with ``-u:´´. (Although I've  
seen ``-E:´´ also – which is a bug in GNU Emacs 22.0.50: the first  
UTF-8 encoded file it opens it does *not* open in UTF-8, but in  
Japanese, at least for me).

--
Greetings

   Pete

The most exciting phrase to hear in science, the one that heralds new  
discoveries, is not "Eureka!" (I found it!) but "That's funny..."
                                       Isaac Asimov

Re: Problems in Displaying endash in Emacs-w3m

[Charles philip Chan]

On  4 Oct 2006, Peter_Dyballa@web.de wrote:

> which looks OK. How is the output when you launch GNU Emacs 22.0.50
> with -Q?

This is really strange, the endash shows fine in a text buffer, but not
emacs-w3m. I know the w3m buffer is in utf-8 mode since I can see Asian
characters (and the "-u" in the modeline). Can you please check a web
page such as http://www.emacswiki.org with emacs-w3m for me and see if
you have the same problem.

Charles

-- 
/* XXX: where the fuck is ->f_vfsmnt? */
        linux-2.6.6/fs/intermezzo/vfs.c

Re: Problems in Displaying endash in Emacs-w3m

[Peter Dyballa]

Am 04.10.2006 um 13:24 schrieb Charles philip Chan:

> Can you please check a web page such as http://www.emacswiki.org  
> with emacs-w3m for me and see if you have the same problem.

Sorry, I can't: I do not use emacs-w3m and I do not have it installed.

--
Greetings

   Pete

Globalisation -- communism from above.

Re: Problems in Displaying endash in Emacs-w3m

[Dieter Wilhelm]

"Charles philip Chan" <cpchan@sympatico.ca> writes:

> On  4 Oct 2006, Peter_Dyballa@web.de wrote:
>
>> which looks OK. How is the output when you launch GNU Emacs 22.0.50
>> with -Q?
>
> This is really strange, the endash shows fine in a text buffer, but not
> emacs-w3m. I know the w3m buffer is in utf-8 mode since I can see Asian
> characters (and the "-u" in the modeline). Can you please check a web
> page such as http://www.emacswiki.org with emacs-w3m for me and see if
> you have the same problem.

I've Emacs 22.050 and w3m installed but I don't know what you mean by
an endash.  Where do I've to look?

>
> Charles

-- 
    Best wishes

    H. Dieter Wilhelm
    Darmstadt, Germany

Re: Problems in Displaying endash in Emacs-w3m

[Peter Dyballa]

Am 07.10.2006 um 15:24 schrieb Dieter Wilhelm:

> I've Emacs 22.050 and w3m installed but I don't know what you mean by
> an endash.  Where do I've to look?

Look here: http://de.wikipedia.org/wiki/Bindestrich

endash is short –, emdash is longer: —, an average dash is this: -.

--
Mit friedvollen Grüßen

   Pete

Upgraded:  Didn't work the first time.

Re: Problems in Displaying endash in Emacs-w3m

[Dieter Wilhelm]

Peter Dyballa <Peter_Dyballa@Web.DE> writes:

> Am 07.10.2006 um 15:24 schrieb Dieter Wilhelm:
>
>> I've Emacs 22.050 and w3m installed but I don't know what you mean by
>> an endash.  Where do I've to look?
>
> Look here: http://de.wikipedia.org/wiki/Bindestrich

Thank you for the pointer.  Maybe this is related to the complaint of
Charles.  W3m displays on this site the hyphen (Bindestrich in German,
which AFAIU is inserted with the hyphen (-) on the keyboard) longer
then the dash (Gedankenstrich in German, used also when omitting text)
but it should be the other way around (As e.g. Firefox does
correctly).

>
> endash is short –, emdash is longer: —, an average dash is this: -.

For me under Gnus endash looks the shortest, emdash and the average
dash the same.

Would you please tell us how you input these characters?  I only know
of this iso-accents-mode for iso-latin-1 for inserting some non-ascii
characters, but the mode is supposed to be obsolete because of some
input methods I don't have any clue.

-- 
    Best wishes

    H. Dieter Wilhelm
    Darmstadt, Germany

Re: Problems in Displaying endash in Emacs-w3m

[Peter Dyballa]

Am 07.10.2006 um 21:26 schrieb Dieter Wilhelm:

> Thank you for the pointer.  Maybe this is related to the complaint of
> Charles.  W3m displays on this site the hyphen (Bindestrich in German,
> which AFAIU is inserted with the hyphen (-) on the keyboard) longer
> then the dash (Gedankenstrich in German, used also when omitting text)
> but it should be the other way around (As e.g. Firefox does
> correctly).

It's not Firefox: it's the font used. Any programme can only use what  
it gets served. If the font is badly designed, you won't these  
subtleties right. (BTW, our German Gedankenstrich is the endash, in  
the U.S.A. it's the emdash; and while we put spaces around it the  
Americans save these.) If you can change the font Firefox to be the  
same as in GNU Emacs it will make the same mistake – but usually  
Firefox will follow the design of the Web page, so would need to make  
your decision override the designer's one, too. Or simply do the  
change in GNU Emacs ... just for this!

I copied the three dashes off the box I was displaying in Camino, a  
more Mac OS X/Aqua/Cocoa version than Firebox (both have problems to  
display text correctly, at least in Mac OS X). With OmniWeb I was  
able to look into the source of the WikiPedia article: they are using  
the right characters at the right spots (and what you see on screen  
or on paper is a glyph: the shape and face of a character from a  
particular font). So we really might have to wait for a detailed  
answer from Charles where exactly he encountered the problem.

>
> Would you please tell us how you input these characters?  I only know
> of this iso-accents-mode for iso-latin-1 for inserting some non-ascii
> characters, but the mode is supposed to be obsolete because of some
> input methods I don't have any clue.

The different dashes have all their Unicode slots:

[c]  Uni   octal  name
------------------------------
[-]  002D  000055 HYPHEN-MINUS
[‒]  2012  020022 FIGURE DASH
[–]  2013  020023 EN DASH
[—]  2014  020024 EM DASH
[﹘]  FE58  177130 SMALL EM DASH

When you (set read-quoted-char-radix 16), which is usually 8, you can  
directly input C-q 2 0 1 2 RET.

The iso-whatsoever modes are completely inadequate in Unicode  
display, but you probably know.

(The FIGURE DASH is meant to be as wide as a digit and to be used for  
example in telephone numbers: Hawaii 5—O, so that these can stay  
printed in long uniform columns.)


I've found that I can install w3m with Fink on my Mac. This has  
happened by now, and now I'll have to learn a bit about browsing the  
Internet inside GNU Emacs! Crazy.


--
Greetings

   Pete

If you're not confused, you're not paying attention.

Re: Problems in Displaying endash in Emacs-w3m

[Reiner Steib]

On Sun, Oct 08 2006, Peter Dyballa wrote:

> When you (set read-quoted-char-radix 16), which is usually 8, you
> can directly input C-q 2 0 1 2 RET.

`M-x ucs-insert RET' is an alternative.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Problems in Displaying endash in Emacs-w3m

[Charles philip Chan]

On  7 Oct 2006, dieter@duenenhof-wilhelm.de wrote:

> I've Emacs 22.050 and w3m installed but I don't know what you mean by
> an endash.  Where do I've to look?

Here is the scoop about endash:

     http://en.wikipedia.org/wiki/Endash#En_dash

For an example page with endash, try:

    http://www.emacswiki.org

Charles

-- 
printk(KERN_ERR "%s: Something Wicked happened! %4.4x.\n",...);
        linux-2.6.6/drivers/net/sundance.c

Re: Problems in Displaying endash in Emacs-w3m

[Peter Dyballa]

Am 08.10.2006 um 18:54 schrieb Charles philip Chan:

> Here is the scoop about endash:
>
>      http://en.wikipedia.org/wiki/Endash#En_dash
>
> For an example page with endash, try:
>
>     http://www.emacswiki.org

I can assure you that in the October version of (Japanese) Carbon  
Emacs (Package) the dashes are coded correctly. And C-u C-x =  
displays something like the right values:

       character: – (342387, #o1234563, #x53973, U+2013)
         charset: mule-unicode-0100-24ff (Unicode characters of the  
range U+0100..U+24FF.)
      code point: #x72 #x73
          syntax: _ 	which means: symbol
     buffer code: #x9C #xF4 #xF2 #xF3
       file code: #xE2 #x80 #x93 (encoded by coding system utf-8)
         display: by this font (glyph code)
      -apple-monaco-medium-r-normal--9-90-72-72-m-90-mac-roman (#xD0)
    Unicode data:
            Name: EN DASH
        Category: dash punctuation
Combining class: Spacing
   Bidi category: Other Neutrals


The decimal, octal, and hexadecimal values are always strange in GNU  
Emacs 21 and 22. Can you check what coding system is displayed for  
you in mode-line? I have "-u:" which stands for UTF-8. Do you have  
set in your .emacs file some non-UTF-8 *-coding-system?

How is it when you launch your Emacs with -Q? For the test you can  
create a small init file that extends the load-path to find w3m,  
requires w3m, and which you can load from the command line with '-l  
<your init file>'.

--
Greetings

   Pete

The future will be much better tomorrow.
                                -- George W. Bush

Re: Problems in Displaying endash in Emacs-w3m

[Charles philip Chan]

On  8 Oct 2006, Peter_Dyballa@web.de wrote:

> The decimal, octal, and hexadecimal values are always strange in GNU
> Emacs 21 and 22. Can you check what coding system is displayed for
> you in mode-line? I have "-u:" which stands for UTF-8. Do you have
> set in your .emacs file some non-UTF-8 *-coding-system?

Yes, all my buffers are in utf-8 (I do have the "-u" in my modline). The
strange thing is that the n-dashes are displayed correctly in ordinary
Emacs buffers, I only have problems in Emacs-w3m. Are the n-dashes
showing for you at http://www.emacswiki.org, for example?

Charles

-- 
# Okay, what on Earth is this one supposed to be used for?
        linux-2.4.0/drivers/char/cp437.uni

Re: Problems in Displaying endash in Emacs-w3m

[Peter Dyballa]

Am 09.10.2006 um 03:19 schrieb Charles philip Chan:

> On  8 Oct 2006, Peter_Dyballa@web.de wrote:
>
>> The decimal, octal, and hexadecimal values are always strange in GNU
>> Emacs 21 and 22. Can you check what coding system is displayed for
>> you in mode-line? I have "-u:" which stands for UTF-8. Do you have
>> set in your .emacs file some non-UTF-8 *-coding-system?
>
> Yes, all my buffers are in utf-8 (I do have the "-u" in my  
> modline). The
> strange thing is that the n-dashes are displayed correctly in ordinary
> Emacs buffers, I only have problems in Emacs-w3m. Are the n-dashes
> showing for you at http://www.emacswiki.org, for example?
>

Well, I cannot say that I *see* them, but at least I can i-search for  
these characters! In this block I can find them right after the first  
word embedded in SPACE (but not in "Emacs-Wiki":

   • SiteMap – This page – the main Emacs-Wiki entry point.
   • HowTo – How to use the Emacs Wiki – especially, how to contribute.
   • Search – Search the wiki in different ways.
   • ElispArea – Upload and download EmacsLisp source code for  
extending and customizing
     Emacs.
   • RecentChanges – Recent changes to the wiki.
   • News – Chronological Web log (blog) about the wiki.
   • Problems – Problems people are having with the wiki.
   • Suggestions – Your suggestions for improvement of the wiki.


(The endashes are saved here in Mail window!) W3m is an awkward mode!  
It destroys all my Meta key bindings. And when I then copy text via  
the Edit menu it makes Carbon Emacs beep. (In the X clients Emacsen  
it behaves better!)


Charles, try to get the file utf8.txt (part of Kermit distribution)  
from, for example, http://ftp.funet.fi/pub/kermit/charsets (or  
directly http://ftp.funet.fi/pub/kermit/charsets/utf8.txt). It's a  
description of the first 64 K characters in Unicode, i.e. its BMP,  
the Basic Multilingual Plane. You can search for 2013 and then  
position the cursor on [–] in the first column and then with C-s C-w  
RET make isearch remember to search for – next time you type C-s C-s.


Again: you don't lose anything when you launch a second GNU Emacs  
with -Q, but you gain one that is working, that is not confused by  
any personal or site specific settings. When this one works fine, as  
it should, then it's likely that some customisation is causing the  
behaviour you complain. To find out which line is the culprit you'll  
have to try the method of binary search in .emacs: first comment  
first half, save, and launch a new Emacs. If the behaviour is still  
faulty kill this Emacs, uncomment the first half, and comment the  
second half instead, and save. Launch new Emacs. If it runs fine it  
means that the culprit is in the commented second half. So uncomment  
first half of the second half (i.e. third quarter of .emacs), save,  
and launch another Emacs. Continue until you've found the line. If  
some instructions are inside of blocks, as I have, it can become  
complicated ...

Do you set-language-environment in .emacs? Try to avoid it! It's  
meant for 7 for 8 bit folks. It's better to have LANG or LC_CTYPE set  
in shell environment.

--
Greetings

   Pete

The day Microsoft makes something that doesn't suck
is the day they start selling vacuum cleaners.
                                     Ernest Jan Plugge

Re: Problems in Displaying endash in Emacs-w3m

[Jim Ottaway]

"Charles philip Chan" <cpchan@sympatico.ca> writes:

> On  8 Oct 2006, Peter_Dyballa@web.de wrote:
>
>> The decimal, octal, and hexadecimal values are always strange in GNU
>> Emacs 21 and 22. Can you check what coding system is displayed for
>> you in mode-line? I have "-u:" which stands for UTF-8. Do you have
>> set in your .emacs file some non-UTF-8 *-coding-system?
>
> Yes, all my buffers are in utf-8 (I do have the "-u" in my modline). The
> strange thing is that the n-dashes are displayed correctly in ordinary
> Emacs buffers, I only have problems in Emacs-w3m. Are the n-dashes
> showing for you at http://www.emacswiki.org, for example?

Do you have w3m set up correctly? I mean the program, rather than the
Emacs mode. I remember that when I switched to utf-8 I had some problems
like this because I had previously set w3m up for latin-1.  You can test
this by visiting some page with problem characters using the w3m program
from a terminal.

Regards,

-- 
Jim Ottaway