problem displaying latin accents with emacs and mac os X

problem displaying latin accents with emacs and mac os X

[Pierre Albarede]

Hello.

There are many problems with non ASCII characters in emacs. I would like 
to share my findings and I also hope to get some help about a little bug 
of my own, please.

I use enhanced-carbon emacs (GNU Emacs 21.3.50.4) on mac os 
X.2.4(powerpc-apple-darwin6.2, kind of BSD unix).

I do not use x windows.

C-h h is an easy way to see how the emacs-mule (multi-lingual 
environment) system performs. For example, many Asian characters work 
fine on my system.

For more detail (jump to \end if you are in a hurry):

\begin{M-x mule says}

###############################################
### Current Status of Multilingual Features ###
###############################################

CONTENTS: Section 1.  General Information
           Section 2.  Display
           Section 3.  Input methods
           Section 4.  Coding systems
           Section 5.  Character sets
           Section 6.  Fontsets

########################################
# Section 1.  General Information
########################################

Version of this emacs:
   GNU Emacs 21.3.50.4 (powerpc-apple-darwin6.2)
  of 2002-12-01 on marx.local.

Configuration options:
   '--with-carbon' '--without-x' 
'--prefix=/Applications/Emacs.app/Contents/Resources'

Multibyte characters awareness:
   default: t
   current-buffer: t

Current language environment: French

########################################
# Section 2.  Display
########################################

Window-system: mac1

Coding system of the terminal: iso-latin-1

########################################
# Section 3.  Input methods
########################################
[...]
French
   french-alt-postfix (`FR<' in mode line)
     French (FranÁais) input method with postfix modifiers
   french-azerty (`AZ@' in mode line)
     French (FranÁais) input method simulating Azerty keyboard
   french-keyboard (`FR@' in mode line)
     French (FranÁais) input method simulating some French keyboard
   french-postfix (`FR<' in mode line)
     French (FranÁais) input method with postfix modifiers
   french-prefix (`FR>' in mode line)
     French (FranÁais) input method with prefix modifiers
[...]
UTF-8
   TeX (`\' in mode line)
     LaTeX-like input method for many characters.
   rfc1345 (`m' in mode line)
     Unicode characters input method using RFC1345 mnemonics (non-ASCII 
only).
   sgml (`&' in mode line)
     Unicode characters input method using SGML entities.
[...]
Default input method: latin-1-prefix
########################################
# Section 4.  Coding systems
########################################

#########################
## LIST OF CODING SYSTEMS
## Each line corresponds to one coding system
[...]
iso-latin-1,iso-8859-1,latin-1:2:1:3:ascii,latin-iso8859-1,-1,-1,0,0,0,0,0,0,0,0,0:ISO 
2022 based 8-bit encoding for Latin-1 (MIME:ISO-8859-1).
[...]
mac-roman:4:M:3: decode-mac-roman,encode-mac-roman:Mac Roman Encoding 
(MIME:MACINTOSH).
[...]
############################
## LIST OF CODING CATEGORIES (ordered by priority)
## CATEGORY:CODING-SYSTEM
##
coding-category-iso-8-1:iso-latin-1
coding-category-iso-8-2:iso-latin-1
coding-category-iso-7-tight:iso-2022-jp
coding-category-iso-7:iso-2022-7bit
coding-category-iso-7-else:iso-2022-7bit-lock
coding-category-iso-8-else:iso-2022-8bit-ss2
coding-category-emacs-mule:emacs-mule
coding-category-raw-text:raw-text
coding-category-sjis:japanese-shift-jis
coding-category-big5:chinese-big5
coding-category-ccl:nil
coding-category-binary:no-conversion
coding-category-utf-16-be:nil
coding-category-utf-16-le:nil
coding-category-utf-8:mule-utf-8

########################################
# Section 5.  Character sets
########################################

#########################
## LIST OF CHARSETS
[...]
244:mule-unicode-0100-24ff:2:96:4:1:0:49:0:Unicode characters of the 
range U+0100..U+24FF.
243:mule-unicode-e000-ffff:2:96:4:1:0:51:0:Unicode characters of the 
range U+E000..U+FFFF.
242:mule-unicode-2500-33ff:2:96:4:1:0:50:0:Unicode characters of the 
range U+2500..U+33FF.
[...]
129:latin-iso8859-1:1:96:2:1:0:65:1:Right-Hand Part of Latin Alphabet 1 
(ISO/IEC 8859-1): ISO-IR-100.
000:ascii:1:94:1:1:0:66:0:ASCII (ISO646 IRV)
[...]

########################################
# Section 6.  Fontsets
########################################

Fontset-Name						  WDxHT Style
------------						  ----- -----
Fontset: -apple-courier-medium-r-normal--14-*-*-*-*-*-fontset-courier
CHARSET or CHAR RANGE	FONT NAME
---------------------	---------
ascii			-apple-courier-medium-r-normal--14-140-75-75-m-140-mac-roman
latin-iso8859-1	 
-apple-courier-medium-r-normal--14-140-75-75-m-140-mac-roman
[...]
mule-unicode-2500-33ff	-*-iso10646-1
mule-unicode-e000-ffff	-*-iso10646-1
mule-unicode-0100-24ff	-*-iso10646-1
[...]
Fontset: -etl-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-mac
CHARSET or CHAR RANGE	FONT NAME
---------------------	---------
ascii			-apple-monaco-medium-r-normal--12-120-75-75-m-120-mac-roman
latin-iso8859-1
   †-?			-*-monaco-*-mac-roman
latin-iso8859-2		-*-iso8859-2
latin-iso8859-3		-*-iso8859-3
latin-iso8859-4		-*-iso8859-4
thai-tis620		-*-*-*-tis620-*
greek-iso8859-7		-*-*-*-iso8859-7
arabic-iso8859-6	-*-*-*-iso8859-6
hebrew-iso8859-8	-*-*-*-iso8859-8
katakana-jisx0201	-*-*-*-jisx0201-*
latin-jisx0201		-*-jisx0201-*
cyrillic-iso8859-5	-*-*-*-iso8859-5
latin-iso8859-9		-*-iso8859-9
[...]
Fontset: -*-*-*-*-*-*-*-*-*-*-*-*-fontset-default
CHARSET or CHAR RANGE	FONT NAME
---------------------	---------
ascii			-apple-monaco-medium-r-*--*-120-*-*-*-*-mac-roman
      [-apple-courier-medium-i-normal--16-154-75-75-m-0-mac-roman]
      [-apple-courier-bold-r-normal--16-154-75-75-m-0-mac-roman]
      [-apple-courier-medium-r-normal--16-154-75-75-m-0-mac-roman]
latin-iso8859-1		-*-iso8859-1
latin-iso8859-2		-*-iso8859-2
latin-iso8859-3		-*-iso8859-3
latin-iso8859-4		-*-iso8859-4

\end{M-x mule says}

Another diagnosis  :

\begin{C-h C says}
Coding system for saving this buffer:
   1 -- iso-latin-1-unix

Default coding system (for new files):
   1 -- iso-latin-1 (alias: iso-8859-1 latin-1)

Coding system for keyboard input:
   1 -- iso-latin-1 (alias: iso-8859-1 latin-1)

Coding system for terminal output:
   1 -- iso-latin-1 (alias: iso-8859-1 latin-1)

Defaults for subprocess I/O:
   decoding: 1 -- iso-latin-1 (alias: iso-8859-1 latin-1)

   encoding: 1 -- iso-latin-1 (alias: iso-8859-1 latin-1)


Priority order for recognizing coding systems when reading files:
   1. iso-latin-1 (alias: iso-8859-1 latin-1)
   2. iso-2022-jp (alias: junet)
   3. iso-2022-7bit
   4. iso-2022-7bit-lock (alias: iso-2022-int-1)
   5. iso-2022-8bit-ss2
   6. emacs-mule
   7. raw-text
   8. japanese-shift-jis (alias: shift_jis sjis)
   9. chinese-big5 (alias: big5 cn-big5)
   10. no-conversion
   11. mule-utf-8 (alias: utf-8)

   Other coding systems cannot be distinguished automatically
   from these, and therefore cannot be recognized automatically
   with the present coding system priorities.

   The following are decoded correctly but recognized as iso-2022-7bit-lock:
     iso-2022-7bit-ss2 iso-2022-7bit-lock-ss2 iso-2022-cn iso-2022-cn-ext
     iso-2022-jp-2 iso-2022-kr

Particular coding systems specified for certain file names:

   OPERATION	TARGET PATTERN		CODING SYSTEM(s)
   ---------	--------------		----------------
   File I/O	"\\.elc\\'"		(emacs-mule . emacs-mule)
		"\\.utf\\(-8\\)?\\'"	utf-8
		"\\(\\`\\|/\\)loaddefs.el\\'"
					(raw-text . raw-text-unix)
		"\\.tar\\'"		(no-conversion . no-conversion)
		"\\.po[tx]?\\'\\|\\.po\\."
					po-find-file-coding-system
		""			(undecided)
   Process I/O	nothing specified
   Network I/O	nothing specified

[back]

\end{C-h C says}

emacs is clever enough to configure itself from the environment variable 
LANG=fr_FR (to see it: M-x shell setenv).

All should be fine to write English and French in the iso-latin-1, 
iso-latin-9, mac-roman or utf-8 codings. (I am happy with only 
iso-latin-1, since I have conversion tools outside emacs - I can comment 
on that if you need.)

You can shift to any language preference with
M-x set-language-environment
and check with
M-: current-language-environment
and most variables will be set accordingly.

As emacs is not a converter, a buffer file coding system cannot be 
changed after the file has been found. But you can casually override the 
default file coding system with
C-x RET c universal-coding-system-argument
and check with
M-: buffer-file-coding-system
This might be necessary for a file containing some alien chars.

C-x RET k set-keyboard-coding-system
M-:keyboard-coding-system

C-x RET t...

Character palettes :
M-x list-character-sets

All these commands are very practical and not often mentioned.

---

Now my problem.

The only font displaying iso8859-1 accented characters is the default 
font, of size 14, definitely too small for my 12'' ibook display and my 
eyes.

;;default font: accents OK but size too small
(font . "-*-*-*-*-*--*-*-*-*-*-*-*")

Default font is (M-x describe-font)
-apple-courier-medium-r-normal--14-140-75-75-m-140-mac-roman

;;size OK but accents losts
;;(font . "-apple-courier-medium-r-normal--16-160-75-75-m-160-mac-roman")

Considering that fontsets are based on fonts, I have experimented only 
on fonts.

Other fonts (M-x set-frame-font) are either not better, undefined or 
display garbage.

There is no iso8859-1 font. Yet the default font (mac-roman) can display 
both mac-roman and iso8859-1.

After Reading The Fucking Manual, I have concluded that I missed GNU 
intl fonts. However, I cannot install them, because they rely on x 
ressources.

Thus, I am still looking for a way to display accented iso8859-1 
characters in size 16, and hope to be finished by Christmas, with your 
kind help :=))

Re: problem displaying latin accents with emacs and mac os X

[Peter Ilberg]

Pierre Albarede <albanews@free.fr> writes:
> There is no iso8859-1 font. Yet the default font (mac-roman) can
> display both mac-roman and iso8859-1.
> 
> After Reading The Fucking Manual, I have concluded that I missed GNU
> intl fonts. However, I cannot install them, because they rely on x
> ressources.

I've had the same problem and found the following entry in the
FAQ at http://members.shaw.ca/akochoi-emacs/FAQ.txt quite helpful.
Just change the font to whichever one you prefer, because 9pt
Monaco is really small. You have to use one of the fonts that show
up in the completion list for set-frame-font and which have
non-zero size (as the entry points out). 16pt Courier mightn't be
in that list.

---

** How do I create my own fontset?

To properly display latin-1 characters using 9-pt Monaco, do this:

(create-fontset-from-fontset-spec
"-apple-monaco-medium-r-normal--9-*-*-*-*-*-fontset-monaco,
ascii:-apple-monaco-medium-r-normal--9-90-75-75-m-90-mac-roman,
latin-iso8859-1:-apple-monaco-medium-r-normal--9-90-75-75-m-90-mac-roman")

and set-frame-font to "fontset-monaco".

You can also use any other font that has non-zero size in the list
returned by the function x-list-fonts.

-- Peter Ilberg <peter.ilberg@ni.com>

Re: problem displaying latin accents with emacs and mac os X

[Erik Simon]

GNU Intl fonts, that's right. But they have been converted for Mac OS
X.

Direct link:
<http://members.shaw.ca/akochoi-emacs/Resources/GNU-Fonts.smi.bin>

See <http://members.shaw.ca/akochoi-emacs/>, section Resources at the
bottom.

Works on my Powerbook with exactly the same versions as you.

Cheers
Erik

Re: problem displaying latin accents with emacs and mac os X

[Pierre Albarede]

Thanks a lot to Peter.

My mistake was not to look into fontset. I was afraid it would not work 
because I do not have x, but it does. This sounds logical, since this is 
emacs, not xemacs.

The mac has no iso8859-1 native font. emacs takes glyphs from mac-roman 
fonts to compose a iso8859-1 fontset.

The largest possible monaco size is 14, but, as monaco is bigger than 
courier, this is just large enough for me.

To get it,

(create-fontset-from-fontset-spec
"-apple-monaco-medium-r-normal--14-*-*-*-*-*-fontset-monaco,
ascii:-apple-monaco-medium-r-normal--14-*,
latin-iso8859-1:-apple-monaco-medium-r-normal--14-*")

M-x set-frame-font fontset-monaco

or, in .emacs,

(when (and (boundp carbon-emacsp) carbon-emacsp)
   (setq default-frame-alist '((width . 80)
			      (height . 38)
			      (font . "fontset-monaco")
))
   (setq initial-frame-alist default-frame-alist)
)

-apple-courier does not exist either in size between 14 (too small) and 
18 (too big)

emacs needs fixed-sized
available are andale-mono courier courier new monaco VT100
(check in textedit police menu rather than emacs)
none in size 16
some in size 0
I am too lazy to try size 0.

M-x list-fontsets
Fontset: -apple-courier-medium-r-normal--14-*-*-*-*-*-fontset-courier
Fontset: -*-*-*-*-*-*-*-*-*-*-*-*-fontset-default
Fontset: -etl-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-mac
Fontset: -apple-monaco-medium-r-normal--14-*-*-*-*-*-fontset-monaco

fontset-mac is misleading since there are no etl fonts on the mac.

etl fonts are in the GNU intl fonts package, but install needs some x* 
commands that I do not have.

I somebody wants to discuss this, I am curious. Good bye.

Re: problem displaying latin accents with emacs and mac os X

[Pierre Albarede]

Oups, I missed Simon while offline. I check this. Thanks.

Re: problem displaying latin accents with emacs and mac os X

[Pierre Albarede]

Erik Simon wrote:
 > Direct link:
 > <http://members.shaw.ca/akochoi-emacs/Resources/GNU-Fonts.smi.bin>

I get the fonts. They have the right icon in the finder. I place
them in /Library/fonts

The font menu (in textedit) does not see them. Reboot. Again, the font 
menu does not see them. But emacs does. (Rebooting might have been 
unnecessary.)

M-x set-frame-font
-etl-fixed-medium-r-normal--16-160-75-75-m-160-iso8859-1

Perfect size and French accents :=)

M-x set-fontset-mac

C-h h
European scripts, including Cyrillic and Greek, look good.

Yet size is not 16 but 12.

M-x describe-fontset

Fontset: -etl-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-mac
CHARSET or CHAR RANGE	FONT NAME
---------------------	---------
ascii			-apple-monaco-medium-r-normal--12-120-75-75-m-120-mac-roman
      [-apple-monaco-bold-r-normal--12-116-75-75-m-0-mac-roman]
      [-apple-monaco-medium-r-normal--12-120-75-75-m-120-mac-roman]
latin-iso8859-1
   †-?			-*-monaco-*-mac-roman
latin-iso8859-2		-*-iso8859-2
latin-iso8859-3		-*-iso8859-3
latin-iso8859-4		-*-iso8859-4
thai-tis620		-*-*-*-tis620-*
greek-iso8859-7		-*-*-*-iso8859-7
arabic-iso8859-6	-*-*-*-iso8859-6
hebrew-iso8859-8	-*-*-*-iso8859-8
katakana-jisx0201	-*-*-*-jisx0201-*
latin-jisx0201		-*-jisx0201-*
cyrillic-iso8859-5	-*-*-*-iso8859-5
latin-iso8859-9		-*-iso8859-9
latin-iso8859-15	-*-iso8859-15
latin-iso8859-14	-*-iso8859-14
[...]

Probably fontset-mac was defined before etf were installed. I need to 
rebuild something. What?