8-bit characters input

8-bit characters input

[Marco De Vitis]

Hi, I know this is a FAQ, but I've browsed for hours through the docs 
without finding any real solution.

I can't input 8-bit chars in Emacs when I'm on the command line; 
especially, I need to type Italian accented chars (e.g. àèìòù). I simply 
want that an "è" appears when I press the "è" key; I want to avoid any 
deadkeys and such.
Currently, instead, when I type one of those chars, it seems like some 
control code is inserted (e.g. when I press "à" the cursor moves down 
some lines).

This applies to various situations:
- when logging locally on a Debian Etch machine (Emacs 21.4.1, locale 
en_US.UTF-8);
- when using locally a MacOS X Terminal (Emacs 21.2.1, locale it_IT.UTF-8);
- when logging remotely on that same Debian machine from the same OSX 
Terminal using ssh;
- when logging remotely on that same Debian machine from a Windows XP 
machine using PuTTy.

In all those situations, I can happily input and view 8-bit chars on the 
bash command line, e.g. when reading files with less. But they stop 
showing when I use Emacs.
To say the truth, they don't even work in other command line text 
editors such as vi or pico, so it might not be a strictly-emacs problem, 
but Emacs is my editor of choice in this situations so I need to fix it 
in there.
On the other hand, if I launch X on the Debian machine and start Emacs 
in the GUI environment, accented chars work fine.

By reading the docs, I tried setting some of the suggested options for 
similar problems, to no avail. I suspect that Emacs' docs regarding this 
subject are generally oriented towards more "exotic" configurations and 
needs than mine.

Does anyone here have any clues on how to fix this problem? It would be 
very useful for me. Thanks.

-- 
Ciao,
   Marco.

Re: 8-bit characters input

[Eli Zaretskii]

> From: Marco De Vitis <starless@spin.it>
> Date: Fri, 06 Apr 2007 16:10:22 GMT
> 
> I can't input 8-bit chars in Emacs when I'm on the command line; 
> especially, I need to type Italian accented chars (e.g. àèìòù). I simply 
> want that an "è" appears when I press the "è" key; I want to avoid any 
> deadkeys and such.
> Currently, instead, when I type one of those chars, it seems like some 
> control code is inserted (e.g. when I press "à" the cursor moves down 
> some lines).

Does the following command help?

  C-x RET k latin-1 RET

Re: 8-bit characters input

[Marco De Vitis]

On 06/04/2007 19:38, Eli Zaretskii wrote:

> Does the following command help?
> 
>   C-x RET k latin-1 RET

Thanks but no, nothing changes.

-- 
Ciao,
   Marco.

Re: 8-bit characters input

[Peter Dyballa]

Am 06.04.2007 um 16:10 schrieb Marco De Vitis:

> - when using locally a MacOS X Terminal (Emacs 21.2.1, locale  
> it_IT.UTF-8);

This Apple product is good for 7 bit US-ASCII. Get yourself a Carbon  
Emacs (based on GNU Emacs 22.0.95). Launching it as "/Applications/ 
Emacs.app/Contents/MacOS/Emacs -nw" in Terminal allows you to use the  
keyboard as in any other Mac OS X application.

--
Greetings

   Pete

What’s the difference between OS X and Vista?

Microsoft employees are excited about OS X…

Re: 8-bit characters input

[Marco De Vitis]

On 06/04/2007 23:36, Peter Dyballa wrote:

> This Apple product is good for 7 bit US-ASCII. Get yourself a Carbon 
> Emacs (based on GNU Emacs 22.0.95). Launching it as 
> "/Applications/Emacs.app/Contents/MacOS/Emacs -nw" in Terminal allows 
> you to use the keyboard as in any other Mac OS X application.

Well, thanks, but actually the OSX reference was just for completeness: 
when editing text on OSX I generally use Textmate (and also have 
Aquamacs installed if I need a more integrated Emacs flavour).

My real problem is on the Debian machine where I connect for work.
Any ideas about that?

-- 
Ciao,
   Marco.

Re: 8-bit characters input

[Stefan Monnier]

> I can't input 8-bit chars in Emacs when I'm on the command line; especially,
> I need to type Italian accented chars (e.g. àèìòù). I simply want that an
> "è" appears when I press the "è" key; I want to avoid any deadkeys and such.
> Currently, instead, when I type one of those chars, it seems like some
> control code is inserted (e.g. when I press "à" the cursor moves down some
> lines).

Check C-h l after hitting à to see what char sequence was really sent
to Emacs.  You'll need both the set Emacs's keyboard-coding-system correctly
(typically to utf-8 or latin-1) and to let your terminal application send
the necessary info (among other things, you'll want it to use ESC to encode
the meta modifier (sometimes referred to as the alt modifier)).


        Stefan

Re: 8-bit characters input

[Marco De Vitis]

Il 09/04/2007 22:03, Stefan Monnier ha scritto:

> Check C-h l after hitting à to see what char sequence was really sent

It seems that a C-v is sent instead of à.

> to Emacs.  You'll need both the set Emacs's keyboard-coding-system correctly
> (typically to utf-8 or latin-1) and to let your terminal application send

I tried setting it to utf-8 or latin-1, using both "M-x customize" and 
"C-x RET k", but nothing changed.

> the necessary info (among other things, you'll want it to use ESC to encode
> the meta modifier (sometimes referred to as the alt modifier)).

Meta is already "assigned" to ESC here.

What options/variables/customization group would you suggest to look at?

-- 
Ciao,
   Marco.

Re: 8-bit characters input

[Stefan Monnier]

>> Check C-h l after hitting à to see what char sequence was really sent
> It seems that a C-v is sent instead of à.

Is that all?  There's a good chance that hitting à sends more than one char.

>> to Emacs.  You'll need both the set Emacs's keyboard-coding-system correctly
>> (typically to utf-8 or latin-1) and to let your terminal application send

> I tried setting it to utf-8 or latin-1, using both "M-x customize" and "C-x
> RET k", but nothing changed.

Yes, it looks like right now the problem is in the terminal
configuration itself.

>> the necessary info (among other things, you'll want it to use ESC to encode
>> the meta modifier (sometimes referred to as the alt modifier)).

> Meta is already "assigned" to ESC here.

> What options/variables/customization group would you suggest to look at?

What terminal did you try this with?


        Stefan

Re: 8-bit characters input

[Marco De Vitis]

Il 11/04/2007 5:44, Stefan Monnier ha scritto:

>> It seems that a C-v is sent instead of à.
> 
> Is that all?  There's a good chance that hitting à sends more than one char.

If I just open a new file ("emacs test"), press à, and then press C-h l, 
here is what I see in the Help buffer:

C-v C-h l

> Yes, it looks like right now the problem is in the terminal
> configuration itself.

But the problem happens in two different remote terminals (and also 
locally), and the same 8-bit chars work fine on the command line in 
those same terminals...

> What terminal did you try this with?

OSX Terminal, which is configured as xterm-color.

I now tried also from PuTTy (configured as xterm) on a Windows machine, 
and hitting à appears to send different stuff:

M-C M-SPC C-h l

I currently can't try logging in locally, I'm far from the machine. But 
I can tell you that, while in the remote terminals I get an "End of 
buffer" message in emacs when I press à, in the local terminal some 
chars where actually entered, I can't remember which ones, but I think 
they were a variant of the upper case A plus a symbol, each time I 
pressed à.

Thanks for your help.

-- 
Ciao,
   Marco.

Re: 8-bit characters input

[Peter Dyballa]

Am 11.04.2007 um 08:59 schrieb Marco De Vitis:

>> What terminal did you try this with?
>
> OSX Terminal, which is configured as xterm-color.

Use on Mac OS X a different version of GNU Emacs than the Apple  
supplied one! GNU Emacs versions 22 and 23 work better.

--
Greetings

   Pete

"engineer: a mechanism for converting caffeine into designs"

Re: 8-bit characters input

[Stefan Monnier]

>>> It seems that a C-v is sent instead of à.
>> Is that all?  There's a good chance that hitting à sends more than one char.
> If I just open a new file ("emacs test"), press à, and then press C-h l,
> here is what I see in the Help buffer:

> C-v C-h l

Hmm... that's odd.. And that's with OSX's Terminal.app?
I get full utf-8 coding there, without having changed any part of the config.

>> Yes, it looks like right now the problem is in the terminal
>> configuration itself.

> But the problem happens in two different remote terminals (and also
> locally), and the same 8-bit chars work fine on the command line in those
> same terminals...

>> What terminal did you try this with?
> OSX Terminal, which is configured as xterm-color.

I'm not sure what you mean by "configured as xterm-color".

> I now tried also from PuTTy (configured as xterm) on a Windows machine, and
> hitting à appears to send different stuff:

> M-C M-SPC C-h l

That looks much better.  Actually it looks almost like utf-8.  Have you
tried C-x RET k utf-8 RET when logged in this way?


        Stefan

Re: 8-bit characters input

[Marco De Vitis]

Il 11/04/2007 16:14, Stefan Monnier ha scritto:

>> C-v C-h l
> 
> Hmm... that's odd.. And that's with OSX's Terminal.app?
> I get full utf-8 coding there, without having changed any part of the config.

Yes, OSX's Terminal.app.
I actually have customized the Terminal config somehow a long time ago, 
to be able to correctly input and read utf-8 in Terminal itself...

> I'm not sure what you mean by "configured as xterm-color".

I mean that OSX's Terminal is set, through its preferences, to declare 
the terminal type as xterm-color. Or, in other words, that the env var 
$TERM is set to "xterm-color".

>> I now tried also from PuTTy (configured as xterm) on a Windows machine, and
>> hitting à appears to send different stuff:
> 
>> M-C M-SPC C-h l
> 
> That looks much better.  Actually it looks almost like utf-8.  Have you
> tried C-x RET k utf-8 RET when logged in this way?

I did now, and indeed it works!
And then... it also works in Terminal.app, as long as I open 
Terminal.app's window preferences and deactivate the "Escape non-ASCII 
chars" option in the Emulation section; indeed, when this option is not 
active, pressing "à" in emacs sends the same codes which were sent using 
PuTTy.

So, I can now use 8-bit chars in emacs by putting 
(set-keyboard-coding-system 'utf-8)
in my ~/.emacs file, and deactivating that Terminal option when I'm on 
OSX; the only problem is that I need that option to correctly input 
8-bit chars in Terminal's "normal" usage, i.e. when using it locally, 
without connecting to a remote machine... I'll try to live with it.

Thanks a lot.

-- 
Ciao,
   Marco.

Re: 8-bit characters input

[Stefan Monnier]

>>> C-v C-h l
>> 
>> Hmm... that's odd.. And that's with OSX's Terminal.app?
>> I get full utf-8 coding there, without having changed any part of the config.

> Yes, OSX's Terminal.app.
> I actually have customized the Terminal config somehow a long time ago, to
> be able to correctly input and read utf-8 in Terminal itself...

Hmm... for me it just worked out of the box.

>> I'm not sure what you mean by "configured as xterm-color".

> I mean that OSX's Terminal is set, through its preferences, to declare the
> terminal type as xterm-color. Or, in other words, that the env var $TERM is
> set to "xterm-color".

I see.  Then this setting should have no impact on this present problem.

> in my ~/.emacs file, and deactivating that Terminal option when I'm on OSX;
> the only problem is that I need that option to correctly input 8-bit chars
> in Terminal's "normal" usage, i.e. when using it locally, without connecting
> to a remote machine... I'll try to live with it.

I don't use OSX any more, so I can't check it out, but I've never needed
this "escape non-ascii chars" for anything.  I'd deactivate it and if you
encounter problems when using some other program in Terminal, try to figure
out how to adjust the configuration of that program.


        Stefan

Re: 8-bit characters input

[Marco De Vitis]

Il 13/04/2007 0:41, Stefan Monnier ha scritto:

> I don't use OSX any more, so I can't check it out, but I've never needed
> this "escape non-ascii chars" for anything.  I'd deactivate it and if you
> encounter problems when using some other program in Terminal, try to figure
> out how to adjust the configuration of that program.

When deactivating it, I could not type 8bit chars anymore in Terminal.app.

This is OT now, but I later played around with my customized 
.bash_profile (on OSX), it contained these two lines:

   stty cs8 -istrip -parenb
   bind 'set output-meta on'

I added them long time ago exactly for using 8bit chars in all Terminal 
apps, following some Usenet suggestions.
I now added this one:

   bind 'set convert-meta off'

...and everything seems to be fine now, without the need for the "Escape 
non-ASCII chars" option. Apart from the fact that now 8bit chars seem to 
be broken in the local OSX text-mode emacs, but I don't care about it, 
as I already said ;).

-- 
Ciao,
   Marco.

Re: 8-bit characters input

[Marco De Vitis]

Il 11/04/2007 13:46, Peter Dyballa ha scritto:

> Use on Mac OS X a different version of GNU Emacs than the Apple supplied 
> one! GNU Emacs versions 22 and 23 work better.

AHEM, Peter... :) thanks for your answer, but it's the second time in 
this thread that you misunderstand me: although I noticed that also my 
"local" emacs on OSX has the same behaviour, this is not my real 
problem, as I normally use Textmate for editing text on OSX.
My real problem is when I ssh to a remote Debian machine and use the 
emacs version which is installed on Debian.

-- 
Ciao,
   Marco.