encoding

encoding

[Alan]

I would like to ask for advice on encoding issues, and I apologize for
coming back to this list when I have asked questions about this in the
past.  I have gotten by for the most part, but every once in a while I
am bitten by an encoding issue.

By and large I have no need for multibyte encodings, but I do use
diacritic character modifications, and have an occasional need to use
greek characters, for example.  So I enabled UTF-8 encoding (without
really understanding the full extend of what is implied).

A the text output of a certain program includes the following.  I will
show it both as it appears in the terminal and as it appears in an
emacs buffer:

 - emacs:   19.6500° N, 156.0000° W

 - term:      19.6500° N, 156.0000° W

I am using emacs 23.1 (ubuntu intrepid ibex, emacs-snapshot).

The following is output from the command C-h C

-----------------------------
  U -- utf-8-unix (alias: mule-utf-8-unix)

Coding system for keyboard input:
  = -- no-conversion (alias: binary)

Coding system for terminal output:
  nil
Coding system for inter-client cut and paste:
  nil
Defaults for subprocess I/O:
  decoding: U -- utf-8-unix (alias: mule-utf-8-unix)

  encoding: U -- utf-8-unix (alias: mule-utf-8-unix)

---------------% end output--------------

The text in question is from subprocess I/O

The following are from my .emacs.el :

  (standard-display-8bit 160 255)
  (setq-default coding-system-for-write 'utf-8)

I have had trouble with text copied from a browser, which if I
remember correctly is why I implemented this default coding-system.

Can someone point the way to a simple explanation of the (seeming)
complexities of coding-systems for emacs?  Or suggest how I can set
this up?

Thank you

Alan

Re: encoding

[Eli Zaretskii]

> From: Alan <lngndvs@gmail.com>
> Date: Wed, 7 Jan 2009 16:35:51 -0800 (PST)
> 
> By and large I have no need for multibyte encodings, but I do use
> diacritic character modifications, and have an occasional need to use
> greek characters, for example.  So I enabled UTF-8 encoding (without
> really understanding the full extend of what is implied).

What do you mean by "enabled UTF-8 encoding"?  What did you put in
your .emacs, exactly?  Is there anything there besides the 2 lines you
show in your message (on which I comment below)?

> A the text output of a certain program includes the following.  I will
> show it both as it appears in the terminal and as it appears in an
> emacs buffer:

How did the output of that program get to an Emacs buffer? please
describe the procedure used to insert the program's output into Emacs.

>   (standard-display-8bit 160 255)

Remove this line, it will (or does already) cause you trouble.

>   (setq-default coding-system-for-write 'utf-8)

It's much better to say

  (prefer-coding-system 'utf-8)

> Can someone point the way to a simple explanation of the (seeming)
> complexities of coding-systems for emacs?

That complexity is supposed to be largely transparent in the default
setup. So the simple rule of thumb for dealing with this is not to
make any customizations you don't understand.  I would suggest as the
first step to remove everything you added to your .emacs for any
encoding related problems, and see if any problems persist.  If they
do, come up here and explain them; it's quite possible that the
solutions you found are incorrect, and are now causing more trouble.

Encoding

[sirciny]

I recently changed the default Ubuntu font in my ~/.emacs file to DejaVu Sans
Mono-10 using the following:

(set-face-attribute 'default nil :font "DejaVu Sans Mono-10")

Now, when I edit certain .txt files, I get the following error when Emacs
autosaves:

These default coding systems were tried to encode text
in the buffer ` *Format Temp 0*':
  (emacs-mule-unix (216366 . 157))
However, each of them encountered characters it couldn't encode:
  emacs-mule-unix cannot encode these: 

Click on a character (or switch to this window by `C-x o'
and select the characters by RET) to jump to the place it appears,
where `C-u C-x =' will give information about it.

Select one of the safe coding systems listed below,
or cancel the writing with C-g and edit the buffer
   to remove or modify the problematic characters,
or specify any other coding system (and risk losing
   the problematic characters).

  utf-16le-with-signature utf-8 gb18030 utf-7 utf-16
  utf-16be-with-signature utf-16be utf-16le utf-8-auto
  utf-8-with-signature utf-7-imap utf-8-emacs

---------------------------------------

I have tried setting my default (en?)coding system to utf-8 via the
following:

  (set-language-environment 'utf-8)

  (setq locale-coding-system 'utf-8)
  (set-default-coding-systems 'utf-8)
  (set-terminal-coding-system 'utf-8)
  (set-selection-coding-system 'utf-8)
  (prefer-coding-system 'utf-8)

--------------------------------------

None of the above has worked when placed in ~/.emacs and after restarting
Emacs. How can I get rid of this error and keep my new font (and avoid the
problem if I change my font attributes in the future)?



--
View this message in context: http://emacs.1067599.n5.nabble.com/Encoding-tp288388.html
Sent from the Emacs - Help mailing list archive at Nabble.com.

Re: Encoding

[Peter Dyballa]

Am 11.06.2013 um 00:47 schrieb sirciny:

> How can I get rid of this error and keep my new font (and avoid the
> problem if I change my font attributes in the future)?

There is no connection between the font you've chosen and the encoding issue. If a font is missing the glyphs for characters used in some (exotic?) encoding, then replacement characters, open boxes, are used for them.

The mode-line of a buffer contains in the left corner an indicator for the encoding used.

GNU Emacs learns from environment variables which encoding to use or to prefer, for example LANG or LC_CTYPE. Try to set these! Then you won't need to set anything in GNU Emacs.

You could check GNU Emacs' behaviour when you launch it with -q or -Q and when LANG and/or LC_CTYPE are set to a reasonable value.

--
Greetings

  Pete

Upgraded, adj.:
	Didn't work the first time.

Re: Encoding

[sirciny]

This is what my .emacs file now contains:

(prefer-coding-system 'utf-16-le)

;; EXPERIMENTAL: Set language environment
;;(setq current-language-environment 'latin-1)
;;(setenv 'latin-1)
;;(setenv LANG en_US.8859-1)

(set-language-environment "Latin-1")
(set-keyboard-coding-system 'latin-1)
(set-terminal-coding-system 'latin-1)

I also tried to M-x setenv using LANG and en_US.8859-1, but after doing so
and evaluating this region of my ~/.emacs, I still get the error in my
original post when I autosave. I'm also not sure what to set LC_CTYPE to, or
how to set it... It doesn't auto-complete with setenv, so I assume there's a
different function responsible for changing its value.



--
View this message in context: http://emacs.1067599.n5.nabble.com/Character-encoding-problem-tp288388p288506.html
Sent from the Emacs - Help mailing list archive at Nabble.com.

encoding

[rinefez]

My Emacs 23.2.1 in inferior-lisp entering (from italian keyboard)

#\à

returns

#\LATIN_CAPITAL_LETTER_A_WITH_TILDE

which is wrong cause i entered latin small a with grave accent. In
fact entering

(char-code #\à)

returns 195 (the code point of Ã) instead of 224 (the code point of
à).

Anybody has a clue to how to solve this?

Re: encoding

[Pascal J. Bourguignon]

rinefez <mmj529@gmail.com> writes:

> My Emacs 23.2.1 in inferior-lisp entering (from italian keyboard)
>
> #\à
>
> returns
>
> #\LATIN_CAPITAL_LETTER_A_WITH_TILDE
>
> which is wrong cause i entered latin small a with grave accent. In
> fact entering
>
> (char-code #\à)
>
> returns 195 (the code point of Ã) instead of 224 (the code point of
> à).
>
> Anybody has a clue to how to solve this?

Set the same encoding system for emacs, slime and the inferior lisp.

http://common-lisp.net/project/slime/doc/html/Emacs_002dside-customization.htm

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: encoding

[rinefez]

> Set the same encoding system for emacs, slime and the inferior lisp.
>
> http://common-lisp.net/project/slime/doc/html/Emacs_002dside-customiz...
>
> --
> __Pascal Bourguignon__                    http://www.informatimago.com/
> A bad day in () is better than a good day in {}.


Thanks!
There is a little typo in the link, the correct one is

http://common-lisp.net/project/slime/doc/html/Emacs_002dside-customization.html

Re: encoding

[rinefez]

On Mar 13, 3:07 am, "Pascal J. Bourguignon" <p...@informatimago.com>
wrote:
> rinefez <mmj...@gmail.com> writes:
> > My Emacs 23.2.1 in inferior-lisp entering (from italian keyboard)
>
> > #\
>
> > returns
>
> > #\LATIN_CAPITAL_LETTER_A_WITH_TILDE
>
> > which is wrong cause i entered latin small a with grave accent. In
> > fact entering
>
> > (char-code #\ )
>
> > returns 195 (the code point of ) instead of 224 (the code point of
> > ).
>
> > Anybody has a clue to how to solve this?
>
> Set the same encoding system for emacs, slime and the inferior lisp.
>
> http://common-lisp.net/project/slime/doc/html/Emacs_002dside-customiz...
>
> --
> __Pascal Bourguignon__                    http://www.informatimago.com/
> A bad day in () is better than a good day in {}.

I tried that, adding the following lines to my emacs, but it didn't
work, I still have the same problem:

(setq inferior-lisp-program "c:/home/bin/clisp/full/lisp.exe -B c:/
home/bin/clisp/full -M c:/home/bin/clisp/full/lispinit.mem -E utf-8 -
q"); my Lisp system
...
(setq slime-net-coding-system 'utf-8-unix) ; sets the coding system of
communications between SLIME and Lisp
(setq locale-coding-system 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)
(set-default-coding-systems 'utf-8)
(prefer-coding-system 'utf-8)
(set-language-environment "UTF-8")
(swank:create-server :coding-system "utf-8-unix")

Any help is appreciated. Thanks.

Re: encoding

[Pascal J. Bourguignon]

rinefez <mmj529@gmail.com> writes:

> On Mar 13, 3:07 am, "Pascal J. Bourguignon" <p...@informatimago.com>
> wrote:
>> rinefez <mmj...@gmail.com> writes:
>> > My Emacs 23.2.1 in inferior-lisp entering (from italian keyboard)
>>
>> > #\
>>
>> > returns
>>
>> > #\LATIN_CAPITAL_LETTER_A_WITH_TILDE
>>
>> > which is wrong cause i entered latin small a with grave accent. In
>> > fact entering
>>
>> > (char-code #\ )
>>
>> > returns 195 (the code point of ) instead of 224 (the code point of
>> > ).
>>
>> > Anybody has a clue to how to solve this?
>>
>> Set the same encoding system for emacs, slime and the inferior lisp.
>>
>> http://common-lisp.net/project/slime/doc/html/Emacs_002dside-customiz...
>
> I tried that, adding the following lines to my emacs, but it didn't
> work, I still have the same problem:
>
> (setq inferior-lisp-program "c:/home/bin/clisp/full/lisp.exe -B c:/
> home/bin/clisp/full -M c:/home/bin/clisp/full/lispinit.mem -E utf-8 -
> q"); my Lisp system
> ...
> (setq slime-net-coding-system 'utf-8-unix) ; sets the coding system of
> communications between SLIME and Lisp
> (setq locale-coding-system 'utf-8)
> (set-terminal-coding-system 'utf-8)
> (set-keyboard-coding-system 'utf-8)
> (set-selection-coding-system 'utf-8)
> (set-default-coding-systems 'utf-8)
> (prefer-coding-system 'utf-8)
> (set-language-environment "UTF-8")
> (swank:create-server :coding-system "utf-8-unix")

This is what I have.


> Any help is appreciated. Thanks.

Either ask to slime developers, or try to debug it, tracing the
characters at the various stages.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: encoding

[rinefez]

On Mar 13, 3:07 am, "Pascal J. Bourguignon" <p...@informatimago.com>
wrote:
> rinefez <mmj...@gmail.com> writes:
> > My Emacs 23.2.1 in inferior-lisp entering (from italian keyboard)
>
> > #\
>
> > returns
>
> > #\LATIN_CAPITAL_LETTER_A_WITH_TILDE
>
> > which is wrong cause i entered latin small a with grave accent. In
> > fact entering
>
> > (char-code #\ )
>
> > returns 195 (the code point of ) instead of 224 (the code point of
> > ).
>
> > Anybody has a clue to how to solve this?
>
> Set the same encoding system for emacs, slime and the inferior lisp.
>
> http://common-lisp.net/project/slime/doc/html/Emacs_002dside-customiz...
>
> --
> __Pascal Bourguignon__                    http://www.informatimago.com/
> A bad day in () is better than a good day in {}.

I tried that, adding the following lines to my emacs, but it didn't
work, I still have the same problem:

(setq inferior-lisp-program "c:/home/bin/clisp/full/lisp.exe -B c:/
home/bin/clisp/full -M c:/home/bin/clisp/full/lispinit.mem -E utf-8 -
q"); my Lisp system
...
(setq slime-net-coding-system 'utf-8-unix) ; sets the coding system of
communications between SLIME and Lisp
(setq locale-coding-system 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)
(set-default-coding-systems 'utf-8)
(prefer-coding-system 'utf-8)
(set-language-environment "UTF-8")
(swank:create-server :coding-system "utf-8-unix")

Any help is appreciated. Thanks.

Encoding..

[Rob Kramer]

Hi,

I'm doing some testing with a program that needs to be made aware of UTF-8.
Now, when I cut & paste a good UTF-8 byte string to some other document
(my .cc file to be precise), that string is converted to something else
when I save and check with od.

How can I disable all forms of multi-language encoding/converting so I can
test in peace without fear of my strings becoming translated to something
else?

Cheers!

    Rob

Re: Encoding..

[Kevin Rodgers]

Rob Kramer wrote:
 > I'm doing some testing with a program that needs to be made aware of UTF-8.
 > Now, when I cut & paste a good UTF-8 byte string to some other document
 > (my .cc file to be precise), that string is converted to something else
 > when I save and check with od.
 >
 > How can I disable all forms of multi-language encoding/converting so I can
 > test in peace without fear of my strings becoming translated to something
 > else?

Start Emacs with the --unibyte option or set the EMACS_UNIBYTE
environment variable (see the "Initial Options" and "Enabling Multibyte"
nodes of the Emacs manual); or

Set selection-coding-system to 'no-conversion or 'raw-text (see the
"Specify Coding" node of the Emacs manual).

-- 
Kevin Rodgers

Re: Encoding..

[Kai Grossjohann]

Rob Kramer <robk@starhub.net.sg> writes:

> I'm doing some testing with a program that needs to be made aware of UTF-8.
> Now, when I cut & paste a good UTF-8 byte string to some other document
> (my .cc file to be precise), that string is converted to something else
> when I save and check with od.

Perhaps it is easiest to tell Emacs to save as UTF-8, by doing C-x
RET f utf-8 RET.

Kai