non-ASCII characters in auto-save files

non-ASCII characters in auto-save files

[grivet]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

Hello,
I write a lot of text under WinXP/Emacs/Latex, in French. For some 
unknown reason, emacs sometimes crashes and
I am left with an auto-save file, say #myfile#.
My problem is that, in such a file, all non ASCII characters are 
mangled. I can open #myfile# in Emacs, but accented e's, for instance,
appear as \303\250 or as \201\250 and similarly for all accented letters.
If I try to save #myfile# to disk, Emacs telles me it cannot encode 
these characters. Here is the answer I get when I do C-uC-x = on one
of the offending chars:

c-u c-x =

         character:   (4194216, #o17777650, #x3fffa8)
preferred charset: eight-bit (Raw bytes 128-255)
        code point: 0xA8
            syntax: w     which means: word
       buffer code: #xA8
         file code: not encodable by coding system emacs-mule-unix
           display: no font available

I have two questions:
     - Can Emacs be persuaded to to encode auto-save files in a more 
useful manner ?
     - Is there a systematic way to convert #myfile# to iso-latin-9 (or 
iso-latin-1) code, other than painstakingly
searching and replacing offending chars ?

Thanks in advance for your time and help
JP Grivet



[-- Attachment #2: Type: text/html, Size: 2352 bytes --]

Re: non-ASCII characters in auto-save files

[Peter Dyballa]

Am 25.06.2012 um 14:46 schrieb grivet:

> I have two questions:
>    - Can Emacs be persuaded to to encode auto-save files in a more useful manner ?
>    - Is there a systematic way to convert #myfile# to iso-latin-9 (or iso-latin-1) code, other than painstakingly
> searching and replacing offending chars ?

I have one answer: use file-local variables!

For example in the header:

	%%% -*- mode: LaTeX; coding: iso-latin-9-unix; -*-

Or in the (AUCTeX) footer:

	%%% Local Variables:
	%%% mode: LaTeX
	%%% fill-column: 99999
	%%% coding: iso-latin-9
	%%% End:
	%

In the header you can combine the coding line with a time-stamp line, updated every time you save the file:

	%%%	Time-stamp: <2012-01-15 16:41:38 pete> 


With C-x RET c <encoding name> RET you can set an encoding to be used when you read and open the auto-save file. You can use the same command to set an encoding for saving the file.

--
Greetings

  Pete

There's no place like ~
			– (UNIX Guru)

Re: non-ASCII characters in auto-save files

[Michael Heerdegen]

Peter Dyballa <Peter_Dyballa@Web.DE> writes:

> Am 25.06.2012 um 14:46 schrieb grivet:
>
> > I have two questions:
> >    - Can Emacs be persuaded to to encode auto-save files in a more
> > useful manner ?
> >    - Is there a systematic way to convert #myfile# to iso-latin-9
> > (or iso-latin-1) code, other than painstakingly
> > searching and replacing offending chars ?
>
> I have one answer: use file-local variables!
>
> For example in the header:
>
> 	%%% -*- mode: LaTeX; coding: iso-latin-9-unix; -*-
>
> Or in the (AUCTeX) footer:
>
> 	%%% Local Variables:
> 	%%% mode: LaTeX
> 	%%% fill-column: 99999
> 	%%% coding: iso-latin-9
> 	%%% End:
> 	%

I'm no expert here, but - that doesn't work for autosave files (try
it!).  Looking at the code, it seems that autosaving handles coding
specially.  File local variables are respected when finding the autosave
file, but it is saved with a different encoding then the base file.

But grivet: Why do you have to open those autosave files?  I think the
standard way to use them is via `recover-file' or `recover-session'.
You shouldn't use them directly.


Michael.

Re: non-ASCII characters in auto-save files

[Peter Dyballa]

Am 26.06.2012 um 00:40 schrieb Michael Heerdegen:

> I'm no expert here, but - that doesn't work for autosave files (try
> it!).  Looking at the code, it seems that autosaving handles coding
> specially.  File local variables are respected when finding the autosave
> file, but it is saved with a different encoding then the base file.

That's right! File-local variables can only prevent encoding ambiguities. (With auto-save and other files.)

--
Mit friedvollen Grüßen

  Pete

The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."

Re: non-ASCII characters in auto-save files

[Stefan Monnier]

>     - Can Emacs be persuaded to to encode auto-save files in a more useful
> manner ?

No.  Auto-save files are encoded using Emacs's internal coding-system,
which makes the operation a lot more reliable and efficient (no need to
worry about the case where some char(s) can't be encoded, for example).

>     - Is there a systematic way to convert #myfile# to iso-latin-9 (or
> iso-latin-1) code, other than painstakingly searching and replacing
> offending chars ?

Just open the file with the right coding-system (typically, opening
"myfile" and then hitting M-x recover-this-file RET should do the
trick).


        Stefan

Re: non-ASCII characters in auto-save files

[grivet]

     - Is there a systematic way to convert #myfile# to iso-latin-9 (or
iso-latin-1) code, other than painstakingly searching and replacing
offending chars ?

> Just open the file with the right coding-system (typically, opening
> "myfile" and then hitting M-x recover-this-file RET should do the
> trick).
>
>
>          Stefan
#myfile# c&n be opened in this way but cannot be saved, as the coding 
system is not recognized.
JP Grivet

Re: non-ASCII characters in auto-save files

[Eli Zaretskii]

> Date: Tue, 26 Jun 2012 12:13:21 +0200
> From: grivet <grivet@cnrs-orleans.fr>
> 
> 
>      - Is there a systematic way to convert #myfile# to iso-latin-9 (or
> iso-latin-1) code, other than painstakingly searching and replacing
> offending chars ?

Yes.  

   C-x RET c emacs-internal RET C-x C-f #myfile# RET
   C-x RET f latin-9 RET
   C-x C-s

> > Just open the file with the right coding-system (typically, opening
> > "myfile" and then hitting M-x recover-this-file RET should do the
> > trick).
> >
> >
> >          Stefan
> #myfile# c&n be opened in this way but cannot be saved, as the coding 
> system is not recognized.

You should open myfile, not #myfile#.  Then Emacs will automatically
notice that the auto-save file for it exists, and offer to recover it.

Re: non-ASCII characters in auto-save files

[Xah Lee]

On Jun 25, 4:21 pm, Stefan Monnier <monn...@iro.umontreal.ca> wrote:
> >     - Can Emacs be persuaded to to encode auto-save files in a more useful
> > manner ?
>
> No.  Auto-save files are encoded using Emacs's internal coding-system,
> which makes the operation a lot more reliable and efficient (no need to
> worry about the case where some char(s) can't be encoded, for example).


i thought emacs 23/24 uses a superset of utf-8 as internal encoding?
so that i thought those #autosave# files are encoded in utf-8, and as
long as one open them with utf-8 decoding all chars would show. Is
this not correct?

(i turned of auto-save and auto backup few years ago)

 Xah

Re: non-ASCII characters in auto-save files

[Eli Zaretskii]

> From: Xah Lee <xahlee@gmail.com>
> Date: Wed, 27 Jun 2012 04:52:41 -0700 (PDT)
> 
> i thought emacs 23/24 uses a superset of utf-8 as internal encoding?

It does.

> so that i thought those #autosave# files are encoded in utf-8, and as
> long as one open them with utf-8 decoding all chars would show. Is
> this not correct?

Not 100%, because it's a superset.  That is, there could be codes
there that UTF-8 will fail to decode.

Re: non-ASCII characters in auto-save files

[Xah Lee]

On Jun 27, 9:44 am, Eli Zaretskii <e...@gnu.org> wrote:
> > From: Xah Lee <xah...@gmail.com>
> > Date: Wed, 27 Jun 2012 04:52:41 -0700 (PDT)
>
> > i thought emacs 23/24 uses a superset of utf-8 as internal encoding?
>
> It does.

hi Eli, thanks for confirmation.

> > so that i thought those #autosave# files are encoded in utf-8, and as
> > long as one open them with utf-8 decoding all chars would show. Is
> > this not correct?
>
> Not 100%, because it's a superset.  That is, there could be codes
> there that UTF-8 will fail to decode.

that's interesting. What's some examples? e.g. What chars in emacs's
encoding that's different from utf-8 encoding? I've been wondering why
it needs to be a superset.

 Xah

Re: non-ASCII characters in auto-save files

[Stefan Monnier]

> that's interesting. What's some examples? e.g. What chars in emacs's
> encoding that's different from utf-8 encoding? I've been wondering why
> it needs to be a superset.

The most obvious unavoidable ones are the chars that represent bytes.
So if you read a utf-8 file which has some invalid byte sequence, Emacs
will be able to save it back unchanged (in the buffer, the invalid
sequence is represented by the corresponding sequence of "byte chars").


        Stefan

Re: non-ASCII characters in auto-save files

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 27 Jun 2012 21:07:07 -0400
> 
> > that's interesting. What's some examples? e.g. What chars in emacs's
> > encoding that's different from utf-8 encoding? I've been wondering why
> > it needs to be a superset.
> 
> The most obvious unavoidable ones are the chars that represent bytes.

Yes, and then there are CJK characters that are not unified with their
Unicode codepoints, for historical and cultural reasons.  There's more
detailed description in the ELisp manual, node "Text Representations".