Emacs refusing to switch to iso-latin-1

Emacs refusing to switch to iso-latin-1

[Sébastien Vauban]

Hi,

For a couple of weeks, I've chosen to select utf-8 as my default
coding system. For the sake of facility, I resisted for years,
but I'd have to adapt sooner or later, so...

--8<---------------cut here---------------start------------->8---
;; system locale to use for formatting time values
(setq system-time-locale "en_US.utf8")

;; default coding system (for new files)
(prefer-coding-system 'utf-8)

;; to copy and paste to and from Emacs through the clipboard
(set-selection-coding-system 'utf-8)
--8<---------------cut here---------------end--------------->8---

Doing so, my Important settings are now:

--8<---------------cut here---------------start------------->8---
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t
--8<---------------cut here---------------end--------------->8---

My problem is that, when creating of updating files for some
applications, I sometimes have to output pure iso-latin-1...

I did create a new file, from a scratch buffer. It was thus
considered as utf-8 format.

I told emacs to save the file in iso-latin-1-unix, with `C-x
<RET> f iso-latin-1-unix'.

    Note - BTW, it'd be better if it was called `C-x <RET> s'
    (for saving), like `C-x <RET> r' is used for revert.

I see:

    o   the coding system flag changing from `U' to `1', and
    o   the modified flag from `--' to `**'.

Excellent...

But, when I do my `C-x C-s', just after it's done, it see back
`-U:--' in my modeline.

Checking with `C-u C-x =' or with `hexl-mode', I see that the
file is still in utf-8; its size did not change neither...

So, I can't save in iso-latin-1...

I could reproduce it both under Windows and under Linux. And it
"works" that way 100% of the times.

Thanks for your help!!

Best regards,
  Seb

-- 
Sébastien Vauban

Re: Emacs refusing to switch to iso-latin-1

[Peter Dyballa]

Am 11.02.2008 um 12:25 schrieb Sébastien Vauban:

> I told emacs to save the file in iso-latin-1-unix, with `C-x
> <RET> f iso-latin-1-unix'.
> ...
>
> But, when I do my `C-x C-s', just after it's done, it see back
> `-U:--' in my modeline.


And what happens with C-x RET c, i.e. encoding for next command, and  
you save the before changed buffer?

--
Greetings

   Pete

If we don't succeed, we run the risk of failure."
				– George W. Bush

Re: Emacs refusing to switch to iso-latin-1

[Sébastien Vauban]

Hi Peter,

>> I told emacs to save the file in `iso-latin-1-unix', with
>> `C-x <RET> f iso-latin-1-unix'. [...]
>> But, when I do my `C-x C-s', just after it's done, I see back
>> `-U:--' in my modeline.
>
> And what happens with `C-x <RET> c', i.e. encoding for next
> command, and you save the before changed buffer?

Good idea... Thanks.

Unluckily, it does nothing, just telling me:

    (No changes need to be saved)

BTW, here's a sample file (let's call it `test.xml') that
exhibits the described problem:

--8<---------------cut here---------------start------------->8---
    <enumType name="test">
      <pair value="0">- Sélectionner -</pair>
      <pair value="11">Dû à l'entreprise</pair>
      <pair value="12">Stagiaire occupé</pair>
      <pair value="13">Autres</pair>
    </enumType>
--8<---------------cut here---------------end--------------->8---

Thanks for your help!

Seb

-- 
Sébastien Vauban

Re: Emacs refusing to switch to iso-latin-1

[Peter Dyballa]

Am 11.02.2008 um 15:54 schrieb Sébastien Vauban:

> Unluckily, it does nothing, just telling me:
>
>     (No changes need to be saved)

Therefore I wrote "changed." Make a small change, a SPC character  
added, and then invoke C-x RET c ...

For me C-x RET f <whatsoever> RET works – when I do not have local  
variables that set the encoding.

--
Greetings

   Pete

You can learn many things from children.  How much patience you have,  
for instance.
				– Franklin P. Jones

Re: Emacs refusing to switch to iso-latin-1

[Peter Dyballa]

Am 11.02.2008 um 15:54 schrieb Sébastien Vauban:

> BTW, here's a sample file (let's call it `test.xml') that
> exhibits the described problem:

Does the XML file has a notice about its encoding in the header?

--
Greetings

   Pete

I hope to die before I *have* to use Microsoft Word.
			- Donald E. Knuth, 2001-10-02 in Tübingen.

Re: Emacs refusing to switch to iso-latin-1

[Sven Joachim]

On 2008-02-11 15:54 +0100, Sébastien Vauban wrote:

> BTW, here's a sample file (let's call it `test.xml') that
> exhibits the described problem:
>
>     <enumType name="test">
>       <pair value="0">- Sélectionner -</pair>
>       <pair value="11">Dû à l'entreprise</pair>
>       <pair value="12">Stagiaire occupé</pair>
>       <pair value="13">Autres</pair>
>     </enumType>

Please file a bug with M-x report-emacs-bug.  I could reproduce the
problem with the Emacs trunk version (it told me that iso-latin-1-unix
is not a safe coding system for that file?!), but Emacs 22 works fine.

Sven

Re: Emacs refusing to switch to iso-latin-1

[David Kastrup]

Sébastien Vauban <zthjwsqqafhv@spammotel.com> writes:

> Hi Peter,
>
>>> I told emacs to save the file in `iso-latin-1-unix', with
>>> `C-x <RET> f iso-latin-1-unix'. [...]
>>> But, when I do my `C-x C-s', just after it's done, I see back
>>> `-U:--' in my modeline.
>>
>> And what happens with `C-x <RET> c', i.e. encoding for next
>> command, and you save the before changed buffer?
>
> Good idea... Thanks.
>
> Unluckily, it does nothing, just telling me:
>
>     (No changes need to be saved)
>
> BTW, here's a sample file (let's call it `test.xml') that
> exhibits the described problem:
>
>     <enumType name="test">
>       <pair value="0">- Sélectionner -</pair>
>       <pair value="11">Dû à l'entreprise</pair>
>       <pair value="12">Stagiaire occupé</pair>
>       <pair value="13">Autres</pair>
>     </enumType>
>
> Thanks for your help!

Huh?  That file does not contain an encoding header, and so according to
the XML standard, it _has_ to be encoded in utf-8.  Emacs gets this
right.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Emacs refusing to switch to iso-latin-1

[Lennart Borgman (gmail)]

David Kastrup wrote:
>> BTW, here's a sample file (let's call it `test.xml') that
>> exhibits the described problem:
>>
>>     <enumType name="test">
>>       <pair value="0">- Sélectionner -</pair>
>>       <pair value="11">Dû à l'entreprise</pair>
>>       <pair value="12">Stagiaire occupé</pair>
>>       <pair value="13">Autres</pair>
>>     </enumType>
>>
>> Thanks for your help!
> 
> Huh?  That file does not contain an encoding header, and so according to
> the XML standard, it _has_ to be encoded in utf-8.  Emacs gets this
> right.

But does Emacs tell what it does?

Re: Emacs refusing to switch to iso-latin-1

[Sébastien Vauban]

Hi Pete,

>> Unluckily, it does nothing, just telling me:
>>
>>     (No changes need to be saved)
>
> Therefore I wrote "changed." Make a small change, a SPC
> character added, and then invoke C-x RET c ...

And then... it works... it's saved with the `iso-latin-1-unix'
encoding, as requested.

But, as soon as I make another change, and save, I'm back to the
same problem. It's automatically "converted" once again to
`utf-8'.


> For me C-x RET f <whatsoever> RET works – when I do not have
> local variables that set the encoding.

Of course, I don't have any local variables for that file, as
you could see in the given sample.

Thanks for your help, mate ;-)

Seb

-- 
Sébastien Vauban

Re: Emacs refusing to switch to iso-latin-1

[Sébastien Vauban]

Hi,

>>> BTW, here's a sample file (let's call it `test.xml') that
>>> exhibits the described problem:
>>>
>>>     <enumType name="test">
>>>       <pair value="0">- Sélectionner -</pair>
>>>       <pair value="11">Dû à l'entreprise</pair>
>>>       <pair value="12">Stagiaire occupé</pair>
>>>       <pair value="13">Autres</pair>
>>>     </enumType>
>>>
>>> Thanks for your help!
>>
>> Huh? That file does not contain an encoding header, and so
>> according to the XML standard, it _has_ to be encoded in
>> utf-8. Emacs gets this right.

I did not know about that.

But, for me, this does not explain why my request to "use
iso-latin-1-unix for saving this buffer" is not respected.


> But does Emacs tell what it does?

Not to me. Don't even see that the coding system could be wrong.

Thanks,
  Seb

-- 
Sébastien Vauban

Re: Emacs refusing to switch to iso-latin-1

[David Kastrup]

Sébastien Vauban <zthjwsqqafhv@spammotel.com> writes:

> Hi,
>
>>>> BTW, here's a sample file (let's call it `test.xml') that
>>>> exhibits the described problem:
>>>>
>>>>     <enumType name="test">
>>>>       <pair value="0">- Sélectionner -</pair>
>>>>       <pair value="11">Dû à l'entreprise</pair>
>>>>       <pair value="12">Stagiaire occupé</pair>
>>>>       <pair value="13">Autres</pair>
>>>>     </enumType>
>>>>
>>>> Thanks for your help!
>>>
>>> Huh? That file does not contain an encoding header, and so
>>> according to the XML standard, it _has_ to be encoded in
>>> utf-8. Emacs gets this right.
>
> I did not know about that.
>
> But, for me, this does not explain why my request to "use
> iso-latin-1-unix for saving this buffer" is not respected.

Because your file does not start with
<?xml version="1.0" encoding="ISO-8859-1"?>

Is that so hard to understand?

XML without such a header is encoded in utf-8, period.  Any conforming
XML interpreter will _barf_ if you were able to save in latin-1.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Emacs refusing to switch to iso-latin-1

[Sébastien Vauban]

Hi David and all,

>>>>> BTW, here's a sample file (let's call it `test.xml') that
>>>>> exhibits the described problem:
>>>>>
>>>>>     <enumType name="test">
>>>>>       <pair value="0">- Sélectionner -</pair>
>>>>>       <pair value="11">Dû à l'entreprise</pair>
>>>>>       <pair value="12">Stagiaire occupé</pair>
>>>>>       <pair value="13">Autres</pair>
>>>>>     </enumType>
>>>>>
>>>>> Thanks for your help!
>>>>
>>>> Huh? That file does not contain an encoding header, and so
>>>> according to the XML standard, it _has_ to be encoded in
>>>> utf-8. Emacs gets this right.
>>
>> I did not know about that.
>>
>> But, for me, this does not explain why my request to "use
>> iso-latin-1-unix for saving this buffer" is not respected.
>
> Because your file does not start with
> <?xml version="1.0" encoding="ISO-8859-1"?>
>
> Is that so hard to understand?

I'm quite surprised, yes, because it's one of the first time
Emacs does something in my back, without my knowledge.

I'm not surprised by such behaviors from all the other tools, in
particular the MS ones, but here... yes I am.


> XML without such a header is encoded in utf-8, period. Any
> conforming XML interpreter will _barf_ if you were able to
> save in latin-1.

OK, but this annoys me, still. It's not because that file has a
`.xml' extension that it has to be complete. I would never give
it to a parser like that.

That file, along with others, is part of a correct XML file
that's created by a process, from different sub-sources. The
file I'm editing is one of these sources. So, I can't put a
header in there, otherwise it would appear everywhere in the
resulting XML file.

OK, maybe I can add "local variables" comments in every
sub-file, yes, but I didn't expect from Emacs to take decisions
away from me - but for a default coding system when nothing is
specified (and, even then, it's under my knowledge). Sorry.

Seb

-- 
Sébastien Vauban

Re: Emacs refusing to switch to iso-latin-1

[Stefan Monnier]

>> But, for me, this does not explain why my request to "use
>> iso-latin-1-unix for saving this buffer" is not respected.

> Because your file does not start with
> <?xml version="1.0" encoding="ISO-8859-1"?>

I don't know for a fact that this is the reason, but it's indeed quite
likely: sgml-xml-auto-coding-function seems to enforce it.  The way this
works is that when Emacs saves a file is tries to use the encoding that
it would use when reading the file (this is necessary in order to
ensure that saving and reloading gives the same text, which I hope you
agree is important).  Since XML files are decoded as utf-8 by default,
Emacs has to save it as utf-8 by default.

The bug really is in C-x RET f: it should complain that the file's
header specify utf-8 encoding (in this case by omission) and hence
latin-1 cannot be used.

Please report it as a bug via M-x report-emacs-bug.


        Stefan

Re: Emacs refusing to switch to iso-latin-1

[Stefan Monnier]

> OK, maybe I can add "local variables" comments in every
> sub-file, yes, but I didn't expect from Emacs to take decisions
> away from me - but for a default coding system when nothing is
> specified (and, even then, it's under my knowledge). Sorry.

That's also a good point.  Maybe the default for XML files shouldn't be
utf-8 but `undefined' (i.e. the usual default which tries to
auto-detect the encoding).  It should still work well since utf-8 is
among the first encodings for which we try auto-detection and it has
a very good detector (very few non-utf-8 files are detected as utf-8).

Again, please report this via M-x report-emacs-bug.


        Stefan

Re: Emacs refusing to switch to iso-latin-1 (on the way to be SOLVED)

[Sébastien Vauban]

Hi Stefan,

>>> My request to "use iso-latin-1-unix for saving this buffer"
>>> (`C-x RET f iso-latin-1 RET') is not respected.
>>
>> Because your file does not start with
>> <?xml version="1.0" encoding="ISO-8859-1"?>
>
> [...] When Emacs saves a file, it tries to use the encoding
> that it would use when reading the file (this is necessary in
> order to ensure that saving and reloading gives the same text,
> which I hope you agree is important).

I do agree. Thanks for making this clear to me, by explaining
how it behaves!


> Since XML files are decoded as utf-8 by default, Emacs has to
> save it as utf-8 by default.
>
> The bug really is in `C-x RET f': it should complain that the
> file's header specify utf-8 encoding (in this case by
> omission) and hence iso-latin-1 cannot be used.

Un-der-stood.

I'm happy we do agree that there was something not totally
normal in the way Emacs behaves in this case (failing silently,
instead of telling me why it can't go on with my request).


> Please report it as a bug via M-x report-emacs-bug.

Just did.

Thank you very much to all,
  Seb

-- 
Sébastien Vauban

Re: Emacs refusing to switch to iso-latin-1

[Sébastien Vauban]

Hi Stefan,

>> OK, maybe I can add "local variables" comments in every
>> sub-file, yes, but I didn't expect from Emacs to take
>> decisions away from me - but for a default coding system when
>> nothing is specified (and, even then, it's under my
>> knowledge).
>
> That's also a good point. Maybe the default for XML files
> shouldn't be utf-8 but `undefined' (i.e. the usual default
> which tries to auto-detect the encoding).

I agree with you.


> Again, please report this via M-x report-emacs-bug.

Just did it as well.

Thanks,
  Seb

-- 
Sébastien Vauban