getting out of raw-text encoding

getting out of raw-text encoding

[David Madore]

Hi!

This is with (FSF) Emacs 21.2, compiled with LEIM.  (Though I suppose
the actual Emacs version makes little difference.)

Suppose I have a file which is *mostly* encoded in the iso-8859-1
character set.  However, it has a few occasional characters that are
not iso-8859-1.  For example, it might be actually in the windows-1252
character set (that closely resembles iso-8859-1 but adds 32 extra
characters, with codes from 128 through 159).  A sample could be
produced with

perl -e 'for ($i=32;$i<256;$i++) { print chr($i); } print "\n";' >file

for instance.

When I open the file with Emacs, it quite rightfully detects that it
is not iso-8859-1, and switches to raw-text encoding (non-ASCII
characters are represented by escape sequences).  Now suppose I don't
like that.

I can force the file to be reinterpreted as iso-8859-1 by killing the
buffer and reopening it with C-x RET c iso-8859-1 as prefix.
Basically this does what I want, but suppose I wish to do it without
closing and reopening the file (there may be reasons for this, e.g.,
if the buffer is actually not a file I opened but something that was
produced by an Emacs submodule, like Gnus in attempting to reply to a
news post with a bad encoding).

So what I wish is, to change every raw-text octet value that makes
sense as an iso-8859-1 character by that corresponding character.  I
hope I'm using the correct terminology here.

First of all, doing C-x RET c iso-8859-1 M-x revert-buffer does not
work (it does not do anything, in fact, apparently).  Nor do I
understand why.  But even if it did, it still wouldn't be what I like,
because it would force the buffer to have an associated file.

Usually, when I need to reinterpret using the <bar> encoding a file
that has been read using the <foo> encoding by mistake (typical
example would be when <bar>=utf-8 and <foo>=iso-8859-1) I can do this:
C-x h M-x encode-coding-region <foo> M-x decode-coding-region <bar>
which works by first forcing the buffer back to the octets that encode
it in the <foo> encoding, and then decoding these octets using the
<bar> encoding.  However, when <foo> is raw-text, this does not
function as I expect it (maybe I'm being entirely naive there?), for
the encode-coding-region command does not do anything, and the
decode-coding-region merely adds more junk in front of my escaped
characters.

So, basically, how do I get out of raw-text without killing the
buffer?

Incidentally, is there an implementation of the windows-1252 character
encoding in Emacs somewhere (one that would map iso-8859-1 characters
to iso-8859-1 characters, and all others to the corresponding Unicode
codepoints)?  That would be useful in interacting with these annoying
systems that insist on pretending that windows-1252 "is" iso-8859-1.

Many thanks to those who can Enlighten me in this matter!

-- 
     David A. Madore
    (david.madore@ens.fr,
     http://www.eleves.ens.fr:8080/home/madore/ )

Re: getting out of raw-text encoding

[Kai Großjohann]

david.madore@ens.fr (David Madore) writes:

> Incidentally, is there an implementation of the windows-1252 character
> encoding in Emacs somewhere (one that would map iso-8859-1 characters
> to iso-8859-1 characters, and all others to the corresponding Unicode
> codepoints)?

(codepage-setup "1252") might have done something useful, but it seems
that my Emacs knows 1250 and 1251 but not 1252.  Why?
-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: getting out of raw-text encoding

[Eli Zaretskii]

> From: kai.grossjohann@gmx.net (=?iso-8859-1?q?Kai_Gro=DFjohann?=)
> Newsgroups: gnu.emacs.help
> Date: Sat, 10 May 2003 19:06:36 +0200
> 
> (codepage-setup "1252") might have done something useful, but it seems
> that my Emacs knows 1250 and 1251 but not 1252.  Why?

Because codepage 1252's target charset is latin-iso8859-1, and where
cp1252 and latin-1 intersect, they are identical, while the few
non-Latin-1 characters there cannot be expressed as latin-iso8859-1.

I'd like to remind you that until now, cpNNN encodings could only
express characters of a single charset.  This makes support of
Microsoft's abominations impossible.  (The CVS version of Emacs fixes
that in code-pages.el.)

Therefore, it didn't make sense to introduce cp1252 into Emacs, as the
most we could have done would be for it to be an alias for latin-1.

Re: getting out of raw-text encoding

[Kai Großjohann]

"Eli Zaretskii" <eliz@elta.co.il> writes:

>> From: kai.grossjohann@gmx.net (=?iso-8859-1?q?Kai_Gro=DFjohann?=)
>> Newsgroups: gnu.emacs.help
>> Date: Sat, 10 May 2003 19:06:36 +0200
>> 
>> (codepage-setup "1252") might have done something useful, but it seems
>> that my Emacs knows 1250 and 1251 but not 1252.  Why?
>
> Because codepage 1252's target charset is latin-iso8859-1, and where
> cp1252 and latin-1 intersect, they are identical, while the few
> non-Latin-1 characters there cannot be expressed as latin-iso8859-1.
>
> I'd like to remind you that until now, cpNNN encodings could only
> express characters of a single charset.  This makes support of
> Microsoft's abominations impossible.  (The CVS version of Emacs fixes
> that in code-pages.el.)

Hm?

It would have been possible to introduce another charset for cp1252.
Then in addition to Latin-1 ä and Latin-2 ä and Latin-9 ä we'd also
have cp1252 ä.  That would be in the spirit of current Mule, wouldn't
it?

And then unify-8859-on-{de,en}coding-mode could unify them...

-- 
This line is not blank.

Re: getting out of raw-text encoding

[Eli Zaretskii]

> From: kai.grossjohann@gmx.net (=?iso-8859-1?q?Kai_Gro=DFjohann?=)
> Newsgroups: gnu.emacs.help
> Date: Sat, 10 May 2003 22:06:21 +0200
> >
> > I'd like to remind you that until now, cpNNN encodings could only
> > express characters of a single charset.  This makes support of
> > Microsoft's abominations impossible.  (The CVS version of Emacs fixes
> > that in code-pages.el.)
> 
> Hm?
> 
> It would have been possible to introduce another charset for cp1252.

Yes, and that would create yet another set of charsets which are
largely identical to the corresponding Latin-N charsets.  Given the
bloody mess we have with Latin-N charsets that have only a few
identical codepoints, I'm glad I've made the design decision to treat
cpNNN as encodings, not as charsets.

Look how many people complain here about the Latin-1 vs Latin-9
discrepancy.

Re: getting out of raw-text encoding

[Kai Großjohann]

"Eli Zaretskii" <eliz@elta.co.il> writes:

> Look how many people complain here about the Latin-1 vs Latin-9
> discrepancy.

I know, I know, but now we have unify-8859-on-encoding-mode ;-)

Maybe Latin-1 could be an encoding for the cp1250 charset...  d&r
-- 
This line is not blank.

Re: getting out of raw-text encoding

[Eli Zaretskii]

> From: david.madore@ens.fr (David Madore)
> Newsgroups: gnu.emacs.help
> Date: 10 May 2003 15:21:08 GMT
> 
> So, basically, how do I get out of raw-text without killing the
> buffer?

For every 8-bit value CHAR that you want Emacs to accept in an
otherwise Latin-1 encoded file, put this line into your .emacs:

  (aset latin-extra-code-table CHAR t)

For example, to accept the character whose code is 158, do this
(untested):

  (aset latin-extra-code-table 158 t)

I should caution you that this might bite you in scertain situations,
so I would not recommend doing that for all the characters in the
region 128-159, just for those you actually see frequently.