Why is cp1252 special?

Why is cp1252 special?

[Torsten Bronger]

Hallöchen!

I use recent CVS versions of Gnus and Emacs.

After a couple of non-declared encodings in Outlook postings which I
received with Gnus, I wanted to change the fallback chain from cp437
(I've got a lot of old MSDOS files) to cp1252.  At the moment, my
.emacs contains

(define-coding-system-alias 'cp1252 'windows-1252)
(prefer-coding-system 'cp1252)
(prefer-coding-system 'iso-latin-1)
(prefer-coding-system 'utf-8)

I'm just curious: Why is there some sort of asymmetry between cp1252
and most (if not all) other codepages?  In particular, why is the
first line above necessary?  By contrast, I can say

(prefer-coding-system 'cp1257)

without problems.  A usenet search yielded some information about
the fact that cp1252 *is* special but not why.  Additionally,
news:v9smnfv8ib.fsf@marauder.physik.uni-ulm.de suggests that new
Emacs versions should allow

(codepage-setup 1252)

however, this doesn't work for me in .emacs.

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus            ICQ 264-296-646

Re: Why is cp1252 special?

[Reiner Steib]

On Fri, Oct 07 2005, Torsten Bronger wrote:

> I use recent CVS versions of Gnus and Emacs.
>
> After a couple of non-declared encodings in Outlook postings which I
> received with Gnus, 

Please elaborate what's the problem in Gnus.

> I'm just curious: Why is there some sort of asymmetry between cp1252
> and most (if not all) other codepages?

I don't see any asymmetry:

$ emacs-cvs -Q -f ielm
*** Welcome to IELM ***  Type (describe-mode) for help.
ELISP> (coding-system-p 'cp1252)
nil
ELISP> (coding-system-p 'windows-1252)
t
ELISP> (coding-system-p 'cp1257)
nil
ELISP> (coding-system-p 'windows-1257)
t
ELISP> (prefer-coding-system 'cp1252)
*** Eval error ***  Invalid coding system `cp1252'
ELISP> (prefer-coding-system 'cp1257)
*** Eval error ***  Invalid coding system `cp1257'
ELISP> (prefer-coding-system 'windows-1252)
(windows-1252 . windows-1252)

ELISP> (prefer-coding-system 'windows-1257)
(windows-1257 . windows-1257)


> A usenet search yielded some information about the fact that cp1252
> *is* special but not why.  Additionally,
> news:v9smnfv8ib.fsf@marauder.physik.uni-ulm.de suggests that new
> Emacs versions should allow
>
> (codepage-setup 1252)
>
> however, this doesn't work for me in .emacs.

The above mentioned article is 2 years old.  The information there
probably is not accurate anymore for current CVS Emacs.  I don't
recall that (codepage-setup 1252) ever worked.

BTW: `rs-windows-1252.el' is obsolete; With recent CVS Emacs 22, you
don't even need it's replacement `rs-ucs-coding-system.el':

,----
| 2005-08-05  Kenichi Handa  <handa@m17n.org>
| 
| 	* international/code-pages.el: Add autoload cookies for all coding
| 	systems.
|
| [...] 
|
| 2005-03-04  Reiner Steib  <Reiner.Steib@gmx.de>
| 
| 	* international/code-pages.el (windows-1250, windows-125[2-8])
| 	(iso-8859-10, -13, -16, georgian-ps): Add autoload cookies.
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Why is cp1252 special?

[Torsten Bronger]

Hallöchen!

Reiner Steib <reinersteib+from-uce@imap.cc> writes:

> On Fri, Oct 07 2005, Torsten Bronger wrote:
>
>> I use recent CVS versions of Gnus and Emacs.
>>
>> After a couple of non-declared encodings in Outlook postings which I
>> received with Gnus, 
>
> Please elaborate what's the problem in Gnus.

The problem *was* that with some articles with undeclared encoding,
Gnus used cp850 because my .emacs said

(prefer-coding-system 'cp850)
(prefer-coding-system 'iso-latin-1)
(prefer-coding-system 'utf-8)

>> I'm just curious: Why is there some sort of asymmetry between cp1252
>> and most (if not all) other codepages?
>
> I don't see any asymmetry:
>
> [...]

What I observed is that

(codepage-setup 1257)

works but

(codepage-setup 1252)

yields "Unsupported codepage 1252".

> [...]  I don't recall that (codepage-setup 1252) ever worked.

Could this be because no mapping for 1252 appears in codepage.el?

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus            ICQ 264-296-646

Re: Why is cp1252 special?

[Reiner Steib]

On Fri, Oct 07 2005, Torsten Bronger wrote:

> Reiner Steib <reinersteib+from-uce@imap.cc> writes:
>> Please elaborate what's the problem in Gnus.
>
> The problem *was* that with some articles with undeclared encoding,
> Gnus used cp850 because my .emacs said
>
> (prefer-coding-system 'cp850)
> (prefer-coding-system 'iso-latin-1)
> (prefer-coding-system 'utf-8)

We already have a mechanism in Gnus to say "treat charset `windows-foo'
as `iso-8859-1', if we don't have `windows-foo': 

,----[ <f1> v mm-charset-synonym-alist RET ]
| mm-charset-synonym-alist is a variable defined in `mm-util'.
| Its value is 
| ((big5-hkscs . big5)
|  (unicode . utf-16-le)
|  (ks_c_5601-1987 . cp949))
| 
| Documentation:
| A mapping from invalid charset names to the real charset names.
`----

But (to my knowledge) we don't have a mechanism to say "treat
`iso-8859-1' as `windows-1252'".  `windows-1252' is a superset of
`iso-8859-1' so this would not hurt for correctly labeled `iso-8859-1'
postings, but it would display "labeled as `iso-8859-1', but in fact
it's `windows-1252'" too.  We also don't have "treat unlabeled
(undecided) as `what-ever'".

I will probably install a patch for this in No Gnus:

,----[ M-x customize-variable RET mm-charset-override-alist RET ]
| mm-charset-override-alist: 
| Set:
| [X] (iso-8859-1 . windows-1252)
| [ ] (undecided . windows-1252)
| [...]
| 
| A mapping from undesired charset names to their replacement.
| 
| You may add pair like (iso-8859-1 . windows-1252) here,
| i.e. treat iso-8859-1 as windows-1252.  windows-1252 is a
| superset of iso-8859-1.
`----

> What I observed is that (codepage-setup 1257) works but
> (codepage-setup 1252) yields "Unsupported codepage 1252".
>
>> [...]  I don't recall that (codepage-setup 1252) ever worked.
>
> Could this be because no mapping for 1252 appears in codepage.el?

Probably.  But it's not missing, because `windows-1252' is provided by
`code-pages.el, not by `codepage.el'.

I didn't remember that cp1257 was already present in `codepage.el'
whereas `windows-1252' is provided by `code-pages.el'.  From the
thread <http://thread.gmane.org/gmane.emacs.devel/41495>...

Eli Zaretskii:
| IIRC, `codepage-setup' was declared obsolete because the need to do
| something in order for a coding system to become available was
| deemed a bad idea.  But I don't remember the details, nor what would
| be the Right Way.  The birth of code-pages.el was accompanied by
| heated discussions, so some issues were lost in the dispute and
| never resolved.

Then, Kenichi Handa added the missing autoloads an changed the NEWS
entry:

,----[ etc/NEWS ]
| *** Many new coding systems are available in the `code-pages' library.
| These include complete versions of most of those in codepage.el, based
| on Unicode mappings.  `codepage-setup' is now obsolete and is used
| only in the MS-DOS port of Emacs.  All coding systems defined in
| `code-pages' are auto-loaded.
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Why is cp1252 special?

[Torsten Bronger]

Hallöchen!

Reiner Steib <reinersteib+from-uce@imap.cc> writes:

> [...]
>
>>> [...]  I don't recall that (codepage-setup 1252) ever worked.
>>
>> Could this be because no mapping for 1252 appears in codepage.el?
>
> Probably.  But it's not missing, because `windows-1252' is
> provided by `code-pages.el, not by `codepage.el'.
>
> I didn't remember that cp1257 was already present in `codepage.el'
> whereas `windows-1252' is provided by `code-pages.el'.  From the
> thread <http://thread.gmane.org/gmane.emacs.devel/41495>...
>
> Eli Zaretskii:
> | [...]

Now it's by and large clear to me; apparently I used an obsolete way
in my .emacs.  Thanks for the explanations!

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus            ICQ 264-296-646

Re: Why is cp1252 special?

[Jason Rumney]

Torsten Bronger <bronger@physik.rwth-aachen.de> writes:

> What I observed is that
>
> (codepage-setup 1257)
>
> works but
>
> (codepage-setup 1252)
>
> yields "Unsupported codepage 1252".

codepage-setup is intended for the DOS port of Emacs. It maps a single
ISO coding-system onto a DOS or Windows codepage as best as it
can. Since the whole of Latin-1 maps 1:1 to Windows-1252, cp1252 is
not needed.

Systems other than DOS should use the codepages defined in
code-pages.el, which are autoloaded. These codepages map multiple
coding-systems onto the DOS and Windows codepages to provide full
coverage in both directions. The windows codepages have names
"windows-125X" to distinguish them from the codepages for the DOS port
in codepage.el.