Re: Auto-detection of windows-1252 fails

[Kenichi Handa <handa@ni.aist.go.jp>]

In article <E1JBQZ5-0001Uy-TV@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

> Can you please DTRT on this, and ack?
[...]
> From: Reiner Steib <reinersteib+gmane@imap.cc>
> Date: Sat, 05 Jan 2008 14:22:37 +0100
> Subject: Auto-detection of windows-1252 fails
[...]
> in September/October 2006 we had a long thread on emacs-pretest-bugs
> about auto-detection of windows-1252 text files:

>   Subject: local chars displayed as numbers
>   <http://thread.gmane.org/gmane.emacs.pretest.bugs/14020/>
>   [ I include a summary of this thread below. ]

> windows-1252 files were supposed to be detected automatically in the
> "Latin-1" and "German" language environments.  This doesn't work
> (anymore?) in Emacs 22.1, the Emacs_22 branch and in the trunk.

> * Summary of the September/October 2006 discussion:

>   The following change was installed...
  
>   ,----[ ChangeLog.12 ]
>   | 2006-09-21  Kenichi Handa  <handa@m17n.org>
>   | 
>   | 	* language/european.el ("Latin-1"): Add windows-1252 to
>   | 	coding-priority.
>   | 	("German"): Likewise.
>   `----
  
>   ... and was supposed to result in the following behavior:
  
>   Kenichi Handa wrote in
>   <http://article.gmane.org/gmane.emacs.pretest.bugs/14384>:
  
>   | A file containing a windows-1252 char that doesn't appear in
>   | iso-8859-1 is detected as windows-1252.  Bad effect is that some (or
>   | many) binary files are also detected as windows-1252.
  
>   Some people pointed out that this may lead to the bad effect that some
>   (or many) binary files are also detected as windows-1252.  Eli
>   suggested to implement null-byte detection which should solve this
>   problem.
  
>   In <http://thread.gmane.org/gmane.emacs.pretest.bugs/14020/focus=14384>
>   Kenichi Handa wrote:
>   | Reiner Steib <reinersteib+gmane <at> imap.cc> writes:
>   | 
>   | > (6) Implement null-byte detection (to prevent binary files
>   | >    mis-detected as windows-12xx), keep the current code (windows-1252)
>   | >    and add windows-1254/1255 accordingly.
>   | 
>   | I think that change results in the best behavior.
  
>   ... and Richard agreed on that.  But I don't think this has been done.
>   ("the current code" refers to the 2006-09-21 change, see above.)
  
I've just installed the null-byte detection code and some
improvement on handling latin-extra-code-table in the trunk.
Could you please test the latest code?

>   | >    and add windows-1254/1255 accordingly.

I've not yet done that.  Could someone tell me which to add
where?

> * Additionally, the addition of windows-1252 to "German" has been lost
>   in the emacs-unicode-2 branch:

>   --- european.el	26 Jul 2007 05:27:10 -0000	1.100
>   +++ european.el	25 Dec 2007 10:57:51 -0000	1.86.4.13
>   @@ -277,16 +414,15 @@
   
>    (set-language-info-alist
>     "German" '((tutorial . "TUTORIAL.de")
>   -	    (charset ascii latin-iso8859-1)
>   +	    (charset iso-8859-1)
>    	    (coding-system iso-latin-1 iso-latin-9)
>   -	    (coding-priority iso-latin-1 windows-1252)
>   +	    (coding-priority iso-latin-1)
>   +	    (nonascii-translation . iso-8859-1)
>    	    (input-method . "german-postfix")
  
Oops, I don't know why that change was lost.  I'll fix it
soon as well as the equivalent change for null-byte
detection and latin-extra-code-table handling improvement.

---
Kenichi Handa
handa@ni.aist.go.jp