buffer-charset

buffer-charset

[Francesco Potorti`]

Some time ago I advocated the use of buffer-charset.el (appended) to
ease users' problems when meeting strange characters in their buffers.
I was answered that this was not the right solution.  However, unless
something better has come up in the meantme, I found it very useful in
the last month or so.

I'll give an example.

I take the file in 
 <ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-21.2.91.announce>

and save it into `ann'.  Then 
emacs -q
C-x m
C-c TAB ann RET
C-c C-c

I get a pop-up window proposing a bunch of "safe" coding systems.
Obviously I have no idea why ever I should change my coding system,
given that the file I included is apparently text only.  So I do:

M-x load-library RET buffer-charset RET
M-x highlight-charset-characters RET
TAB

and I get a completion window with these possible charset to higlight:

ascii                              latin-iso8859-1
mule-unicode-0100-24ff

Obviously the last one is strange.  So I select that to highlight, look
in the buffer and discover that there are two such characters, which I
can change or leave alone, if I know what I do (I don't).

In summary, I find this to be a really useful and comfortable user
interface for dealing with these problems, until a better one is
introduced.


===File ~/elisp/buffer-charset.el===========================
;;; buffer-charsets.el --- show usage of charsets in a buffer

;; Copyright (C) 2002 Joanna Pluta<joanna_p@poczta.gazeta.pl>.

;; Keywords: mule, hi-lock

;; This file is [not yet] part of GNU Emacs.

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 2, or (at your option)
;; any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; A copy of the GNU General Public License can be obtained from this
;; program's author (send electronic mail to <thomas.link@a1.net>) or
;; from the Free Software Foundation, Inc., 675 Mass Ave, Cambridge,
;; MA 02139, USA.

;;; Commentary:

;;   - display-buffer-charsets
;;     Displays list of charsets used in current buffer.

;;   - show-buffer-charset-characters C-x w c

;;     Uses hi-lock-mode to highlight in current buffer characters
;;     from chosen charset.
;;     Creates and adds to local history (hi-lock-interactive-patterns)
;;     regexpx matching characters from a given charset.
;;     Characters from different charsets can be highlighted by
;;     different colors according to chosen face.

;;   - unhighlight-charset  C-x w u
;;     unhighlights characters from given charset
;;



;;; Code:

;;;

(require 'hi-lock)

(defun charset-chars-regexp (charset)
 (let ((dim (charset-dimension charset))
	(chars (charset-chars charset))
	(plane (charset-iso-graphic-plane charset))
	min
	max)
      (cond ((eq charset 'eight-bit-control)
	   (setq min 128 max 159))
	  ((eq charset 'eight-bit-graphic)
	   (setq min 160 max 255))
	  (t
	   (if (= chars 94)
	       (setq min 33 max 126)
	     (setq min 32 max 127))
	   (or (= plane 0)
	       (setq min (+ min 128) max (+ max 128)))))
    (if (= dim 1)
	(format "[%c-%c]" (make-char charset min) (make-char charset max))
     (format "[%c-%c]"
	     (make-char charset min min) (make-char charset max max)))))

(defun buffer-charsets ()
 (find-charset-region (point-min) (point-max)))

(defun display-buffer-charsets ()
  "Displays list of charsets used in current buffer"
  (interactive)
  (let ((charsets (buffer-charsets))
	(curr-buf-name (current-buffer)))
    (with-output-to-temp-buffer "*Buffer Charsets*"
      (save-excursion
	(set-buffer standard-output)
	(insert
	 (format "Buffer %s uses the following charsets:\n" curr-buf-name))
	(while charsets
	  (insert (symbol-name (car charsets)))
	  (insert "\n")
	  (setq charsets (cdr charsets)))))))

(defun charset-alist (charset-list)
 (let ((l (charset-list))
       charset-alist)
  (while l
    (setq charset-alist (cons (list (symbol-name (car l))) charset-alist))
    (setq l (cdr l)))
  charset-alist))


(defalias 'highlight-charset-characters 'show-buffer-charset-characters)
(defun show-buffer-charset-characters (charset face)
 "Uses hi-lock-mode to highlight by face characters of charset."
  (interactive
   (let ((completion-ignore-case t))
   (list
    (completing-read "Charset:"
		     (charset-alist (buffer-charsets))  nil t nil nil)
    (hi-lock-read-face-name))))
    (highlight-regexp (charset-chars-regexp (intern charset)) face))

(defun unhighlight-charset (charset)
 (interactive
  (let ((completion-ignore-case t))
   (list
    (completing-read "Charset:"
		     (charset-alist (buffer-charsets))  nil t nil nil))))
    (unhighlight-regexp (charset-chars-regexp  (intern charset))))

(define-key hi-lock-map "\C-xwc" 'show-buffer-charset-characters)
(define-key hi-lock-map "\C-xwu" 'unhighlight-charset)

;;; mule-utils.el ends here
============================================================

Re: buffer-charset

[Kenichi Handa]

In article <E17zdSK-00032n-00@pot.cnuce.cnr.it>, Francesco Potorti` <pot@gnu.org> writes:
> I take the file in 
>  <ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-21.2.91.announce>

> and save it into `ann'.  Then 
> emacs -q
> C-x m
> C-c TAB ann RET
> C-c C-c

> I get a pop-up window proposing a bunch of "safe" coding systems.
> Obviously I have no idea why ever I should change my coding system,
> given that the file I included is apparently text only.  So I do:

In HEAD trunk, select-safe-coding-system is improved.  It
shows problematic characters in *Warning* buffer, and you
can click on them to move point to that character.

This works well on saving a buffer, but on sending a mail
(by sendmail-send-it), select-safe-coding-system (via
select-message-coding-system) is run on "sendmail temp"
buffer, thus that buffer disappears when you type C-g.

The attached patch is to run select-message-coding-system on
the current buffer (i.e. *mail*) in advance.  Could you
please try it with HEAD?

To the maintainer of sendmail.el:
  Do you think it breaks something?  If no, can I install that patch?

---
Ken'ichi HANDA
handa@m17n.org

*** sendmail.el.~1.258.~	Mon Sep 23 09:17:51 2002
--- sendmail.el	Fri Oct 11 09:17:14 2002
***************
*** 808,816 ****
  	(tembuf (generate-new-buffer " sendmail temp"))
  	(multibyte enable-multibyte-characters)
  	(case-fold-search nil)
! 	(coding (and (local-variable-p 'buffer-file-coding-system)
! 		     buffer-file-coding-system))
! 	selected-coding
  ;;;	resend-to-addresses
  	delimline
  	fcc-was-found
--- 808,814 ----
  	(tembuf (generate-new-buffer " sendmail temp"))
  	(multibyte enable-multibyte-characters)
  	(case-fold-search nil)
! 	(selected-coding (select-message-coding-system))
  ;;;	resend-to-addresses
  	delimline
  	fcc-was-found
***************
*** 830,836 ****
  	  (unless multibyte
  	    (set-buffer-multibyte nil))
  	  (insert-buffer-substring mailbuf)
- 	  (set-buffer-file-coding-system coding)
  	  (goto-char (point-max))
  	  ;; require one newline at the end.
  	  (or (= (preceding-char) ?\n)
--- 828,833 ----
***************
*** 954,960 ****
  		   (not (re-search-forward "^MIME-version:" delimline t))
  		   (progn (skip-chars-forward "\0-\177")
  			  (/= (point) (point-max)))
! 		   (setq selected-coding (select-message-coding-system))
  		   (setq charset
  			 (coding-system-get selected-coding 'mime-charset))
  		   (goto-char delimline)
--- 951,957 ----
  		   (not (re-search-forward "^MIME-version:" delimline t))
  		   (progn (skip-chars-forward "\0-\177")
  			  (/= (point) (point-max)))
! 		   selected-coding
  		   (setq charset
  			 (coding-system-get selected-coding 'mime-charset))
  		   (goto-char delimline)
***************
*** 983,991 ****
  \\|^resent-cc:\\|^resent-bcc:"
  				   delimline t))
  	      (let* ((default-directory "/")
! 		     (coding-system-for-write
! 		      (or selected-coding
! 			  (select-message-coding-system)))
  		     (args 
  		      (append (list (point-min) (point-max)
  				    program
--- 980,986 ----
  \\|^resent-cc:\\|^resent-bcc:"
  				   delimline t))
  	      (let* ((default-directory "/")
! 		     (coding-system-for-write selected-coding)
  		     (args 
  		      (append (list (point-min) (point-max)
  				    program

Re: buffer-charset

[Richard Stallman]

    To the maintainer of sendmail.el:
      Do you think it breaks something?  If no, can I install that patch?

I see no problem in your patch.

Re: buffer-charset

[Simon Josefsson]

Kenichi Handa <handa@m17n.org> writes:

> To the maintainer of sendmail.el:
>   Do you think it breaks something?  If no, can I install that patch?

Please install it.  The MIME handling in sendmail.el is poor though
(e.g., no RFC 2047 header encoding, no encoding of body if server
isn't 8bit clean, no support for non-unifiable coding systems in the
same buffer), but fixing it is rather non-trivial.  Perhaps the manual
could discuss this and suggest MIME savvy users to use Message
instead, I'll see if I can come up with some text.

Re: buffer-charset

[Kenichi Handa]

In article <E17zz9c-0000UN-00@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:
>     To the maintainer of sendmail.el:
>       Do you think it breaks something?  If no, can I install that patch?

> I see no problem in your patch.

Ok, I've just installed it in HEAD.

---
Ken'ichi HANDA
handa@m17n.org