* Subsets of Unicode and coding systems
@ 2011-12-06 11:38 Ulrich Mueller
2011-12-06 12:08 ` Eli Zaretskii
0 siblings, 1 reply; 4+ messages in thread
From: Ulrich Mueller @ 2011-12-06 11:38 UTC (permalink / raw)
To: emacs-devel
Hi,
Sometimes I want to check if a file saved in UTF-8 encoding contains
only characters from a certain subset of the Unicode character
repertoire, like the MES-* [1] or WGL4 [2] subsets. (For example, for
things published on the WWW one might rather avoid exotic characters
for better compatibility.)
My idea was now to define a coding system e.g. for MES-2 and set the
buffer-file-coding-system accordingly, so that Emacs would check it
upon saving the file.
Now I have the following questions:
1. Is using coding systems for this purpose a reasonable approach
at all, or is there a better way to achieve this?
2. Where can I find documentation how to define a coding system?
The Elisp Reference Manual isn't really helpful, it only says:
"How to define a coding system is an arcane matter, and is not
documented here."
Ulrich
[1] <http://www.evertype.com/standards/iso10646/pdf/cwa13873.pdf>
[2] <http://en.wikipedia.org/wiki/Windows_Glyph_List_4>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Subsets of Unicode and coding systems
2011-12-06 11:38 Subsets of Unicode and coding systems Ulrich Mueller
@ 2011-12-06 12:08 ` Eli Zaretskii
2011-12-06 12:24 ` Werner LEMBERG
0 siblings, 1 reply; 4+ messages in thread
From: Eli Zaretskii @ 2011-12-06 12:08 UTC (permalink / raw)
To: Ulrich Mueller; +Cc: emacs-devel
> Date: Tue, 6 Dec 2011 12:38:57 +0100
> From: Ulrich Mueller <ulm@gentoo.org>
>
> Sometimes I want to check if a file saved in UTF-8 encoding contains
> only characters from a certain subset of the Unicode character
> repertoire, like the MES-* [1] or WGL4 [2] subsets. (For example, for
> things published on the WWW one might rather avoid exotic characters
> for better compatibility.)
>
> My idea was now to define a coding system e.g. for MES-2 and set the
> buffer-file-coding-system accordingly, so that Emacs would check it
> upon saving the file.
>
> Now I have the following questions:
> 1. Is using coding systems for this purpose a reasonable approach
> at all, or is there a better way to achieve this?
I don't think defining a new coding-system is the best way to go about
this. I would suggest to define a new category instead, see
"(elisp)Categories" (which includes a working example), and then make
a category table (which is just a special type of char-table) where
the relevant characters are marked with the appropriate categories.
Then you can simply scan the buffer with a regexp that uses "\\Cx"
where "x" is the letter you assign to your category. (You will
probably need a separate letter for each of the MES-* and WGL4 sets.)
> 2. Where can I find documentation how to define a coding system?
You don't want to know that ;-)
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Subsets of Unicode and coding systems
2011-12-06 12:08 ` Eli Zaretskii
@ 2011-12-06 12:24 ` Werner LEMBERG
0 siblings, 0 replies; 4+ messages in thread
From: Werner LEMBERG @ 2011-12-06 12:24 UTC (permalink / raw)
To: eliz; +Cc: ulm, emacs-devel
>> 2. Where can I find documentation how to define a coding system?
>
> You don't want to know that ;-)
Tsk, tsk, tsk. This is at the border of heresy.
Werner
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Subsets of Unicode and coding systems
@ 2011-12-08 4:16 Kenichi Handa
0 siblings, 0 replies; 4+ messages in thread
From: Kenichi Handa @ 2011-12-08 4:16 UTC (permalink / raw)
To: emacs-devel; +Cc: ulm
> Date: Tue, 6 Dec 2011 12:38:57 +0100
> From: Ulrich Mueller <ulm@gentoo.org>
>
> Sometimes I want to check if a file saved in UTF-8 encoding contains
> only characters from a certain subset of the Unicode character
> repertoire, like the MES-* [1] or WGL4 [2] subsets. (For example, for
> things published on the WWW one might rather avoid exotic characters
> for better compatibility.)
>
> My idea was now to define a coding system e.g. for MES-2 and set the
> buffer-file-coding-system accordingly, so that Emacs would check it
> upon saving the file.
>
> Now I have the following questions:
> 1. Is using coding systems for this purpose a reasonable approach
> at all, or is there a better way to achieve this?
> 2. Where can I find documentation how to define a coding system?
I agree with Eli, defining proper categories provides more
flexible usage, but FYI, something like this defines the
coding system utf-8-mes-2.
(let ((repertory '((0 . #x017f) #x018f #x0192 #x01b4
(#x01de . #x01ef) (#x01fa . #x01ff)
;;; add more here
))
list)
(dolist (elm repertory)
(if (consp elm)
(let ((from (car elm))
(to (cdr elm)))
(while (<= from to)
(setq list (cons from (cons from list)))
(setq from (1+ from))))
(setq list (cons elm (cons elm list)))))
(define-charset 'unicode-mes-2 "MES-2 subset of Unicode"
:code-space [0 255 0 255]
:ascii-compatible-p t
:map (apply 'vector (nreverse list)))
(define-coding-system 'utf-8-mes-2 "UTF-8 for MES-2 subset"
:mnemonic ?M
:coding-type 'utf-8
:charset-list '(unicode-mes-2)
:mime-charset 'utf-8))
See the docstring of define-charset and define-coding-system
to understand the code.
---
Kenichi Handa
handa@m17n.org
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2011-12-08 4:16 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-12-06 11:38 Subsets of Unicode and coding systems Ulrich Mueller
2011-12-06 12:08 ` Eli Zaretskii
2011-12-06 12:24 ` Werner LEMBERG
-- strict thread matches above, loose matches on Subject: below --
2011-12-08 4:16 Kenichi Handa
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.