From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: Kenichi Handa Newsgroups: gmane.emacs.devel Subject: Re: Cyrillic vs UTF-8 Date: Thu, 1 May 2003 17:27:42 +0900 (JST) Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: <200305010827.RAA15024@etlken.m17n.org> References: <1858-Fri25Apr2003194023+0300-eliz@elta.co.il> <200304260811.RAA08227@etlken.m17n.org> NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Trace: main.gmane.org 1051777677 10709 80.91.224.249 (1 May 2003 08:27:57 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Thu, 1 May 2003 08:27:57 +0000 (UTC) Cc: jas@extundo.com Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Thu May 01 10:27:55 2003 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 19B9Q3-0002mX-00 for ; Thu, 01 May 2003 10:27:55 +0200 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.12 #1 (Debian)) id 19B9Za-0001zb-00 for ; Thu, 01 May 2003 10:37:46 +0200 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 19B9QV-0006Tf-00 for emacs-devel@quimby.gnus.org; Thu, 01 May 2003 04:28:23 -0400 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.10.13) id 19B9Pv-0006SQ-00 for emacs-devel@gnu.org; Thu, 01 May 2003 04:27:47 -0400 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.10.13) id 19B9Pu-0006RW-00 for emacs-devel@gnu.org; Thu, 01 May 2003 04:27:47 -0400 Original-Received: from tsukuba.m17n.org ([192.47.44.130]) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 19B9Pt-0006Qj-00; Thu, 01 May 2003 04:27:45 -0400 Original-Received: from fs.m17n.org (fs.m17n.org [192.47.44.2])h418Rho29975; Thu, 1 May 2003 17:27:43 +0900 (JST) (envelope-from handa@m17n.org) Original-Received: from etlken.m17n.org (etlken.m17n.org [192.47.44.125]) h418RgA05756; Thu, 1 May 2003 17:27:43 +0900 (JST) Original-Received: (from handa@localhost) by etlken.m17n.org (8.8.8+Sun/3.7W-2001040620) id RAA15024; Thu, 1 May 2003 17:27:42 +0900 (JST) Original-To: rms@gnu.org In-reply-to: (message from Richard Stallman on Mon, 28 Apr 2003 00:38:00 -0400) User-Agent: SEMI/1.14.3 (Ushinoya) FLIM/1.14.2 (Yagi-Nishiguchi) APEL/10.2 Emacs/21.2.92 (sparc-sun-solaris2.6) MULE/5.0 (SAKAKI) Original-cc: d.love@dl.ac.uk Original-cc: eliz@elta.co.il Original-cc: emacs-devel@gnu.org X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1b5 Precedence: list List-Id: Emacs development discussions. List-Help: List-Post: List-Subscribe: , List-Archive: List-Unsubscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:13586 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:13586 In article , Richard Stallman writes: > By the way, all these problems are solved in emacs-unicode. > Could you report on what work is needed before we > can release this code? Dave has compiled the current problems in the file emacs-unicode/README.unicode. Some of them (especially serious ones) are already fixed. Dave, do you have anything else to add to that file? I think the most difficult task for releasing that code is to merge the changes into HEAD. Emacs-unicode was branched on 2002-03-01, and since then, there were a lot of changes in HEAD. --- Ken'ichi HANDA handa@m17n.org --- README.unicode --- -*-mode: text; coding: latin-1;= -*- Problems, fixmes and other issues in the emacs-unicode branch ------------------------------------------------------------- Notes by fx to record various things of variable importance. handa needs to check them -- don't take too seriously, especially with regard to completeness. _Do take seriously that you don't want this branch unless you're actually working on it; you risk your data by actually using it._ If you just want to edit Unicode and/or unify iso-8859 et al, see the existing support and the extra stuff at , mostly now in the CVS trunk. (Editing support is mostly orthogonal to the internal representation.) * SINGLE_BYTE_CHAR_P returns true for Latin-1 characters, which has undesirable effects. E.g.: (multibyte-string-p (let ((s "x")) (aset s 0 ?=C2=A3) s)) =3D> nil (multibyte-string-p (concat [?=C2=A3])) =3D> nil (text-char-description ?=C2=A3) =3D> "M-#" These examples are all fixed by the change of 2002-10-14, but there still exist questionalble SINGLE_BYTE_CHAR_P in the code. * Rationalize character syntax and its relationship to the Unicode database. (Applies mainly to symbol an punctuation syntax.) * Fontset handling and customization needs work. We want to relate fonts to scripts, probably based on the Unicode blocks. The presence of small-repertoire 10646-encoded fonts in XFree 4 is a pain, not currently worked round. With the change on 2002-07-26, multiple fonts can be specified in a fontset for a specific range of characters. Each range can also be specified by script. Before using ISO10646 fonts, Emacs checks their repertories to avoid such fonts that don't have a glyph for a specific character. * Work is also needed on charset and coding system priorities. * The relevant bits of latin1-disp.el need porting (and probably re-naming/updating). See also cyril-util.el. * Quail files need more work now the encoding is irrelevant. * What to do with the old coding categories stuff? * The preferred-coding-system property of charsets should probably be junked unless it can be made more useful now. * find-multibyte-characters needs looking at. * Implement Korean cp949/UHC, BIG5-HKSCS and any other important missing charsets. * Check up on definition of alternativnj. * Lazy-load tables for unify-charset somehow? Actually, Emacs clear out all charset maps and unify-map just before dumping, and their are loaded again on demand the dumped emacs. But, those maps (char tables) generated while temacs is running can't be get rid of from the dumped emacs. * Translation tables for {en,de}code currently aren't supported. This should be fixed by the changes of 2002-10-14. * Defining CCL coding systems currently doesn't work. This should be fixed by the changes of 2003-01-30. * iso-2022 charsets get unified on i/o. With the change on 2003-01-06, decoding routines put `charset' property to decoded text, and iso-2022 encoder pay attention to it. Thus, for instance, reading and writing by iso-2022-7bit preserve the original designation sequences. The property name `preferred-charset' may be better? We may have to utilize this property to decide a font. * Revisit locale processing: look at treating the language and charset parts separately. (Language should affect things like speling and calendar, but that's not a Unicode issue.) * Handle Unicode combining characters usefully, e.g. diacritics, and handle more scripts specifically (=C3=A0 la Devanagari). There are issues with canonicalization. * Bidi is a separate issue with no support currently. * We need tabular input methods, e.g. for maths symbols. (Not specific to Unicode.) * Need multibyte text in menus, e.g. for the above. (Not specific to Unicode.) * There's currently no support for Unicode normalization. * Populate char-width-table correctly for Unicode chanaracters and worry about what happens when double-width charsets covering non-CJK characters are unified. * Emacs 20/21 .elc files are currently not loadable. It may or may not be possible to do this properly. With the change on 2002-07-24, elc files generated by Emacs 20.3 and later are correctly loaded (including those containing multibyte characters and compressed). But, elc files generated by 20.2 and the primer are still not loadable. Is it really worth working on it? * Rmail won't work with non-ASCII text. Encoding issues for Babyl files need sorting out, but rms says Babyl will go before this is released. * Gnus still needs some attention, and we need to get changes accepted by Gnus maintainers... * There are type errors lurking, e.g. in Fcheck_coding_systems_region. Define ENABLE_CHECKING to find them. * You can grep the code for lots of fixmes. * Old auto-save files, and similar files, such as Gnus drafts, containing non-ASCII characters probably won't be re-read correctly.