From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#13041: 24.2; diacritic-fold-search Date: Sun, 02 Dec 2012 20:16:17 +0200 Message-ID: <83wqx0s4jy.fsf@gnu.org> References: <20121130182205.C722F14B8D@panix1.panix.com> <87hao69b5r.fsf@mail.jurta.org> <20665.8224.844876.619203@panix5.panix.com> <87hao6zko4.fsf@mail.jurta.org> <83fw3qtboc.fsf@gnu.org> <87hao5jqu3.fsf@mail.jurta.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1354472273 31627 80.91.229.3 (2 Dec 2012 18:17:53 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 2 Dec 2012 18:17:53 +0000 (UTC) Cc: perin@panix.com, 13041@debbugs.gnu.org, perin@acm.org To: Juri Linkov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Dec 02 19:18:05 2012 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1TfE7B-0007VJ-GN for geb-bug-gnu-emacs@m.gmane.org; Sun, 02 Dec 2012 19:18:05 +0100 Original-Received: from localhost ([::1]:44453 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TfE6z-0000as-Ra for geb-bug-gnu-emacs@m.gmane.org; Sun, 02 Dec 2012 13:17:53 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:54880) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TfE6v-0000US-B0 for bug-gnu-emacs@gnu.org; Sun, 02 Dec 2012 13:17:51 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TfE6r-0007aZ-47 for bug-gnu-emacs@gnu.org; Sun, 02 Dec 2012 13:17:49 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:40109) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TfE6r-0007aF-0V for bug-gnu-emacs@gnu.org; Sun, 02 Dec 2012 13:17:45 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.72) (envelope-from ) id 1TfE94-0005rK-08 for bug-gnu-emacs@gnu.org; Sun, 02 Dec 2012 13:20:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 02 Dec 2012 18:20:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 13041 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 13041-submit@debbugs.gnu.org id=B13041.135447236122452 (code B ref 13041); Sun, 02 Dec 2012 18:20:01 +0000 Original-Received: (at 13041) by debbugs.gnu.org; 2 Dec 2012 18:19:21 +0000 Original-Received: from localhost ([127.0.0.1]:50357 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TfE8O-0005q4-KI for submit@debbugs.gnu.org; Sun, 02 Dec 2012 13:19:20 -0500 Original-Received: from mtaout22.012.net.il ([80.179.55.172]:53401) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TfE8L-0005pw-GX for 13041@debbugs.gnu.org; Sun, 02 Dec 2012 13:19:18 -0500 Original-Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MEF004001DEGX00@a-mtaout22.012.net.il> for 13041@debbugs.gnu.org; Sun, 02 Dec 2012 20:16:25 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MEF004081FC37A0@a-mtaout22.012.net.il>; Sun, 02 Dec 2012 20:16:25 +0200 (IST) In-reply-to: <87hao5jqu3.fsf@mail.jurta.org> X-012-Sender: halo1@inter.net.il X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:67790 Archived-At: > From: Juri Linkov > Cc: perin@panix.com, 13041@debbugs.gnu.org, perin@acm.org > Date: Sun, 02 Dec 2012 02:27:32 +0200 > > I'm surprised to see case mappings hard-coded in > lisp/international/characters.el instead of using the properties > `uppercase' and `lowercase' during creation of case tables. My guess is that this is because the code in characters.el was written long before we had access to Unicode character properties in Emacs, and in fact before Emacs was switched to character representation based on Unicode codepoints. And no one bothered to rewrite that code since then; volunteers are welcome. > (defvar decomposition-table nil) > > (defun make-decomposition-table () > (let ((table (standard-case-table)) > canon) > (setq canon (copy-sequence table)) > (let ((c #x0000) d) > (while (<= c #xFFFD) > (make-decomposition-table-1 canon c c) > (setq c (1+ c)))) > (set-char-table-extra-slot table 1 canon) > (set-char-table-extra-slot table 2 nil) > (setq decomposition-table table))) > > (defun make-decomposition-table-1 (canon c0 c1) > (let ((d (get-char-code-property c1 'decomposition))) > (when d > (unless (characterp (car d)) (pop d)) > (if (eq c1 (car d)) > (aset canon c0 (car d)) > (make-decomposition-table-1 canon c0 (car d)))))) > > (make-decomposition-table) > > Then a new Isearch command (the existing `isearch-toggle-case-fold' > can't be used because it enables/disables the standard case table) > could toggle between the current case table and the decomposition > case table using > > (set-case-table decomposition-table) > > After evaluating this, Isearch correctly finds all related characters > in every row of this example: > > http://hex-machina.com/scripts/yui/3.3.0pr1/api/unicode-data-accentfold.js.html > > But it seems using the case table for decomposition has one limitation. > I see no way to ignore combining accent characters in the case table, > i.e. to map combining accent characters to nothing. These characters > have the general-category "Mn (Mark, Nonspacing)", so they should be ignored > in the search. IMO, using case tables for this is evil. If I want to "fold" diacritics in search, that doesn't necessarily mean I want to fold the letter-case as well. I might want doing that, or I might not; these are two orthogonal features. So we need a separate kind of char-table, one that could be installed in addition to the case table, and one that will interpret nil as an indication to ignore the character during search. Then we will be able to ignore combining accents, as we indeed should. We also need to modify the searching primitives to consult this new table, in addition to case table. IOW, I don't think we can implement this feature entirely in Lisp. Some changes are needed on the C level as well.