unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* ignoring combining diacritics in isearch
@ 2022-11-23 17:27 Robert Pluim
  2022-11-23 17:45 ` Juri Linkov
  2022-11-23 18:02 ` Eli Zaretskii
  0 siblings, 2 replies; 3+ messages in thread
From: Robert Pluim @ 2022-11-23 17:27 UTC (permalink / raw)
  To: emacs-devel

Over on Stack Overflow, someone has been trying to get char-folded
isearch working for Arabic, and has been having some issues because
char-folding only works for equivalent characters, not base characters
followed by combining characters. So eg searching for 'ee' when the
buffer contains

    éé

(thatʼs 'e' followed by COMBINING ACUTE ACCENT) fails.

The following patch fixes that, but itʼs a bit of a sledgehammer (the
"\\c^*" bit probably needs to be configurable, because there are
diacritic-like codepoints in Arabic that are not combining, such as
U+0640 ARABIC TATWEEL)

diff --git c/lisp/char-fold.el i/lisp/char-fold.el
index 43e3cd45ec..8e9fdd7f37 100644
--- c/lisp/char-fold.el
+++ i/lisp/char-fold.el
@@ -209,7 +209,11 @@
         ;; is used by `describe-char-fold-equivalences'.
         (map-char-table
          (lambda (char decomp-list)
-           (let ((re (regexp-opt (cons (char-to-string char) decomp-list))))
+           (let ((re
+                  (concat "\\(?:"
+                          (string-join (cons (char-to-string char) decomp-list)
+                                       "\\c^*\\|")
+                          "\\c^*\\)")))
              (aset equiv char re)))
          equiv))
       equiv)))

Thoughts?

Robert
-- 



^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: ignoring combining diacritics in isearch
  2022-11-23 17:27 ignoring combining diacritics in isearch Robert Pluim
@ 2022-11-23 17:45 ` Juri Linkov
  2022-11-23 18:02 ` Eli Zaretskii
  1 sibling, 0 replies; 3+ messages in thread
From: Juri Linkov @ 2022-11-23 17:45 UTC (permalink / raw)
  To: Robert Pluim; +Cc: emacs-devel

> Over on Stack Overflow, someone has been trying to get char-folded
> isearch working for Arabic, and has been having some issues because
> char-folding only works for equivalent characters, not base characters
> followed by combining characters. So eg searching for 'ee' when the
> buffer contains
>
>     éé
>
> (thatʼs 'e' followed by COMBINING ACUTE ACCENT) fails.

It doesn't fail when I tried.  What am I doing wrong?



^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: ignoring combining diacritics in isearch
  2022-11-23 17:27 ignoring combining diacritics in isearch Robert Pluim
  2022-11-23 17:45 ` Juri Linkov
@ 2022-11-23 18:02 ` Eli Zaretskii
  1 sibling, 0 replies; 3+ messages in thread
From: Eli Zaretskii @ 2022-11-23 18:02 UTC (permalink / raw)
  To: Robert Pluim; +Cc: emacs-devel

> From: Robert Pluim <rpluim@gmail.com>
> Date: Wed, 23 Nov 2022 18:27:25 +0100
> 
> Over on Stack Overflow, someone has been trying to get char-folded
> isearch working for Arabic, and has been having some issues because
> char-folding only works for equivalent characters, not base characters
> followed by combining characters. So eg searching for 'ee' when the
> buffer contains
> 
>     éé
> 
> (thatʼs 'e' followed by COMBINING ACUTE ACCENT) fails.
> 
> The following patch fixes that, but itʼs a bit of a sledgehammer (the
> "\\c^*" bit probably needs to be configurable, because there are
> diacritic-like codepoints in Arabic that are not combining, such as
> U+0640 ARABIC TATWEEL)

Yes, this is definitely not the way.  There are many more "foldings" that
Latin scripts don't know about.  For example, it should be possible to fold
the initial, medial, and final forms of letters that exist in some scripts
(including Arabic).

I think we've all but reached the limit to which this quasi-folding via
regexps can be stretched.  Writing regexp by hand or semi-mechanically based
on Unicode properties can only go this far.  _Real_ character folding cannot
work this way.  We should work on infrastructure for folding text for search
purposes, and then we can build features on top of that.



^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2022-11-23 18:02 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-23 17:27 ignoring combining diacritics in isearch Robert Pluim
2022-11-23 17:45 ` Juri Linkov
2022-11-23 18:02 ` Eli Zaretskii

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).