From: Nathan Trapuzzano <nbtrap@nbtrap.com>
To: Eli Zaretskii <eliz@gnu.org>
Cc: 17130@debbugs.gnu.org
Subject: bug#17130: 24.4.50; Deficient Unicode case folding
Date: Sat, 29 Mar 2014 14:31:52 -0400 [thread overview]
Message-ID: <8761mwua93.fsf@nbtrap.com> (raw)
In-Reply-To: <831txkewil.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 29 Mar 2014 20:37:38 +0300")
Eli Zaretskii <eliz@gnu.org> writes:
>> > So you would need to search all characters to find those which have σ
>> > in the CANONICALIZE slot -- not very efficient, to say the least.
>>
>> Doesn't this already happen?
>
> No, not when that slot is used for case-insensitive search. You just
> use it to get the canonical equivalent, i.e. use the one-way mapping
> that it provides.
I still don't get it. What I say below may explain why.
>> If not, then what is the CANONICALIZE slot doing that couldn't be
>> done with the regular upcase/downcase slots by themselves?
>
> If that slot is "trivial", i.e. contains the lower-case variant of the
> character, then indeed this slot doesn't add information, I think,
> only utility. But it doesn't have to contain the lower-case variant.
I know. But if Emacs doesn't do Unicode folding, what is there other
than lower/upper variants?
>> > IOW, what you suggest will provide a one-way mapping, whereas we need
>> > a two-way mapping.
>>
>> Not sure I follow. Seems to me the CANONICALIZE slot is sufficient, at
>> least in principle.
>
> It is sufficient for mapping a character to its canonical equivalent,
> but not finding the non-canonical variants of a canonical character.
> IOW, it is not well suited to finding ς given just σ.
Finding the non-canonical variants is not something that happens (at
least in principle) during case-insensitive matching. You convert both
the matching string and the string being matched into their canonical
equivalents and see if they match. You never UNfold. Case folding is
by definition a one-way operation.
>> That said, making M-l downcase terminal Σ to ς would be a nice feature
>> that could be enabled, e.g., by enabling a minor mode or by modifying
>> some *-functions variable of functions that get called before the normal
>> behavior of M-l is applied, etc. But it shouldn't have anything to do
>> with Unicode-compliant case-insensitive searching.
>
> For searching, you only need the CANONICALIZE slot. But what about
> replacing the search string while keeping the letter case in the
> replacement? For that, CANONICALIZE alone is not enough, you need the
> reverse mapping.
There is no reverse mapping when it comes to folding. There can't be,
since multiple characters can fold into the same character.
I don't fully understand what "case-replace" does (e.g. case being a
property of characters and not strings, what does it mean to "preserve
case" when replacing a string of length x with a string of length y
where x != y), but I don't think Unicode folding would complicate it.
There are three cases in Unicode: lower, upper, and title. Upper and
title already overlap for the vast majority of codepoints, so there you
already have problems with a case-preserving replace. That said "fold"
is not a case in Unicode; it's a one-way mapping of non-overlapping sets
of characters to a canonical equivalent, so it makes no sense to talk
about preserving case with respect to case folding.
Notandum: I was wrong about Unicode saying nothing about character
ordering for non-combining characters. The "special casing" document
(ftp://ftp.unicode.org/Public/UCD/latest/ucd/SpecialCasing.txt) contains
context- and language- dependent case rules for certain characters,
including final sigma. Notably, the document says that Σ in terminal
position should (or "may"--I'm not really sure about how to interpret
the document) downcase to ς. That said, the document has _nothing_ to
do with case _folding_, which is always context- and language-
independent.
Rightly interpreted, therefore, case _conversion_ (such as in
case-preserving replace) and case-insensitive _searching_ (i.e. case
folding), according to Unicode, are orthogonal. We don't have to
address both at the same time.
>> Given the point about ß, you're probably right. Unless we can make
>> entries in the CANONICALIZE slot be strings rather than code points.
>
> This is Lisp; a vector slot can contain any Lisp object. But using
> CANONICALIZE for what you want would be wrong, I think, because it
> will screw up case-insensitive search, which expects to find there a
> single character.
Right, that's what I meant. Putting strings there would break
something.
next prev parent reply other threads:[~2014-03-29 18:31 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-03-28 12:07 bug#17130: 24.4.50; Deficient Unicode case folding Nathan Trapuzzano
2014-03-28 15:51 ` Eli Zaretskii
2014-03-28 19:31 ` nbtrap
2014-03-29 6:45 ` Eli Zaretskii
[not found] ` <87ob0pnptc.fsf@nbtrap.com>
2014-03-29 13:15 ` Eli Zaretskii
2014-03-29 14:03 ` Nathan Trapuzzano
2014-03-29 14:45 ` Eli Zaretskii
2014-03-29 15:29 ` Nathan Trapuzzano
2014-03-29 17:37 ` Eli Zaretskii
2014-03-29 18:31 ` Nathan Trapuzzano [this message]
2014-03-29 18:36 ` Nathan Trapuzzano
2014-03-29 19:51 ` Eli Zaretskii
2014-03-29 20:15 ` Nathan Trapuzzano
2014-03-30 2:45 ` Eli Zaretskii
2014-03-29 19:50 ` Eli Zaretskii
2014-03-29 20:01 ` Nathan Trapuzzano
2019-09-29 14:23 ` Lars Ingebrigtsen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8761mwua93.fsf@nbtrap.com \
--to=nbtrap@nbtrap.com \
--cc=17130@debbugs.gnu.org \
--cc=eliz@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.