From: Eli Zaretskii <eliz@gnu.org>
To: awrhygty@outlook.com
Cc: 65996-done@debbugs.gnu.org
Subject: bug#65996: 29.1; UCS normalization is wrong
Date: Sat, 16 Sep 2023 12:21:42 +0300 [thread overview]
Message-ID: <83sf7eiic9.fsf@gnu.org> (raw)
In-Reply-To: <TYZPR01MB39203C7CDB53D90424AF00A6C3F6A@TYZPR01MB3920.apcprd01.prod.exchangelabs.com> (awrhygty@outlook.com)
> From: awrhygty@outlook.com
> Date: Fri, 15 Sep 2023 21:49:38 +0900
>
>
> UCS normalization is wrong for some characters.
>
> (1) NFD/NFKD decompostion is not done
> U+1112E 𑄮 CHAKMA VOWEL SIGN O
> U+1112F 𑄯 CHAKMA VOWEL SIGN AU
> U+1134B 𑍋 GRANTHA VOWEL SIGN OO
> U+1134C 𑍌 GRANTHA VOWEL SIGN AU
> U+114BB 𑒻 TIRHUTA VOWEL SIGN AI
> U+114BC 𑒼 TIRHUTA VOWEL SIGN O
> U+114BE 𑒾 TIRHUTA VOWEL SIGN AU
> U+115BA 𑖺 SIDDHAM VOWEL SIGN O
> U+115BB 𑖻 SIDDHAM VOWEL SIGN AU
> U+11938 𑤸 DIVES AKURU VOWEL SIGN O
>
> (let ((s "\U0001112E\U0001112F\U0001134B\U0001134C\
> \U000114BB\U000114BC\U000114BE\U000115BA\U000115BB\U00011938"))
> (require 'ucs-normalize)
> (list (equal s (ucs-normalize-NFD-string s))
> (equal s (ucs-normalize-NFKD-string s))))
> =>(t t)
>
> (2) NFKC/NFKD replacement is not done
> U+1E030..U+1E06D Cyrillic MODIFIER LETTER or SUBSCRIPT
> U+1EE00..U+1EEBB ARABIC MATHEMATICAL *
> U+1FBF0..U+1FBF9 SEGMENTED DIGIT *
>
> (let* ((f (lambda (cell)
> (apply #'string (number-sequence (car cell) (cdr cell)))))
> (s (mapconcat f '((#x1E030 . #x1E06D)
> (#x1EE00 . #x1EEBB)
> (#x1FBF0 . #x1FBF9)))))
> (require 'ucs-normalize)
> (list (equal s (ucs-normalize-NFKC-string s))
> (equal s (ucs-normalize-NFKD-string s))))
> =>(t t)
Thanks, fixed on the emacs-29 branch.
Once again, if (as I'm guessing) you found these problems by examining
the data in ucs-normalize.el, it would have greatly helped if you'd
pointed to the problematic data in your report. Reverse-engineering
the sources of the problem from the behavior takes time, especially
when the relevant code is not trivial and was written by someone else.
prev parent reply other threads:[~2023-09-16 9:21 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-15 12:49 bug#65996: 29.1; UCS normalization is wrong awrhygty
2023-09-16 9:21 ` Eli Zaretskii [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=83sf7eiic9.fsf@gnu.org \
--to=eliz@gnu.org \
--cc=65996-done@debbugs.gnu.org \
--cc=awrhygty@outlook.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.