From: awrhygty@outlook.com
To: 65996@debbugs.gnu.org
Subject: bug#65996: 29.1; UCS normalization is wrong
Date: Fri, 15 Sep 2023 21:49:38 +0900 [thread overview]
Message-ID: <TYZPR01MB39203C7CDB53D90424AF00A6C3F6A@TYZPR01MB3920.apcprd01.prod.exchangelabs.com> (raw)
UCS normalization is wrong for some characters.
(1) NFD/NFKD decompostion is not done
U+1112E 𑄮 CHAKMA VOWEL SIGN O
U+1112F 𑄯 CHAKMA VOWEL SIGN AU
U+1134B 𑍋 GRANTHA VOWEL SIGN OO
U+1134C 𑍌 GRANTHA VOWEL SIGN AU
U+114BB 𑒻 TIRHUTA VOWEL SIGN AI
U+114BC 𑒼 TIRHUTA VOWEL SIGN O
U+114BE 𑒾 TIRHUTA VOWEL SIGN AU
U+115BA 𑖺 SIDDHAM VOWEL SIGN O
U+115BB 𑖻 SIDDHAM VOWEL SIGN AU
U+11938 𑤸 DIVES AKURU VOWEL SIGN O
(let ((s "\U0001112E\U0001112F\U0001134B\U0001134C\
\U000114BB\U000114BC\U000114BE\U000115BA\U000115BB\U00011938"))
(require 'ucs-normalize)
(list (equal s (ucs-normalize-NFD-string s))
(equal s (ucs-normalize-NFKD-string s))))
=>(t t)
(2) NFKC/NFKD replacement is not done
U+1E030..U+1E06D Cyrillic MODIFIER LETTER or SUBSCRIPT
U+1EE00..U+1EEBB ARABIC MATHEMATICAL *
U+1FBF0..U+1FBF9 SEGMENTED DIGIT *
(let* ((f (lambda (cell)
(apply #'string (number-sequence (car cell) (cdr cell)))))
(s (mapconcat f '((#x1E030 . #x1E06D)
(#x1EE00 . #x1EEBB)
(#x1FBF0 . #x1FBF9)))))
(require 'ucs-normalize)
(list (equal s (ucs-normalize-NFKC-string s))
(equal s (ucs-normalize-NFKD-string s))))
=>(t t)
In GNU Emacs 29.1 (build 2, x86_64-w64-mingw32) of 2023-08-02 built on
AVALON
Windowing system distributor 'Microsoft Corp.', version 10.0.19045
System Description: Microsoft Windows 10 Pro (v10.0.2009.19045.3448)
Configured using:
'configure --with-modules --without-dbus --with-native-compilation=aot
--without-compress-install --with-tree-sitter CFLAGS=-O2'
Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB
(NATIVE_COMP present but libgccjit not available)
Important settings:
value of $LANG: JPN
locale-coding-system: cp932
Major mode: Lisp Interaction
Minor modes in effect:
highlight-changes-visible-mode: t
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
line-number-mode: t
indent-tabs-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
Load-path shadows:
None found.
Features:
(misearch multi-isearch comp comp-cstr warnings icons rx emoji-labels
emoji multisession sqlite transient format-spec edmacro kmacro cl-extra
gnutls network-stream nsm mailalias smtpmail textsec uni-scripts url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util url-parse auth-source cl-seq
eieio eieio-core cl-macs json map url-vars idna-mapping ucs-normalize
uni-confusable textsec-check cl-print byte-opt gv bytecomp byte-compile
debug backtrace find-func hilit-chg wid-edit thingatpt help-fns
radix-tree help-mode pp shadow sort mail-extr emacsbug message mailcap
yank-media puny dired dired-loaddefs rfc822 mml mml-sec password-cache
epa derived epg rfc6068 epg-config gnus-util text-property-search
time-date subr-x mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils term/bobcat japan-util
rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel dos-w32 ls-lisp disp-table
term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty
make-network-process native-compile emacs)
Memory information:
((conses 16 331760 49630)
(symbols 48 14840 3)
(strings 32 66748 8954)
(string-bytes 1 1357518)
(vectors 16 55924)
(vector-slots 8 1637738 128446)
(floats 8 68 385)
(intervals 56 7100 2925)
(buffers 984 18))
next reply other threads:[~2023-09-15 12:49 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-15 12:49 awrhygty [this message]
2023-09-16 9:21 ` bug#65996: 29.1; UCS normalization is wrong Eli Zaretskii
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=TYZPR01MB39203C7CDB53D90424AF00A6C3F6A@TYZPR01MB3920.apcprd01.prod.exchangelabs.com \
--to=awrhygty@outlook.com \
--cc=65996@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).