unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
From: awrhygty@outlook.com
To: 65996@debbugs.gnu.org
Subject: bug#65996: 29.1; UCS normalization is wrong
Date: Fri, 15 Sep 2023 21:49:38 +0900	[thread overview]
Message-ID: <TYZPR01MB39203C7CDB53D90424AF00A6C3F6A@TYZPR01MB3920.apcprd01.prod.exchangelabs.com> (raw)


UCS normalization is wrong for some characters.

(1) NFD/NFKD decompostion is not done
    U+1112E 𑄮 CHAKMA VOWEL SIGN O
    U+1112F 𑄯 CHAKMA VOWEL SIGN AU
    U+1134B 𑍋 GRANTHA VOWEL SIGN OO
    U+1134C 𑍌 GRANTHA VOWEL SIGN AU
    U+114BB 𑒻 TIRHUTA VOWEL SIGN AI
    U+114BC 𑒼 TIRHUTA VOWEL SIGN O
    U+114BE 𑒾 TIRHUTA VOWEL SIGN AU
    U+115BA 𑖺 SIDDHAM VOWEL SIGN O
    U+115BB 𑖻 SIDDHAM VOWEL SIGN AU
    U+11938 𑤸 DIVES AKURU VOWEL SIGN O

    (let ((s "\U0001112E\U0001112F\U0001134B\U0001134C\
    \U000114BB\U000114BC\U000114BE\U000115BA\U000115BB\U00011938"))
      (require 'ucs-normalize)
      (list (equal s (ucs-normalize-NFD-string s))
            (equal s (ucs-normalize-NFKD-string s))))
    =>(t t)

(2) NFKC/NFKD replacement is not done
    U+1E030..U+1E06D Cyrillic MODIFIER LETTER or SUBSCRIPT
    U+1EE00..U+1EEBB ARABIC MATHEMATICAL *
    U+1FBF0..U+1FBF9 SEGMENTED DIGIT *

    (let* ((f (lambda (cell)
                (apply #'string (number-sequence (car cell) (cdr cell)))))
           (s (mapconcat f '((#x1E030 . #x1E06D)
                             (#x1EE00 . #x1EEBB)
                             (#x1FBF0 . #x1FBF9)))))
      (require 'ucs-normalize)
      (list (equal s (ucs-normalize-NFKC-string s))
            (equal s (ucs-normalize-NFKD-string s))))
    =>(t t)


In GNU Emacs 29.1 (build 2, x86_64-w64-mingw32) of 2023-08-02 built on
 AVALON
Windowing system distributor 'Microsoft Corp.', version 10.0.19045
System Description: Microsoft Windows 10 Pro (v10.0.2009.19045.3448)

Configured using:
 'configure --with-modules --without-dbus --with-native-compilation=aot
 --without-compress-install --with-tree-sitter CFLAGS=-O2'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

(NATIVE_COMP present but libgccjit not available)

Important settings:
  value of $LANG: JPN
  locale-coding-system: cp932

Major mode: Lisp Interaction

Minor modes in effect:
  highlight-changes-visible-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(misearch multi-isearch comp comp-cstr warnings icons rx emoji-labels
emoji multisession sqlite transient format-spec edmacro kmacro cl-extra
gnutls network-stream nsm mailalias smtpmail textsec uni-scripts url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util url-parse auth-source cl-seq
eieio eieio-core cl-macs json map url-vars idna-mapping ucs-normalize
uni-confusable textsec-check cl-print byte-opt gv bytecomp byte-compile
debug backtrace find-func hilit-chg wid-edit thingatpt help-fns
radix-tree help-mode pp shadow sort mail-extr emacsbug message mailcap
yank-media puny dired dired-loaddefs rfc822 mml mml-sec password-cache
epa derived epg rfc6068 epg-config gnus-util text-property-search
time-date subr-x mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils term/bobcat japan-util
rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel dos-w32 ls-lisp disp-table
term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 331760 49630)
 (symbols 48 14840 3)
 (strings 32 66748 8954)
 (string-bytes 1 1357518)
 (vectors 16 55924)
 (vector-slots 8 1637738 128446)
 (floats 8 68 385)
 (intervals 56 7100 2925)
 (buffers 984 18))





             reply	other threads:[~2023-09-15 12:49 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-15 12:49 awrhygty [this message]
2023-09-16  9:21 ` bug#65996: 29.1; UCS normalization is wrong Eli Zaretskii

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/emacs/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=TYZPR01MB39203C7CDB53D90424AF00A6C3F6A@TYZPR01MB3920.apcprd01.prod.exchangelabs.com \
    --to=awrhygty@outlook.com \
    --cc=65996@debbugs.gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).