From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: awrhygty@outlook.com Newsgroups: gmane.emacs.bugs Subject: bug#65996: 29.1; UCS normalization is wrong Date: Fri, 15 Sep 2023 21:49:38 +0900 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="11972"; mail-complaints-to="usenet@ciao.gmane.io" To: 65996@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Sep 15 14:51:18 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qh8I1-0002x1-B1 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 15 Sep 2023 14:51:17 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qh8Hj-0007Hm-0L; Fri, 15 Sep 2023 08:50:59 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qh8Hg-0007HW-Uv for bug-gnu-emacs@gnu.org; Fri, 15 Sep 2023 08:50:56 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qh8Hg-0007X4-Le for bug-gnu-emacs@gnu.org; Fri, 15 Sep 2023 08:50:56 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qh8Hm-0007ML-Ju for bug-gnu-emacs@gnu.org; Fri, 15 Sep 2023 08:51:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: awrhygty@outlook.com Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 15 Sep 2023 12:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 65996 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.169478220828202 (code B ref -1); Fri, 15 Sep 2023 12:51:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 15 Sep 2023 12:50:08 +0000 Original-Received: from localhost ([127.0.0.1]:42398 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qh8Gt-0007Kn-SI for submit@debbugs.gnu.org; Fri, 15 Sep 2023 08:50:08 -0400 Original-Received: from lists.gnu.org ([2001:470:142::17]:48258) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qh8Gq-0007KF-OX for submit@debbugs.gnu.org; Fri, 15 Sep 2023 08:50:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qh8Ge-00074T-NQ for bug-gnu-emacs@gnu.org; Fri, 15 Sep 2023 08:49:52 -0400 Original-Received: from mail-psaapc01olkn2078.outbound.protection.outlook.com ([40.92.52.78] helo=APC01-PSA-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qh8Gc-000780-Pa for bug-gnu-emacs@gnu.org; Fri, 15 Sep 2023 08:49:52 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UuRSLe1V5BCB7yxZpvwcS42kc5yUuSY4L/nEApOtUFfpgNEv6SzxjqixXbZ0HhzRlS5SD60fDk+mYrJqg/6ic4F1Mni6RlUfkIWMr28DxIsAR77HaexoiRSTtOl7yhAr7oxSgR3SFmXPjasYHy8t6/Qqvji68HnppOYdKHyMyAaYBiKwR5Vhj0volt+uLP6+bHN8smskuuxawWrA6HqmJ8BrUBfdKJCw4rLdtYuiFUA532wMTO0vfepNLxDosbkKtNyOTxnJdPAZlA9BKpTFSUanpsp3mHt3+nk0qHvytojBkaJ9qsPkE72fuxiJULCKmG0vNRciMnUwED7Uhhs2xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Cl7QLsl2bmqAZ8gvIhMD6nJ2qzmJca0f67pnn/JR9JA=; b=QBjUlajkOlSYgS0yiTh+OgAhhT62uNd0Vpm7u0all38ibK5OYbEvu0CsGit32V3EYbq63mm8PAQ+k5GgEnXiFlwQ/B2xJ1So2RWhw1jq6oaLT1SpSvBGhNp4NNQGU0aQFP99h0ymB5EZw4NOEQc/aHQcL1b7wpiAUV2HWS9osIiMXdTUyt5bmqcHE9rNlESHqgUX5zP27XliCswCdFoYkYCoVMbL6iA3uldO/kb9AK8OOhw9rpqJxkN6T6zNSaH1lWbEzFPa1neVW8TsbeLGCmueujv/FvKm9j8tnoHMTI1gZCfZln0bPyR5pUCG4FDbAb/nh0zYZ3aDIUYZerIGow== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cl7QLsl2bmqAZ8gvIhMD6nJ2qzmJca0f67pnn/JR9JA=; b=WuuB9dcsognAJTIwAEFmOtdDhs5DQyuhSMU/tcvH2YWQltq0nNtwQKRHOtF/LQGs+BWIfhUKH15tIsPpT+qFRuOpBYRrYMaLsDCapIYjUPEc3rm42hOboWzdCL6xOWlVxaU0n8kpAPyp5YNzQ8nwmuY981XLWnUzED8ci3Mf3mP2wRyJJfBqtTR5LjPzhg8ZUvwrXoNXl71SEeAfdXhr346lLhW6KYXtTXNwxYEKY4Q+aic3STQPywifPRBgIsfrAJqYkv5fFKNenKG2jgD3eMEuPqffl5dHlBXBLOSi9ChqQpS7ZuGadQ3SI1f7mMLVIFKPIACiYXJcYjJG3dk1Xg== Original-Received: from TYZPR01MB3920.apcprd01.prod.exchangelabs.com (2603:1096:400:30::11) by TYSPR01MB5679.apcprd01.prod.exchangelabs.com (2603:1096:400:412::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.21; Fri, 15 Sep 2023 12:49:40 +0000 Original-Received: from TYZPR01MB3920.apcprd01.prod.exchangelabs.com ([fe80::4ba1:740d:9058:2c68]) by TYZPR01MB3920.apcprd01.prod.exchangelabs.com ([fe80::4ba1:740d:9058:2c68%4]) with mapi id 15.20.6792.021; Fri, 15 Sep 2023 12:49:40 +0000 X-TMN: [bjAdSiu2TsUNawgBJAcpOiKq3bzCmtyc] X-ClientProxiedBy: TYCP286CA0194.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:382::8) To TYZPR01MB3920.apcprd01.prod.exchangelabs.com (2603:1096:400:30::11) X-Microsoft-Original-Message-ID: <86fs3f38kd.fsf@outlook.com> X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TYZPR01MB3920:EE_|TYSPR01MB5679:EE_ X-MS-Office365-Filtering-Correlation-Id: 9b0545ad-96c4-4c4a-5172-08dbb5ea39ba X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7IW873a6WWsXMJ0yS2M3NLWwtBkCZQU5PQt/aW0lCOaeIDbC52+K6ybik/vXtu0DvDSTLN1k9bHEl2YQhCFiH1Qho9cF+tb64bo3uGH9033n5HSFTOJtspReipG7GArBCmMP9CB5/1HPEYXjkp8w4gvr/O0k3NC9f3U8O1t3j92Pd3OcijxZhbtkP87pgtfCY+4JbRqrdWUyWYZQ+FJcOMgX9xP/weLJ79QBi6dRO2PFv5i8f86A/gvlK5ISODYVwjj32ihmT+cbNFIBd7RjUU1eKtX/mYYsHHjv29Z57irth9TDeh4HhRm4dlEzT5Ylud4lxmnCb/fTiUPFCMvTkMz+E8QMabKmHyFm4LPms2rl07WV/4nBje0W258ScTbFcTouY0f/8OC3zb/USFEwMX1gHXMkQ9JFfhyPUYlM1WPB59K33OSHVmXAj0inviHp4iqjGeRddZYjD4kWWGw3Kp+KHYrv+yUIMV8AiYECCoyCFYyD42re4Ldbl8Vi/qcqKccHaL7IjmH0p1VlJVuN9YgT5P+RXSXtX/T9Ir6ZV8j2uJzUyx0EbIcPh7SDbmlS X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: F6Ca/69VawfhExoQJIbr2XhiqfjI8rk04SAZnh27mFgRTZY6WyOVMhk7pJAeVF0u0H3AGtsWFjwKxDkYm+wfV9loQbk2Eb0GmUNScpFv4+J14NWaDuUVbq7hI7vw02ZGYw2Z48zaQfBy62y7sMPd9wG1grO/o9b3KELovwVnruLxQT/4HgjUEwDcRRxOyhTqDSkbwfb9IikmlWcCElzGohQrRmoR8E1p/2/dfyXGcB8Xjb2Ys+OFMab9ydnkPEeKanQufVViLgfM80W+RWM/Uzor5hF5IS5wLB7GNqsIPErO0/0epaOVRUc7JwOOOxTEVePv/rD+6J0JFI7+DFHcuxTjEiDo4urEry5ZmV9B+G2CkJQ+dRZHgDR8h6Swj/dqNYaWM0or7uiuLPIDkbUNDkutVTWyT5bRbOq+M/wcN8aXwT5cntxL4uOvuLBvZ7vIREuKZsXiSsjyKUKN1SToNwPVbU7Iy+NbeEk6wwm3pRbh7RGg5etDRoAb76UySd04n5W5oaINR5ZzTNzBYI5Gid8pZxUxDW3zSwYdgBXIBQ9p7qGYO66MhLtE/iETKkicAq7wbbXRGRpZ/kPa5pdKmurl67uQlQvrvxxOPXqON1cH/mYJoAdg0Pu7+tE8uRb9v9h1K6rpPSwPyp5BPqYzvC2PG6pdTxV3xQtkQCvWPdpR9hwN/LgXG85H3YheN0Sv87ydDC4rlQGR2HWQzE+Dgv9Lf2UjNiuwQwLw/V9sWpFaDVT61J4n1ZrbuE FkZ8IByMkQUXPIQ3wA5B7naTK9TTcl9aqX40mSVPAHD2q+OEbv/iz5uRZTxy5uVh+5wS/RrTQ4as1hpgBQ2NmbwWomqYPtLDG9 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b0545ad-96c4-4c4a-5172-08dbb5ea39ba X-MS-Exchange-CrossTenant-AuthSource: TYZPR01MB3920.apcprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2023 12:49:40.0164 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYSPR01MB5679 Received-SPF: pass client-ip=40.92.52.78; envelope-from=awrhygty@outlook.com; helo=APC01-PSA-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:270533 Archived-At: UCS normalization is wrong for some characters. (1) NFD/NFKD decompostion is not done U+1112E =F0=91=84=AE CHAKMA VOWEL SIGN O U+1112F =F0=91=84=AF CHAKMA VOWEL SIGN AU U+1134B =F0=91=8D=8B GRANTHA VOWEL SIGN OO U+1134C =F0=91=8D=8C GRANTHA VOWEL SIGN AU U+114BB =F0=91=92=BB TIRHUTA VOWEL SIGN AI U+114BC =F0=91=92=BC TIRHUTA VOWEL SIGN O U+114BE =F0=91=92=BE TIRHUTA VOWEL SIGN AU U+115BA =F0=91=96=BA SIDDHAM VOWEL SIGN O U+115BB =F0=91=96=BB SIDDHAM VOWEL SIGN AU U+11938 =F0=91=A4=B8 DIVES AKURU VOWEL SIGN O (let ((s "\U0001112E\U0001112F\U0001134B\U0001134C\ \U000114BB\U000114BC\U000114BE\U000115BA\U000115BB\U00011938")) (require 'ucs-normalize) (list (equal s (ucs-normalize-NFD-string s)) (equal s (ucs-normalize-NFKD-string s)))) =3D>(t t) (2) NFKC/NFKD replacement is not done U+1E030..U+1E06D Cyrillic MODIFIER LETTER or SUBSCRIPT U+1EE00..U+1EEBB ARABIC MATHEMATICAL * U+1FBF0..U+1FBF9 SEGMENTED DIGIT * (let* ((f (lambda (cell) (apply #'string (number-sequence (car cell) (cdr cell))))) (s (mapconcat f '((#x1E030 . #x1E06D) (#x1EE00 . #x1EEBB) (#x1FBF0 . #x1FBF9))))) (require 'ucs-normalize) (list (equal s (ucs-normalize-NFKC-string s)) (equal s (ucs-normalize-NFKD-string s)))) =3D>(t t) In GNU Emacs 29.1 (build 2, x86_64-w64-mingw32) of 2023-08-02 built on AVALON Windowing system distributor 'Microsoft Corp.', version 10.0.19045 System Description: Microsoft Windows 10 Pro (v10.0.2009.19045.3448) Configured using: 'configure --with-modules --without-dbus --with-native-compilation=3Daot --without-compress-install --with-tree-sitter CFLAGS=3D-O2' Configured features: ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB (NATIVE_COMP present but libgccjit not available) Important settings: value of $LANG: JPN locale-coding-system: cp932 Major mode: Lisp Interaction Minor modes in effect: highlight-changes-visible-mode: t tooltip-mode: t global-eldoc-mode: t eldoc-mode: t show-paren-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t line-number-mode: t indent-tabs-mode: t transient-mark-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t Load-path shadows: None found. Features: (misearch multi-isearch comp comp-cstr warnings icons rx emoji-labels emoji multisession sqlite transient format-spec edmacro kmacro cl-extra gnutls network-stream nsm mailalias smtpmail textsec uni-scripts url url-proxy url-privacy url-expand url-methods url-history url-cookie generate-lisp-file url-domsuf url-util url-parse auth-source cl-seq eieio eieio-core cl-macs json map url-vars idna-mapping ucs-normalize uni-confusable textsec-check cl-print byte-opt gv bytecomp byte-compile debug backtrace find-func hilit-chg wid-edit thingatpt help-fns radix-tree help-mode pp shadow sort mail-extr emacsbug message mailcap yank-media puny dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068 epg-config gnus-util text-property-search time-date subr-x mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils term/bobcat japan-util rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic indonesian philippine cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget keymap hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty make-network-process native-compile emacs) Memory information: ((conses 16 331760 49630) (symbols 48 14840 3) (strings 32 66748 8954) (string-bytes 1 1357518) (vectors 16 55924) (vector-slots 8 1637738 128446) (floats 8 68 385) (intervals 56 7100 2925) (buffers 984 18))