bug#50402: 27.0.90; column-number-mode breaks Farsi/Arabic character shaping

bug#50402: 27.0.90; column-number-mode breaks Farsi/Arabic character shaping

[Mohammad Razavi]

As you may know in scripts such as Farsi (Persian) or Arabic characters
may change shape depending on their adjacent characters
(http://www.unicode.org/versions/Unicode1.0.0/V2appA.pdf).


This behavior works fine in emacs; but if you enable
"column-number-mode" and copy/paste Farsi/Arabic script into the buffer
character shaping will not work correctly.


To reproduce the problem, you can run


emacs -q --eval '(progn (setq column-number-mode t) (switch-to-buffer
"foobar"))'


and then copy (only one of) the strings from below:


افغانستان


ایران


(The first word is "Afghanistan" and the second one is "Iran" in Farsi
script)


and paste it on the emacs buffer. You will see:


ﺎﻔﻏﺎﻨﺴﺗﺎﻧ


ﺎﯾﺭﺎﻧ


in which the character shaping is broken. If it didn't work with this
simple words try long Farsi/Arabic texts.


This problem only exists if  column-number-mode is enabled. Strangely,
if you type the words usually it works fine but if you copy/paste from
somewhere else it will not work. Also if you type some word and then
copy/paste the same work the character shaping works fine.



In GNU Emacs 27.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.14)
of 2020-03-29 built on 30bc0080ed46
Repository revision: c5f255d68156926923232b1edadf50faac527861
Repository branch: HEAD
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Ubuntu 20.04.3 LTS

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...

Configured using:
'configure --build x86_64-linux-gnu --prefix=/opt/emacs
--with-mailutils=yes --with-sound=alsa --without-gconf --with-x=yes
--with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
-fstack-protector-strong -Wformat -Werror=format-security -Wall'
'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD PDUMPER
LCMS2 GMP

Important settings:
value of $LANG: en_US.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
column-number-mode: t
line-number-mode: t
transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 45662 7148)
(symbols 48 5981 1)
(strings 32 16013 1905)
(string-bytes 1 518187)
(vectors 16 10056)
(vector-slots 8 129239 10066)
(floats 8 20 39)
(intervals 56 214 0)
(buffers 1000 13))

bug#50402: 27.0.90; column-number-mode breaks Farsi/Arabic character shaping

[Eli Zaretskii]

> From: Mohammad Razavi <sepent@gmail.com>
> Date: Sun, 5 Sep 2021 14:43:02 +0430
> 
> As you may know in scripts such as Farsi (Persian) or Arabic characters
> may change shape depending on their adjacent characters
> (http://www.unicode.org/versions/Unicode1.0.0/V2appA.pdf).
> 
> 
> This behavior works fine in emacs; but if you enable
> "column-number-mode" and copy/paste Farsi/Arabic script into the buffer
> character shaping will not work correctly.

This is bug#41005, which was solved in Emacs 27.1.  You are running a
pretest of Emacs 27.1, from before that bug was solved.  Please
upgrade to Emacs 27.1 or 27.2, and I believe the problem you see
should go away.

Thanks.

bug#50402: 27.0.90; column-number-mode breaks Farsi/Arabic character shaping

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> This is bug#41005, which was solved in Emacs 27.1.  You are running a
> pretest of Emacs 27.1, from before that bug was solved.  Please
> upgrade to Emacs 27.1 or 27.2, and I believe the problem you see
> should go away.

This was a month ago, but there was no response, so I'm assuming this
works, and I'm closing this bug report.  If there's still an issue,
please respond to the debbugs address and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no