bug#49562: 27.2; Crash with specific mode-line-format in BiDi processing

bug#49562: 27.2; Crash with specific mode-line-format in BiDi processing

[martin.bruestel]

[-- Attachment #1: Type: text/plain, Size: 4842 bytes --]


Emacs crashed when visiting specific pages in a browser using EXWM.
Turns out this was triggered when the buffer was renamed accordingly.
During redisplay, the crash happens for certain strings which are set as
`mode-line-format`.  I created an example to reproduce this with a stock
Emacs configuration, see the attached elisp file.  For me, this only
triggers when the window is large enough, I suspect smaller windows will
prevent the problematic part of the mode-line-format string to be
processed.

To reproduce:

1. Start Emacs using 'emacs' (not tested with 'emacs -Q')
2. Maximize Frame
3. M-x ielm
4. (load "/path/to/attached/break-emacs.el")
5. M-x crash-test-mode-line-format
6. Observe crash

The output of 'bt full' is attached in 'gdb.txt'

In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.16.0)
Windowing system distributor 'The X.Org Foundation', version 11.0.12010000
System Description: NixOS 20.09 (Nightingale)

Configured using:
 'configure
 --prefix=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2
 --bindir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/bin
 --sbindir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/sbin
 --includedir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/include
 --oldincludedir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/include
 --mandir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/share/man
 --infodir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/share/info
 --docdir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/share/doc/emacs
 --libdir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/lib
 --libexecdir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/libexec
 --localedir=/nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/share/locale
 --disable-build-details --with-modules --with-x-toolkit=gtk3 --with-xft
 --with-cairo'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY
INOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER GMP

Important settings:
  value of $EMACSLOADPATH: 
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Load-path shadows:
/run/current-system/sw/share/emacs/site-lisp/site-start hides /nix/store/2csx4hz6ab0i6d5h4b7z1afwsydbx02n-emacs-packages-deps/share/emacs/site-lisp/site-start
/run/current-system/sw/share/emacs/site-lisp/site-start hides /nix/store/l8wil534cr2bp339s27k6p2i8zyi5y6k-emacs-27.2/share/emacs/site-lisp/site-start

Features:
(shadow sort mail-extr emacsbug sendmail notmuch notmuch-tree
notmuch-jump notmuch-hello wid-edit notmuch-show notmuch-print
notmuch-crypto notmuch-mua notmuch-message notmuch-draft
notmuch-maildir-fcc notmuch-address notmuch-company notmuch-parser
notmuch-wash diff-mode easy-mmode coolj notmuch-query goto-addr
thingatpt icalendar diary-lib diary-loaddefs cal-menu calendar
cal-loaddefs notmuch-tag crm notmuch-lib notmuch-compat pcase hl-line
message rmc puny dired dired-loaddefs format-spec rfc822 mml mailabbrev
gmm-utils mailheader mm-view mml-smime mml-sec epa derived epg
epg-config gnus-util rmail rmail-loaddefs mail-utils
text-property-search time-date smime dig mm-decode mm-bodies mm-encode
mailcap mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
package easymenu browse-url url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 83001 9776)
 (symbols 48 9884 1)
 (strings 32 28953 2047)
 (string-bytes 1 1044840)
 (vectors 16 15878)
 (vector-slots 8 199051 14998)
 (floats 8 27 44)
 (intervals 56 319 0)
 (buffers 1000 14))


[-- Attachment #2: gdb.txt.gz --]
[-- Type: application/octet-stream, Size: 14600 bytes --]

[-- Attachment #3: break-emacs.el.gz --]
[-- Type: application/octet-stream, Size: 2021 bytes --]

[-- Attachment #4: Type: text/plain, Size: 150 bytes --]


-- 
Martin Brüstel
Research Assistant
TU Dresden, Processor Design Chair, Center for Advancing Electronics Dresden (cfaed)
Tel: +49 351 43726

bug#49562: 27.2; Crash with specific mode-line-format in BiDi processing

[Eli Zaretskii]

> From: <martin.bruestel@tu-dresden.de>
> Date: Wed, 14 Jul 2021 18:40:57 +0200
> 
> Emacs crashed when visiting specific pages in a browser using EXWM.
> Turns out this was triggered when the buffer was renamed accordingly.
> During redisplay, the crash happens for certain strings which are set as
> `mode-line-format`.  I created an example to reproduce this with a stock
> Emacs configuration, see the attached elisp file.  For me, this only
> triggers when the window is large enough, I suspect smaller windows will
> prevent the problematic part of the mode-line-format string to be
> processed.
> 
> To reproduce:
> 
> 1. Start Emacs using 'emacs' (not tested with 'emacs -Q')
> 2. Maximize Frame
> 3. M-x ielm
> 4. (load "/path/to/attached/break-emacs.el")
> 5. M-x crash-test-mode-line-format
> 6. Observe crash

The "evil" mode-line string includes invalid use of bidi formatting
controls: you have there a U+202A LEFT-TO-RIGHT EMBEDDING without a
matching U+202C POP DIRECTIONAL FORMATTING.  Removing the former or
adding the latter avoids the crash.

I will look into avoiding the crash even with the unbalanced control
characters.  I guess this imbalance triggers a subtle bug somewhere,
which is somehow related to the use of (space (:align-to ...)) display
spec.  Hmm...

bug#49562: 27.2; Crash with specific mode-line-format in BiDi processing

[Martin Brüstel]

[-- Attachment #1: Type: text/plain, Size: 1551 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: <martin.bruestel@tu-dresden.de>
>> Date: Wed, 14 Jul 2021 18:40:57 +0200
>> 
>> Emacs crashed when visiting specific pages in a browser using EXWM.
>> Turns out this was triggered when the buffer was renamed accordingly.
>> During redisplay, the crash happens for certain strings which are set as
>> `mode-line-format`.  I created an example to reproduce this with a stock
>> Emacs configuration, see the attached elisp file.  For me, this only
>> triggers when the window is large enough, I suspect smaller windows will
>> prevent the problematic part of the mode-line-format string to be
>> processed.
>
> The "evil" mode-line string includes invalid use of bidi formatting
> controls: you have there a U+202A LEFT-TO-RIGHT EMBEDDING without a
> matching U+202C POP DIRECTIONAL FORMATTING.  Removing the former or
> adding the latter avoids the crash.

Thanks for finding the cause!  It turns out, the control chars are
actually already in the window title of the browser window.  The
imbalance results from truncating the name to a certain length.

Is there any built-in functionality to either

1. "sanitize" a string to remove bidi formatting in a sane way, or
2. have `substring` functionality that can somehow deal with these
   formatting controls, or
3. restore balance of a possibly broken bidi formatted string?


-- 
Martin Brüstel
Research Assistant
TU Dresden, Processor Design Chair, Center for Advancing Electronics Dresden (cfaed)
Tel: +49 351 43726

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5341 bytes --]

bug#49562: 27.2; Crash with specific mode-line-format in BiDi processing

[Eli Zaretskii]

> From: Martin Brüstel <martin.bruestel@tu-dresden.de>
> CC: <49562@debbugs.gnu.org>
> Date: Thu, 15 Jul 2021 11:30:27 +0200
> 
> Is there any built-in functionality to either
> 
> 1. "sanitize" a string to remove bidi formatting in a sane way, or
> 2. have `substring` functionality that can somehow deal with these
>    formatting controls, or
> 3. restore balance of a possibly broken bidi formatted string?

Not as you describe it, no.  You could remove all bidi formatting
controls with subst-char-in-string or with string-match/replace-match,
and then use bidi-string-mark-left-to-right to solve many problems
with bidirectional display of mixed RTL and LTR text.  But note that
bidi-string-mark-left-to-right doesn't exactly wrap the string in an
embedding, it does something smarter.

bug#49562: 27.2; Crash with specific mode-line-format in BiDi processing

[Eli Zaretskii]

> From: Martin Brüstel <martin.bruestel@tu-dresden.de>
> CC: <49562@debbugs.gnu.org>
> Date: Thu, 15 Jul 2021 11:30:27 +0200
> 
> > The "evil" mode-line string includes invalid use of bidi formatting
> > controls: you have there a U+202A LEFT-TO-RIGHT EMBEDDING without a
> > matching U+202C POP DIRECTIONAL FORMATTING.  Removing the former or
> > adding the latter avoids the crash.
> 
> Thanks for finding the cause!  It turns out, the control chars are
> actually already in the window title of the browser window.  The
> imbalance results from truncating the name to a certain length.

This tricky bug is now fixed on the master branch.  It actually
revealed a design flaw in the original code.

Thanks.