bug#35132: 25.2; Single-quote in numbers not handled properly by font-lock

bug#35132: 25.2; Single-quote in numbers not handled properly by font-lock

[Aidan Gauland]

[-- Attachment #1: Type: text/plain, Size: 4620 bytes --]

As of C++14, the single-quote character can be used as a digit separator (see
<http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3781.pdf>). This case
is not handled by c++-mode font-lock, which can mess up parsing of the entire
buffer in code using digit separators. For example, in the attached example
program, the first three zeroes on line 5 have the string face, and the text
from the next digit separator to the end of the buffer also has the string face,
overriding all other syntax highlighting for the rest of the code.

Regards,
Aidan Gauland


In GNU Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
of 2017-09-23, modified by Debian built on lgw01-amd64-050
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.2 LTS

Configured using:
'configure --build x86_64-linux-gnu --prefix=/usr
--sharedstatedir=/var/lib --libexecdir=/usr/lib
--localstatedir=/var/lib --infodir=/usr/share/info
--mandir=/usr/share/man --with-pop=yes
--enable-locallisppath=/etc/emacs25:/etc/emacs:/usr/local/share/emacs/25.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.2/site-lisp:/usr/share/emacs/site-lisp
--with-sound=alsa --without-gconf --build x86_64-linux-gnu
--prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/lib
--localstatedir=/var/lib --infodir=/usr/share/info
--mandir=/usr/share/man --with-pop=yes
--enable-locallisppath=/etc/emacs25:/etc/emacs:/usr/local/share/emacs/25.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.2/site-lisp:/usr/share/emacs/site-lisp
--with-sound=alsa --without-gconf --with-x=yes --with-x-toolkit=gtk3
--with-toolkit-scroll-bars 'CFLAGS=-g -O2
-fdebug-prefix-map=/build/emacs25-jYekUr/emacs25-25.2+1=. -fstack-protector-strong
-Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
-D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
 value of $LC_MONETARY: en_NZ.UTF-8
 value of $LC_NUMERIC: en_NZ.UTF-8
 value of $LC_TIME: en_NZ.UTF-8
 value of $LANG: en_GB.UTF-8
 value of $XMODIFIERS: @im=ibus
 locale-coding-system: utf-8-unix

Major mode: C++/l

Minor modes in effect:
 tooltip-mode: t
 global-eldoc-mode: t
 electric-indent-mode: t
 mouse-wheel-mode: t
 tool-bar-mode: t
 menu-bar-mode: t
 file-name-shadow-mode: t
 global-font-lock-mode: t
 font-lock-mode: t
 blink-cursor-mode: t
 auto-composition-mode: t
 auto-encryption-mode: t
 auto-compression-mode: t
 line-number-mode: t
 transient-mark-mode: t
 abbrev-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Quit
Type C-x 1 to delete the help window, C-M-v to scroll help.
Char: ' (39, #o47, #x27) point=50 of 101 (49%) column=11
forward-sexp: Scan error: "Unbalanced parentheses", 50, 102

Load-path shadows:
None found.

Features:
(shadow sort mail-extr pp wid-edit descr-text emacsbug message dired
format-spec rfc822 mml mml-sec password-cache epg epg-config gnus-util
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util help-fns
mail-prsvr mail-utils cl-extra help-mode cc-mode cc-fonts easymenu
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
cl-loaddefs pcase cl-lib time-date mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 196193 24068)
(symbols 48 33542 1)
(miscs 40 440 182)
(strings 32 74421 2521)
(string-bytes 1 1834089)
(vectors 16 15511)
(vector-slots 8 533534 58931)
(floats 8 170 134)
(intervals 56 345 32)
(buffers 976 19))

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: c-sep-bug.cpp --]
[-- Type: text/x-c++src; name="c-sep-bug.cpp", Size: 109 bytes --]

#include <iostream>

int main(void)
{
  int x = 1'000'000'000;
  std::cout << x << std::endl;

  return 0;
}

bug#35132: 25.2; Single-quote in numbers not handled properly by font-lock

[Basil L. Contovounesios]

"Aidan Gauland" <aidalgol@fastmail.net> writes:

> As of C++14, the single-quote character can be used as a digit separator (see
> <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3781.pdf>). This case
> is not handled by c++-mode font-lock, which can mess up parsing of the entire
> buffer in code using digit separators. For example, in the attached example
> program, the first three zeroes on line 5 have the string face, and the text
> from the next digit separator to the end of the buffer also has the string face,
> overriding all other syntax highlighting for the rest of the code.

Support for this was added in Emacs 26.1[1], which was released 28th May
2018.

[1]: Make C++ digit separators work.  Amend the handling of single quotes generally
  2017-07-01 15:43:07 +0000
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=59d07875df9d44568d93a7517853e6a5ccaf1e5b

-- 
Basil