bug#28098: 26.0.50; bad C fontification

bug#28098: 26.0.50; bad C fontification

[Marti Bolivar]

[-- Attachment #1: Type: text/plain, Size: 3418 bytes --]

1. From emacs -Q, open this file:


https://github.com/zephyrproject-rtos/zephyr/blob/2de59023dc726e61244eb7cca17252294016c65a/subsys/net/lib/http/http_client.c

2. Observe incorrect syntax highlighting at beginning of file. Screenshot:

   https://postimg.org/image/tzy8qgjjz/

   (Note that loading a C file with just the #ifdefs and initial
   comment which fail to highlight above doesn't reproduce the issue.)


In GNU Emacs 26.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.11)
 of 2017-08-10 built on plop
Repository revision: 81656add8117e8d1b7faab18b330d0706462b433
Windowing system distributor 'The X.Org Foundation', version 11.0.11903000
System Description: Ubuntu 17.04

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...
mwheel-scroll: Beginning of buffer [21 times]
Making completion list...

Configured using:
 'configure --without-pop'

Configured features:
XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS NOTIFY GNUTLS FREETYPE XFT
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: C/*l

Minor modes in effect:
  diff-auto-refine-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message subr-x puny seq byte-opt gv
bytecomp byte-compile cconv dired dired-loaddefs format-spec rfc822 mml
mml-sec password-cache epa derived epg epg-config gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils vc-git diff-mode easy-mmode cc-mode
cc-fonts easymenu cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs cl-loaddefs cl-lib time-date mule-util tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 123261 11555)
 (symbols 48 22839 1)
 (miscs 40 60 140)
 (strings 32 35859 1155)
 (string-bytes 1 1079093)
 (vectors 16 17814)
 (vector-slots 8 527552 12359)
 (floats 8 52 134)
 (intervals 56 1378 3)
 (buffers 992 13))

[-- Attachment #2: Type: text/html, Size: 4779 bytes --]

bug#28098: 26.0.50; bad C fontification

[Alan Mackenzie]

Hello, Marti.

In article <mailman.13159.1502810889.21957.bug-gnu-emacs@gnu.org> you wrote:
> [-- text/plain, encoding 7bit, charset: UTF-8, 98 lines --]

> 1. From emacs -Q, open this file:


> https://github.com/zephyrproject-rtos/zephyr/blob/2de59023dc726e61244eb7cca17252294016c65a/subsys/net/lib/http/http_client.c

> 2. Observe incorrect syntax highlighting at beginning of file. Screenshot:

>    https://postimg.org/image/tzy8qgjjz/

More precisely, the first ~500 bytes, which consist entirely of comments
and preprocessor directives doesn't get fontified at all.

>    (Note that loading a C file with just the #ifdefs and initial
>    comment which fail to highlight above doesn't reproduce the issue.)

Thanks.  The fact that it's ~500 bytes which don't get fontified suggest
that something is interfering with the first jit-lock chunk.  I'll be
looking into it.  Give me a bit of time.

> In GNU Emacs 26.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.11)
>  of 2017-08-10 built on plop
> Repository revision: 81656add8117e8d1b7faab18b330d0706462b433
> Windowing system distributor 'The X.Org Foundation', version 11.0.11903000
> System Description: Ubuntu 17.04

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28098: 26.0.50; bad C fontification

[Alan Mackenzie]

Hello again, Marti.

In article <mailman.13159.1502810889.21957.bug-gnu-emacs@gnu.org> you wrote:
> [-- text/plain, encoding 7bit, charset: UTF-8, 98 lines --]

> 1. From emacs -Q, open this file:


> https://github.com/zephyrproject-rtos/zephyr/blob/2de59023dc726e61244eb7cca17252294016c65a/subsys/net/lib/http/http_client.c

> 2. Observe incorrect syntax highlighting at beginning of file. Screenshot:

>    https://postimg.org/image/tzy8qgjjz/

>    (Note that loading a C file with just the #ifdefs and initial
>    comment which fail to highlight above doesn't reproduce the issue.)

I think the following patch should fix the bug.  Would you please apply
it, try it out, and either confirm to me that it fixes the bug, or tell
me what's still wrong:


diff -r 9533dc4cbda3 cc-mode.el
--- a/cc-mode.el	Thu Jul 27 17:37:02 2017 +0000
+++ b/cc-mode.el	Wed Aug 16 20:48:54 2017 +0000
@@ -1529,10 +1529,13 @@
     (c-backward-syntactic-ws)
     (when (setq pos1 (c-on-identifier))
       (goto-char pos1)
-      (when (and (c-forward-declarator)
-		 (eq (c-forward-token-2) 0))
-	(c-backward-syntactic-ws)
-	(point)))))
+      (let ((lim (save-excursion
+		   (and (c-beginning-of-macro)
+			(progn (c-end-of-macro) (point))))))
+	(when (and (c-forward-declarator lim)
+		   (eq (c-forward-token-2 1 nil lim) 0))
+	  (c-backward-syntactic-ws)
+	  (point))))))
 
 (defun c-change-expand-fl-region (beg end old-len)
   ;; Expand the region (c-new-BEG c-new-END) to an after-change font-lock

Thanks for taking the trouble to report this bug.

> In GNU Emacs 26.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.11)
>  of 2017-08-10 built on plop
> Repository revision: 81656add8117e8d1b7faab18b330d0706462b433
> Windowing system distributor 'The X.Org Foundation', version 11.0.11903000
> System Description: Ubuntu 17.04

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28098: 26.0.50; bad C fontification

[Marti Bolivar]

[-- Attachment #1.1: Type: text/plain, Size: 2416 bytes --]

Hi Alan,

Thanks for the patch.

On Wed, Aug 16, 2017 at 1:57 PM, Alan Mackenzie <acm@muc.de> wrote:

> Hello again, Marti.
>
> In article <mailman.13159.1502810889.21957.bug-gnu-emacs@gnu.org> you
> wrote:
> > [-- text/plain, encoding 7bit, charset: UTF-8, 98 lines --]
>
> > 1. From emacs -Q, open this file:
>
>
> > https://github.com/zephyrproject-rtos/zephyr/blob/
> 2de59023dc726e61244eb7cca17252294016c65a/subsys/net/lib/http/http_client.c
>
> > 2. Observe incorrect syntax highlighting at beginning of file.
> Screenshot:
>
> >    https://postimg.org/image/tzy8qgjjz/
>
> >    (Note that loading a C file with just the #ifdefs and initial
> >    comment which fail to highlight above doesn't reproduce the issue.)
>
> I think the following patch should fix the bug.  Would you please apply
> it, try it out, and either confirm to me that it fixes the bug, or tell
> me what's still wrong:
>
>
> diff -r 9533dc4cbda3 cc-mode.el
> --- a/cc-mode.el        Thu Jul 27 17:37:02 2017 +0000
> +++ b/cc-mode.el        Wed Aug 16 20:48:54 2017 +0000
> @@ -1529,10 +1529,13 @@
>      (c-backward-syntactic-ws)
>      (when (setq pos1 (c-on-identifier))
>        (goto-char pos1)
> -      (when (and (c-forward-declarator)
> -                (eq (c-forward-token-2) 0))
> -       (c-backward-syntactic-ws)
> -       (point)))))
> +      (let ((lim (save-excursion
> +                  (and (c-beginning-of-macro)
> +                       (progn (c-end-of-macro) (point))))))
> +       (when (and (c-forward-declarator lim)
> +                  (eq (c-forward-token-2 1 nil lim) 0))
> +         (c-backward-syntactic-ws)
> +         (point))))))
>
>  (defun c-change-expand-fl-region (beg end old-len)
>    ;; Expand the region (c-new-BEG c-new-END) to an after-change font-lock
>

I couldn't manage to get this to apply, but I made the changes manually;
git format-patch output for the relevant commit is attached. This did
indeed resolve the issue.


>
> Thanks for taking the trouble to report this bug.
>
>
Thanks for taking the time to investigate and fix it.

Marti


> > In GNU Emacs 26.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.11)
> >  of 2017-08-10 built on plop
> > Repository revision: 81656add8117e8d1b7faab18b330d0706462b433
> > Windowing system distributor 'The X.Org Foundation', version
> 11.0.11903000
> > System Description: Ubuntu 17.04
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>
>

[-- Attachment #1.2: Type: text/html, Size: 3896 bytes --]

[-- Attachment #2: 0001-Test-c-fontification-fix.patch --]
[-- Type: text/x-patch, Size: 1196 bytes --]

From 95a39b33959b98f4b314c1c441060682bf49aa44 Mon Sep 17 00:00:00 2001
From: Marti Bolivar <marti.f.bolivar@gmail.com>
Date: Mon, 21 Aug 2017 11:47:29 -0700
Subject: [PATCH] Test c fontification fix

From: Alan Mackenzie <acm@muc.de>
---
 lisp/progmodes/cc-mode.el | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/lisp/progmodes/cc-mode.el b/lisp/progmodes/cc-mode.el
index 0bf89b9a36..f27ec8722c 100644
--- a/lisp/progmodes/cc-mode.el
+++ b/lisp/progmodes/cc-mode.el
@@ -1549,10 +1549,13 @@ c-fl-decl-end
     (c-backward-syntactic-ws)
     (when (setq pos1 (c-on-identifier))
       (goto-char pos1)
-      (when (and (c-forward-declarator)
-		 (eq (c-forward-token-2) 0))
-	(c-backward-syntactic-ws)
-	(point)))))
+      (let ((lim (save-excursion
+                  (and (c-beginning-of-macro)
+                       (progn (c-end-of-macro) (point))))))
+       (when (and (c-forward-declarator lim)
+                  (eq (c-forward-token-2 1 nil lim) 0))
+         (c-backward-syntactic-ws)
+         (point))))))
 
 (defun c-change-expand-fl-region (_beg _end _old-len)
   ;; Expand the region (c-new-BEG c-new-END) to an after-change font-lock
-- 
2.11.0

bug#28098: 26.0.50; bad C fontification

[Alan Mackenzie]

On Mon, Aug 21, 2017 at 12:32:30 -0700, Marti Bolivar wrote:
> Hi Alan,

> Thanks for the patch.

[ .... ]

> I couldn't manage to get this to apply, but I made the changes manually;
> git format-patch output for the relevant commit is attached. This did
> indeed resolve the issue.

Sorry about that and thanks for taking the trouble!

I've committed the patch, and am closing the bug with this email.

> Marti

-- 
Alan Mackenzie (Nuremberg, Germany).