bug#20240: 25.0.50; CC Mode broken fontification and commenting

bug#20240: 25.0.50; CC Mode broken fontification and commenting

[Mark Oteiza]

From emacs -Q:

1. C-x C-f foo.c RET
2. Insert: { RET }
3. Select the first line and M-;

Only `/* ` is inserted at the beginning of the buffer, which is wrong.

4. Undo: C-/

The braces remain fontified as comments.

In GNU Emacs 25.0.50.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw scroll bars)
 of 2015-03-31 on logos
Repository revision: 8d1d8fc474bd9ed22cbe8b80125c8ce89dc62b57
Configured using:
 `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --without-gconf --with-x-toolkit=lucid
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 --param=ssp-buffer-size=4' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

bug#20240: 25.0.50; CC Mode broken fontification and commenting

[Alan Mackenzie]

Hello, Mark.

In article <mailman.3120.1427885528.31049.bug-gnu-emacs@gnu.org> you wrote:

>>From emacs -Q:

> 1. C-x C-f foo.c RET
> 2. Insert: { RET }
> 3. Select the first line and M-;

> Only `/* ` is inserted at the beginning of the buffer, which is wrong.

It actually signals an error at that point, which is indeed wrong.

I've just committed a fix to the Emacs repository.  Could you please update
your sources, rebuild, and confirm that the fix works, please.

Thanks for taking the trouble to report this bug.

> 4. Undo: C-/

> The braces remain fontified as comments.

> In GNU Emacs 25.0.50.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw scroll bars)
> of 2015-03-31 on logos

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#20240: 25.0.50; CC Mode broken fontification and commenting

[Mark Oteiza]

Alan Mackenzie <acm@muc.de> writes:

> In article <mailman.3120.1427885528.31049.bug-gnu-emacs@gnu.org> you wrote:
>>>From emacs -Q:
>
>> 1. C-x C-f foo.c RET
>> 2. Insert: { RET }
>> 3. Select the first line and M-;
>
>> Only `/* ` is inserted at the beginning of the buffer, which is wrong.
>
> It actually signals an error at that point, which is indeed wrong.

Ah, I missed it then, thanks.

> I've just committed a fix to the Emacs repository.  Could you please update
> your sources, rebuild, and confirm that the fix works, please.
>
> Thanks for taking the trouble to report this bug.

You're welcome. You fixed the example I gave, but as it turns out my
example was too minimal. Hopefully this is better: uncommenting the
commented line in the following will signal a jit-lock error

int main(void) {
  for(;;) {
    /* return 0; */
  }
}

Repeating M-; on that line, eventually I get `return 0;` uncommented but
fontified as a comment.

bug#20240: 25.0.50; CC Mode broken fontification and commenting

[Alan Mackenzie]

Hello, Stefan and Mark.

In article <mailman.3163.1427918769.31049.bug-gnu-emacs@gnu.org> you wrote:
> Alan Mackenzie <acm@muc.de> writes:

>> In article <mailman.3120.1427885528.31049.bug-gnu-emacs@gnu.org> you wrote:
>>>>From emacs -Q:
>>
>>> 1. C-x C-f foo.c RET
>>> 2. Insert: { RET }
>>> 3. Select the first line and M-;
>>
>>> Only `/* ` is inserted at the beginning of the buffer, which is wrong.
>>
>> It actually signals an error at that point, which is indeed wrong.

> Ah, I missed it then, thanks.

>> I've just committed a fix to the Emacs repository.  Could you please update
>> your sources, rebuild, and confirm that the fix works, please.
>>
>> Thanks for taking the trouble to report this bug.

> You're welcome. You fixed the example I gave, but as it turns out my
> example was too minimal. Hopefully this is better: uncommenting the
> commented line in the following will signal a jit-lock error

> int main(void) {
>  for(;;) {
>    /* return 0; */
>  }
> }

> Repeating M-; on that line, eventually I get `return 0;` uncommented but
> fontified as a comment.

That is actually a different (though similar) bug.

Stefan, do you have any objections to the patch below, which widens the
buffer in jit-lock-after-change before setting the 'fontified text
properties?  Mark's (latest) bug is happening because `comment-dwim' is
narrowing to an existing comment which isn't at BOL, and
jit-lock-after-change is erroring because it's trying to set the text
properties at col 0, but can't (because of the narrowing).

Like you said a while back, narrowing can cause problems.


diff --git a/lisp/jit-lock.el b/lisp/jit-lock.el
index 0faabeb..d5651c6 100644
--- a/lisp/jit-lock.el
+++ b/lisp/jit-lock.el
@@ -650,12 +650,14 @@ will take place when text is fontified stealthily."
     (let ((jit-lock-start start)
           (jit-lock-end end))
       (with-buffer-prepared-for-jit-lock
-          (run-hook-with-args 'jit-lock-after-change-extend-region-functions
-                              start end old-len)
-          ;; Make sure we change at least one char (in case of deletions).
-          (setq jit-lock-end (min (max jit-lock-end (1+ start)) (point-max)))
-          ;; Request refontification.
-          (put-text-property jit-lock-start jit-lock-end 'fontified nil))
+          (save-restriction
+           (widen)
+           (run-hook-with-args 'jit-lock-after-change-extend-region-functions
+                               start end old-len)
+           ;; Make sure we change at least one char (in case of deletions).
+           (setq jit-lock-end (min (max jit-lock-end (1+ start)) (point-max)))
+           ;; Request refontification.
+           (put-text-property jit-lock-start jit-lock-end 'fontified nil)))
       ;; Mark the change for deferred contextual refontification.
       (when jit-lock-context-unfontify-pos
         (setq jit-lock-context-unfontify-pos


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#20240: 25.0.50; CC Mode broken fontification and commenting

[Stefan Monnier]

> -          (run-hook-with-args 'jit-lock-after-change-extend-region-functions
> -                              start end old-len)
> -          ;; Make sure we change at least one char (in case of deletions).
> -          (setq jit-lock-end (min (max jit-lock-end (1+ start)) (point-max)))
> -          ;; Request refontification.
> -          (put-text-property jit-lock-start jit-lock-end 'fontified nil))
> +          (save-restriction
> +           (widen)
> +           (run-hook-with-args 'jit-lock-after-change-extend-region-functions
> +                               start end old-len)
> +           ;; Make sure we change at least one char (in case of deletions).
> +           (setq jit-lock-end (min (max jit-lock-end (1+ start)) (point-max)))
> +           ;; Request refontification.
> +           (put-text-property jit-lock-start jit-lock-end 'fontified nil)))

Widening during the call to
jit-lock-after-change-extend-region-functions seems wrong.  But I guess
there can be meaningful cases where
jit-lock-after-change-extend-region-functions could return positions
outside of the narrowed region, so I'd be OK with widening during the
subsequent `put-text-property'.


        Stefan

bug#20240: 25.0.50; CC Mode broken fontification and commenting

[Alan Mackenzie]

Hello, Stefan and Mark.

On Fri, Apr 03, 2015 at 06:09:12PM -0400, Stefan Monnier wrote:
> > -          (run-hook-with-args 'jit-lock-after-change-extend-region-functions
> > -                              start end old-len)
> > -          ;; Make sure we change at least one char (in case of deletions).
> > -          (setq jit-lock-end (min (max jit-lock-end (1+ start)) (point-max)))
> > -          ;; Request refontification.
> > -          (put-text-property jit-lock-start jit-lock-end 'fontified nil))
> > +          (save-restriction
> > +           (widen)
> > +           (run-hook-with-args 'jit-lock-after-change-extend-region-functions
> > +                               start end old-len)
> > +           ;; Make sure we change at least one char (in case of deletions).
> > +           (setq jit-lock-end (min (max jit-lock-end (1+ start)) (point-max)))
> > +           ;; Request refontification.
> > +           (put-text-property jit-lock-start jit-lock-end 'fontified nil)))

> Widening during the call to
> jit-lock-after-change-extend-region-functions seems wrong.  But I guess
> there can be meaningful cases where
> jit-lock-after-change-extend-region-functions could return positions
> outside of the narrowed region, so I'd be OK with widening during the
> subsequent `put-text-property'.

Thanks.  I've just committed that change.

Mark, would you please update your Emacs and build it again, then test
whether or not your bug has finally been fixed, then let us know.  Then
I can close the bug.  Thanks!

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#20240: 25.0.50; CC Mode broken fontification and commenting

[Mark Oteiza]

Alan Mackenzie <acm@muc.de> writes:
> Thanks.  I've just committed that change.
>
> Mark, would you please update your Emacs and build it again, then test
> whether or not your bug has finally been fixed, then let us know.  Then
> I can close the bug.  Thanks!

Fixed.  Thank you!