bug#36492: c-mode fails with errors during fontification

bug#36492: c-mode fails with errors during fontification

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 5094 bytes --]

Opening this message in Gnus fails with the following backtrace
after setting (setq diff-font-lock-syntax 'hunk-also)
because c-mode fails to fontify the attachment.

Debugger entered--Lisp error: (wrong-type-argument integer-or-marker-p nil)
  c-forward-decl-or-cast-1(0 top nil)
  (setq decl-or-cast (c-forward-decl-or-cast-1 match-pos context last-cast-end))
  (if (eq context 'not-decl) ...)
  (if (or (and (eq (get-text-property (point) 'face) 'font-lock-keyword-face) ...) ...) t ...)
  (if (if (or (and (eq (get-text-property (point) 'face) 'font-lock-keyword-face) ...) ...) t ...) ...)
  (lambda (match-pos inside-macro &optional toplev) ...)(0 nil t)
  c-find-decl-spots(11 "[[:alpha:]_]" (nil font-lock-type-face font-lock-constant-face font-lock-keyword-face) ...)
  (let (start-pos context ...) ... nil)
  (save-restriction (let (start-pos context ...) ... nil))
  (progn (save-restriction (let (start-pos context ...) ... nil)))
  (if (< (point) limit) (progn (save-restriction (let (start-pos context ...) ... nil))))
  c-font-lock-declarations(11)
  font-lock-fontify-keywords-region(1 11 nil)
  font-lock-default-fontify-region(1 11 nil)
  funcall(font-lock-default-fontify-region 1 11 nil)
  (let (new-beg new-end new-region case-fold-search) ...)
  c-font-lock-fontify-region(1 11 nil)
  font-lock-fontify-region(1 11)
  #f(compiled-function (beg end) #<bytecode 0x1fd6e1a05171>)(1 11)
  font-lock-ensure(1 11)
  diff-syntax-fontify-props(#("a/test.c" 0 8 (face (diff-file-header diff-header))) "int i)\n{\n1" (1 3) t)
  diff-syntax-fontify-hunk(78 111 t)
  diff-syntax-fontify(78 111)
  #f(compiled-function (beg end) #<bytecode 0x158f39208a39>)(78 111)
  diff--iterate-hunks(113 #f(compiled-function (beg end) #<bytecode 0x158f39208a39>))
  diff--font-lock-syntax(113)
  font-lock-fontify-keywords-region(1 113 nil)
  font-lock-default-fontify-region(1 113 nil)
  font-lock-fontify-region(1 113)
  #f(compiled-function (beg end) #<bytecode 0x1fd6e1a05171>)(1 113)
  font-lock-ensure()
  (if (eq major-mode 'fundamental-mode) nil (font-lock-ensure))
  (progn (if mode (let ((#:wconfig (current-window-configuration))) ...) ...) ...)
  (condition-case #:err (progn ...) ((debug error) (message "Error: %S" #:err) nil))
  (let ((font-lock-verbose nil) (enable-local-variables nil)) ...)
  (progn (buffer-disable-undo) (mm-enable-multibyte) ...)
  (unwind-protect (progn (buffer-disable-undo) (mm-enable-multibyte) ...) ...)
  (save-current-buffer (set-buffer #:temp-buffer) (unwind-protect ...))
  (let ((#:temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer ...))
  (let ((charset (mail-content-type-get (nth 1 handle) 'charset)) text coding-system ovs) ...)
  mm-display-inline-fontify(... diff-mode)
  mm-display-patch-inline(...)
  mm-display-inline(...)
  gnus-mime-display-single(...)
  gnus-mime-display-part(...)
  mapcar(gnus-mime-display-part (...))
  gnus-mime-display-mixed((...))
  gnus-mime-display-part((#("multipart/mixed" 0 15 ...) ...))
  gnus-display-mime()
  gnus-article-prepare-display()
  gnus-article-prepare(5460 nil)
  funcall(gnus-article-prepare 5460 nil)
  (prog1 (funcall (or gnus-summary-display-article-function ...) article all-header) ...)
  (if (null article) nil (prog1 (funcall (or gnus-summary-display-article-function ...) article all-header)))
  gnus-summary-display-article(5460 nil)
  (progn (gnus-summary-display-article article all-headers) ...)
  (if (or (and gnus-single-article-buffer ...)) ... 'old)
  (save-current-buffer (set-buffer gnus-summary-buffer) ...)
  (let ((article (or article (gnus-summary-article-number))) ...) ...)
  gnus-summary-select-article(nil nil pseudo)
  (eq (gnus-summary-select-article nil nil 'pseudo) 'old)
  (if (eq (gnus-summary-select-article nil nil 'pseudo) 'old) ...)
  gnus-summary-scroll-up(1)
  funcall-interactively(gnus-summary-scroll-up 1)
  call-interactively(gnus-summary-scroll-up nil nil)
  command-execute(gnus-summary-scroll-up)

I don't know if it's possible to fix c-mode to not raise the error
during fontification.  But at least the patch below will avoid this error
while displaying such attachments in Gnus.  Normally calling font-lock-ensure
ignores the fontification errors when called from font-lock.
But in mm-display-inline-fontify font-lock-ensure is called directly
from top-level, so this patch ignores its errors.

diff --git a/lisp/gnus/mm-view.el b/lisp/gnus/mm-view.el
index 6ffa1fc168..ebaf8435c0 100644
--- a/lisp/gnus/mm-view.el
+++ b/lisp/gnus/mm-view.el
@@ -500,7 +500,7 @@ mm-display-inline-fontify
 	      (setq mode major-mode)))
 	  ;; Do not fontify if the guess mode is fundamental.
 	  (unless (eq major-mode 'fundamental-mode)
-	    (font-lock-ensure))))
+	    (ignore-errors (font-lock-ensure)))))
       (setq text (buffer-string))
       (when (eq mode 'diff-mode)
 	(setq ovs (mapcar (lambda (ov) (list ov (overlay-start ov)

Then visiting such attachments will just leave this line in *Messages*
without wreaking much havoc:

  Error during redisplay: (jit-lock-function 1) signaled (wrong-type-argument integer-or-marker-p nil)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test.diff --]
[-- Type: text/x-diff, Size: 112 bytes --]

diff --git a/test.c b/test.c
index aaa..bbb 100644
--- a/test.c
+++ b/test.c
@@ -1,3 +2,3 @@
 int i)
 {
-1
+2

bug#36492: c-mode fails with errors during fontification

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 220 bytes --]

> Opening this message in Gnus fails with the following backtrace
> because c-mode fails to fontify the attachment.

Actually, with the attachment in this message the bug in c-mode
font-lock is much easier to reproduce.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test.c --]
[-- Type: text/x-csrc, Size: 9 bytes --]

int i)
{

bug#36492: c-mode fails with errors during fontification

[Alan Mackenzie]

Hello, Juri.

In article <mailman.154.1562272763.2688.bug-gnu-emacs@gnu.org> you wrote:
> [-- text/plain, encoding 7bit, charset: US-ASCII, 6 lines --]

>> Opening this message in Gnus fails with the following backtrace
>> because c-mode fails to fontify the attachment.

> Actually, with the attachment in this message the bug in c-mode
> font-lock is much easier to reproduce.

Yes, I can reproduce the bug in that buffer with
(font-lock-fontify-region (point-min) (point-max).  I suspect it will be
fairly easy to diagnose and fix.

> [-- text/x-csrc, encoding 7bit, charset: US-ASCII, 3 lines, name: test.c --]

-- 
Alan Mackezie (Nuremberg, Germany).

bug#36492: c-mode fails with errors during fontification

[Alan Mackenzie]

In article <mailman.154.1562272763.2688.bug-gnu-emacs@gnu.org> you wrote:
> [-- text/plain, encoding 7bit, charset: US-ASCII, 6 lines --]

>> Opening this message in Gnus fails with the following backtrace
>> because c-mode fails to fontify the attachment.

> Actually, with the attachment in this message the bug in c-mode
> font-lock is much easier to reproduce.

That is one ugly piece of erroneous C.  ;-)

What triggers the bug is the ) without a preceding (.  A variable
recording the position of the opening ( is still set to nil, and we
tried to use it in the given buffer with the unbalanced ).

> [-- text/x-csrc, encoding 7bit, charset: US-ASCII, 3 lines, name: test.c --]

The following patch fixes it, I hope.  Would you please do the usual,
and confirm that it does indeed fix the bug (or tell me what's still
wrong).

Thanks!


diff -r eb4ee9bb0682 cc-engine.el
--- a/cc-engine.el	Thu Jul 04 13:01:08 2019 +0000
+++ b/cc-engine.el	Sat Jul 06 14:16:43 2019 +0000
@@ -9480,6 +9480,7 @@
 			   (not got-prefix)
 			   (or (eq context 'top) make-top)
 			   (eq (char-after) ?\))
+			   after-paren-pos
 			   (or (memq at-type '(nil maybe))
 			       (not got-identifier)
 			       (save-excursion
@@ -9516,7 +9517,7 @@
 	    ;; (con|de)structors in C++ and `c-typeless-decl-kwds'
 	    ;; style declarations.  That isn't applicable in an
 	    ;; arglist context, though.
-	    (when (and (> paren-depth 0)
+	    (when (and (> paren-depth 0) ; ensures `after-paren-pos' is non-nil
 		       (not got-prefix-before-parens)
 		       (not (eq at-type t))
 		       (or backup-at-type


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#36492: c-mode fails with errors during fontification

[Juri Linkov]

>>> Opening this message in Gnus fails with the following backtrace
>>> because c-mode fails to fontify the attachment.
>
>> Actually, with the attachment in this message the bug in c-mode
>> font-lock is much easier to reproduce.
>
> That is one ugly piece of erroneous C.  ;-)
>
> What triggers the bug is the ) without a preceding (.  A variable
> recording the position of the opening ( is still set to nil, and we
> tried to use it in the given buffer with the unbalanced ).
>
>> [-- text/x-csrc, encoding 7bit, charset: US-ASCII, 3 lines, name: test.c --]
>
> The following patch fixes it, I hope.  Would you please do the usual,
> and confirm that it does indeed fix the bug (or tell me what's still
> wrong).

Thanks, Alan.  Now there is no error anymore.

I wonder if ignore-errors around font-lock-ensure in mm-display-inline-fontify
is still necessary to ensure no more bugs in other modes disrupt Gnus
by erroneous code in attachments?  Or maybe raising such errors will help
to find fontification bugs sooner?

bug#36492: c-mode fails with errors during fontification

[Alan Mackenzie]

Hello, Juri.

On Mon, Jul 08, 2019 at 00:52:02 +0300, Juri Linkov wrote:
> >>> Opening this message in Gnus fails with the following backtrace
> >>> because c-mode fails to fontify the attachment.

> >> Actually, with the attachment in this message the bug in c-mode
> >> font-lock is much easier to reproduce.

> > That is one ugly piece of erroneous C.  ;-)

> > What triggers the bug is the ) without a preceding (.  A variable
> > recording the position of the opening ( is still set to nil, and we
> > tried to use it in the given buffer with the unbalanced ).

> >> [-- text/x-csrc, encoding 7bit, charset: US-ASCII, 3 lines, name: test.c --]

> > The following patch fixes it, I hope.  Would you please do the usual,
> > and confirm that it does indeed fix the bug (or tell me what's still
> > wrong).

> Thanks, Alan.  Now there is no error anymore.

Thanks.  I've committed the fix, and I'm closing the bug.

> I wonder if ignore-errors around font-lock-ensure in mm-display-inline-fontify
> is still necessary to ensure no more bugs in other modes disrupt Gnus
> by erroneous code in attachments?  Or maybe raising such errors will help
> to find fontification bugs sooner?

My feeling is that it's better to let the errors happen, so that we can
debug them.  But on the other hand, it's not me that gets inconvenienced
by such an error (I don't use gnus).

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#36492: c-mode fails with errors during fontification

[Noam Postavsky]

On Mon, 8 Jul 2019 at 08:54, Alan Mackenzie <acm@muc.de> wrote:

> > I wonder if ignore-errors around font-lock-ensure in mm-display-inline-fontify
> > is still necessary to ensure no more bugs in other modes disrupt Gnus
> > by erroneous code in attachments?  Or maybe raising such errors will help
> > to find fontification bugs sooner?
>
> My feeling is that it's better to let the errors happen, so that we can
> debug them.  But on the other hand, it's not me that gets inconvenienced
> by such an error (I don't use gnus).

Rather than ignore-errors, perhaps using with-demoted-errors would
still let the errors be noticed without too much inconvenience?

bug#36492: c-mode fails with errors during fontification

[Dmitry Gutov]

On 08.07.2019 15:53, Alan Mackenzie wrote:
>> I wonder if ignore-errors around font-lock-ensure in mm-display-inline-fontify
>> is still necessary to ensure no more bugs in other modes disrupt Gnus
>> by erroneous code in attachments?  Or maybe raising such errors will help
>> to find fontification bugs sooner?
> My feeling is that it's better to let the errors happen, so that we can
> debug them.  But on the other hand, it's not me that gets inconvenienced
> by such an error (I don't use gnus).

I'd like to second that.

And with similar errors taken care of, it might become feasible to use 
e.g. c-mode with mmm-mode.

bug#36492: c-mode fails with errors during fontification

[Juri Linkov]

>> > I wonder if ignore-errors around font-lock-ensure in mm-display-inline-fontify
>> > is still necessary to ensure no more bugs in other modes disrupt Gnus
>> > by erroneous code in attachments?  Or maybe raising such errors will help
>> > to find fontification bugs sooner?
>>
>> My feeling is that it's better to let the errors happen, so that we can
>> debug them.  But on the other hand, it's not me that gets inconvenienced
>> by such an error (I don't use gnus).
>
> Rather than ignore-errors, perhaps using with-demoted-errors would
> still let the errors be noticed without too much inconvenience?

I tried this change:

diff --git a/lisp/gnus/mm-view.el b/lisp/gnus/mm-view.el
index 6ffa1fc168..c833f2ed01 100644
--- a/lisp/gnus/mm-view.el
+++ b/lisp/gnus/mm-view.el
@@ -500,7 +500,7 @@ mm-display-inline-fontify
 	      (setq mode major-mode)))
 	  ;; Do not fontify if the guess mode is fundamental.
 	  (unless (eq major-mode 'fundamental-mode)
-	    (font-lock-ensure))))
+	    (with-demoted-errors (font-lock-ensure)))))
       (setq text (buffer-string))
       (when (eq mode 'diff-mode)
 	(setq ovs (mapcar (lambda (ov) (list ov (overlay-start ov)

But it pops up the same error backtrace buffer.

bug#36492: c-mode fails with errors during fontification

[Noam Postavsky]

Juri Linkov <juri@linkov.net> writes:

>> Rather than ignore-errors, perhaps using with-demoted-errors would
>> still let the errors be noticed without too much inconvenience?
>
> I tried this change:
>
>  	  (unless (eq major-mode 'fundamental-mode)
> -	    (font-lock-ensure))))
> +	    (with-demoted-errors (font-lock-ensure)))))
>        (setq text (buffer-string))
>        (when (eq mode 'diff-mode)
>  	(setq ovs (mapcar (lambda (ov) (list ov (overlay-start ov)
>
> But it pops up the same error backtrace buffer.

Right, if you have debug-on-error set, with-demoted-errors won't stop
the debugger.  But you should be able to continue with 'c' as normal, or
if you don't have debug-on-error set, you only get a message.