Fix to handling of C++ digit separators, improvements in handling of single quotes.

Fix to handling of C++ digit separators, improvements in handling of single quotes.

[Alan Mackenzie]

Hello, Emacs.

Up until half an hour ago, the handling of C++ digit separators (') was
broken.  They were find for when you'd just loaded a file, but on typing
in digits and separators they were broken.

I've rewritten this part of CC Mode, in the process improving the
handling of single quotes generally.  Now, single quotes are fontified
with font-lock-warning-face unless they form part of a valid character
constant, or are digit separators in C++.

Just as a matter of interest, single quotes inside strings and comments
now get the same syntax-table text properties that they do in plain text.
It's just simpler this way.

The code involved is moderately involved, with special cases needed to
handle things like '\'', '\\', or char foo [] = {',',',',',',','};.  So
if there are any problems with it (in particular, large areas of text
following a quotey bit getting misfontified with font-lock-string-face),
please let me know.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Fix to handling of C++ digit separators, improvements in handling of single quotes.

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 576 bytes --]

On Sat, Jul 1, 2017 at 12:10 PM, Alan Mackenzie <acm@muc.de> wrote:
> The code involved is moderately involved, with special cases needed to
> handle things like '\'', '\\', or char foo [] = {',',',',',',','};.  So
> if there are any problems with it (in particular, large areas of text
> following a quotey bit getting misfontified with font-lock-string-face),
> please let me know.

I think this change breaks tests/lisp/electric-tests.el, here's a
reduced test case (backtrace attached)

(with-temp-buffer
  (c++-mode)
  (insert " (((()) ")
  (goto-char 4)
  (insert ")"))

[-- Attachment #2: c-quotes.backtrace --]
[-- Type: application/octet-stream, Size: 1619 bytes --]

Debugger entered--Lisp error: (args-out-of-range 0 0)
  get-text-property(0 syntax-table)
  c-clear-char-property-with-value-on-char-function(0 4 syntax-table (1) 39)
  c-parse-quotes-before-change(4 4)
  #f(compiled-function (fn) #<bytecode>)(c-parse-quotes-before-change)
  mapc(#f(compiled-function (fn) #<bytecode>) (c-extend-region-for-CPP c-before-change-check-raw-strings c-before-change-check-<>-operators c-depropertize-CPP c-invalidate-macro-cache c-truncate-bs-cache c-parse-quotes-before-change))
  c-before-change(4 4)
  insert(")")
  (progn (c++-mode) (insert " (((()) ") (goto-char 4) (insert ")"))
  (unwind-protect (progn (c++-mode) (insert " (((()) ") (goto-char 4) (insert ")")) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (c++-mode) (insert " (((()) ") (goto-char 4) (insert ")")) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
  (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (c++-mode) (insert " (((()) ") (goto-char 4) (insert ")")) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
  eval((let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (c++-mode) (insert " (((()) ") (goto-char 4) (insert ")")) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))) nil)
  elisp--eval-last-sexp(nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)

Re: Fix to handling of C++ digit separators, improvements in handling of single quotes.

[Alan Mackenzie]

Hello, Noam.

On Sat, Jul 01, 2017 at 23:35:41 -0400, Noam Postavsky wrote:
> On Sat, Jul 1, 2017 at 12:10 PM, Alan Mackenzie <acm@muc.de> wrote:
> > The code involved is moderately involved, with special cases needed to
> > handle things like '\'', '\\', or char foo [] = {',',',',',',','};.  So
> > if there are any problems with it (in particular, large areas of text
> > following a quotey bit getting misfontified with font-lock-string-face),
> > please let me know.

> I think this change breaks tests/lisp/electric-tests.el, here's a
> reduced test case (backtrace attached)

> (with-temp-buffer
>   (c++-mode)
>   (insert " (((()) ")
>   (goto-char 4)
>   (insert ")"))

Yes.  Sorry about that.

The problem is in c-quoted-number-head-before-point, where I've used
(match-beginning 0) without first checking that a search was successful.
In effect, I was using an uninitialised variable.

I'll fix it later on today, when I've got a clearer head and plenty of
time.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Fix to handling of C++ digit separators, improvements in handling of single quotes.

[Alan Mackenzie]

Hello, Noam.

On Sat, Jul 01, 2017 at 23:35:41 -0400, Noam Postavsky wrote:
> On Sat, Jul 1, 2017 at 12:10 PM, Alan Mackenzie <acm@muc.de> wrote:
> > The code involved is moderately involved, with special cases needed to
> > handle things like '\'', '\\', or char foo [] = {',',',',',',','};.  So
> > if there are any problems with it (in particular, large areas of text
> > following a quotey bit getting misfontified with font-lock-string-face),
> > please let me know.

> I think this change breaks tests/lisp/electric-tests.el, here's a
> reduced test case (backtrace attached)

> (with-temp-buffer
>   (c++-mode)
>   (insert " (((()) ")
>   (goto-char 4)
>   (insert ")"))

This should be fixed, now.

Obviously, I should have run the test myself.  I'll be doing this in the
future.

Thanks for reporting the bug to me in such an easy to digest way.  :-)

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Fix to handling of C++ digit separators, improvements in handling of single quotes.

[Noam Postavsky]

On Sun, Jul 2, 2017 at 9:05 AM, Alan Mackenzie <acm@muc.de> wrote:
>
> This should be fixed, now.

Thanks, it works.

> Obviously, I should have run the test myself.  I'll be doing this in the
> future.

We came ahead this time anyway, since you uncovered a little bug I had
introduced in the test machinery :)