bug#48837: Interaction between completion-moide and cc-mode new in 27.2

bug#48837: Interaction between completion-moide and cc-mode new in 27.2

[John C. Ruttenberg]

[-- Attachment #1: Type: text/plain, Size: 3959 bytes --]

This bug occurs in 27.2 but not in 26.3

In a clean emacs "emacs -q --no-site-file":
1. Enable dynamic completion "(dynamic-completion-mode)"
2. Open a c++ source file
3. Text should be:

foo() {
  std::cerr <
  int i = static_cast<unsigned>(z);
}

   4. Position the cursor directly after the "<" and type "< " (two
characters "<" " "
   5. I get this error message: "Scan error: containing expression ends
prematurely, 22, 22"



In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20)
 of 2021-06-02 built on catb
Windowing system distributor 'AT&T Laboratories Cambridge', version
11.0.3332
System Description: Ubuntu 20.04.2 LTS

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Loading completions from file /home/ruttenberg/.emacs.d/completions . . .
Loading completions from file /home/ruttenberg/.emacs.d/completions . . .
Done.
You can run the command ‘dynamic-completion-mode’ with M-x dy-m RET
Loading completions from file /home/ruttenberg/.emacs.d/completions . . .
Done.
Error in pre-command-hook (completion-before-command): (scan-error
"Containing expression ends prematurely" 16 16)
symbol-before-point: Scan error: "Containing expression ends prematurely",
17, 17
Configured using:
 'configure --prefix=/opt/emacs-27.2'

Configured features:
XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY
LIBSELINUX GNUTLS FREETYPE HARFBUZZ XFT ZLIB TOOLKIT_SCROLL_BARS GTK3
X11 XDBE XIM MODULES THREADS PDUMPER GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: C++//l

Minor modes in effect:
  dynamic-completion-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml mml-sec password-cache epa derived epg epg-config
gnus-util rmail rmail-loaddefs text-property-search time-date subr-x seq
byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils cc-mode cc-fonts
easymenu cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs cl-loaddefs cl-lib cus-start cus-load completion tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 76735 9686)
 (symbols 48 9655 1)
 (strings 32 23478 2066)
 (string-bytes 1 843632)
 (vectors 16 13035)
 (vector-slots 8 166467 11092)
 (floats 8 22 45)
 (intervals 56 208 3)
 (buffers 1000 14))

[-- Attachment #2: Type: text/html, Size: 4500 bytes --]

bug#48837: Interaction between completion-moide and cc-mode new in 27.2

[Alan Mackenzie]

Hello, John.

Thanks for taking the trouble to report this bug, and thanks even more
for cutting the test case down to an easy to work with minimum.

On Fri, Jun 04, 2021 at 16:13:47 -0400, John C. Ruttenberg wrote:
> This bug occurs in 27.2 but not in 26.3

> In a clean emacs "emacs -q --no-site-file":
> 1. Enable dynamic completion "(dynamic-completion-mode)"
> 2. Open a c++ source file
> 3. Text should be:

> foo() {
>   std::cerr <
>   int i = static_cast<unsigned>(z);
> }

>    4. Position the cursor directly after the "<" and type "< " (two
> characters "<" " "
>    5. I get this error message: "Scan error: containing expression ends
> prematurely, 22, 22"

The cause of this was that the "<" had wrongly been marked as an opening
angle bracket.  A backward `scan-sexps' Lisp instruction bumped into
this barrier of an angle bracket and gave the error.

The solution is to correct the reason for the spurious marking.  That
was that "static_cast" (along with three other newish ..._cast keywords)
weren't being properly handled.  This should be corrected in the
supplied patch (which actually fixes a bit more).

Would you please apply the following patch then byte-compile CC Mode in
its entirety (as a Lisp macro has been changed).  CC Mode is in
..../emacs/lisp/progmodes/cc-*.el

Then please load the new CC Mode into your Emacs 27.2, test it on your
real source code, and either confirm that the bug is fixed, or tell me
where it is still going wrong.  If you want any help with the patching
or byte compiling, feel free to send me private email.  Thanks!



diff -r 92a4592886a1 cc-engine.el
--- a/cc-engine.el	Sun Apr 25 17:26:38 2021 +0000
+++ b/cc-engine.el	Sun Jun 20 18:23:16 2021 +0000
@@ -6965,8 +6965,10 @@
 	(c-go-list-forward))
       (when (equal (c-get-char-property (1- (point)) 'syntax-table)
 		   c->-as-paren-syntax) ; should always be true.
-	(c-unmark-<->-as-paren (1- (point))))
-      (c-unmark-<->-as-paren pos))))
+	(c-unmark-<->-as-paren (1- (point)))
+	(c-truncate-lit-pos-cache (1- (point))))
+      (c-unmark-<->-as-paren pos)
+      (c-truncate-lit-pos-cache pos))))
 
 (defun c-clear->-pair-props (&optional pos)
   ;; POS (default point) is at a > character.  If it is marked with
@@ -6982,8 +6984,10 @@
 	(c-go-up-list-backward))
       (when (equal (c-get-char-property (point) 'syntax-table)
 			c-<-as-paren-syntax) ; should always be true.
-	(c-unmark-<->-as-paren (point)))
-      (c-unmark-<->-as-paren pos))))
+	(c-unmark-<->-as-paren (point))
+	(c-truncate-lit-pos-cache (point)))
+      (c-unmark-<->-as-paren pos)
+      (c-truncate-lit-pos-cache pos))))
 
 (defun c-clear-<>-pair-props (&optional pos)
   ;; POS (default point) is at a < or > character.  If it has an
@@ -7016,7 +7020,8 @@
 		 (equal (c-get-char-property (1- (point)) 'syntax-table)
 			c->-as-paren-syntax)) ; should always be true.
 	(c-unmark-<->-as-paren (1- (point)))
-	(c-unmark-<->-as-paren pos))
+	(c-unmark-<->-as-paren pos)
+	(c-truncate-lit-pos-cache pos))
       t)))
 
 (defun c-clear->-pair-props-if-match-before (lim &optional pos)
@@ -7037,6 +7042,7 @@
 		 (equal (c-get-char-property (point) 'syntax-table)
 			c-<-as-paren-syntax)) ; should always be true.
 	(c-unmark-<->-as-paren (point))
+	(c-truncate-lit-pos-cache (point))
 	(c-unmark-<->-as-paren pos))
       t)))
 
@@ -8078,13 +8084,14 @@
 	;; bracket arglist.  It's propagated through the return value
 	;; on successful completion.
 	(c-record-found-types c-record-found-types)
+	(syntax-table-prop-on-< (c-get-char-property (point) 'syntax-table))
 	;; List that collects the positions after the argument
 	;; separating ',' in the arglist.
 	arg-start-pos)
     ;; If the '<' has paren open syntax then we've marked it as an angle
     ;; bracket arglist before, so skip to the end.
     (if (and (not c-parse-and-markup-<>-arglists)
-	     (c-get-char-property (point) 'syntax-table))
+	     syntax-table-prop-on-<)
 
 	(progn
 	  (forward-char)
@@ -8167,8 +8174,20 @@
 			(c-put-c-type-property (1- (car arg-start-pos))
 					       'c-<>-arg-sep)
 			(setq arg-start-pos (cdr arg-start-pos)))
+		      (when (and (not syntax-table-prop-on-<)
+				 (c-get-char-property (1- (point))
+						      'syntax-table))
+			;; Clear the now spuriously matching < of its
+			;; syntax-table property.  This could happen on
+			;; inserting "_cast" into "static <" with C-y.
+			(save-excursion
+			  (and (c-go-list-backward)
+			       (eq (char-after) ?<)
+			       (c-truncate-lit-pos-cache (point))
+			       (c-unmark-<->-as-paren (point)))))
 		      (c-mark-<-as-paren start)
-		      (c-mark->-as-paren (1- (point))))
+		      (c-mark->-as-paren (1- (point)))
+		      (c-truncate-lit-pos-cache start))
 		    (setq res t)
 		    nil))		; Exit the loop.
 
diff -r 92a4592886a1 cc-langs.el
--- a/cc-langs.el	Sun Apr 25 17:26:38 2021 +0000
+++ b/cc-langs.el	Sun Jun 20 18:23:16 2021 +0000
@@ -2730,7 +2730,8 @@
 `c-recognize-<>-arglists' for details.  That language constant is
 assumed to be set if this isn't nil."
   t    nil
-  c++  '("template")
+  c++  '("template" "const_cast" "dynamic_cast" "reinterpret_cast"
+	 "static_cast")
   idl  '("fixed" "string" "wstring"))
 
 (c-lang-defconst c-<>-sexp-kwds


> In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20)
>  of 2021-06-02 built on catb
> Windowing system distributor 'AT&T Laboratories Cambridge', version
> 11.0.3332
> System Description: Ubuntu 20.04.2 LTS

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#48837: Interaction between completion-moide and cc-mode new in 27.2

[Alan Mackenzie]

Hello, John.

On Sat, Jun 26, 2021 at 16:17:41 -0400, John C. Ruttenberg wrote:
> Thanks.  The patch succeeded and the bug seems to be fixed.

Thanks!  I've committed the patch to the emacs-27 branch at savannah,
from where it should reach the master branch in a few days, and to stand
alone CC Mode and XEmacs.  I'm closing the bug with this post.

> I just submitted another bug but it says it needs moderation.  Did you
> see it?

By submitted, do you mean to bug-gnu-emacs@gnu.org?  If so, the "needs
moderation" sounds very strange indeed.  I don't know why this has
happened, and I didn't see your new bug last time I looked at the online
list (this morning, European time).  Can I suggest you try submitting it
again?

> Here is the text of the email, if not:

> Here is my C++ file:

> class __attribute__((packed)) y : z {

> Typing a new line after the  "{" indents to under the "z":

> class __attribute__((packed)) y : z {
>                                   ^ here

> I think it should indent the same as if "__attribute__((packed))" were not
> present.

This bug seems already to be fixed in the upcoming Emacs 28.1 and the
head of the standalone CC Mode.  The bug is present in Emacs 27.2.  With
a bit of bisection, I should be able to find the patch which fixed it.
With any luck, I'll be able to send you a patch for Emacs 27.2.

> This appears to be fragile in surprising ways.  If I spell "__attribute__"
> wrong (like __attribute), it does not reproduce.

> It also does not reproduce if the source file is:

> {
>   class __attribute__((packed)) y : z {

> But if the source file is:

> {
>   class __attribute__((packed)) y : z {

> then it does reproduce.

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#48837: Interaction between completion-moide and cc-mode new in 27.2

[John C. Ruttenberg]

[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]

My mistake on the "submission".  I sent the email to sourceforge.  But I am
glad the bug has been fixed.

On Sun, Jun 27, 2021 at 9:36 AM Alan Mackenzie <acm@muc.de> wrote:

> Hello, John.
>
> On Sat, Jun 26, 2021 at 16:17:41 -0400, John C. Ruttenberg wrote:
> > Thanks.  The patch succeeded and the bug seems to be fixed.
>
> Thanks!  I've committed the patch to the emacs-27 branch at savannah,
> from where it should reach the master branch in a few days, and to stand
> alone CC Mode and XEmacs.  I'm closing the bug with this post.
>
> > I just submitted another bug but it says it needs moderation.  Did you
> > see it?
>
> By submitted, do you mean to bug-gnu-emacs@gnu.org?  If so, the "needs
> moderation" sounds very strange indeed.  I don't know why this has
> happened, and I didn't see your new bug last time I looked at the online
> list (this morning, European time).  Can I suggest you try submitting it
> again?
>
> > Here is the text of the email, if not:
>
> > Here is my C++ file:
>
> > class __attribute__((packed)) y : z {
>
> > Typing a new line after the  "{" indents to under the "z":
>
> > class __attribute__((packed)) y : z {
> >                                   ^ here
>
> > I think it should indent the same as if "__attribute__((packed))" were
> not
> > present.
>
> This bug seems already to be fixed in the upcoming Emacs 28.1 and the
> head of the standalone CC Mode.  The bug is present in Emacs 27.2.  With
> a bit of bisection, I should be able to find the patch which fixed it.
> With any luck, I'll be able to send you a patch for Emacs 27.2.
>
> > This appears to be fragile in surprising ways.  If I spell
> "__attribute__"
> > wrong (like __attribute), it does not reproduce.
>
> > It also does not reproduce if the source file is:
>
> > {
> >   class __attribute__((packed)) y : z {
>
> > But if the source file is:
>
> > {
> >   class __attribute__((packed)) y : z {
>
> > then it does reproduce.
>
> [ .... ]
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>

[-- Attachment #2: Type: text/html, Size: 2689 bytes --]