bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[Eason Huang]

Hello Emacs dev team,

I found that the c++-ts-mode conflict with electric-pair-mode.
My build is based on commit a15cd55044c6bb4d4e9f9aec23b8b5b9cbcb38ac

May be the issue was introduced by this commit:
https://git.savannah.gnu.org/cgit/emacs.git/commit/?h=emacs-29&id=480f41c7deb4d3a45a862a5308950b35085f835d

I can confirm that Before the commit 480f41c7, It work well.

steps to reproduce:

1. emacs -Q, start Emacs
2. C-x, C-f ~/test.cpp, open any cpp file.
3. M-x c++-ts-mode, enable c++ tree-sitter support
4. M-x electric-pair-mode, enable electric-pair-mode
4. Type the following code in the test.cpp buffer.
```
#include <
```
you will the the result as below:
```
#include <<
```

expected result is:
```
#include <>
```


--
Eason Huang


In GNU Emacs 29.0.60 (build 1, x86_64-w64-mingw32) of 2022-12-15 built
 on DESKTOP-VIHFE84
Windowing system distributor 'Microsoft Corp.', version 10.0.19044
System Description: Microsoft Windows 10 Pro (v10.0.2009.19044.2364)

Configured using:
 'configure --with-native-compilation=aot'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

Important settings:
  value of $LANG: CHS
  locale-coding-system: cp936

Major mode: C++

Minor modes in effect:
  electric-pair-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils cus-start
cus-load elec-pair c-ts-mode treesit cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs cl-loaddefs comp
comp-cstr warnings icons subr-x rx cl-seq cl-macs gv cl-extra help-mode
bytecomp byte-compile cl-lib china-util rmc iso-transl tooltip cconv
eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads w32notify w32 lcms2 multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 117001 10015)
 (symbols 48 10094 0)
 (strings 32 29095 1843)
 (string-bytes 1 947838)
 (vectors 16 19056)
 (vector-slots 8 428832 17970)
 (floats 8 49 205)
 (intervals 56 295 0)
 (buffers 984 14))

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[Daniel Martín via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eason Huang <aqua0210@foxmail.com> writes:

> Hello Emacs dev team,
>
> I found that the c++-ts-mode conflict with electric-pair-mode.
> My build is based on commit a15cd55044c6bb4d4e9f9aec23b8b5b9cbcb38ac
>
> May be the issue was introduced by this commit:
> https://git.savannah.gnu.org/cgit/emacs.git/commit/?h=emacs-29&id=480f41c7deb4d3a45a862a5308950b35085f835d
>
> I can confirm that Before the commit 480f41c7, It work well.
>
> steps to reproduce:
>
> 1. emacs -Q, start Emacs
> 2. C-x, C-f ~/test.cpp, open any cpp file.
> 3. M-x c++-ts-mode, enable c++ tree-sitter support
> 4. M-x electric-pair-mode, enable electric-pair-mode
> 4. Type the following code in the test.cpp buffer.
> ```
> #include <
> ```
> you will the the result as below:
> ```
> #include <<
> ```
>
> expected result is:
> ```
> #include <>
> ```
>

Thanks for the report.  I think the bug is that we need a second
character in the syntax table entry, to set the correct matching
delimiter.

Does the following patch solve this issue and bug#60049?

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index ff2ff63fd8..11b2b9408c 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -86,8 +86,8 @@ c-ts-mode--syntax-table
 (defvar c++-ts-mode--syntax-table
   (let ((table (make-syntax-table c-ts-mode--syntax-table)))
     ;; Template delimiters.
-    (modify-syntax-entry ?<  "("     table)
-    (modify-syntax-entry ?>  ")"     table)
+    (modify-syntax-entry ?<  "(>"     table)
+    (modify-syntax-entry ?>  ")<"     table)
     table)
   "Syntax table for `c++-ts-mode'.")
 

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

On Thu, Dec 15, 2022 at 11:34 AM Eason Huang <aqua0210@foxmail.com> wrote:
>
> Hello Emacs dev team,
>
> I found that the c++-ts-mode conflict with electric-pair-mode.
> My build is based on commit a15cd55044c6bb4d4e9f9aec23b8b5b9cbcb38ac
>
> May be the issue was introduced by this commit:
>
https://git.savannah.gnu.org/cgit/emacs.git/commit/?h=emacs-29&id=480f41c7deb4d3a45a862a5308950b35085f835d
>
> I can confirm that Before the commit 480f41c7, It work well.

Does "it work well" mean it used to auto-pair '<' to '>'?  I don't
have a tree-sitter build to check atm, but I find this odd.

João

[-- Attachment #2: Type: text/html, Size: 964 bytes --]

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 816 bytes --]

On Thu, Dec 15, 2022 at 12:25 PM Daniel Martín via Bug reports for GNU
Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org> wrote:

>
> Thanks for the report.  I think the bug is that we need a second
> character in the syntax table entry, to set the correct matching
> delimiter.
>
> Does the following patch solve this issue and bug#60049?
>
>
I don't know how this can work if '<' is going to be used to input
the less-than operator.  I think more complex syntax-propertization
functionality is going to be needed here.  I don't have a tree-sitter
build to test, but I'd say that the tree-sitter backend should be asked
about what kind of '<' and '>' we're talking about.  AFAICT, just
saying that '<' has the delimiter syntax in C++ is wrong. It's not
like '(' or '{.'

João

[-- Attachment #2: Type: text/html, Size: 1265 bytes --]

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[Eason Huang]

João Távora <joaotavora@gmail.com> writes:

> On Thu, Dec 15, 2022 at 11:34 AM Eason Huang <aqua0210@foxmail.com> wrote:
>>
>> Hello Emacs dev team,
>>
>> I found that the c++-ts-mode conflict with electric-pair-mode.
>> My build is based on commit a15cd55044c6bb4d4e9f9aec23b8b5b9cbcb38ac
>>
>> May be the issue was introduced by this commit:
>> https://git.savannah.gnu.org/cgit/emacs.git/commit/?h=emacs-29&id=480f41c7deb4d3a45a862a5308950b35085f835d
>>
>> I can confirm that Before the commit 480f41c7, It work well.
>
> Does "it work well" mean it used to auto-pair '<' to '>'?  I don't
> have a tree-sitter build to check atm, but I find this odd.

Yes, I mean it works as expected , type '<' will get auto-pair '<' to
'>'.


-- 
Eason Huang

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[Daniel Martín via Bug reports for GNU Emacs, the Swiss army knife of text editors]

João Távora <joaotavora@gmail.com> writes:

> I don't know how this can work if '<' is going to be used to input
> the less-than operator.  I think more complex syntax-propertization
> functionality is going to be needed here.  I don't have a tree-sitter
> build to test, but I'd say that the tree-sitter backend should be asked
> about what kind of '<' and '>' we're talking about.  AFAICT, just
> saying that '<' has the delimiter syntax in C++ is wrong. It's not
> like '(' or '{.'

Yes, you are right.  I think for now it's better to consider them as
punctuation.

A better idea may be to add a syntax-table text property to "<" and ">"
when they define a C++ template (or a Java generic).  Tree-sitter can
match something like a "template_argument_list" node easily, but I
wonder how to keep this information in sync with the buffer text in the
most efficient way?

I thought about adding a custom parser notifier via
treesit-parser-add-notifier and remove/add the text properties for the
template argument list delimiters.  WDYT?  Of course, C would not use
this extra processing at all.  I've CC'ed Yuan for feedback.

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1855 bytes --]

On Thu, Dec 15, 2022 at 9:34 PM Daniel Martín <mardani29@yahoo.es> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> > I don't know how this can work if '<' is going to be used to input
> > the less-than operator.  I think more complex syntax-propertization
> > functionality is going to be needed here.  I don't have a tree-sitter
> > build to test, but I'd say that the tree-sitter backend should be asked
> > about what kind of '<' and '>' we're talking about.  AFAICT, just
> > saying that '<' has the delimiter syntax in C++ is wrong. It's not
> > like '(' or '{.'
>
> Yes, you are right.  I think for now it's better to consider them as
> punctuation.
>
> A better idea may be to add a syntax-table text property to "<" and ">"
> when they define a C++ template (or a Java generic).  Tree-sitter can
> match something like a "template_argument_list" node easily, but I
> wonder how to keep this information in sync with the buffer text in the
> most efficient way?

Supposedly this is what I understand tree-sitter to be very very good
at.  My completely naive and absolutely ignorant understanding of the
new tree sitter modes is that once you insert something into the buffer,
tree sitter is immediately notified, recomputes the syntax tree very
efficiently and incrementally, then Emacs can immediately take advantage
of that in the `syntax-propertize-function`, applying the correct syntax
to each separate manifestation of the '<' and '>' characters.  Hopefully
all of this runs in time for post-self-insert-hook to see the correct syntax
class and thus electric-pair-mode, which relies on p-s-i-h can do the
right thing each time.

Again, this is completely my vapourware idea of how all this was
implemented.  I'll also CC: Stefan who is my syntax-propertize
and electric-*-mode sensei.

João

[-- Attachment #2: Type: text/html, Size: 2419 bytes --]

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[Eason Huang]

João Távora <joaotavora@gmail.com> writes:

> On Thu, Dec 15, 2022 at 9:34 PM Daniel Martín <mardani29@yahoo.es> wrote:

> Does the following patch solve this issue and bug#60049?

> diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
> index ff2ff63fd8..11b2b9408c 100644
> --- a/lisp/progmodes/c-ts-mode.el
> +++ b/lisp/progmodes/c-ts-mode.el
> @@ -86,8 +86,8 @@ c-ts-mode--syntax-table
> (defvar c++-ts-mode--syntax-table
>   (let ((table (make-syntax-table c-ts-mode--syntax-table)))
>     ;; Template delimiters.
>-    (modify-syntax-entry ?<  "("     table)
>-    (modify-syntax-entry ?>  ")"     table)
>+    (modify-syntax-entry ?<  "(>"     table)
>+    (modify-syntax-entry ?>  ")<"     table)
>     table)
>   "Syntax table for `c++-ts-mode'.")

Hi Daniel,

Sorry for the delay feedback.

I tried your patch and it fixs bug#60049, but I just notice that `show-paren-mode` don't
play well with c++-ts-mode too (before and after this patch).

with the code snippet as below:

```
#include <iostream>

int main (){
    std::cout << "Test" << '\n';
    return 0;
}
```

With `show-paren-mode` enabled, then put cursor after the '}', the last
 '<' will be matched and highlighted, the expected match is the '{'.

I hope this information will help you to understand the issue.

-- 
Eason Huang

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[Eason Huang]

Hi all,

I can confirmed that the latest commit (909091d7578b7225601b202fb9257dedae879e9a)
 of Emacs-29 have been fix this bug#60087,  and `show-paren-mode` also get the expected result
 when c++-ts-mode enabled.

I think we can close this bug now.

----

Eason Huang

> 2022年12月27日 17:03，Eason Huang <aqua0210@foxmail.com> 写道：
> 
> João Távora <joaotavora@gmail.com> writes:
> 
>> On Thu, Dec 15, 2022 at 9:34 PM Daniel Martín <mardani29@yahoo.es> wrote:
> 
>> Does the following patch solve this issue and bug#60049?
> 
>> diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
>> index ff2ff63fd8..11b2b9408c 100644
>> --- a/lisp/progmodes/c-ts-mode.el
>> +++ b/lisp/progmodes/c-ts-mode.el
>> @@ -86,8 +86,8 @@ c-ts-mode--syntax-table
>> (defvar c++-ts-mode--syntax-table
>>  (let ((table (make-syntax-table c-ts-mode--syntax-table)))
>>    ;; Template delimiters.
>> -    (modify-syntax-entry ?<  "("     table)
>> -    (modify-syntax-entry ?>  ")"     table)
>> +    (modify-syntax-entry ?<  "(>"     table)
>> +    (modify-syntax-entry ?>  ")<"     table)
>>    table)
>>  "Syntax table for `c++-ts-mode'.")
> 
> Hi Daniel,
> 
> Sorry for the delay feedback.
> 
> I tried your patch and it fixs bug#60049, but I just notice that `show-paren-mode` don't
> play well with c++-ts-mode too (before and after this patch).
> 
> with the code snippet as below:
> 
> ```
> #include <iostream>
> 
> int main (){
>    std::cout << "Test" << '\n';
>    return 0;
> }
> ```
> 
> With `show-paren-mode` enabled, then put cursor after the '}', the last
> '<' will be matched and highlighted, the expected match is the '{'.
> 
> I hope this information will help you to understand the issue.
> 

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[Eli Zaretskii]

> Cc: casouri@gmail.com, Stefan Monnier <monnier@iro.umontreal.ca>,
>  60087@debbugs.gnu.org, Daniel Martín <mardani29@yahoo.es>
> From: Eason Huang <aqua0210@foxmail.com>
> Date: Thu, 29 Dec 2022 14:10:13 +0800
> 
> Hi all,
> 
> I can confirmed that the latest commit (909091d7578b7225601b202fb9257dedae879e9a)
>  of Emacs-29 have been fix this bug#60087,  and `show-paren-mode` also get the expected result
>  when c++-ts-mode enabled.
> 
> I think we can close this bug now.

Thanks, done.

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[Yuan Fu]

> On Dec 28, 2022, at 10:10 PM, Eason Huang <aqua0210@foxmail.com> wrote:
> 
> Hi all,
> 
> I can confirmed that the latest commit (909091d7578b7225601b202fb9257dedae879e9a)
> of Emacs-29 have been fix this bug#60087,  and `show-paren-mode` also get the expected result
> when c++-ts-mode enabled.
> 
> I think we can close this bug now.

Sorry guys, I completely missed this bug report :-(

Yuan

bug#60087: 29.0.60; c++-ts-mode conflict with electric-pair-mode

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

I wish that we would either re-open this bug or open a new one
to study the pairing or not-pairing of `<` and `>` with Tree-sitter
and electric-pair-mode.

One of the high hopes that I had for tree-sitter modes is that
it would help electric-pair-mode decide what to do depending
on the syntactic situation.  So it would pair < and > in template
argument lists but not when < or > are being used as
comparison operators.

I wish Stefan or Yuan Fu would comment here.

João

[-- Attachment #2: Type: text/html, Size: 701 bytes --]