bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode

bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode

[Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi:

Using this code:

```
int main()
{
  abort(); /* 1 */
  abort(); /* 1 */
}
```

There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.

When there is a comment at the end of the line, if I do `mark-sexp`
(C-M-SPC) consecutively I get this selected regions:

-----------------------------
1.
abort();

2.
abort(); /* 1 */

3.
abort(); /* 1 */
abort

4.
abort(); /* 1 */
abort()

5.
abort(); /* 1 */
abort();

6.
abort(); /* 1 */
abort(); /* 1 */

-------------------------------



But when there is NOT trailing comment

```
int main()
{
  abort();
  abort();
}
```

-------------------------------
1.
abort();

2.
abort();
abort();
-------------------------------


It looks like in the fist example after 3 the sexp definition is more fine
grained (similar to the previous c++-mode behavior) and it selects
separately:
 the function name,
 the arguments
 the semicolon
 the comment

But if there is no comment at the end, it always considers the complete
line as a sexp (including the ;).

For my use case I would prefer the old behavior because it is consistent
with the current sexp definition in all emacs (with maybe the exception
of python-mode).  Because it is easier to copy function names or
function calls with a few movements.

However, if it is too difficult to reproduce the old behavior; then the
new one may be implemented consistently.

Best,
Ergus



In GNU Emacs 31.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version
 3.24.43, cairo version 1.18.0) of 2024-08-07 built on RTX
Repository revision: dd93c32c82c65f7c7b26e8c7a8e4fa8425cdbfe0
Repository branch: project
System Description: Arch Linux

Configured using:
 'configure --prefix=/home/ergo/.local/ --with-mailutils --with-pgtk
 --with-modules --with-cairo --with-harfbuzz
 --with-native-compilation=aot
 '--program-transform-name=s/^ctags$/ctags.emacs/''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: C++//

Minor modes in effect:
  fancy-compilation-mode: t
  global-auto-revert-mode: t
  electric-pair-mode: t
  whitespace-mode: t
  flyspell-mode: t
  completion-preview-mode: t
  diff-hl-margin-local-mode: t
  diff-hl-margin-mode: t
  diff-hl-mode: t
  corfu-terminal-mode: t
  global-corfu-mode: t
  corfu-mode: t
  project-multi-mode: t
  gtags-mode: t
  repeat-mode: t
  xterm-mouse-mode: t
  xclip-mode: t
  override-global-mode: t
  winner-mode: t
  save-place-mode: t
  delete-selection-mode: t
  savehist-mode: t
  global-display-fill-column-indicator-mode: t
  display-fill-column-indicator-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  which-key-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/mnt/casa/gits/emacs_clones/cuda-mode/cuda-mode hides /home/ergo/.config/emacs/elpa/cuda-mode-20240716.1936/cuda-mode
/mnt/casa/gits/emacs_clones/gtags-mode/gtags-mode hides /home/ergo/.config/emacs/elpa/gtags-mode-1.8/gtags-mode
/home/ergo/.config/emacs/elpa/transient-20240805.1231/transient hides /home/ergo/.local/share/emacs/31.0.50/lisp/transient

Features:
(shadow sort mail-extr fancy-compilation compile comint ansi-osc
ansi-color comp-run comp-common emacsbug message mailcap yank-media puny
rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util
text-property-search time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils autorevert filenotify ffap
thingatpt url-parse auth-source eieio eieio-core icons password-cache
json map url-vars elec-pair whitespace flyspell-correct flyspell ispell
completion-preview diff-hl-margin diff-hl-dired citre-lang-fileref
citre-tags citre-ctags citre-readtags citre-readtags-tables
citre-backend-interface citre-common-tag rx citre-common-util dired-x
dired dired-loaddefs diff-hl log-view pcvs-util vc-dir ewoc vc
vc-dispatcher diff-mode track-changes corfu-terminal popon corfu
project-multi-mode gtags-mode cl-macs files-x xref project c++-ts-mode
c-ts-mode c-ts-common term/tmux term/xterm xterm init repeat
markdown-ts-mode subr-x treesit cape compat use-package-ensure
use-package-diminish xt-mouse xclip edmacro kmacro byte-opt gv
use-package-bind-key bind-key cl-extra help-mode simple-16-theme winner
ring saveplace delsel savehist easy-mmode display-fill-column-indicator
display-line-numbers diminish which-key cl-seq use-package-core
cl-loaddefs cl-lib bytecomp byte-compile disp-table info
0blayout-autoloads ac-emoji-autoloads ac-haskell-process-autoloads
ac-html-autoloads arduino-cli-mode-autoloads auctex-autoloads tex-site
auto-complete-autoloads avy-zap-autoloads avy-autoloads
better-shell-autoloads caml-autoloads cape-autoloads citre-autoloads
clang-format-autoloads cobol-mode-autoloads compile-multi-autoloads
corfu-terminal-autoloads corfu-autoloads crdt-autoloads
csv-mode-autoloads cuda-mode-autoloads d-mode-autoloads
deadgrep-autoloads debbugs-autoloads diff-hl-autoloads
diminish-autoloads dired-sidebar-autoloads dired-subtree-autoloads
dired-hacks-utils-autoloads dumb-jump-autoloads e2ansi-autoloads
emamux-autoloads esup-autoloads evil-collection-autoloads
annalist-autoloads evil-leader-autoloads evil-autoloads
face-explorer-autoloads fancy-compilation-autoloads flx-autoloads
flycheck-julia-autoloads flycheck-rust-autoloads flycheck-autoloads
flymake-nasm-autoloads flymake-quickdef-autoloads
flyspell-correct-autoloads git-modes-autoloads git-timemachine-autoloads
gnuplot-autoloads google-c-style-autoloads goto-chg-autoloads
groovy-mode-autoloads gtags-mode-autoloads haskell-mode-autoloads
highlight-indent-guides-autoloads i3wm-config-mode-autoloads
ibuffer-sidebar-autoloads iedit-autoloads imenu-list-autoloads
julia-ts-mode-autoloads julia-mode-autoloads languagetool-autoloads
lice-autoloads lorem-ipsum-autoloads lua-mode-autoloads magit-autoloads
git-commit-autoloads magit-section-autoloads markdown-mode-autoloads
markdown-ts-mode-autoloads modern-cpp-font-lock-autoloads
move-dup-autoloads multiple-cursors-autoloads mutt-mode-autoloads
nasm-mode-autoloads neotree-autoloads nftables-mode-autoloads
nginx-mode-autoloads notmuch-autoloads objed-autoloads
opencl-mode-autoloads paradox-autoloads phi-search-autoloads
pkg-info-autoloads epl-autoloads pkgbuild-mode-autoloads
platformio-mode-autoloads async-autoloads popon-autoloads
popup-autoloads projectile-autoloads projection-autoloads
protobuf-mode-autoloads protobuf-ts-mode-autoloads
ptemplate-templates-autoloads ptemplate-autoloads scopeline-autoloads
shell-command+-autoloads slime-autoloads macrostep-autoloads
sphinx-mode-autoloads f-autoloads dash-autoloads s-autoloads
spinner-autoloads ssh-config-mode-autoloads string-inflection-autoloads
sudo-edit-autoloads systemd-autoloads tmux-mode-autoloads
transient-autoloads tsc-autoloads urgrep-autoloads vdiff-autoloads
hydra-autoloads lv-autoloads vterm-toggle-autoloads vterm-autoloads
vundo-autoloads with-editor-autoloads xclip-autoloads
yasnippet-snippets-autoloads yasnippet-autoloads early-init rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win
term/common-win touch-screen pgtk-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty move-toolbar make-network-process native-compile emacs)

Memory information:
((conses 16 185298 34784) (symbols 48 15017 0)
 (strings 32 50296 12386) (string-bytes 1 1766229) (vectors 16 19239)
 (vector-slots 8 220612 9451) (floats 8 95 112) (intervals 56 967 0)
 (buffers 992 13))

bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode

[Eli Zaretskii]

> Date: Thu, 08 Aug 2024 16:45:42 +0200
> From:  Ergus via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> 
> Hi:
> 
> Using this code:
> 
> ```
> int main()
> {
>   abort(); /* 1 */
>   abort(); /* 1 */
> }
> ```
> 
> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
> 
> When there is a comment at the end of the line, if I do `mark-sexp`
> (C-M-SPC) consecutively I get this selected regions:
> 
> -----------------------------
> 1.
> abort();
> 
> 2.
> abort(); /* 1 */
> 
> 3.
> abort(); /* 1 */
> abort
> 
> 4.
> abort(); /* 1 */
> abort()
> 
> 5.
> abort(); /* 1 */
> abort();
> 
> 6.
> abort(); /* 1 */
> abort(); /* 1 */
> 
> -------------------------------
> 
> 
> 
> But when there is NOT trailing comment
> 
> ```
> int main()
> {
>   abort();
>   abort();
> }
> ```
> 
> -------------------------------
> 1.
> abort();
> 
> 2.
> abort();
> abort();
> -------------------------------
> 
> 
> It looks like in the fist example after 3 the sexp definition is more fine
> grained (similar to the previous c++-mode behavior) and it selects
> separately:
>  the function name,
>  the arguments
>  the semicolon
>  the comment
> 
> But if there is no comment at the end, it always considers the complete
> line as a sexp (including the ;).
> 
> For my use case I would prefer the old behavior because it is consistent
> with the current sexp definition in all emacs (with maybe the exception
> of python-mode).  Because it is easier to copy function names or
> function calls with a few movements.
> 
> However, if it is too difficult to reproduce the old behavior; then the
> new one may be implemented consistently.

Yuan, any comments or suggestions?

FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
source code is not well-defined.

bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode

[Yuan Fu]

> On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> Date: Thu, 08 Aug 2024 16:45:42 +0200
>> From:  Ergus via "Bug reports for GNU Emacs,
>> the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> 
>> Hi:
>> 
>> Using this code:
>> 
>> ```
>> int main()
>> {
>>  abort(); /* 1 */
>>  abort(); /* 1 */
>> }
>> ```
>> 
>> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
>> 
>> When there is a comment at the end of the line, if I do `mark-sexp`
>> (C-M-SPC) consecutively I get this selected regions:
>> 
>> -----------------------------
>> 1.
>> abort();
>> 
>> 2.
>> abort(); /* 1 */
>> 
>> 3.
>> abort(); /* 1 */
>> abort
>> 
>> 4.
>> abort(); /* 1 */
>> abort()
>> 
>> 5.
>> abort(); /* 1 */
>> abort();
>> 
>> 6.
>> abort(); /* 1 */
>> abort(); /* 1 */
>> 
>> -------------------------------
>> 
>> 
>> 
>> But when there is NOT trailing comment
>> 
>> ```
>> int main()
>> {
>>  abort();
>>  abort();
>> }
>> ```
>> 
>> -------------------------------
>> 1.
>> abort();
>> 
>> 2.
>> abort();
>> abort();
>> -------------------------------
>> 
>> 
>> It looks like in the fist example after 3 the sexp definition is more fine
>> grained (similar to the previous c++-mode behavior) and it selects
>> separately:
>> the function name,
>> the arguments
>> the semicolon
>> the comment
>> 
>> But if there is no comment at the end, it always considers the complete
>> line as a sexp (including the ;).
>> 
>> For my use case I would prefer the old behavior because it is consistent
>> with the current sexp definition in all emacs (with maybe the exception
>> of python-mode).  Because it is easier to copy function names or
>> function calls with a few movements.
>> 
>> However, if it is too difficult to reproduce the old behavior; then the
>> new one may be implemented consistently.
> 
> Yuan, any comments or suggestions?
> 
> FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
> source code is not well-defined.

Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.

Yuan

bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode

[Eli Zaretskii]

Ping!  Any progress with this?

> From: Yuan Fu <casouri@gmail.com>
> Date: Sat, 10 Aug 2024 22:10:01 -0700
> Cc: Ergus <spacibba@aol.com>,
>  72525@debbugs.gnu.org
> 
> 
> 
> > On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> Date: Thu, 08 Aug 2024 16:45:42 +0200
> >> From:  Ergus via "Bug reports for GNU Emacs,
> >> the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> >> 
> >> 
> >> Hi:
> >> 
> >> Using this code:
> >> 
> >> ```
> >> int main()
> >> {
> >>  abort(); /* 1 */
> >>  abort(); /* 1 */
> >> }
> >> ```
> >> 
> >> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
> >> 
> >> When there is a comment at the end of the line, if I do `mark-sexp`
> >> (C-M-SPC) consecutively I get this selected regions:
> >> 
> >> -----------------------------
> >> 1.
> >> abort();
> >> 
> >> 2.
> >> abort(); /* 1 */
> >> 
> >> 3.
> >> abort(); /* 1 */
> >> abort
> >> 
> >> 4.
> >> abort(); /* 1 */
> >> abort()
> >> 
> >> 5.
> >> abort(); /* 1 */
> >> abort();
> >> 
> >> 6.
> >> abort(); /* 1 */
> >> abort(); /* 1 */
> >> 
> >> -------------------------------
> >> 
> >> 
> >> 
> >> But when there is NOT trailing comment
> >> 
> >> ```
> >> int main()
> >> {
> >>  abort();
> >>  abort();
> >> }
> >> ```
> >> 
> >> -------------------------------
> >> 1.
> >> abort();
> >> 
> >> 2.
> >> abort();
> >> abort();
> >> -------------------------------
> >> 
> >> 
> >> It looks like in the fist example after 3 the sexp definition is more fine
> >> grained (similar to the previous c++-mode behavior) and it selects
> >> separately:
> >> the function name,
> >> the arguments
> >> the semicolon
> >> the comment
> >> 
> >> But if there is no comment at the end, it always considers the complete
> >> line as a sexp (including the ;).
> >> 
> >> For my use case I would prefer the old behavior because it is consistent
> >> with the current sexp definition in all emacs (with maybe the exception
> >> of python-mode).  Because it is easier to copy function names or
> >> function calls with a few movements.
> >> 
> >> However, if it is too difficult to reproduce the old behavior; then the
> >> new one may be implemented consistently.
> > 
> > Yuan, any comments or suggestions?
> > 
> > FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
> > source code is not well-defined.
> 
> Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.
> 
> Yuan

bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode

[Yuan Fu]

> On Aug 24, 2024, at 1:28 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> Ping!  Any progress with this?
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sat, 10 Aug 2024 22:10:01 -0700
>> Cc: Ergus <spacibba@aol.com>,
>> 72525@debbugs.gnu.org
>> 
>> 
>> 
>>> On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>>> Date: Thu, 08 Aug 2024 16:45:42 +0200
>>>> From:  Ergus via "Bug reports for GNU Emacs,
>>>> the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>>>> 
>>>> 
>>>> Hi:
>>>> 
>>>> Using this code:
>>>> 
>>>> ```
>>>> int main()
>>>> {
>>>> abort(); /* 1 */
>>>> abort(); /* 1 */
>>>> }
>>>> ```
>>>> 
>>>> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
>>>> 
>>>> When there is a comment at the end of the line, if I do `mark-sexp`
>>>> (C-M-SPC) consecutively I get this selected regions:
>>>> 
>>>> -----------------------------
>>>> 1.
>>>> abort();
>>>> 
>>>> 2.
>>>> abort(); /* 1 */
>>>> 
>>>> 3.
>>>> abort(); /* 1 */
>>>> abort
>>>> 
>>>> 4.
>>>> abort(); /* 1 */
>>>> abort()
>>>> 
>>>> 5.
>>>> abort(); /* 1 */
>>>> abort();
>>>> 
>>>> 6.
>>>> abort(); /* 1 */
>>>> abort(); /* 1 */
>>>> 
>>>> -------------------------------
>>>> 
>>>> 
>>>> 
>>>> But when there is NOT trailing comment
>>>> 
>>>> ```
>>>> int main()
>>>> {
>>>> abort();
>>>> abort();
>>>> }
>>>> ```
>>>> 
>>>> -------------------------------
>>>> 1.
>>>> abort();
>>>> 
>>>> 2.
>>>> abort();
>>>> abort();
>>>> -------------------------------
>>>> 
>>>> 
>>>> It looks like in the fist example after 3 the sexp definition is more fine
>>>> grained (similar to the previous c++-mode behavior) and it selects
>>>> separately:
>>>> the function name,
>>>> the arguments
>>>> the semicolon
>>>> the comment
>>>> 
>>>> But if there is no comment at the end, it always considers the complete
>>>> line as a sexp (including the ;).
>>>> 
>>>> For my use case I would prefer the old behavior because it is consistent
>>>> with the current sexp definition in all emacs (with maybe the exception
>>>> of python-mode).  Because it is easier to copy function names or
>>>> function calls with a few movements.
>>>> 
>>>> However, if it is too difficult to reproduce the old behavior; then the
>>>> new one may be implemented consistently.
>>> 
>>> Yuan, any comments or suggestions?
>>> 
>>> FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
>>> source code is not well-defined.
>> 
>> Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.
>> 
>> Yuan

I know the reason for the inconsistency now. Treesit-forward-sexp first checks whether point is in a “text” node, ie, comment or string; if so, it uses the default/normal forward-sexp function; if not, it uses the parse tree to go over sexp.

In the first example, because there’s a comment right before point, treesit-forward-sexp thinks it’s in a text node, and used the normal forward-sexp function, which moved point after the next symbol.

In the second example, because there’s no comment anymore, treesit-forward-sexp uses the parse tree to move the point; and since the next node after point is the statement line, it moves point over the whole line. When using the parse tree to move point, treesit-forward-sexp always moves in the same “level” in which that point is. Eg, when point is between two lines, treesit-forward-sexp moves over lines; if point is inside an argument list between two arguments, treesit-forward-sexp moves over each argument.

If you want to just select the identifier or other more fine-grained movement, IMHO it’s probably better to use forward-word.

I fixed the inconsistency so now treesit-forward-sexp in both example moves over the whole line. The fix is pushed to emacs-30.

Yuan

bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Mon, 26 Aug 2024 19:53:51 -0700
> Cc: Ergus <spacibba@aol.com>,
>  72525@debbugs.gnu.org
> 
> 
> 
> > On Aug 24, 2024, at 1:28 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> > Ping!  Any progress with this?
> > 
> >> From: Yuan Fu <casouri@gmail.com>
> >> Date: Sat, 10 Aug 2024 22:10:01 -0700
> >> Cc: Ergus <spacibba@aol.com>,
> >> 72525@debbugs.gnu.org
> >> 
> >> 
> >> 
> >>> On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> >>> 
> >>>> Date: Thu, 08 Aug 2024 16:45:42 +0200
> >>>> From:  Ergus via "Bug reports for GNU Emacs,
> >>>> the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> >>>> 
> >>>> 
> >>>> Hi:
> >>>> 
> >>>> Using this code:
> >>>> 
> >>>> ```
> >>>> int main()
> >>>> {
> >>>> abort(); /* 1 */
> >>>> abort(); /* 1 */
> >>>> }
> >>>> ```
> >>>> 
> >>>> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
> >>>> 
> >>>> When there is a comment at the end of the line, if I do `mark-sexp`
> >>>> (C-M-SPC) consecutively I get this selected regions:
> >>>> 
> >>>> -----------------------------
> >>>> 1.
> >>>> abort();
> >>>> 
> >>>> 2.
> >>>> abort(); /* 1 */
> >>>> 
> >>>> 3.
> >>>> abort(); /* 1 */
> >>>> abort
> >>>> 
> >>>> 4.
> >>>> abort(); /* 1 */
> >>>> abort()
> >>>> 
> >>>> 5.
> >>>> abort(); /* 1 */
> >>>> abort();
> >>>> 
> >>>> 6.
> >>>> abort(); /* 1 */
> >>>> abort(); /* 1 */
> >>>> 
> >>>> -------------------------------
> >>>> 
> >>>> 
> >>>> 
> >>>> But when there is NOT trailing comment
> >>>> 
> >>>> ```
> >>>> int main()
> >>>> {
> >>>> abort();
> >>>> abort();
> >>>> }
> >>>> ```
> >>>> 
> >>>> -------------------------------
> >>>> 1.
> >>>> abort();
> >>>> 
> >>>> 2.
> >>>> abort();
> >>>> abort();
> >>>> -------------------------------
> >>>> 
> >>>> 
> >>>> It looks like in the fist example after 3 the sexp definition is more fine
> >>>> grained (similar to the previous c++-mode behavior) and it selects
> >>>> separately:
> >>>> the function name,
> >>>> the arguments
> >>>> the semicolon
> >>>> the comment
> >>>> 
> >>>> But if there is no comment at the end, it always considers the complete
> >>>> line as a sexp (including the ;).
> >>>> 
> >>>> For my use case I would prefer the old behavior because it is consistent
> >>>> with the current sexp definition in all emacs (with maybe the exception
> >>>> of python-mode).  Because it is easier to copy function names or
> >>>> function calls with a few movements.
> >>>> 
> >>>> However, if it is too difficult to reproduce the old behavior; then the
> >>>> new one may be implemented consistently.
> >>> 
> >>> Yuan, any comments or suggestions?
> >>> 
> >>> FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
> >>> source code is not well-defined.
> >> 
> >> Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.
> >> 
> >> Yuan
> 
> I know the reason for the inconsistency now. Treesit-forward-sexp first checks whether point is in a “text” node, ie, comment or string; if so, it uses the default/normal forward-sexp function; if not, it uses the parse tree to go over sexp.
> 
> In the first example, because there’s a comment right before point, treesit-forward-sexp thinks it’s in a text node, and used the normal forward-sexp function, which moved point after the next symbol.
> 
> In the second example, because there’s no comment anymore, treesit-forward-sexp uses the parse tree to move the point; and since the next node after point is the statement line, it moves point over the whole line. When using the parse tree to move point, treesit-forward-sexp always moves in the same “level” in which that point is. Eg, when point is between two lines, treesit-forward-sexp moves over lines; if point is inside an argument list between two arguments, treesit-forward-sexp moves over each argument.
> 
> If you want to just select the identifier or other more fine-grained movement, IMHO it’s probably better to use forward-word.
> 
> I fixed the inconsistency so now treesit-forward-sexp in both example moves over the whole line. The fix is pushed to emacs-30.

Thanks.

No further comments, so I assume the bug is fixed, and I'm therefore
closing it.