bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes

[Yuan Fu <casouri@gmail.com>]

> On Sep 20, 2024, at 10:06 PM, Mickey Petersen <mickey@masteringemacs.org> wrote:
> 
> 
> 
> Examples with javascript-mode. It holds for all modes i tested with a
> TS equivalent. Let -!- be the starting point and ^N be the subsequent
> position after a movement command.
> 
> -!-export const add = (a, b) => a + b;
> 
> Repeated `C-M-f' yields
> 
> export const add = (a, b) => a + b;
> 
>      ^1    ^2  ^3       ^4   ^5  ^6
> 
> 
> In other words, it works as it always has.
> 
> Meanwhile, in `js-ts-mode':
> 
> export const add = (a, b) => a + b;
>                ^1       ^2   ^3  ^4
> 
> From ^1 and back with `C-M-b'
> 
> export const add-!- = (a, b) => a + b;
> 
> export const add = (a, b) => a + b;
>             ^1
> 
> At this point, `C-M-b' no longer goes back. It is stuck.
> 
> 
> Another example:
> 
> -!-console.log("Addition result:", result1);
> 
> With `C-M-f':
> 
> console.log("Addition result:", result1);
> 
>       ^1                               ^2
> 
> 
> This affects every single -sexp function that uses either
> `forward-sexp-function' or `transpose-sexp-function' to do its job.
> 
> Thanks.
> 

I’m aware of this problem and it’s quite inconvenient at times, but right now I don’t have a good solution for it. Ideas are welcome.

Basically tree-sitter’s sexp movement works on subtrees. It determines the position of the point in the whole parse tree and goes forward/back across the next subtree in the parse tree. If there’s no more sibling subtrees in the same level to move over, sexp movement stops like in lisp. The parse tree is invisible and often groups token in unexpected ways, so many times the sexp movement isn’t intuitive.

We might need to add a user option so people can easily turn off tree-sitter sexp movement, since it isn’t a strict upgrade from the generic sexp movement—it’s more of a different flavored sexp movement.

Yuan