bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes

[Theodor Thornhill via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Juri Linkov <juri@linkov.net> writes:

>>> The command is 'forward-list'.  With 'list' it will do in ts-modes
>>> the same that 'forward-list' already does in non-ts modes.
>>
>> One issue I've had when exploring these things earlier is that I don't
>> know the "spec" for how navigation in programming languages should work
>> in Emacs. I mean, there are lots of examples in the source code, but no
>> true specification. Should we possibly try to define some sane defaults
>> here, so that it will be simpler to deal with idiosyncrasies of the tree
>> sitter parsers? Lisp isn't in my experience the best language to define
>> this, as it is just too simple.
>
> Very simple: I'm striving to provide compatibility between ts-modes
> with their predecessor non-ts modes.  So the de-facto "spec" is the
> existing non-ts modes that provide canonical reference implementation.
> This helps the users to more smoothly switch to ts-modes.
>

Yes, I agree on this goal, absolutely. That was also my initial intent
and attempt when we started doing this. There are still many places
where either the canonical modes are inconsistent but has so much
history that we're so used to it and forget. If we blindly try to copy
everything one to one we might also adopt some stuff that doesn't really
make sense.

However, deciding on what makes sense is hard, which is why I'm
suggesting we try to define a "spec". It needn't really be much more
than reference suggestions for mode implementors at first, then maybe
some intersection of features crystallize across all modes.

>> Also, many of the functions are related to word processing, like
>> sentence, word, paragraph etc, which aren't really useful in
>> programming, imo. What is a sentence in java, lisp, or python, for
>> example? Also, maybe sexp is too lisp/ast-like of a term, so much so
>> that it is hard to reason about what a sexp is or isn't.
>
> Indeed, sentences and paragraphs are even more fuzzy concepts when applied
> to programming languages.  But some modes make sense of these things
> nevertheless.  For example, in both c-mode and c-ts-mode 'M-e'
> moves to the end of statement that usually end with a semicolon;
> so your intuition about these things looks correct.

Yes, but imo more importantly, disregarding any confusing naming we have
a very rich set of operations available to us that I think should be as
distinct as possible:

- C-M-f
- M-f
- M-a
- M-e
- C-f
- C-a
- C-e
- more

If we'll aim to get the absolute most out of the parse tree
possibilities we should strive to make these distinct, at least down the
line. Some of these are simpler to categorize than others, such as a
sentence meaning "some expression that ends that isn't a function or a
class definition".