bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes

[Theodor Thornhill via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Eli Zaretskii <eliz@gnu.org> writes:

>> X-Spam-Status: No, score=-1 tagged_above=-10 required=5 tests=[ALL_TRUSTED=-1]
>>  autolearn=ham autolearn_force=no
>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: casouri@gmail.com, mickey@masteringemacs.org, 73404@debbugs.gnu.org,
>>  monnier@iro.umontreal.ca
>> Date: Sun, 05 Jan 2025 12:46:03 +0100
>> 
>> Juri Linkov <juri@linkov.net> writes:
>> 
>> > The command is 'forward-list'.  With 'list' it will do in ts-modes
>> > the same that 'forward-list' already does in non-ts modes.
>> 
>> One issue I've had when exploring these things earlier is that I don't
>> know the "spec" for how navigation in programming languages should work
>> in Emacs. I mean, there are lots of examples in the source code, but no
>> true specification. Should we possibly try to define some sane defaults
>> here, so that it will be simpler to deal with idiosyncrasies of the tree
>> sitter parsers? Lisp isn't in my experience the best language to define
>> this, as it is just too simple. Also, many of the functions are related
>> to word processing, like sentence, word, paragraph etc, which aren't
>> really useful in programming, imo. What is a sentence in java, lisp, or
>> python, for example? Also, maybe sexp is too lisp/ast-like of a term, so
>> much so that it is hard to reason about what a sexp is or isn't.
>
> Is this goal achievable in practice,l given the sometimes radical
> differences between languages?  Maybe we only can have examples and
> some dwim-ish behavior in most cases?

I'm not sure if it is achievable or not, but I think it'd be nice to
avoid any "hand-waviness" to confuse us while we talk about
sexps/lists/etc. I could try to document thoroughly how lisp-modes and
cc modes implement these and see if we have some glaring discrepancies
between just those two?

I'm just thinking that describing the as-is could be very beneficial, so
that we know we're talking the same abstractions. It could start just as
a scratch/movements.org file, not touching any code until you, stefan
and others agree that we're talking about the same stuff.

Then we can at least consult this document when we stumble upon things
like jsx-text, whitespace etc. After this we can proceed with mickey's
combobulate and friends to inform us on what the best way to deal with
it is? Maybe we can discover som simplifications in core too.

WDYT?

Theo