Re: Plug treesit.el into other emacs constructs

[Theodor Thornhill <theo@thornhill.no>]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>>>In this case, yes.  But in other cases it will move at different levels
>>>>>of the tree.  E.g.:
>>>>>
>>>>>   int x = f (b + 4, c * 7 - z * 2, d, e);
>>>>>
>>>>>It will sometimes move over the whole instruction, and other times over
>>>>>just a single variable or over a whole argument or over just a "factor".
>>>>>This depends on where point is when `forward/backward-sexp` is called.
>>>>
>>>> Yeah. I think this example shows what I find unintuitive. If point is right
>>>> before the first comma, and we transpose-sexps, it could end up swapping
>>>> 4 for c * 7 - z * 2, which would rarely make sense in this context.
>>>
>>> If so, that would be a bug in `transpose-sexp`, agreed.
>>> I'm talking here about `forward/backward-sexp`.
>>> The two are linked, but we shouldn't use one to justify a bug in the other.
>>
>> Sure, but I think they necessarily needs to be viewed as a whole.  If we
>> drop tree-sitter or SMIE (which I actually know pretty well) for one
>> moment, the cc-mode based java-mode would exhibit the exact behavior I
>> described.
>
> Really?  When I try it out in CC-mode's java-mode, I get from
>
>     int x = f (b + 4|, c * 7 - z * 2, d, e);
> to
>     int x = f (b + c, 4| * 7 - z * 2, d, e);
>
> which is not completely non-sensical, but is somewhere between a bug and
> a misfeature.

Yeah, I misspoke, sorry.  I get the same, and obviously agree.  I
believe I wanted to say something like "I get the same behavior with
transpose-sexps as with transpose-word", or something like that.

>
>> If it's a bug in tranpsose-sexps it is definitely an issue
>> with forward/backward-sexp, because in every situation the positions to
>> be swapped is just "backward-sexp - forward-sexp - forward-sexp -
>> backward-sexp", right?
>
> The way I see it, the problem with sexp movement and infix syntax is
> that a given buffer position maps to several positions at different
> levels in the AST (contrary to Lisp style syntax where there is no such
> ambiguity).
>
> So for every command, we need to decide/guess at which level of the AST
> the user wants to operate.  For `transpose-sexp` we have more
> information than for `forward/backward-sexp` because some of those
> positions are "non-sensical" in the sense that they would end up swapping
> subtrees that live at different levels or that do not share their
> immediate parent.
>
> For this reason, what we should do with `transpose-sexp` is not necessarily
> exactly the same as what we should do with `backward/forward-sexp`.
>

Yeah, I agree.  I could create a treesit-transpose-sexps that doesn't
use forward-sexp and uses the 'special (which probably should be
documented) argument, similarly to how it's implemented now.

>> And the thing in the middle, usually a comma,
>> operators or other is the space between that doesn't move.  I also
>> observe this fixme inside of transpose-words:
>>
>>   ;; FIXME: `foo a!nd bar' should transpose into `bar and foo'.
>>
>> I read this more like it's how transpose-sexps should behave on text.
>
> IIRC I wrote this when I was working on the SMIE `transpose-sexp` code :-)
>
>> Wouldn't it make sense to make transpose-sexps actually do what that
>> fixme asks?
>
> I obviously agree, since I wrote that fixme.
>

Great.  It seems there has been almost no development, nor documentation
done in this area for a long time.  Should I try to improve on this part
of the code while I'm at it?

>> And why is the
>>
>> 		 (cons (progn (funcall mover x) (point))
>> 		       (progn (funcall mover (- x)) (point)))
>>
>> in this form, and not some pseudo-code like:
>> (cons '(backward-thing-from-start-point forward-thing-point)
>>       '(forward-thing-from-start-point backward-thing-point))
>
> Sorry, haven't looked at the code in a while.
> Not sure what you're getting at here.  I suspect that in the case of
> tree-sitter you'd ideally want to implement `transpose-sexp` directly
> rather than via something like `forward/backward-sexp`:
> - Go from point to a node in the tree.
> - Find the node whose children we want to swap.
> - Find the bounds of those two children.
> - Do the actual textual swap.
>

Yeah, that's what I'm thinking too.  I'm just thinking we should be
clear on what a word/sexp/sentence/paragraph/defun etc is in non-lisp
and non-human languages.


>> Now I'm having issues where movement over sexps ends up not in the
>> same place.
>
> Same place as?
>

IIRC there's no guarantee that the movement sequence used for
transpose-sexp moves over the same blocks of code, so in non-lisp
languages there's no real semantic to go from.  I'll find an example.

Theo