Re: Plug treesit.el into other emacs constructs

[Stefan Monnier <monnier@iro.umontreal.ca>]

>>>>In this case, yes.  But in other cases it will move at different levels
>>>>of the tree.  E.g.:
>>>>
>>>>   int x = f (b + 4, c * 7 - z * 2, d, e);
>>>>
>>>>It will sometimes move over the whole instruction, and other times over
>>>>just a single variable or over a whole argument or over just a "factor".
>>>>This depends on where point is when `forward/backward-sexp` is called.
>>>
>>> Yeah. I think this example shows what I find unintuitive. If point is right
>>> before the first comma, and we transpose-sexps, it could end up swapping
>>> 4 for c * 7 - z * 2, which would rarely make sense in this context.
>>
>> If so, that would be a bug in `transpose-sexp`, agreed.
>> I'm talking here about `forward/backward-sexp`.
>> The two are linked, but we shouldn't use one to justify a bug in the other.
>
> Sure, but I think they necessarily needs to be viewed as a whole.  If we
> drop tree-sitter or SMIE (which I actually know pretty well) for one
> moment, the cc-mode based java-mode would exhibit the exact behavior I
> described.

Really?  When I try it out in CC-mode's java-mode, I get from

    int x = f (b + 4|, c * 7 - z * 2, d, e);
to
    int x = f (b + c, 4| * 7 - z * 2, d, e);

which is not completely non-sensical, but is somewhere between a bug and
a misfeature.

> If it's a bug in tranpsose-sexps it is definitely an issue
> with forward/backward-sexp, because in every situation the positions to
> be swapped is just "backward-sexp - forward-sexp - forward-sexp -
> backward-sexp", right?

The way I see it, the problem with sexp movement and infix syntax is
that a given buffer position maps to several positions at different
levels in the AST (contrary to Lisp style syntax where there is no such
ambiguity).

So for every command, we need to decide/guess at which level of the AST
the user wants to operate.  For `transpose-sexp` we have more
information than for `forward/backward-sexp` because some of those
positions are "non-sensical" in the sense that they would end up swapping
subtrees that live at different levels or that do not share their
immediate parent.

For this reason, what we should do with `transpose-sexp` is not necessarily
exactly the same as what we should do with `backward/forward-sexp`.

> And the thing in the middle, usually a comma,
> operators or other is the space between that doesn't move.  I also
> observe this fixme inside of transpose-words:
>
>   ;; FIXME: `foo a!nd bar' should transpose into `bar and foo'.
>
> I read this more like it's how transpose-sexps should behave on text.

IIRC I wrote this when I was working on the SMIE `transpose-sexp` code :-)

> Wouldn't it make sense to make transpose-sexps actually do what that
> fixme asks?

I obviously agree, since I wrote that fixme.

> And why is the
>
> 		 (cons (progn (funcall mover x) (point))
> 		       (progn (funcall mover (- x)) (point)))
>
> in this form, and not some pseudo-code like:
> (cons '(backward-thing-from-start-point forward-thing-point)
>       '(forward-thing-from-start-point backward-thing-point))

Sorry, haven't looked at the code in a while.
Not sure what you're getting at here.  I suspect that in the case of
tree-sitter you'd ideally want to implement `transpose-sexp` directly
rather than via something like `forward/backward-sexp`:
- Go from point to a node in the tree.
- Find the node whose children we want to swap.
- Find the bounds of those two children.
- Do the actual textual swap.

> Now I'm having issues where movement over sexps ends up not in the
> same place.

Same place as?


        Stefan