bug#62086: 29.0.60; ruby-ts-mode regressions

[Yuan Fu <casouri@gmail.com>]

> On Apr 12, 2023, at 2:56 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> On 13/04/2023 00:50, Yuan Fu wrote:
>>> On Apr 12, 2023, at 1:13 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
>>> 
>>> On 12/04/2023 18:31, Dmitry Gutov wrote:
>>>> On 12/04/2023 10:05, Yuan Fu wrote:
>>>>> Actually, would it make sense to define sexp as “anything but some very small punctuation and delimiters”?
>>>> Pretty much. If I understood you correctly.
>>>> E.g. in ruby-ts-mode identifiers and numbers are also sexps.
>>> 
>>> Allow me to update that.
>>> 
>>> From the previous threads, for ruby-ts-mode at least, we seem to have concluded that it's best to treat those nodes as sexps which have visible boundaries that are visible and don't overlay exactly the boundaries of the contained nodes.
>>> 
>>> For example, we now exclude statement nodes and binary expression nodes because both make forward/backward-sexp less obvious and predictable: you move point to the beginning of 'a + b', press C-M-f, and if the jump happens over the whole expression, this is just as likely to mismatch the user's intention (which might have wanted to only jump over 'a'). So these are the node we rule out.
>> User might as well want to move over the whole expression, since they can use forward-word if they want to move over smaller elements. But I guess that’s just personal preferences.
> 
> forward-word works for minor elements, but the sub-expression can be, for example, a parenthesized expression (with "real" parens).
> 
> It's definitely something that can be discussed, but the above guideline seems to me like something that puts the user more in control. Because as handy jumping over statements can be, it's usually not what one is trying to do.
> 
>>> The easiest choice would be to go back to treating only braces/brackets/parens are sexp delimiters, but in Ruby, at least, we have lots of constructs that are delimited with keywords (such as 'if', 'def', 'end'), so that doesn't work. Maybe it'll work better in C/C++, where you mostly need to be able to differentiate between different types of angle brackets.
>> To clarify, my point is to define sexp by exclusion rather than inclusion, ie, defining a set of nodes that are not sexp, rather than defining a set of nodes that are sexp. I mentioned delimiters because they are excluded from sexp, not because they delimit sexp.
> 
> Yes, that can work. Only when the excluded type names a one-char long, though, because Elisp has no lookahead. In ruby-ts-mode there are longer excluded types.

Actually, I’m working on extending the “pattern” treesit-search-forward and friends can accept. Right now it has to be a regexp or a pred function. I plan to extend it to regexp | function | (regexp . function) | (or <pattern>…) | (not <pattern>…) | (verbatim string)

I’m not yet sure about the performance implication of the recursive patterns (or and not). And I’m not sure if verbatim is necessary, but I guess having it wouldn’t hurt.

Yuan