Question on getting tree-sitter matches

Question on getting tree-sitter matches

[Abin Simon]

Hey,

How would I get the set of matches from tree-sitter with items from the
same query together.

For example if I have a query of the following form:

    (function_declaration (identifier) @id) @func

elisp-tree-sitter returns this as the output of `tsc-query-matches`:

    [(0 . [
           (func . #<user-ptr ptr=0x4f7fa40 finalizer=0x7f2173b15780>)
           (id . #<user-ptr ptr=0x4f7fa00 finalizer=0x7f2173b15780>)
          ]
     )
     (0 . [
           (func . #<user-ptr ptr=0x10043a0 finalizer=0x7f2173b15780>)
           (id . #<user-ptr ptr=0x7b9a660 finalizer=0x7f2173b15780>)
          ]
     )
    ]


This groups the matches from the same "query" together.

I could only find `treesit-query-capture` which just returns a list of
captures items in the following form:

    (
     (func . #<treesit-node function_declaration in 441-791>)
     (id . #<treesit-node identifier in 446-460>)
     (func . #<treesit-node function_declaration in 865-1052>)
     (id . #<treesit-node identifier in 870-888>)
    )

Re: Question on getting tree-sitter matches

[Yuan Fu]

> On Mar 12, 2023, at 7:32 AM, Abin Simon <mail@meain.io> wrote:
> 
> Hey,
> 
> How would I get the set of matches from tree-sitter with items from the
> same query together.
> 
> For example if I have a query of the following form:
> 
>    (function_declaration (identifier) @id) @func
> 
> elisp-tree-sitter returns this as the output of `tsc-query-matches`:
> 
>    [(0 . [
>           (func . #<user-ptr ptr=0x4f7fa40 finalizer=0x7f2173b15780>)
>           (id . #<user-ptr ptr=0x4f7fa00 finalizer=0x7f2173b15780>)
>          ]
>     )
>     (0 . [
>           (func . #<user-ptr ptr=0x10043a0 finalizer=0x7f2173b15780>)
>           (id . #<user-ptr ptr=0x7b9a660 finalizer=0x7f2173b15780>)
>          ]
>     )
>    ]
> 
> 
> This groups the matches from the same "query" together.
> 
> I could only find `treesit-query-capture` which just returns a list of
> captures items in the following form:
> 
>    (
>     (func . #<treesit-node function_declaration in 441-791>)
>     (id . #<treesit-node identifier in 446-460>)
>     (func . #<treesit-node function_declaration in 865-1052>)
>     (id . #<treesit-node identifier in 870-888>)
>    )
> 

Unfortunately, that’s currently not possible. Could you elaborate on why you need such feature?

Thanks,
Yuan

Re: Question on getting tree-sitter matches

[Abin Simon]

> Unfortunately, that’s currently not possible. Could you elaborate on why you need such feature?

This came up when working with https://github.com/meain/evil-textobj-tree-sitter.

In this repo, I pull queries from Neovim repo which dos similar things,
https://github.com/nvim-treesitter/nvim-treesitter-textobjects. They
have custom predicates added and one of them is `#make-range!` which is
used to combine two captures together to form a new capture.

You can find an example at
https://github.com/nvim-treesitter/nvim-treesitter-textobjects/blob/5b2bcb9ca8315879181f468b37a897100d631005/queries/perl/textobjects.scm#L5

((patter_matcher_m
   (start_delimiter) @_start
   (end_delimiter) @_end) @regex.outer
 (#make-range! "regex.inner" @_start @_end))

In here, they use `#make-range!` to form a `regex.inner` predicate by
combining `@_start` and `@_end` markers' ranges. Since we do not have
support for this predicate, I was working around this by converting the
query into the following:

https://github.com/meain/evil-textobj-tree-sitter/blob/66819ee8547e439f003fcb0e1647acade194bd1a/queries/perl/textobjects.scm#LL5-L8C3

((patter_matcher_m
   (start_delimiter)  @regex.inner._start
   (end_delimiter)  @regex.inner._end) @regex.outer
 )

Once we have this, we use `regex.inner._start` and `regex.inner._end` to
do something similar (but we need to be able to restrict the search to a
single match group).

You can find the relevant part of the code from the plugin here:

https://github.com/meain/evil-textobj-tree-sitter/blob/66819ee8547e439f003fcb0e1647acade194bd1a/evil-textobj-tree-sitter-core.el#L157

Re: Question on getting tree-sitter matches

[Yuan Fu]

> On Mar 12, 2023, at 8:29 PM, Abin Simon <mail@meain.io> wrote:
> 
>> Unfortunately, that’s currently not possible. Could you elaborate on why you need such feature?
> 
> This came up when working with https://github.com/meain/evil-textobj-tree-sitter.
> 
> In this repo, I pull queries from Neovim repo which dos similar things,
> https://github.com/nvim-treesitter/nvim-treesitter-textobjects. They
> have custom predicates added and one of them is `#make-range!` which is
> used to combine two captures together to form a new capture.
> 
> You can find an example at
> https://github.com/nvim-treesitter/nvim-treesitter-textobjects/blob/5b2bcb9ca8315879181f468b37a897100d631005/queries/perl/textobjects.scm#L5
> 
> ((patter_matcher_m
>   (start_delimiter) @_start
>   (end_delimiter) @_end) @regex.outer
> (#make-range! "regex.inner" @_start @_end))
> 
> In here, they use `#make-range!` to form a `regex.inner` predicate by
> combining `@_start` and `@_end` markers' ranges. Since we do not have
> support for this predicate, I was working around this by converting the
> query into the following:
> 
> https://github.com/meain/evil-textobj-tree-sitter/blob/66819ee8547e439f003fcb0e1647acade194bd1a/queries/perl/textobjects.scm#LL5-L8C3
> 
> ((patter_matcher_m
>   (start_delimiter)  @regex.inner._start
>   (end_delimiter)  @regex.inner._end) @regex.outer
> )
> 
> Once we have this, we use `regex.inner._start` and `regex.inner._end` to
> do something similar (but we need to be able to restrict the search to a
> single match group).
> 
> You can find the relevant part of the code from the plugin here:
> 
> https://github.com/meain/evil-textobj-tree-sitter/blob/66819ee8547e439f003fcb0e1647acade194bd1a/evil-textobj-tree-sitter-core.el#L157

I see. We can add an optional argument to treesit-query-capture to toggle the “grouping” of matches, WDYT?

Yuan

Re: Question on getting tree-sitter matches

[Abin Simon]

> I see. We can add an optional argument to treesit-query-capture to toggle the “grouping” of matches, WDYT?

That would work for my current usecase.

With that said, I would like to point out that elisp-tree-sitter has two
methods with one named captures returning a list of captures and one
named matches which returns the "grouped" captures(along with pattern
index). I'm not sure if these names have any semantic meaning.  From a
quick scan of Neovim docs, they too seem to have two functions with
similar names.

Re: Question on getting tree-sitter matches

[Yuan Fu]

> On Mar 14, 2023, at 9:24 PM, Abin Simon <mail@meain.io> wrote:
> 
>> I see. We can add an optional argument to treesit-query-capture to toggle the “grouping” of matches, WDYT?
> 
> That would work for my current usecase.
> 
> With that said, I would like to point out that elisp-tree-sitter has two
> methods with one named captures returning a list of captures and one
> named matches which returns the "grouped" captures(along with pattern
> index). I'm not sure if these names have any semantic meaning.  From a
> quick scan of Neovim docs, they too seem to have two functions with
> similar names.

For your particular use, would it be better to add a separate function or a flag to treesit-query-capture?

Yuan

Re: Question on getting tree-sitter matches

[Abin Simon]

> For your particular use, would it be better to add a separate function or a flag to treesit-query-capture?

In my particular case, I don't think one is better than the other. Both
works equally well.

Re: Question on getting tree-sitter matches

[Ergus]

Hi Yuan:

Just a question. I have seen many new languages added to the tree-sitter
repo.

Is the main goal to add support for all those languages in vanilla, or
they were added to support the external packages I see in melpa/elpa??

Thanks in advance,
Ergus

Re: Question on getting tree-sitter matches

[Yuan Fu]

> On Mar 20, 2023, at 1:27 PM, Ergus <spacibba@aol.com> wrote:
> 
> Hi Yuan:
> 
> Just a question. I have seen many new languages added to the tree-sitter
> repo.
> 
> Is the main goal to add support for all those languages in vanilla, or
> they were added to support the external packages I see in melpa/elpa??
> 
> Thanks in advance,
> Ergus
> 

I assume you mean tree-sitter-module, that’s an unofficial script mainly for convenience. And those languages are added only for convince, so probably for melpa. Author of elixir-ts-mode is in the process of adding it to Emacs, though.

Yuan

Re: Question on getting tree-sitter matches

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Mon, 20 Mar 2023 16:25:53 -0700
> Cc: emacs-devel@gnu.org
> 
> Author of elixir-ts-mode is in the process of adding it to Emacs, though.

Correction: it's already there, on the master branch.