hideshow support for treesitter

hideshow support for treesitter

[Pranshu Sharma]

[-- Attachment #1: Type: text/plain, Size: 192 bytes --]

Hello all

Are there any plans to add treesitter support for hideshow, it seems easy
enough to do and also very helpful, especially for langs that don't
excessively use parens, such as python

[-- Attachment #2: Type: text/html, Size: 241 bytes --]

Re: hideshow support for treesitter

[Juri Linkov]

> Are there any plans to add treesitter support for hideshow, it seems easy
> enough to do and also very helpful, especially for langs that don't
> excessively use parens, such as python

Indeed, it's easy to do this.  You just need to override
hs-find-block-beginning-func, hs-find-next-block-func, etc.
with treesit functions like 'treesit-outline-search'.

Re: hideshow support for treesitter

[Pranshu Sharma]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

Is there a preferred way to handle this treesit wise. I know some packages
(eg imenu) require major modes to define their support for it, but others
like ispell do it themselves. I personally thing the latter would be better
in this case.

On Thu, 5 Sept 2024 at 16:36, Juri Linkov <juri@linkov.net> wrote:

> >>> Are there any plans to add treesitter support for hideshow, it seems
> easy
> >>> enough to do and also very helpful, especially for langs that don't
> >>> excessively use parens, such as python
> >>
> >> Indeed, it's easy to do this.  You just need to override
> >> hs-find-block-beginning-func, hs-find-next-block-func, etc.
> >> with treesit functions like 'treesit-outline-search'.
> >
> > Ah I see, then I think treesit.el should contain code for this, because
> > this seems extremely useful task that would not make sense if each tree
> > sitter based major mode was to implement this separately
>
> Or hideshow code for treesit could be in hideshow.el as well.
> The reason why 'treesit-outline-search' was added to treesit.el
> was because treesit.el already contained imenu support.  But
> imenu support could be in imenu.el as well.
>

[-- Attachment #2: Type: text/html, Size: 1573 bytes --]

Re: hideshow support for treesitter

[Juri Linkov]

>>>>> Are there any plans to add treesitter support for hideshow, it seems easy
>>>>> enough to do and also very helpful, especially for langs that don't
>>>>> excessively use parens, such as python
>>>>
>>>> Indeed, it's easy to do this.  You just need to override
>>>> hs-find-block-beginning-func, hs-find-next-block-func, etc.
>>>> with treesit functions like 'treesit-outline-search'.
>>>
>>> Ah I see, then I think treesit.el should contain code for this, because
>>> this seems extremely useful task that would not make sense if each tree
>>> sitter based major mode was to implement this separately
>>
>> Or hideshow code for treesit could be in hideshow.el as well.
>> The reason why 'treesit-outline-search' was added to treesit.el
>> was because treesit.el already contained imenu support.  But
>> imenu support could be in imenu.el as well.
>
> Is there a preferred way to handle this treesit wise. I know some packages
> (eg imenu) require major modes to define their support for it, but others
> like ispell do it themselves. I personally thing the latter would be better
> in this case.

I looked at hideshow.el and found such a list:

(defvar hs-special-modes-alist
  (mapcar #'purecopy
  '((c-mode "{" "}" "/[*/]" nil nil)
    (c-ts-mode "{" "}" "/[*/]" nil nil)
    (c++-mode "{" "}" "/[*/]" nil nil)
    (c++-ts-mode "{" "}" "/[*/]" nil nil)
    (bibtex-mode ("@\\S(*\\(\\s(\\)" 1))
    (java-mode "{" "}" "/[*/]" nil nil)
    (java-ts-mode "{" "}" "/[*/]" nil nil)
    (js-mode "{" "}" "/[*/]" nil)
    (js-ts-mode "{" "}" "/[*/]" nil)
    (lua-ts-mode "{\\|\\[\\[" "}\\|\\]\\]" "--" nil)
    (mhtml-mode "{\\|<[^/>]*?" "}\\|</[^/>]*[^/]>" "<!--" mhtml-forward nil)

But this is useless for tree-sitter.

Line-oriented treesit-outline-search relied on treesit-simple-imenu-settings
that defines header lines for imenu.  And every ts-mode sets own value of
treesit-simple-imenu-settings.  But this can't be used for hideshow.

So whoever will implement hideshow support for treesitter
will need to design a setting like buffer-local
treesit-simple-imenu-settings for ts-modes.

Re: hideshow support for treesitter

[Pranshu Sharma]

[-- Attachment #1: Type: text/plain, Size: 2437 bytes --]

I am working to try get the hide show working for tree sitter, however one
thing that is missing in treesit is sexp navigation functions, hideshow
requires sexp function.

On Fri, 6 Sept 2024 at 03:04, Juri Linkov <juri@linkov.net> wrote:

> >>>>> Are there any plans to add treesitter support for hideshow, it seems
> easy
> >>>>> enough to do and also very helpful, especially for langs that don't
> >>>>> excessively use parens, such as python
> >>>>
> >>>> Indeed, it's easy to do this.  You just need to override
> >>>> hs-find-block-beginning-func, hs-find-next-block-func, etc.
> >>>> with treesit functions like 'treesit-outline-search'.
> >>>
> >>> Ah I see, then I think treesit.el should contain code for this, because
> >>> this seems extremely useful task that would not make sense if each tree
> >>> sitter based major mode was to implement this separately
> >>
> >> Or hideshow code for treesit could be in hideshow.el as well.
> >> The reason why 'treesit-outline-search' was added to treesit.el
> >> was because treesit.el already contained imenu support.  But
> >> imenu support could be in imenu.el as well.
> >
> > Is there a preferred way to handle this treesit wise. I know some
> packages
> > (eg imenu) require major modes to define their support for it, but others
> > like ispell do it themselves. I personally thing the latter would be
> better
> > in this case.
>
> I looked at hideshow.el and found such a list:
>
> (defvar hs-special-modes-alist
>   (mapcar #'purecopy
>   '((c-mode "{" "}" "/[*/]" nil nil)
>     (c-ts-mode "{" "}" "/[*/]" nil nil)
>     (c++-mode "{" "}" "/[*/]" nil nil)
>     (c++-ts-mode "{" "}" "/[*/]" nil nil)
>     (bibtex-mode ("@\\S(*\\(\\s(\\)" 1))
>     (java-mode "{" "}" "/[*/]" nil nil)
>     (java-ts-mode "{" "}" "/[*/]" nil nil)
>     (js-mode "{" "}" "/[*/]" nil)
>     (js-ts-mode "{" "}" "/[*/]" nil)
>     (lua-ts-mode "{\\|\\[\\[" "}\\|\\]\\]" "--" nil)
>     (mhtml-mode "{\\|<[^/>]*?" "}\\|</[^/>]*[^/]>" "<!--" mhtml-forward
> nil)
>
> But this is useless for tree-sitter.
>
> Line-oriented treesit-outline-search relied on
> treesit-simple-imenu-settings
> that defines header lines for imenu.  And every ts-mode sets own value of
> treesit-simple-imenu-settings.  But this can't be used for hideshow.
>
> So whoever will implement hideshow support for treesitter
> will need to design a setting like buffer-local
> treesit-simple-imenu-settings for ts-modes.
>

[-- Attachment #2: Type: text/html, Size: 3431 bytes --]

Re: hideshow support for treesitter

[Yuan Fu]

> 
> On Fri, 6 Sept 2024 at 03:04, Juri Linkov <juri@linkov.net> wrote:
> >>>>> Are there any plans to add treesitter support for hideshow, it seems easy
> >>>>> enough to do and also very helpful, especially for langs that don't
> >>>>> excessively use parens, such as python
> >>>>
> >>>> Indeed, it's easy to do this.  You just need to override
> >>>> hs-find-block-beginning-func, hs-find-next-block-func, etc.
> >>>> with treesit functions like 'treesit-outline-search'.
> >>>
> >>> Ah I see, then I think treesit.el should contain code for this, because
> >>> this seems extremely useful task that would not make sense if each tree
> >>> sitter based major mode was to implement this separately
> >>
> >> Or hideshow code for treesit could be in hideshow.el as well.
> >> The reason why 'treesit-outline-search' was added to treesit.el
> >> was because treesit.el already contained imenu support.  But
> >> imenu support could be in imenu.el as well.
> >
> > Is there a preferred way to handle this treesit wise. I know some packages
> > (eg imenu) require major modes to define their support for it, but others
> > like ispell do it themselves. I personally thing the latter would be better
> > in this case.
> 
> I looked at hideshow.el and found such a list:
> 
> (defvar hs-special-modes-alist
>   (mapcar #'purecopy
>   '((c-mode "{" "}" "/[*/]" nil nil)
>     (c-ts-mode "{" "}" "/[*/]" nil nil)
>     (c++-mode "{" "}" "/[*/]" nil nil)
>     (c++-ts-mode "{" "}" "/[*/]" nil nil)
>     (bibtex-mode ("@\\S(*\\(\\s(\\)" 1))
>     (java-mode "{" "}" "/[*/]" nil nil)
>     (java-ts-mode "{" "}" "/[*/]" nil nil)
>     (js-mode "{" "}" "/[*/]" nil)
>     (js-ts-mode "{" "}" "/[*/]" nil)
>     (lua-ts-mode "{\\|\\[\\[" "}\\|\\]\\]" "--" nil)
>     (mhtml-mode "{\\|<[^/>]*?" "}\\|</[^/>]*[^/]>" "<!--" mhtml-forward nil)
> 
> But this is useless for tree-sitter.
> 
> Line-oriented treesit-outline-search relied on treesit-simple-imenu-settings
> that defines header lines for imenu.  And every ts-mode sets own value of
> treesit-simple-imenu-settings.  But this can't be used for hideshow.
> 
> So whoever will implement hideshow support for treesitter
> will need to design a setting like buffer-local
> treesit-simple-imenu-settings for ts-modes.

> On Sep 5, 2024, at 6:18 PM, Pranshu Sharma <pranshusharma198@gmail.com> wrote:
> 
> I am working to try get the hide show working for tree sitter, however one thing that is missing in treesit is sexp navigation functions, hideshow requires sexp function.

Master branch should have them, you can check out treesit-forward-sexp. Major modes need to define sexp in treesit-thing-settings for it to work though. Also, hideshow can just use forward-sexp, it’ll use treesit-forward-sexp behind the scenes.

With that said, last time I tried to make hideshow work with tree-sitter, I found that hideshow is not generic enough, it insists on its way of using regexp and search back and forth for block beginning and ends. If you want to make it work for tree-sitter, you might need to add another layer of abstraction and make hideshow more generic on how it finds the region to fold.

Also just a note, for tree-sitter integration, I always try to make tree-sitter work for another package/library/feature, rather than  the other way around. So if possible, hideshow.el should not require treesit or even be aware of it, treesit should set the necessary variables to work with hideshow.

Yuan

Re: hideshow support for treesitter

[pranshu sharma]

Yuan Fu <casouri@gmail.com> writes:

>> 
>> On Fri, 6 Sept 2024 at 03:04, Juri Linkov <juri@linkov.net> wrote:
>> >>>>> Are there any plans to add treesitter support for hideshow, it seems easy
>> >>>>> enough to do and also very helpful, especially for langs that don't
>> >>>>> excessively use parens, such as python
>> >>>>
>> >>>> Indeed, it's easy to do this.  You just need to override
>> >>>> hs-find-block-beginning-func, hs-find-next-block-func, etc.
>> >>>> with treesit functions like 'treesit-outline-search'.
>> >>>
>> >>> Ah I see, then I think treesit.el should contain code for this, because
>> >>> this seems extremely useful task that would not make sense if each tree
>> >>> sitter based major mode was to implement this separately
>> >>
>> >> Or hideshow code for treesit could be in hideshow.el as well.
>> >> The reason why 'treesit-outline-search' was added to treesit.el
>> >> was because treesit.el already contained imenu support.  But
>> >> imenu support could be in imenu.el as well.
>> >
>> > Is there a preferred way to handle this treesit wise. I know some packages
>> > (eg imenu) require major modes to define their support for it, but others
>> > like ispell do it themselves. I personally thing the latter would be better
>> > in this case.
>> 
>> I looked at hideshow.el and found such a list:
>> 
>> (defvar hs-special-modes-alist
>>   (mapcar #'purecopy
>>   '((c-mode "{" "}" "/[*/]" nil nil)
>>     (c-ts-mode "{" "}" "/[*/]" nil nil)
>>     (c++-mode "{" "}" "/[*/]" nil nil)
>>     (c++-ts-mode "{" "}" "/[*/]" nil nil)
>>     (bibtex-mode ("@\\S(*\\(\\s(\\)" 1))
>>     (java-mode "{" "}" "/[*/]" nil nil)
>>     (java-ts-mode "{" "}" "/[*/]" nil nil)
>>     (js-mode "{" "}" "/[*/]" nil)
>>     (js-ts-mode "{" "}" "/[*/]" nil)
>>     (lua-ts-mode "{\\|\\[\\[" "}\\|\\]\\]" "--" nil)
>>     (mhtml-mode "{\\|<[^/>]*?" "}\\|</[^/>]*[^/]>" "<!--" mhtml-forward nil)
>> 
>> But this is useless for tree-sitter.
>> 
>> Line-oriented treesit-outline-search relied on treesit-simple-imenu-settings
>> that defines header lines for imenu.  And every ts-mode sets own value of
>> treesit-simple-imenu-settings.  But this can't be used for hideshow.
>> 
>> So whoever will implement hideshow support for treesitter
>> will need to design a setting like buffer-local
>> treesit-simple-imenu-settings for ts-modes.
>
>> On Sep 5, 2024, at 6:18 PM, Pranshu Sharma <pranshusharma198@gmail.com> wrote:
>> 
>> I am working to try get the hide show working for tree sitter, however one thing that is missing in treesit is sexp navigation functions, hideshow requires sexp function.
>
> Master branch should have them, you can check out
> treesit-forward-sexp. Major modes need to define sexp in
> treesit-thing-settings for it to work though. Also, hideshow can just
> use forward-sexp, it’ll use treesit-forward-sexp behind the scenes.

I played around with sexp navigation in haskell-ts-mode, and found that
is is currently useless for ambiguous scenarios like:
---
f x = 2_
---
Where '_' represents the point, now. In this place if we do backward
sexp, there are 2 valid places it would go, we can see this by if we
put parens around the function:
---
(f x = (2)_)_
---
Now either the point could go to front of 2. or in front of f, but it
will choose to go infront off 'f', which is kind of unpredictable.
This is not a problem with treesit, rather treesitter itself.

Btw, are there been any plans on a treesit version  of backwards-up-list?

>
> With that said, last time I tried to make hideshow work with
> tree-sitter, I found that hideshow is not generic enough, it insists
> on its way of using regexp and search back and forth for block
> beginning and ends. If you want to make it work for tree-sitter, you
> might need to add another layer of abstraction and make hideshow more
> generic on how it finds the region to fold.

I so far found python mode which managed to impliment code folding
without parens, but haven't looked into it too much yet.

>
> Also just a note, for tree-sitter integration, I always try to make
> tree-sitter work for another package/library/feature, rather than the
> other way around. So if possible, hideshow.el should not require
> treesit or even be aware of it, treesit should set the necessary
> variables to work with hideshow.
>
> Yuan

Pranshu

Re: hideshow support for treesitter

[Yuan Fu]

> On Sep 15, 2024, at 7:28 AM, pranshu sharma <pranshusharma366@gmail.com> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>>> 
>>> On Fri, 6 Sept 2024 at 03:04, Juri Linkov <juri@linkov.net> wrote:
>>>>>>>> Are there any plans to add treesitter support for hideshow, it seems easy
>>>>>>>> enough to do and also very helpful, especially for langs that don't
>>>>>>>> excessively use parens, such as python
>>>>>>> 
>>>>>>> Indeed, it's easy to do this.  You just need to override
>>>>>>> hs-find-block-beginning-func, hs-find-next-block-func, etc.
>>>>>>> with treesit functions like 'treesit-outline-search'.
>>>>>> 
>>>>>> Ah I see, then I think treesit.el should contain code for this, because
>>>>>> this seems extremely useful task that would not make sense if each tree
>>>>>> sitter based major mode was to implement this separately
>>>>> 
>>>>> Or hideshow code for treesit could be in hideshow.el as well.
>>>>> The reason why 'treesit-outline-search' was added to treesit.el
>>>>> was because treesit.el already contained imenu support.  But
>>>>> imenu support could be in imenu.el as well.
>>>> 
>>>> Is there a preferred way to handle this treesit wise. I know some packages
>>>> (eg imenu) require major modes to define their support for it, but others
>>>> like ispell do it themselves. I personally thing the latter would be better
>>>> in this case.
>>> 
>>> I looked at hideshow.el and found such a list:
>>> 
>>> (defvar hs-special-modes-alist
>>>  (mapcar #'purecopy
>>>  '((c-mode "{" "}" "/[*/]" nil nil)
>>>    (c-ts-mode "{" "}" "/[*/]" nil nil)
>>>    (c++-mode "{" "}" "/[*/]" nil nil)
>>>    (c++-ts-mode "{" "}" "/[*/]" nil nil)
>>>    (bibtex-mode ("@\\S(*\\(\\s(\\)" 1))
>>>    (java-mode "{" "}" "/[*/]" nil nil)
>>>    (java-ts-mode "{" "}" "/[*/]" nil nil)
>>>    (js-mode "{" "}" "/[*/]" nil)
>>>    (js-ts-mode "{" "}" "/[*/]" nil)
>>>    (lua-ts-mode "{\\|\\[\\[" "}\\|\\]\\]" "--" nil)
>>>    (mhtml-mode "{\\|<[^/>]*?" "}\\|</[^/>]*[^/]>" "<!--" mhtml-forward nil)
>>> 
>>> But this is useless for tree-sitter.
>>> 
>>> Line-oriented treesit-outline-search relied on treesit-simple-imenu-settings
>>> that defines header lines for imenu.  And every ts-mode sets own value of
>>> treesit-simple-imenu-settings.  But this can't be used for hideshow.
>>> 
>>> So whoever will implement hideshow support for treesitter
>>> will need to design a setting like buffer-local
>>> treesit-simple-imenu-settings for ts-modes.
>> 
>>> On Sep 5, 2024, at 6:18 PM, Pranshu Sharma <pranshusharma198@gmail.com> wrote:
>>> 
>>> I am working to try get the hide show working for tree sitter, however one thing that is missing in treesit is sexp navigation functions, hideshow requires sexp function.
>> 
>> Master branch should have them, you can check out
>> treesit-forward-sexp. Major modes need to define sexp in
>> treesit-thing-settings for it to work though. Also, hideshow can just
>> use forward-sexp, it’ll use treesit-forward-sexp behind the scenes.
> 
> I played around with sexp navigation in haskell-ts-mode, and found that
> is is currently useless for ambiguous scenarios like:
> ---
> f x = 2_
> ---
> Where '_' represents the point, now. In this place if we do backward
> sexp, there are 2 valid places it would go, we can see this by if we
> put parens around the function:
> ---
> (f x = (2)_)_
> ---
> Now either the point could go to front of 2. or in front of f, but it
> will choose to go infront off 'f', which is kind of unpredictable.
> This is not a problem with treesit, rather treesitter itself.

Yeah, there has been discussion around sexp navigation with tree-sitter. I’ll go a step further and say the problem is us trying to retrofit the concept of sexp on non-lisp languages, where not everything is enclosed with parens—the definition of sexp outside of lisp is very fuzzy. As you shown here, where to move to is entirely subjective. Also, most languages doesn’t have delimiters like paren around syntactic constructs, making which sexp to move over ambiguous. 

My personal opinion is to make sexp navigation in non-lisp languages work like list navigation: statements, arguments in argument list, blocks, fields.

The conclusion from previous discussions, OTOH, is to define sexp as “the largest syntactic construct before/after point”. 

> Btw, are there been any plans on a treesit version  of backwards-up-list?

There are plans for generalizing all structural navigation commands and plug tree-inter into them. Right now most of the navigation functions (like backwards-up-list) are not generic. So the first step is to update them and allow other package/major mode to define custom behavior for them. 

There’s already some progress made here, like the addition of forward-sexp-function and transpose-sexp-function. Other navigation commands can follow the same pattern.

Yuan