outline-mode treesitter support?

outline-mode treesitter support?

[Huan Nguyen]

Are there any plans on making outline-mode take tree sitter nodes instead of regex?
Is it possible to add that functionality or would a new mode be necessary.

Re: outline-mode treesitter support?

[Eli Zaretskii]

> From: Huan Nguyen <nguyenthieuhuan@gmail.com>
> Date: Tue, 19 Dec 2023 12:43:40 +0100
> 
> Are there any plans on making outline-mode take tree sitter nodes instead of regex?

Is there a tree-sitter grammar available somewhere for Outline mode?

> Is it possible to add that functionality or would a new mode be necessary.

It depends on how well the existing mode's features map into a
tree-sitter based one.

Re: outline-mode treesitter support?

[Huan Nguyen]

> On Dec 19, 2023, at 14:00, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Huan Nguyen <nguyenthieuhuan@gmail.com>
>> Date: Tue, 19 Dec 2023 12:43:40 +0100
>> 
>> Are there any plans on making outline-mode take tree sitter nodes instead of regex?
> 
> Is there a tree-sitter grammar available somewhere for Outline mode?
I am not aware of any. I am writing a treesitter based mode for a markup language.
It has been manageable to write increase/decrease heading levels and move headings up/down yourself. For folding I am was looking how I could do it but I only found outline-mode.
> 
>> Is it possible to add that functionality or would a new mode be necessary.
> 
> It depends on how well the existing mode's features map into a
> tree-sitter based one.
The features should be the same but they should work on treesitter nodes instead of regexp. The advantage is that it will be more “correct”.

Take the example nodes: 
(heading1 (content ...)) 
You may wish to hide the child content.

(heading1 
	(content
		(heading2 (content …))))
You may wish to promote heading2, removing it from its parent heading.

The problem I see in that is that not all treesitter grammars have this tree like structure. 
Doing it yourself is not that hard as I said before, so it is not a high priority for me.

Re: outline-mode treesitter support?

[Eli Zaretskii]

> From: Huan Nguyen <nguyenthieuhuan@gmail.com>
> Date: Tue, 19 Dec 2023 14:25:01 +0100
> Cc: emacs-devel@gnu.org
> 
> > On Dec 19, 2023, at 14:00, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> From: Huan Nguyen <nguyenthieuhuan@gmail.com>
> >> Date: Tue, 19 Dec 2023 12:43:40 +0100
> >> 
> >> Are there any plans on making outline-mode take tree sitter nodes instead of regex?
> > 
> > Is there a tree-sitter grammar available somewhere for Outline mode?
> I am not aware of any. I am writing a treesitter based mode for a markup language.

Someone will have to write a grammar before an Emacs mode will be
practical.  For other TS modes, we didn't write the grammar: we used
existing ones.

Re: outline-mode treesitter support?

[Juri Linkov]

> Are there any plans on making outline-mode take tree sitter nodes instead of regex?

The last time this was discussed a year ago, and still no progress:
https://lists.gnu.org/archive/html/emacs-devel/2022-12/msg01010.html

> Is it possible to add that functionality or would a new mode be necessary.

Yes, now this is possible to do after 'outline-search-function'
was added in Emacs 29.  So you can use it to find the next outline heading
that corresponds to the next tree sitter node.

Re: outline-mode treesitter support?

[Huan Nguyen]

> On Dec 19, 2023, at 18:14, Juri Linkov <juri@linkov.net> wrote:
> 
>> Are there any plans on making outline-mode take tree sitter nodes instead of regex?
> 
> The last time this was discussed a year ago, and still no progress:
> https://lists.gnu.org/archive/html/emacs-devel/2022-12/msg01010.html
> 
>> Is it possible to add that functionality or would a new mode be necessary.
> 
> Yes, now this is possible to do after 'outline-search-function'
> was added in Emacs 29.  So you can use it to find the next outline heading
> that corresponds to the next tree sitter node.

‘outline-search-function’ should be documented in C-h f outline-mode help buffer.

Re: outline-mode treesitter support?

[Juri Linkov]

>>> Are there any plans on making outline-mode take tree sitter nodes instead of regex?
>>
>> The last time this was discussed a year ago, and still no progress:
>> https://lists.gnu.org/archive/html/emacs-devel/2022-12/msg01010.html
>>
>>> Is it possible to add that functionality or would a new mode be necessary.
>>
>> Yes, now this is possible to do after 'outline-search-function'
>> was added in Emacs 29.  So you can use it to find the next outline heading
>> that corresponds to the next tree sitter node.
>
> ‘outline-search-function’ should be documented in C-h f outline-mode help buffer.

‘outline-mode’ is irrelevant for tree sitter.  You can use tree sitter only with
‘outline-minor-mode’.

OK, one way to extract treesit information for outline-minor-mode is by using
‘treesit-simple-imenu’ because outline headings should correspond to imenu entries.

So I looked how to do this for example in ruby-ts-mode, but it's strange
that ruby-ts-mode doesn't use ‘treesit-simple-imenu-settings’: all ts-modes
use ‘treesit-simple-imenu-settings’, only ruby-ts-mode doesn't use it.

This means we can't use ‘treesit-simple-imenu’ for outline headings,
and need to implement a separate feature similar to ‘treesit-simple-imenu’.

Re: outline-mode treesitter support?

[Juri Linkov]

> OK, one way to extract treesit information for outline-minor-mode is by using
> ‘treesit-simple-imenu’ because outline headings should correspond to imenu entries.

I don't know if this is a good idea, but with

  (setq-local outline-search-function #'outline-search-imenu
              outline-level (lambda () 1))

this should do the trick:

#+begin_src emacs-lisp
(defun outline-search-imenu (&optional bound move backward looking-at)
  (unless imenu--index-alist
    (imenu--make-index-alist))
  (let* ((imenu-index (cdar imenu--index-alist))
         (imenu-positions (mapcar (lambda (i) (cdr i)) imenu-index)))
    (if looking-at
        (when (member (point-marker) imenu-positions)
          (set-match-data (list (pos-bol) (pos-eol)))
          t)
      (let ((found (if backward
                       (seq-find (lambda (p) (< p (pos-bol))) (nreverse imenu-positions))
                     (seq-find (lambda (p) (> p (pos-eol))) imenu-positions))))
        (if found
            (if (or (not bound) (if backward (>= found bound) (<= found bound)))
                (progn
                  (goto-char found)
                  (goto-char (pos-bol))
                  (set-match-data (list (point) (pos-eol)))
                  t)
              (when move (goto-char bound))
              nil)
          (when move (goto-char (or bound (if backward (point-min) (point-max)))))
          nil)))))
#+end_src

Re: outline-mode treesitter support?

[Huan Nguyen]

> On Dec 20, 2023, at 18:08, Juri Linkov <juri@linkov.net> wrote:
> 
>> OK, one way to extract treesit information for outline-minor-mode is by using
>> ‘treesit-simple-imenu’ because outline headings should correspond to imenu entries.
> 
> I don't know if this is a good idea, but with
> 
>  (setq-local outline-search-function #'outline-search-imenu
>              outline-level (lambda () 1))
> 
> this should do the trick:
> 
> #+begin_src emacs-lisp
> (defun outline-search-imenu (&optional bound move backward looking-at)
>  (unless imenu--index-alist
>    (imenu--make-index-alist))
>  (let* ((imenu-index (cdar imenu--index-alist))
>         (imenu-positions (mapcar (lambda (i) (cdr i)) imenu-index)))
>    (if looking-at
>        (when (member (point-marker) imenu-positions)
>          (set-match-data (list (pos-bol) (pos-eol)))
>          t)
>      (let ((found (if backward
>                       (seq-find (lambda (p) (< p (pos-bol))) (nreverse imenu-positions))
>                     (seq-find (lambda (p) (> p (pos-eol))) imenu-positions))))
>        (if found
>            (if (or (not bound) (if backward (>= found bound) (<= found bound)))
>                (progn
>                  (goto-char found)
>                  (goto-char (pos-bol))
>                  (set-match-data (list (point) (pos-eol)))
>                  t)
>              (when move (goto-char bound))
>              nil)
>          (when move (goto-char (or bound (if backward (point-min) (point-max)))))
>          nil)))))
> #+end_src

Thanks for the example!

Re: outline-mode treesitter support?

[Juri Linkov]

>>> OK, one way to extract treesit information for outline-minor-mode is by using
>>> ‘treesit-simple-imenu’ because outline headings should correspond to imenu entries.
>>
>> I don't know if this is a good idea, but with
>>
>>  (setq-local outline-search-function #'outline-search-imenu
>>              outline-level (lambda () 1))
>>
>> this should do the trick:
>>
>> (defun outline-search-imenu (&optional bound move backward looking-at)
>>  (unless imenu--index-alist
>>    (imenu--make-index-alist))
>>  (let* ((imenu-index (cdar imenu--index-alist))
>> [...]
>
> Thanks for the example!

After trying to use this, I came to conclusion that imenu--index-alist
is unsuitable for outline-minor-mode because imenu has a different
hierarchy than the source files, so outline-level can be only 1.
But without different levels for outline-level it's quite useless.

However, in the opposite direction mapping the outline hierarchy
to imenu is possible, and outline-imenu-generic-expression
does this, but only to a flat list that is sorted alphabetically.

Re: outline-mode treesitter support?

[Juri Linkov]

>>> (defun outline-search-imenu (&optional bound move backward looking-at)
>>>  (unless imenu--index-alist
>>>    (imenu--make-index-alist))
>>>  (let* ((imenu-index (cdar imenu--index-alist))
>
> After trying to use this, I came to conclusion that imenu--index-alist
> is unsuitable for outline-minor-mode because imenu has a different
> hierarchy than the source files, so outline-level can be only 1.
> But without different levels for outline-level it's quite useless.

Many ts-modes are already defining treesit-simple-imenu-settings that
can be reused or outlines.  We just need to add levels to all entries
found for imenu.

So here is a complete working implementation.  Then
with e.g. `M-x c-ts-mode` and `M-x outline-minor-mode`
it will put outlines on declarations and definitions.

#+begin_src emacs-lisp
(add-hook
 'prog-mode-hook
 (lambda ()
   (when (bound-and-true-p treesit-simple-imenu-settings)
     (setq-local treesit-outline-predicate
                 (lambda (node)
                   (seq-some
                    (lambda (setting)
                      (and (string-match-p (nth 1 setting) (treesit-node-type node))
                           (funcall (nth 2 setting) node)))
                    treesit-simple-imenu-settings)))
     (setq-local treesit-outline-levels
                 (treesit-outline-levels
                  (treesit-induce-sparse-tree
                   (treesit-buffer-root-node)
                   treesit-outline-predicate)
                  0))
     (setq-local outline-search-function #'treesit-search-outline
                 outline-level
                 (lambda ()
                   (or (alist-get (point) treesit-outline-levels nil nil
                                  (lambda (m k) (eq (marker-position m) k)))
                       1))))))

(defun treesit-outline-levels (node level)
  (let* ((ts-node (car node))
         (children (cdr node))
         (subtrees (mapcan (lambda (node)
                             (treesit-outline-levels node (1+ level)))
                           children))
         (marker (when ts-node
                   (set-marker (make-marker)
                               (save-excursion
                                 (goto-char (treesit-node-start ts-node))
                                 (search-forward (or (treesit-defun-name ts-node) ""))
                                 (pos-bol))))))
    (cond
     ((null ts-node)
      subtrees)
     (subtrees
      (cons (cons marker level) subtrees))
     (t
      (list (cons marker level))))))

(defun treesit-search-outline (&optional bound move backward looking-at)
  (let ((positions (mapcar #'car treesit-outline-levels)))
    (if looking-at
        (when (member (point-marker) positions)
          (set-match-data (list (pos-bol) (pos-eol)))
          t)
      (let ((found (if backward
                       (seq-find (lambda (p) (< p (pos-bol))) (nreverse positions))
                     (seq-find (lambda (p) (> p (pos-eol))) positions))))
        (if found
            (if (or (not bound) (if backward (>= found bound) (<= found bound)))
                (progn
                  (goto-char found)
                  (goto-char (pos-bol))
                  (set-match-data (list (point) (pos-eol)))
                  t)
              (when move (goto-char bound))
              nil)
          (when move (goto-char (or bound (if backward (point-min) (point-max)))))
          nil)))))
#+end_src