Re: treesit: how to get it to parse multiple languages

[Andrew De Angelis <bobodeangelis@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 5574 bytes --]

Thanks so much!

I took a look at the Emacs 30 manual and it's a lot clearer, it's perfect!
I think one thing that would truly be ideal is if there is a major mode out
there that already implements multiple-language functionalities using
treesitter. Seeing all the components in action would be quite helpful: the
simple HTML examples are very clarifying but they can only do so much.

Do you all know if such a mode exists? `(ripgrep-regexp "local-parser"
source-directory)` on the master branch only shows me matches in
`treesit.el` itself (and associated ChangeLog / manual).
If it doesn't exist yet I'm happy to give it a knack when implementing the
notebook mode. I might have run into some more questions then :)

On Tue, Nov 5, 2024 at 1:47 AM Yuan Fu <casouri@gmail.com> wrote:

>
>
> > On Nov 4, 2024, at 4:02 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> >
> >> From: Andrew De Angelis <bobodeangelis@gmail.com>
> >> Date: Sun, 3 Nov 2024 13:28:57 -0500
> >>
> >> I'm trying to get a better understanding of treesit.el, and I've
> stumbled on a couple of things that make me
> >> think the manual is either outdated/faulty, or just not entirely clear
> and I'm missing something.
> >>
> >> The latter is most likely, but I'd appreciate any help in figuring out
> what exactly is wrong in my
> >> approach/setup. I would be happy to contribute to the manual, if
> needed, to ensure it is clearer.
> >>
> >> This is the relevant section of the manual:
> >>
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Multiple-Languages.html
> >> I've started out with simply trying to recreate the setup described in
> the manual, but I've run into some
> >> issues.
> >> Here's what I've done so far:
> >> - I've defined a very simple `html-ts-mode`, using the elisp functions
> from the manual:
> >> https://github.com/andrewdea/poc-html-ts-mode/blob/main/html-ts-mode.el
> >> - I activate this mode when visiting the example.html file (which is
> also copied from the manual):
> >> https://github.com/andrewdea/poc-html-ts-mode/blob/main/example.html
> >> - the queries seem to be working as expected: when I'm in a buffer
> visiting example.html, evaluating
> >> `(treesit-query-capture 'html css-query)` and `(treesit-query-capture
> 'html js-query)` return the expected
> >> nodes
> >> - ISSUE: `treesit-update-ranges` doesn't seem to be working as
> expected: even if I call it multiple times, the
> >> parser for the whole buffer seems to still be 'html.
> `(treesit-language-at (point))` always returns 'html, even
> >> when I'm inside the nodes captured by the css-query or js-query.
> >>
> >> Some additional context: the reason I'm looking into tree-sitter (and
> its functionalities to support multiple
> >> languages) is to potentially use it to fontify markdown code blocks and
> to improve emacs support for python
> >> notebooks. For markdown, I was trying a similar approach to the HTML
> one described in the manual, but ran
> >> into other similar issues:
> >>
> https://www.reddit.com/r/emacs/comments/1gcrv8k/syntaxhighlighting_codeblocks_in_markdown/
> .
> >> I'm just including this as context.
> >>
> >> Let me know if any of this is not clear.
> >>
> >> Thanks in advance for all your help!
> >
> > Yuan, can you help Andrew?
>
> Ah yes, thanks for the ping. Andrew, I take that your problem is with
> treesit-language-at, right? Specifically, it doesn’t return expected
> results. That’s because for treesit-language-at to work, major mode needs
> to define treesit-language-at-function.
>
> This confusion has came up a couple times now, evidently
> treesit-language-at is not very intuitive. Hopefully it’ll be fixed by our
> updated manual for Emacs 30. In Emacs 30, we define
> treesit-language-at-function in the example code:
>
>    Emacs automates this process in ‘treesit-update-ranges’.  A
> multi-language major mode should set ‘treesit-range-settings’ so that
> ‘treesit-update-ranges’ knows how to perform this process automatically.
> Major modes should use the helper function ‘treesit-range-rules’ to
> generate a value that can be assigned to ‘treesit-range-settings’.  The
> settings in the following example directly translate into operations
> shown above.
>
>      (setq treesit-range-settings
>            (treesit-range-rules
>             :embed 'javascript
>             :host 'html
>             '((script_element (raw_text) @capture))
>             :embed 'css
>             :host 'html
>             '((style_element (raw_text) @capture))))
>
>      ;; Major modes with multiple languages should always set
>      ;; `treesit-language-at-point-function' (which see).
>      (setq treesit-language-at-point-function
>            (lambda (pos)
>              (let* ((node (treesit-node-at pos 'html))
>                     (parent (treesit-node-parent node)))
>                (cond
>                 ((and node parent
>                       (equal (treesit-node-type node) "raw_text")
>                       (equal (treesit-node-type parent) "script_element"))
>                  'javascript)
>                 ((and node parent
>                       (equal (treesit-node-type node) "raw_text")
>                       (equal (treesit-node-type parent) "style_element"))
>                  'css)
>                 (t 'html)))))
>
> And FYI, in Emacs 30 we added local parsers, that might make implementing
> code/markdown blocks in a notebook easier.
>
> Yuan

[-- Attachment #2: Type: text/html, Size: 7221 bytes --]