Tree-sitter, using treesit-range-rules for embedded languages

Tree-sitter, using treesit-range-rules for embedded languages

[Wilhelm]

Is there perhaps an example using treesit-range-rules? To following
attempt to set treesit-range-settings does not seem to have the desired
effect.

(defvar heex--treesit-range-rules
  (treesit-range-rules
   :embed 'elixir
   :host 'heex
   '((directive (partial_expression_value) @cap)
     (directive (expression_value) @cap)
     (expression (expression_value) @cap))))

(setq-local treesit-range-settings heex--treesit-range-rules)

The following error also pops up when modifying the buffer when removing
buffer content:

Error during redisplay: (jit-lock-function 1) signaled
(treesit-range-invalid "RANGE is either overlapping, out-of-order or
out-of-range" ((11 . 20)))

The full attempt is here: https://github.com/wkirschbaum/elixir-mode/blob/main/heex-mode.el

Re: Tree-sitter, using treesit-range-rules for embedded languages

[Yuan Fu]

> On Nov 22, 2022, at 9:24 AM, Wilhelm <wkirschbaum@gmail.com> wrote:
> 
> Is there perhaps an example using treesit-range-rules? To following
> attempt to set treesit-range-settings does not seem to have the desired
> effect.
> 
> (defvar heex--treesit-range-rules
>  (treesit-range-rules
>   :embed 'elixir
>   :host 'heex
>   '((directive (partial_expression_value) @cap)
>     (directive (expression_value) @cap)
>     (expression (expression_value) @cap))))
> 
> (setq-local treesit-range-settings heex--treesit-range-rules)
> 
> The following error also pops up when modifying the buffer when removing
> buffer content:
> 
> Error during redisplay: (jit-lock-function 1) signaled
> (treesit-range-invalid "RANGE is either overlapping, out-of-order or
> out-of-range" ((11 . 20)))
> 
> The full attempt is here: https://github.com/wkirschbaum/elixir-mode/blob/main/heex-mode.el
> 

Thanks Wilhelm. It would be nice if you can provide some example source file of heex for me to experiment with. I tried to find some on the internet but didn’t get very far :-)

Yuan

Re: Tree-sitter, using treesit-range-rules for embedded languages

[Wilhelm Hugo Kirschbaum]

On Tuesday, 22 November 2022 21:28:48 SAST Yuan Fu wrote:
> > On Nov 22, 2022, at 9:24 AM, Wilhelm <wkirschbaum@gmail.com> wrote:
> > 
> > Is there perhaps an example using treesit-range-rules? To following
> > attempt to set treesit-range-settings does not seem to have the desired
> > effect.
> > 
> > (defvar heex--treesit-range-rules
> > 
> >  (treesit-range-rules
> >  
> >   :embed 'elixir
> >   :host 'heex
> >   
> >   '((directive (partial_expression_value) @cap)
> >   
> >     (directive (expression_value) @cap)
> >     (expression (expression_value) @cap))))
> > 
> > (setq-local treesit-range-settings heex--treesit-range-rules)
> > 
> > The following error also pops up when modifying the buffer when removing
> > buffer content:
> > 
> > Error during redisplay: (jit-lock-function 1) signaled
> > (treesit-range-invalid "RANGE is either overlapping, out-of-order or
> > out-of-range" ((11 . 20)))
> > 
> > The full attempt is here:
> > https://github.com/wkirschbaum/elixir-mode/blob/main/heex-mode.el
> Thanks Wilhelm. It would be nice if you can provide some example source file
> of heex for me to experiment with. I tried to find some on the internet but
> didn’t get very far :-)
> 
> Yuan

Thanks for having a look as I was not sure if this was meant to be used yet. 

A heex file can be as simple as this: 

#sample.html.heex
<div>
  <%= Enum.map([1, 2], &(&1 + 1)) %>
</div>

and the directive <%= (partial_expression_value) %> should then be elixir., 
not heex.

When checking with (treesit-language-at (point))  for the embed it still show.

Wilhelm

Re: Tree-sitter, using treesit-range-rules for embedded languages

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 2458 bytes --]



> On Nov 22, 2022, at 11:52 AM, Wilhelm Hugo Kirschbaum <wilhelm@floatpays.co.za> wrote:
> 
> On Tuesday, 22 November 2022 21:28:48 SAST Yuan Fu wrote:
>>> On Nov 22, 2022, at 9:24 AM, Wilhelm <wkirschbaum@gmail.com> wrote:
>>> 
>>> Is there perhaps an example using treesit-range-rules? To following
>>> attempt to set treesit-range-settings does not seem to have the desired
>>> effect.
>>> 
>>> (defvar heex--treesit-range-rules
>>> 
>>> (treesit-range-rules
>>> 
>>>  :embed 'elixir
>>>  :host 'heex
>>> 
>>>  '((directive (partial_expression_value) @cap)
>>> 
>>>    (directive (expression_value) @cap)
>>>    (expression (expression_value) @cap))))
>>> 
>>> (setq-local treesit-range-settings heex--treesit-range-rules)
>>> 
>>> The following error also pops up when modifying the buffer when removing
>>> buffer content:
>>> 
>>> Error during redisplay: (jit-lock-function 1) signaled
>>> (treesit-range-invalid "RANGE is either overlapping, out-of-order or
>>> out-of-range" ((11 . 20)))
>>> 
>>> The full attempt is here:
>>> https://github.com/wkirschbaum/elixir-mode/blob/main/heex-mode.el
>> Thanks Wilhelm. It would be nice if you can provide some example source file
>> of heex for me to experiment with. I tried to find some on the internet but
>> didn’t get very far :-)
>> 
>> Yuan
> 
> Thanks for having a look as I was not sure if this was meant to be used yet. 
> 
> A heex file can be as simple as this: 
> 
> #sample.html.heex
> <div>
>  <%= Enum.map([1, 2], &(&1 + 1)) %>
> </div>
> 
> and the directive <%= (partial_expression_value) %> should then be elixir., 
> not heex.
> 
> When checking with (treesit-language-at (point))  for the embed it still show.
> 
> Wilhelm

Cool. Please see if this works. Admittedly the process is a little “raw”. That’s mainly due to me not wanting to commit to a framework before we have a lot of real-world experience with multi-language major modes. Please let me know if you think how could this be improved. For starters, maybe someways to “inherit” font-lock rules with treesit-font-lock-rules? Maybe the feature-list should have independent configuration for each language, so we don’t have feature-namespace conflicts? (Right now we rename all elixir features to elixir-<feature>.)

I’ve also fixed the warning you see (which is due to treesit.el, not your code), so you’ll need to pull master.

Thanks!
Yuan


[-- Attachment #2: heex.patch --]
[-- Type: application/octet-stream, Size: 2315 bytes --]

From 89eaff15030bf627b39a4e98801103ab622cfac6 Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Tue, 22 Nov 2022 12:24:31 -0800
Subject: [PATCH] Enable elixir font-lock rules in heex-mode

---
 heex-mode.el | 32 +++++++++++++++++++++++++++-----
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/heex-mode.el b/heex-mode.el
index a32b9ee..7425ed8 100644
--- a/heex-mode.el
+++ b/heex-mode.el
@@ -18,6 +18,7 @@
 
 ;; Code:
 
+(require 'elixir-mode)
 (require 'treesit)
 
 (eval-when-compile
@@ -257,11 +258,32 @@
 
   (cond
    ((treesit-ready-p 'heex)
-    (progn (if (treesit-ready-p 'elixir)
-               ;; (setq-local treesit-range-settings
-               ;;             heex--treesit-range-rules)
-             )
-           (treesit-major-mode-setup)))
+    (when (treesit-ready-p 'elixir)
+      (setq-local treesit-range-settings heex--treesit-range-rules)
+      (setq-local treesit-font-lock-settings
+                  (append heex--treesit-font-lock-settings
+                          ;; Add elixir font-lock rules, rename each
+                          ;; rules’ feature name to be
+                          ;; elixir-<feature>.
+                          (mapcar
+                           (lambda (rule)
+                             (list (nth 0 rule)
+                                   (nth 1 rule)
+                                   (intern (format "elixir-%s"
+                                                   (nth 2 rule)))
+                                   (nth 3 rule)))
+                           elixir--treesit-font-lock-settings)))
+      ;; Add elixir features to the feature list.
+      (setq-local treesit-font-lock-feature-list
+                  '(( doctype comment elixir-comment elixir-string )
+                    ( bracket tag attribute keyword string
+                      elixir-keyword elixir-unary-operator elixir-operator
+                      elixir-doc)
+                    ( component
+                      elixir-call elixir-constant)
+                    ( elixir-sigil elixir-string-escape)
+                    ( elixir-string-interpolation))))
+    (treesit-major-mode-setup))
 
    (t
     (message "Tree-sitter for Heex isn't available")))
-- 
2.33.1


[-- Attachment #3: Type: text/plain, Size: 2 bytes --]

Re: Tree-sitter, using treesit-range-rules for embedded languages

[Wilhelm Kirschbaum]

On Tuesday, 22 November 2022 22:39:54 SAST Yuan Fu wrote:
> > On Nov 22, 2022, at 11:52 AM, Wilhelm Hugo Kirschbaum
> > <wilhelm@floatpays.co.za> wrote:> 
> > On Tuesday, 22 November 2022 21:28:48 SAST Yuan Fu wrote:
> >>> On Nov 22, 2022, at 9:24 AM, Wilhelm <wkirschbaum@gmail.com> wrote:
> >>> 
> >>> Is there perhaps an example using treesit-range-rules? To following
> >>> attempt to set treesit-range-settings does not seem to have the desired
> >>> effect.
> >>> 
> >>> (defvar heex--treesit-range-rules
> >>> 
> >>> (treesit-range-rules
> >>> 
> >>>  :embed 'elixir
> >>>  :host 'heex
> >>>  
> >>>  '((directive (partial_expression_value) @cap)
> >>>  
> >>>    (directive (expression_value) @cap)
> >>>    (expression (expression_value) @cap))))
> >>> 
> >>> (setq-local treesit-range-settings heex--treesit-range-rules)
> >>> 
> >>> The following error also pops up when modifying the buffer when removing
> >>> buffer content:
> >>> 
> >>> Error during redisplay: (jit-lock-function 1) signaled
> >>> (treesit-range-invalid "RANGE is either overlapping, out-of-order or
> >>> out-of-range" ((11 . 20)))
> >>> 
> >>> The full attempt is here:
> >>> https://github.com/wkirschbaum/elixir-mode/blob/main/heex-mode.el
> >> 
> >> Thanks Wilhelm. It would be nice if you can provide some example source
> >> file of heex for me to experiment with. I tried to find some on the
> >> internet but didn’t get very far :-)
> >> 
> >> Yuan
> > 
> > Thanks for having a look as I was not sure if this was meant to be used
> > yet.
> > 
> > A heex file can be as simple as this:
> > 
> > #sample.html.heex
> > <div>
> > 
> >  <%= Enum.map([1, 2], &(&1 + 1)) %>
> > 
> > </div>
> > 
> > and the directive <%= (partial_expression_value) %> should then be
> > elixir.,
> > not heex.
> > 
> > When checking with (treesit-language-at (point))  for the embed it still
> > show.
> > 
> > Wilhelm
> 
> Cool. Please see if this works. Admittedly the process is a little “raw”.
> That’s mainly due to me not wanting to commit to a framework before we have
> a lot of real-world experience with multi-language major modes. Please let
> me know if you think how could this be improved. For starters, maybe
> someways to “inherit” font-lock rules with treesit-font-lock-rules? Maybe
> the feature-list should have independent configuration for each language,
> so we don’t have feature-namespace conflicts? (Right now we rename all
> elixir features to elixir-<feature>.)
> 
> I’ve also fixed the warning you see (which is due to treesit.el, not your
> code), so you’ll need to pull master.
> 
> Thanks!
> Yuan

This is great, thank you! The font lock for the embedded language seems to 
work with some quirks, but will have a look in the next couple of days to 
pinpoint any specific issues.

Wilhelm