treesit-range-settings with ':local' : I missed something or it's a bug?

treesit-range-settings with ':local' : I missed something or it's a bug?

[Vincenzo Pupillo]

Hi, 
in the php-ts-mode I am writing, I am trying to use the tree-sitter-phpdoc parser 
(I had tried before but without success, and I currently use a font block based 
on regular expressions). 
tree-sitter-phpdoc requires a single doc block (a single /** */ doc block and 
nothing else, no /* */ or #). I wrote these range rules:
(setq-local treesit-range-settings
		      (treesit-range-rules
		       :embed 'phpdoc
		       :host 'php
		       :local t
		       '(((comment) @cap
			 (:match "/\\*\\*" @cap)))

		       :embed 'html
		       :host 'php
		       '((program (text) @cap)
			 (text_interpolation (text) @cap))

		       :embed 'javascript
		       :host 'html
		       :offset '(1 . -1)
		       '((script_element
			  (start_tag (tag_name))
			  (raw_text) @cap))

		       :embed 'css
		       :host 'html
		       :offset '(1 . -1)
		       '((style_element
			  (start_tag (tag_name))
			  (raw_text) @cap))))

With html, js and css it works fine. With phpdoc I tried with or without :local. 
Without :local the parse intervals are all null. Without :local on the other hand they are correct.
With this simple php snippet:
<?php

/**
 * Test class
 * @author v <v.pupillo@gmail.com>
 */
class Test {
    /** @see http://example.com the lib */
    function test() {
	echo "prova";
    }
}

Without :local the field rules are:
((#<treesit-parser for phpdoc> ((1 . 1) (8 . 64) (82 . 120))) (#<treesit-
parser for html> ((1 . 1) (1 . 2))) (#<treesit-parser for css> ((1 . 1))) 
(#<treesit-parser for javascript> ((1 . 1))) (#<treesit-parser for php> nil))

With :local the result is:
((#<treesit-parser for phpdoc> nil) (#<treesit-parser for html> nil) 
(#<treesit-parser for css> nil) (#<treesit-parser for javascript> nil) 
(#<treesit-parser for php> nil))


With :local the treesit-language-at breaks, and js or css rules 
are also applied to php code :-(

I tried tracking the node location sent from the parser to the font-lock-rules I wrote for phpdoc, 
with :local the result is:

phpdoc-block node-start= 8 node-end= 64 start= 1 end= 166
phpdoc-block node-start= 82 node-end= 120 start= 1 end= 166
phpdoc-block node-start= 1 node-end= 166 start= 1 end= 166

the first two are right, the last one is not.

Any idea?
Tested with:
GNU Emacs 30.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.39, cairo version 1.18.0) of 2024-01-11


Thanks

p.s. the function I wrote to get the ranges is:
(defun php-ts-mode--get-parser-ranges ()
  "Return the ranges covered by the parsers.

`php-ts-mode' use 5 parsers, this function returns, for the
current buffer, the ranges covered by each parser.
Usefull for debugging."
  (let ((ranges))
    (if (not (treesit-parser-list))
	(message "At least one parser must be initialized"))
    (cl-loop
     for parser in (treesit-parser-list)
     do (push (list parser (treesit-parser-included-ranges parser)) ranges)
     finally return ranges)))

Re: treesit-range-settings with ':local' : I missed something or it's a bug?

[Vincenzo Pupillo]

yes, it is a bug. I will send a patch to debbugs.gnu.org in the next few days.

V.

In data giovedì 11 gennaio 2024 12:15:24 CET, Vincenzo Pupillo ha scritto:
> Hi, 
> in the php-ts-mode I am writing, I am trying to use the tree-sitter-phpdoc parser 
> (I had tried before but without success, and I currently use a font block based 
> on regular expressions). 
> tree-sitter-phpdoc requires a single doc block (a single /** */ doc block and 
> nothing else, no /* */ or #). I wrote these range rules:
> (setq-local treesit-range-settings
> 		      (treesit-range-rules
> 		       :embed 'phpdoc
> 		       :host 'php
> 		       :local t
> 		       '(((comment) @cap
> 			 (:match "/\\*\\*" @cap)))
> 
> 		       :embed 'html
> 		       :host 'php
> 		       '((program (text) @cap)
> 			 (text_interpolation (text) @cap))
> 
> 		       :embed 'javascript
> 		       :host 'html
> 		       :offset '(1 . -1)
> 		       '((script_element
> 			  (start_tag (tag_name))
> 			  (raw_text) @cap))
> 
> 		       :embed 'css
> 		       :host 'html
> 		       :offset '(1 . -1)
> 		       '((style_element
> 			  (start_tag (tag_name))
> 			  (raw_text) @cap))))
> 
> With html, js and css it works fine. With phpdoc I tried with or without :local. 
> Without :local the parse intervals are all null. Without :local on the other hand they are correct.
> With this simple php snippet:
> <?php
> 
> /**
>  * Test class
>  * @author v <v.pupillo@gmail.com>
>  */
> class Test {
>     /** @see http://example.com the lib */
>     function test() {
> 	echo "prova";
>     }
> }
> 
> Without :local the field rules are:
> ((#<treesit-parser for phpdoc> ((1 . 1) (8 . 64) (82 . 120))) (#<treesit-
> parser for html> ((1 . 1) (1 . 2))) (#<treesit-parser for css> ((1 . 1))) 
> (#<treesit-parser for javascript> ((1 . 1))) (#<treesit-parser for php> nil))
> 
> With :local the result is:
> ((#<treesit-parser for phpdoc> nil) (#<treesit-parser for html> nil) 
> (#<treesit-parser for css> nil) (#<treesit-parser for javascript> nil) 
> (#<treesit-parser for php> nil))
> 
> 
> With :local the treesit-language-at breaks, and js or css rules 
> are also applied to php code :-(
> 
> I tried tracking the node location sent from the parser to the font-lock-rules I wrote for phpdoc, 
> with :local the result is:
> 
> phpdoc-block node-start= 8 node-end= 64 start= 1 end= 166
> phpdoc-block node-start= 82 node-end= 120 start= 1 end= 166
> phpdoc-block node-start= 1 node-end= 166 start= 1 end= 166
> 
> the first two are right, the last one is not.
> 
> Any idea?
> Tested with:
> GNU Emacs 30.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.39, cairo version 1.18.0) of 2024-01-11
> 
> 
> Thanks
> 
> p.s. the function I wrote to get the ranges is:
> (defun php-ts-mode--get-parser-ranges ()
>   "Return the ranges covered by the parsers.
> 
> `php-ts-mode' use 5 parsers, this function returns, for the
> current buffer, the ranges covered by each parser.
> Usefull for debugging."
>   (let ((ranges))
>     (if (not (treesit-parser-list))
> 	(message "At least one parser must be initialized"))
>     (cl-loop
>      for parser in (treesit-parser-list)
>      do (push (list parser (treesit-parser-included-ranges parser)) ranges)
>      finally return ranges)))
> 
> 
> 
> 

Re: treesit-range-settings with ':local' : I missed something or it's a bug?

[Yuan Fu]

> On Jan 11, 2024, at 3:15 AM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> Hi, 
> in the php-ts-mode I am writing, I am trying to use the tree-sitter-phpdoc parser 
> (I had tried before but without success, and I currently use a font block based 
> on regular expressions). 
> tree-sitter-phpdoc requires a single doc block (a single /** */ doc block and 
> nothing else, no /* */ or #). I wrote these range rules:
> (setq-local treesit-range-settings
>      (treesit-range-rules
>       :embed 'phpdoc
>       :host 'php
>       :local t
>       '(((comment) @cap
> (:match "/\\*\\*" @cap)))
> 
>       :embed 'html
>       :host 'php
>       '((program (text) @cap)
> (text_interpolation (text) @cap))
> 
>       :embed 'javascript
>       :host 'html
>       :offset '(1 . -1)
>       '((script_element
>  (start_tag (tag_name))
>  (raw_text) @cap))
> 
>       :embed 'css
>       :host 'html
>       :offset '(1 . -1)
>       '((style_element
>  (start_tag (tag_name))
>  (raw_text) @cap))))
> 
> With html, js and css it works fine. With phpdoc I tried with or without :local. 
> Without :local the parse intervals are all null. Without :local on the other hand they are correct.
> With this simple php snippet:
> <?php
> 
> /**
> * Test class
> * @author v <v.pupillo@gmail.com>
> */
> class Test {
>    /** @see http://example.com the lib */
>    function test() {
> echo "prova";
>    }
> }
> 
> Without :local the field rules are:
> ((#<treesit-parser for phpdoc> ((1 . 1) (8 . 64) (82 . 120))) (#<treesit-
> parser for html> ((1 . 1) (1 . 2))) (#<treesit-parser for css> ((1 . 1))) 
> (#<treesit-parser for javascript> ((1 . 1))) (#<treesit-parser for php> nil))
> 
> With :local the result is:
> ((#<treesit-parser for phpdoc> nil) (#<treesit-parser for html> nil) 
> (#<treesit-parser for css> nil) (#<treesit-parser for javascript> nil) 
> (#<treesit-parser for php> nil))
> 
> 
> With :local the treesit-language-at breaks, and js or css rules 
> are also applied to php code :-(
> 
> I tried tracking the node location sent from the parser to the font-lock-rules I wrote for phpdoc, 
> with :local the result is:
> 
> phpdoc-block node-start= 8 node-end= 64 start= 1 end= 166
> phpdoc-block node-start= 82 node-end= 120 start= 1 end= 166
> phpdoc-block node-start= 1 node-end= 166 start= 1 end= 166
> 
> the first two are right, the last one is not.
> 
> Any idea?
> Tested with:
> GNU Emacs 30.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.39, cairo version 1.18.0) of 2024-01-11
> 
> 
> Thanks
> 
> p.s. the function I wrote to get the ranges is:
> (defun php-ts-mode--get-parser-ranges ()
>  "Return the ranges covered by the parsers.
> 
> `php-ts-mode' use 5 parsers, this function returns, for the
> current buffer, the ranges covered by each parser.
> Usefull for debugging."
>  (let ((ranges))
>    (if (not (treesit-parser-list))
> (message "At least one parser must be initialized"))
>    (cl-loop
>     for parser in (treesit-parser-list)
>     do (push (list parser (treesit-parser-included-ranges parser)) ranges)
>     finally return ranges)))

IIUC your problem is that treesit-language-at doesn’t work, right? Have you assigned treesit-language-at-function? People often assume treesit-language-at works automatically when they define treesit-range-rules. But you actually need to define treesit-language-at-function.

Yuan

Re: treesit-range-settings with ':local' : I missed something or it's a bug?

[Vincenzo Pupillo]

[-- Attachment #1.1: Type: text/plain, Size: 5992 bytes --]

No. I defined a function for "treesit-language-at", which works correctly 
without tree-sitter-phodoc parser (the version without phpdoc is here https://
github.com/vpxyz/php-ts-mode)
I find two problems: 
1. treesit-range-rules does not reset the variable "local" at the end of pcase 
as it does with "host", "embed" and "offset". If there is a parser with the 
:local attribute, all parsers defined after it are marked as :local 
Without fix: 
treesit-range-settings is a variable defined in ‘~/Projects/Emacs/php-ts-mode/
treesit.el’.

Its value is
((#<treesit-compiled-query> phpdoc t nil)
 (#<treesit-compiled-query> html t nil)
 (#<treesit-compiled-query> javascript t (1 . -1))
 (#<treesit-compiled-query> css t (1 . -1)))
Local in buffer index4.php; global value is nil

with fix:
treesit-range-settings is a variable defined in ‘~/Projects/Emacs/php-ts-mode/
treesit.el’.

Its value is
((#<treesit-compiled-query> phpdoc t nil)
 (#<treesit-compiled-query> html nil nil)
 (#<treesit-compiled-query> javascript nil (1 . -1))
 (#<treesit-compiled-query> css nil (1 . -1)))
Local in buffer index4.php; global value is nil

2.  treesit-font-lock-fontify-region: the variable "global-parser" is set with 
all parsers defined, including local parsers. As result the root-nodes 
variable, without fix are ("document node" is  a phpdoc node): 
root-nodes (#<treesit-node document in 517-621> #<treesit-node document in 
672-810> #<treesit-node program in 1-1057> #<treesit-node program in 79-138> 
#<treesit-node stylesheet in 167-223> #<treesit-node fragment in 1-271> 
#<treesit-node document in 1-1057>)

with fix:  
root-nodes (#<treesit-node document in 517-621> #<treesit-node document in 
672-810> #<treesit-node program in 1-1057> #<treesit-node program in 79-138> 
#<treesit-node stylesheet in 167-223> #<treesit-node fragment in 1-271>)
 
you can see the difference in the attached screenshots.
The patch I wrote is not a solution (it's trivial) . I think there are other 
"moving parts" to take into account.

Thanks.


In data sabato 27 gennaio 2024 05:32:02 CET, Yuan Fu ha scritto:
> > On Jan 11, 2024, at 3:15 AM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> > 
> > Hi,
> > in the php-ts-mode I am writing, I am trying to use the tree-sitter-phpdoc
> > parser (I had tried before but without success, and I currently use a
> > font block based on regular expressions).
> > tree-sitter-phpdoc requires a single doc block (a single /** */ doc block
> > and nothing else, no /* */ or #). I wrote these range rules:
> > (setq-local treesit-range-settings
> > 
> >      (treesit-range-rules
> >      
> >       :embed 'phpdoc
> >       :host 'php
> >       :local t
> >       
> >       '(((comment) @cap
> > 
> > (:match "/\\*\\*" @cap)))
> > 
> >       :embed 'html
> >       :host 'php
> >       
> >       '((program (text) @cap)
> > 
> > (text_interpolation (text) @cap))
> > 
> >       :embed 'javascript
> >       :host 'html
> >       :offset '(1 . -1)
> >       
> >       '((script_element
> >  
> >  (start_tag (tag_name))
> >  (raw_text) @cap))
> >  
> >       :embed 'css
> >       :host 'html
> >       :offset '(1 . -1)
> >       
> >       '((style_element
> >  
> >  (start_tag (tag_name))
> >  (raw_text) @cap))))
> > 
> > With html, js and css it works fine. With phpdoc I tried with or without
> > :local. Without :local the parse intervals are all null. Without :local
> > on the other hand they are correct. With this simple php snippet:
> > <?php
> > 
> > /**
> > * Test class
> > * @author v <v.pupillo@gmail.com>
> > */
> > class Test {
> > 
> >    /** @see http://example.com the lib */
> >    function test() {
> > 
> > echo "prova";
> > 
> >    }
> > 
> > }
> > 
> > Without :local the field rules are:
> > ((#<treesit-parser for phpdoc> ((1 . 1) (8 . 64) (82 . 120))) (#<treesit-
> > parser for html> ((1 . 1) (1 . 2))) (#<treesit-parser for css> ((1 . 1)))
> > (#<treesit-parser for javascript> ((1 . 1))) (#<treesit-parser for php>
> > nil))
> > 
> > With :local the result is:
> > ((#<treesit-parser for phpdoc> nil) (#<treesit-parser for html> nil)
> > (#<treesit-parser for css> nil) (#<treesit-parser for javascript> nil)
> > (#<treesit-parser for php> nil))
> > 
> > 
> > With :local the treesit-language-at breaks, and js or css rules
> > are also applied to php code :-(
> > 
> > I tried tracking the node location sent from the parser to the
> > font-lock-rules I wrote for phpdoc, with :local the result is:
> > 
> > phpdoc-block node-start= 8 node-end= 64 start= 1 end= 166
> > phpdoc-block node-start= 82 node-end= 120 start= 1 end= 166
> > phpdoc-block node-start= 1 node-end= 166 start= 1 end= 166
> > 
> > the first two are right, the last one is not.
> > 
> > Any idea?
> > Tested with:
> > GNU Emacs 30.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.39,
> > cairo version 1.18.0) of 2024-01-11
> > 
> > 
> > Thanks
> > 
> > p.s. the function I wrote to get the ranges is:
> > (defun php-ts-mode--get-parser-ranges ()
> > 
> >  "Return the ranges covered by the parsers.
> > 
> > `php-ts-mode' use 5 parsers, this function returns, for the
> > current buffer, the ranges covered by each parser.
> > Usefull for debugging."
> > 
> >  (let ((ranges))
> >  
> >    (if (not (treesit-parser-list))
> > 
> > (message "At least one parser must be initialized"))
> > 
> >    (cl-loop
> >    
> >     for parser in (treesit-parser-list)
> >     do (push (list parser (treesit-parser-included-ranges parser)) ranges)
> >     finally return ranges)))
> 
> IIUC your problem is that treesit-language-at doesn’t work, right? Have you
> assigned treesit-language-at-function? People often assume
> treesit-language-at works automatically when they define
> treesit-range-rules. But you actually need to define
> treesit-language-at-function.
> 
> Yuan


[-- Attachment #1.2: treesit_local_withoutfix.png --]
[-- Type: image/png, Size: 17487 bytes --]

[-- Attachment #1.3: treesit_local_withfix.png --]
[-- Type: image/png, Size: 17947 bytes --]

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: treesit-range-settings with ':local' : I missed something or it's a bug?

[Yuan Fu]

> On Jan 27, 2024, at 2:23 AM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> No. I defined a function for "treesit-language-at", which works correctly 
> without tree-sitter-phodoc parser (the version without phpdoc is here https://
> github.com/vpxyz/php-ts-mode)
> I find two problems: 
> 1. treesit-range-rules does not reset the variable "local" at the end of pcase 
> as it does with "host", "embed" and "offset". If there is a parser with the 
> :local attribute, all parsers defined after it are marked as :local 
> Without fix: 
> treesit-range-settings is a variable defined in ‘~/Projects/Emacs/php-ts-mode/
> treesit.el’.
> 
> Its value is
> ((#<treesit-compiled-query> phpdoc t nil)
> (#<treesit-compiled-query> html t nil)
> (#<treesit-compiled-query> javascript t (1 . -1))
> (#<treesit-compiled-query> css t (1 . -1)))
> Local in buffer index4.php; global value is nil
> 
> with fix:
> treesit-range-settings is a variable defined in ‘~/Projects/Emacs/php-ts-mode/
> treesit.el’.
> 
> Its value is
> ((#<treesit-compiled-query> phpdoc t nil)
> (#<treesit-compiled-query> html nil nil)
> (#<treesit-compiled-query> javascript nil (1 . -1))
> (#<treesit-compiled-query> css nil (1 . -1)))
> Local in buffer index4.php; global value is nil

Thanks, I pushed a fix for it.

> 2.  treesit-font-lock-fontify-region: the variable "global-parser" is set with 
> all parsers defined, including local parsers. As result the root-nodes 
> variable, without fix are ("document node" is  a phpdoc node): 
> root-nodes (#<treesit-node document in 517-621> #<treesit-node document in 
> 672-810> #<treesit-node program in 1-1057> #<treesit-node program in 79-138> 
> #<treesit-node stylesheet in 167-223> #<treesit-node fragment in 1-271> 
> #<treesit-node document in 1-1057>)
> 
> with fix:  
> root-nodes (#<treesit-node document in 517-621> #<treesit-node document in 
> 672-810> #<treesit-node program in 1-1057> #<treesit-node program in 79-138> 
> #<treesit-node stylesheet in 167-223> #<treesit-node fragment in 1-271>)
> 
> you can see the difference in the attached screenshots.
> The patch I wrote is not a solution (it's trivial) . I think there are other 
> "moving parts" to take into account.
> 
> Thanks.

Actually, local parsers are not included in the return value of (treesit-parser-list). By default, treesit-parser-list returns all the parsers whose tag is nil, but all the local parsers carry a tag of ‘embedded. To actually return all the parsers in the buffer you need to use (treesit-parser-list nil nil t), ie, pass t to the TAG parameter. 

I pulled your php-ts-mode_phpdoc.el and played around with it. I found the root cause to be the call to 

(treesit-parser-create ‘phpdoc)

In the major mode body. This creates a global phpdoc parser that fontifies everything in doc face.

Removing that, plus the fix for #1 that I just pushed to master, should fix the font-lock problem you are observing.

I couldn’t fine the patch you mentioned in the thread so I don’t know if this is what your patch does.

Yuan

Re: treesit-range-settings with ':local' : I missed something or it's a bug?

[Vincenzo Pupillo]

[-- Attachment #1.1: Type: text/plain, Size: 2389 bytes --]

Hi Yuan,
In data domenica 28 gennaio 2024 08:09:02 CET, Yuan Fu ha scritto:
> 
> Actually, local parsers are not included in the return value of
> (treesit-parser-list). By default, treesit-parser-list returns all the
> parsers whose tag is nil, but all the local parsers carry a tag of
> ‘embedded. To actually return all the parsers in the buffer you need to use
> (treesit-parser-list nil nil t), ie, pass t to the TAG parameter.
> 
> I pulled your php-ts-mode_phpdoc.el and played around with it. I found the
> root cause to be the call to
> 
> (treesit-parser-create ‘phpdoc)
> 
> In the major mode body. This creates a global phpdoc parser that fontifies
> everything in doc face.
> 
> Removing that, plus the fix for #1 that I just pushed to master, should fix
> the font-lock problem you are observing.
It seems to work, but just try to indent the entire buffer, and the problem 
reappears. Before indenting: 
((#<treesit-parser for html> ((1 . 271))) (#<treesit-parser for css> ((161 . 
223))) (#<treesit-parser for javascript> ((73 . 138))) (#<treesit-parser for 
php> nil) (#<treesit-parser for phpdoc> ((517 . 621))) (#<treesit-parser for 
phpdoc> ((672 . 810))) (#<treesit-parser for phpdoc> ((939 . 1032))) 
(#<treesit-parser for phpdoc> ((1157 . 1223))))

after indenting the whole buffer:

((#<treesit-parser for html> ((1 . 271))) (#<treesit-parser for css> ((161 . 
223))) (#<treesit-parser for javascript> ((73 . 138))) (#<treesit-parser for 
php> nil) (#<treesit-parser for phpdoc> ((517 . 621))) (#<treesit-parser for 
phpdoc> ((672 . 810))) (#<treesit-parser for phpdoc> ((939 . 1032))) 
(#<treesit-parser for phpdoc> ((1157 . 1223))) (#<treesit-parser for phpdoc> 
nil))
 
As you can see the list of parsers has changed, and the latest one is:
(#<treesit-parser for phpdoc> nil)  .

To reproduce the problem:
1. open a php file (for e.g. the php file attached)
2. Indent the entire buffer
3. add a comment line inside a document block

The attached screenshot shows the result.
> 
> I couldn’t fine the patch you mentioned in the thread so I don’t know if
> this is what your patch does.

The modification I attached (that's the one I was talking about that I forgot 
to attach) solves the problem, but downstream, not upstream (I am not familiar 
with how treesit.el works). 
> 
> Yuan
Thanks 

V.


[-- Attachment #1.2: tressit_indent_font-lock_issue.png --]
[-- Type: image/png, Size: 10731 bytes --]

[-- Attachment #1.3: 0001-Remove-local-parser-from-global-parser-list-when-fon.patch --]
[-- Type: text/x-patch, Size: 1870 bytes --]

From c84464e0a8e3d5086af8bf2ff8f510a503f13374 Mon Sep 17 00:00:00 2001
From: Vincenzo Pupillo <vincenzo.pupillo@unimi.it>
Date: Sun, 28 Jan 2024 21:17:51 +0100
Subject: [PATCH] Remove local-parser from global-parser list when fontify
 region

The list of global parsers used by treesit-font-lock-fontify-region
also contains local parsers (parsers embedded with the :local
attribute). As a result, the font-lock range of local parsers overlaps
with others.

* lisp/treesit.el (treesit-font-lock-fontify-region): Drop local
parsers from the global-parsers list.
---
 lisp/treesit.el | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 96222ed81cb..47f06c37793 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1233,7 +1233,20 @@ treesit-font-lock-fontify-region
   (treesit-update-ranges start end)
   (font-lock-unfontify-region start end)
   (let* ((local-parsers (treesit-local-parsers-on start end))
-         (global-parsers (treesit-parser-list))
+         (global-parsers (cl-remove-if
+                          (lambda (parser)
+                            (let ((range-settings treesit-range-settings)
+                                  (finded nil))
+                              (while (and range-settings (not finded))
+                                (let* ((range-setting (pop range-settings))
+                                       (language (nth 1 range-setting))
+                                       (local (nth 2 range-setting)))
+                                  (setq finded
+					(and
+					 local
+					 (eq (treesit-parser-language parser) language)))))
+			      finded))
+                          (treesit-parser-list)))
          (root-nodes
           (mapcar #'treesit-parser-root-node
                   (append local-parsers global-parsers))))
-- 
2.43.0


[-- Attachment #1.4: index4.php --]
[-- Type: application/x-php, Size: 1285 bytes --]

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: treesit-range-settings with ':local' : I missed something or it's a bug?

[Yuan Fu]

> On Jan 29, 2024, at 6:04 AM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> Hi Yuan,
> In data domenica 28 gennaio 2024 08:09:02 CET, Yuan Fu ha scritto:
>> 
>> Actually, local parsers are not included in the return value of
>> (treesit-parser-list). By default, treesit-parser-list returns all the
>> parsers whose tag is nil, but all the local parsers carry a tag of
>> ‘embedded. To actually return all the parsers in the buffer you need to use
>> (treesit-parser-list nil nil t), ie, pass t to the TAG parameter.
>> 
>> I pulled your php-ts-mode_phpdoc.el and played around with it. I found the
>> root cause to be the call to
>> 
>> (treesit-parser-create ‘phpdoc)
>> 
>> In the major mode body. This creates a global phpdoc parser that fontifies
>> everything in doc face.
>> 
>> Removing that, plus the fix for #1 that I just pushed to master, should fix
>> the font-lock problem you are observing.
> It seems to work, but just try to indent the entire buffer, and the problem 
> reappears. Before indenting: 
> ((#<treesit-parser for html> ((1 . 271))) (#<treesit-parser for css> ((161 . 
> 223))) (#<treesit-parser for javascript> ((73 . 138))) (#<treesit-parser for 
> php> nil) (#<treesit-parser for phpdoc> ((517 . 621))) (#<treesit-parser for 
> phpdoc> ((672 . 810))) (#<treesit-parser for phpdoc> ((939 . 1032))) 
> (#<treesit-parser for phpdoc> ((1157 . 1223))))
> 
> after indenting the whole buffer:
> 
> ((#<treesit-parser for html> ((1 . 271))) (#<treesit-parser for css> ((161 . 
> 223))) (#<treesit-parser for javascript> ((73 . 138))) (#<treesit-parser for 
> php> nil) (#<treesit-parser for phpdoc> ((517 . 621))) (#<treesit-parser for 
> phpdoc> ((672 . 810))) (#<treesit-parser for phpdoc> ((939 . 1032))) 
> (#<treesit-parser for phpdoc> ((1157 . 1223))) (#<treesit-parser for phpdoc> 
> nil))
> 
> As you can see the list of parsers has changed, and the latest one is:
> (#<treesit-parser for phpdoc> nil)  .
> 
> To reproduce the problem:
> 1. open a php file (for e.g. the php file attached)
> 2. Indent the entire buffer
> 3. add a comment line inside a document block
> 
> The attached screenshot shows the result.

That’s due to a bug in treesit--indent-1. I’ve pushed a fix for it to master. It should solve your problem. BTW, you don’t need to call flush-syntax-ppss anymore, we fixed that bug a while back.

Yuan

Re: treesit-range-settings with ':local' : I missed something or it's a bug?

[Vincenzo Pupillo]

[-- Attachment #1: Type: text/plain, Size: 2661 bytes --]

Thanks Yuan, 
now work correctly.
V.

In data mercoledì 31 gennaio 2024 07:32:47 CET, Yuan Fu ha scritto:
> > On Jan 29, 2024, at 6:04 AM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> > 
> > Hi Yuan,
> > 
> > In data domenica 28 gennaio 2024 08:09:02 CET, Yuan Fu ha scritto:
> >> Actually, local parsers are not included in the return value of
> >> (treesit-parser-list). By default, treesit-parser-list returns all the
> >> parsers whose tag is nil, but all the local parsers carry a tag of
> >> ‘embedded. To actually return all the parsers in the buffer you need to
> >> use
> >> (treesit-parser-list nil nil t), ie, pass t to the TAG parameter.
> >> 
> >> I pulled your php-ts-mode_phpdoc.el and played around with it. I found
> >> the
> >> root cause to be the call to
> >> 
> >> (treesit-parser-create ‘phpdoc)
> >> 
> >> In the major mode body. This creates a global phpdoc parser that
> >> fontifies
> >> everything in doc face.
> >> 
> >> Removing that, plus the fix for #1 that I just pushed to master, should
> >> fix
> >> the font-lock problem you are observing.
> > 
> > It seems to work, but just try to indent the entire buffer, and the
> > problem
> > reappears. Before indenting:
> > ((#<treesit-parser for html> ((1 . 271))) (#<treesit-parser for css> ((161
> > . 223))) (#<treesit-parser for javascript> ((73 . 138)))
> > (#<treesit-parser for php> nil) (#<treesit-parser for phpdoc> ((517 .
> > 621))) (#<treesit-parser for phpdoc> ((672 . 810))) (#<treesit-parser for
> > phpdoc> ((939 . 1032))) (#<treesit-parser for phpdoc> ((1157 . 1223))))
> > 
> > after indenting the whole buffer:
> > 
> > ((#<treesit-parser for html> ((1 . 271))) (#<treesit-parser for css> ((161
> > . 223))) (#<treesit-parser for javascript> ((73 . 138)))
> > (#<treesit-parser for php> nil) (#<treesit-parser for phpdoc> ((517 .
> > 621))) (#<treesit-parser for phpdoc> ((672 . 810))) (#<treesit-parser for
> > phpdoc> ((939 . 1032))) (#<treesit-parser for phpdoc> ((1157 . 1223)))
> > (#<treesit-parser for phpdoc> nil))
> > 
> > As you can see the list of parsers has changed, and the latest one is:
> > (#<treesit-parser for phpdoc> nil)  .
> > 
> > To reproduce the problem:
> > 1. open a php file (for e.g. the php file attached)
> > 2. Indent the entire buffer
> > 3. add a comment line inside a document block
> > 
> > The attached screenshot shows the result.
> 
> That’s due to a bug in treesit--indent-1. I’ve pushed a fix for it to
> master. It should solve your problem. BTW, you don’t need to call
> flush-syntax-ppss anymore, we fixed that bug a while back.
> 
> Yuan


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: treesit-range-settings with ':local' : I missed something or it's a bug?

[Yuan Fu]

> On Jan 31, 2024, at 12:05 PM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> Thanks Yuan, 
> now work correctly.
> V.

Great, thanks for your report! Feel free to reach out to me if you have any problem using treesit.el

Yuan