Define treesit-font-lock-level as buffer local

Define treesit-font-lock-level as buffer local

[Vincenzo Pupillo]

Ciao,
I use tree-sitter for different languages, for some of them I prefer font lock level 3, for others I prefer level 4. 
treesit-font-lock-level, however, is not defined as buffer local and so there is no elegant way to have 
different levels for different languages. 
Would it be possible to define treesit-font-lock-level as a buffer local?

Thank you.

V.

Re: Define treesit-font-lock-level as buffer local

[Eli Zaretskii]

> From: Vincenzo Pupillo <v.pupillo@gmail.com>
> Date: Thu, 07 Nov 2024 11:30:03 +0100
> 
> Ciao,
> I use tree-sitter for different languages, for some of them I prefer font lock level 3, for others I prefer level 4. 
> treesit-font-lock-level, however, is not defined as buffer local and so there is no elegant way to have 
> different levels for different languages. 
> Would it be possible to define treesit-font-lock-level as a buffer local?

Is it a problem for you to do that in your init file?

Also, we could support a value of that variable which is a list, like
we do with font-lock-maximum-decoration.  This will be more useful
than making the variable buffer-local, since presumably your
preferences are per-mode, not per-buffer.  Yuan, WDYT?

Re: Define treesit-font-lock-level as buffer local

[Vincenzo Pupillo]

In data giovedì 7 novembre 2024 12:03:49 Ora standard dell’Europa centrale, Eli Zaretskii ha scritto:
> > From: Vincenzo Pupillo <v.pupillo@gmail.com>
> > Date: Thu, 07 Nov 2024 11:30:03 +0100
> > 
> > Ciao,
> > I use tree-sitter for different languages, for some of them I prefer font lock level 3, for others I prefer level 4. 
> > treesit-font-lock-level, however, is not defined as buffer local and so there is no elegant way to have 
> > different levels for different languages. 
> > Would it be possible to define treesit-font-lock-level as a buffer local?
> 
> Is it a problem for you to do that in your init file?

No, it's not a problem.

> Also, we could support a value of that variable which is a list, like
> we do with font-lock-maximum-decoration.  This will be more useful
> than making the variable buffer-local, since presumably your
> preferences are per-mode, not per-buffer.  
Yes, this would be a better solution.

> Yuan, WDYT?
> 

V.

Re: Define treesit-font-lock-level as buffer local

[Yuan Fu]

> On Nov 7, 2024, at 3:08 AM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> In data giovedì 7 novembre 2024 12:03:49 Ora standard dell’Europa centrale, Eli Zaretskii ha scritto:
>>> From: Vincenzo Pupillo <v.pupillo@gmail.com>
>>> Date: Thu, 07 Nov 2024 11:30:03 +0100
>>> 
>>> Ciao,
>>> I use tree-sitter for different languages, for some of them I prefer font lock level 3, for others I prefer level 4. 
>>> treesit-font-lock-level, however, is not defined as buffer local and so there is no elegant way to have 
>>> different levels for different languages. 
>>> Would it be possible to define treesit-font-lock-level as a buffer local?
>> 
>> Is it a problem for you to do that in your init file?
> 
> No, it's not a problem.
> 
>> Also, we could support a value of that variable which is a list, like
>> we do with font-lock-maximum-decoration.  This will be more useful
>> than making the variable buffer-local, since presumably your
>> preferences are per-mode, not per-buffer.  
> Yes, this would be a better solution.
> 
>> Yuan, WDYT?
>> 

Perhaps even per-language, for multi-language modes. I think it’s a valid use case. IMO, specify the level by language is better than mode. For single language modes, using the language is equivalent to using the mode; for multi-language modes, using the language allows more flexibility. 

Yuan

Re: Define treesit-font-lock-level as buffer local

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Sat, 9 Nov 2024 00:36:31 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  emacs-devel@gnu.org
> 
> >> Also, we could support a value of that variable which is a list, like
> >> we do with font-lock-maximum-decoration.  This will be more useful
> >> than making the variable buffer-local, since presumably your
> >> preferences are per-mode, not per-buffer.  
> > Yes, this would be a better solution.
> > 
> >> Yuan, WDYT?
> >> 
> 
> Perhaps even per-language, for multi-language modes. I think it’s a valid use case. IMO, specify the level by language is better than mode. For single language modes, using the language is equivalent to using the mode; for multi-language modes, using the language allows more flexibility. 

Patches welcome.

Re: Define treesit-font-lock-level as buffer local

[Yuan Fu]

> On Nov 9, 2024, at 12:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sat, 9 Nov 2024 00:36:31 -0800
>> Cc: Eli Zaretskii <eliz@gnu.org>,
>> emacs-devel@gnu.org
>> 
>>>> Also, we could support a value of that variable which is a list, like
>>>> we do with font-lock-maximum-decoration.  This will be more useful
>>>> than making the variable buffer-local, since presumably your
>>>> preferences are per-mode, not per-buffer.  
>>> Yes, this would be a better solution.
>>> 
>>>> Yuan, WDYT?
>>>> 
>> 
>> Perhaps even per-language, for multi-language modes. I think it’s a valid use case. IMO, specify the level by language is better than mode. For single language modes, using the language is equivalent to using the mode; for multi-language modes, using the language allows more flexibility. 
> 
> Patches welcome.

I’ll prepare one soon.

Yuan

Re: Define treesit-font-lock-level as buffer local

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Sun, 10 Nov 2024 00:04:00 -0800
> Cc: Vincenzo Pupillo <v.pupillo@gmail.com>,
>  emacs-devel@gnu.org
> 
> 
> 
> > On Nov 9, 2024, at 12:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri@gmail.com>
> >> Date: Sat, 9 Nov 2024 00:36:31 -0800
> >> Cc: Eli Zaretskii <eliz@gnu.org>,
> >> emacs-devel@gnu.org
> >> 
> >>>> Also, we could support a value of that variable which is a list, like
> >>>> we do with font-lock-maximum-decoration.  This will be more useful
> >>>> than making the variable buffer-local, since presumably your
> >>>> preferences are per-mode, not per-buffer.  
> >>> Yes, this would be a better solution.
> >>> 
> >>>> Yuan, WDYT?
> >>>> 
> >> 
> >> Perhaps even per-language, for multi-language modes. I think it’s a valid use case. IMO, specify the level by language is better than mode. For single language modes, using the language is equivalent to using the mode; for multi-language modes, using the language allows more flexibility. 
> > 
> > Patches welcome.
> 
> I’ll prepare one soon.

Did you have time to work on this?

Tree-sitter central configuration variable

[Yuan Fu]

> On Nov 23, 2024, at 4:20 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sun, 10 Nov 2024 00:04:00 -0800
>> Cc: Vincenzo Pupillo <v.pupillo@gmail.com>,
>> emacs-devel@gnu.org
>> 
>> 
>> 
>>> On Nov 9, 2024, at 12:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>>> From: Yuan Fu <casouri@gmail.com>
>>>> Date: Sat, 9 Nov 2024 00:36:31 -0800
>>>> Cc: Eli Zaretskii <eliz@gnu.org>,
>>>> emacs-devel@gnu.org
>>>> 
>>>>>> Also, we could support a value of that variable which is a list, like
>>>>>> we do with font-lock-maximum-decoration.  This will be more useful
>>>>>> than making the variable buffer-local, since presumably your
>>>>>> preferences are per-mode, not per-buffer.  
>>>>> Yes, this would be a better solution.
>>>>> 
>>>>>> Yuan, WDYT?
>>>>>> 
>>>> 
>>>> Perhaps even per-language, for multi-language modes. I think it’s a valid use case. IMO, specify the level by language is better than mode. For single language modes, using the language is equivalent to using the mode; for multi-language modes, using the language allows more flexibility. 
>>> 
>>> Patches welcome.
>> 
>> I’ll prepare one soon.
> 
> Did you have time to work on this?

Actually, I want to expand this to something that allows users to configure tree-sitter modes and toggle on/off tree-sitter features.

Since Emacs 29, I see many people ask about how to configure tree-sitter modes by setting some variable. It seems that people much prefer setting a variable than adding a major mode hook that calls some functions. Also, admittedly adding custom font-lock or indent rules aren’t very straightforward for users.

In Emacs 29, we went with the major-mode hook approach for customization since the major mode inheritance situation wasn’t yet clear, the hook approach provides most flexibility, and we don’t really know what we want. But at this point I think we can add another layer of convenience. As long as this convenience layer handles 90% of the use-cases and doesn’t add any confusion, it’ll be a net-gain.

What do you guys think about something like this:

(setq treesit-global-configuration
      '((c-ts-mode
         ;; Set treesit-font-lock-level to 4
         (font-lock-level . 4)
         ;; Disable tree-sitter’s outline support
         (outline . disable)
         ;; Enable these features on top of the default ones.
         (font-lock-enable . (function property variable))
         ;; Disable these features.
         (font-lock-disable . (definition))
         ;; Add extra font-lock rules
         (font-lock-extra-rules
          ( :feature 'my-rules
            :language 'c
            ((some_query) @some-face)))
         (simple-indent-extra-rules
          (c
           (matcher anchor offset))
          (d
           (matcher anchor offset)))
         )))

This config will apply to c-ts-mode or its derived mode. Users can add extra font-lock and indent rules by setting this variable.

One thing I don’t like is how it handles languages. In this POC language-specific settings are nested under the mode. I’m ok with mode-language hierarchy, but the nesting adds a lot of nesting levels to the variable. And the language nesting isn’t consistent, some settings have language nesting, some don’t.

Implementation wise, it shouldn’t be difficult. We just make treesit-major-mode-setup aware of this variable and use it to override stuff when setting things up.

Yuan

Re: Tree-sitter central configuration variable

[Eshel Yaron]

Hi,

Yuan Fu <casouri@gmail.com> writes:

> Actually, I want to expand this to something that allows users to
> configure tree-sitter modes and toggle on/off tree-sitter features.
>
[...]
>
> What do you guys think about something like this:
>
> (setq treesit-global-configuration
>       '((c-ts-mode
>          ;; Set treesit-font-lock-level to 4
>          (font-lock-level . 4)
[...]
>          )))
>

FWIW, I think tree-sitter should be treated as an implementation detail,
not a user-facing concept.  Tree-sitter empowers major mode developers,
but users shouldn't have to know or care about tree-sitter to benefit
from it.  So a TS-specific configuration user option is not the best
approach IMO.  (For the same reason, I'm not a big fan of the "powered
by tree-sitter" that we have today in the docstring of various modes.)

Instead, when it comes to settings that apply across major modes, I
think we should focus on knobs that make sense whether or not a mode is
TS-based, and have the TS-based modes respect such universal settings.


Just my 2c,

Eshel

Re: Tree-sitter central configuration variable

[Yuan Fu]

> On Nov 28, 2024, at 11:12 PM, Eshel Yaron <me@eshelyaron.com> wrote:
> 
> Hi,
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>> Actually, I want to expand this to something that allows users to
>> configure tree-sitter modes and toggle on/off tree-sitter features.
>> 
> [...]
>> 
>> What do you guys think about something like this:
>> 
>> (setq treesit-global-configuration
>>      '((c-ts-mode
>>         ;; Set treesit-font-lock-level to 4
>>         (font-lock-level . 4)
> [...]
>>         )))
>> 
> 
> FWIW, I think tree-sitter should be treated as an implementation detail,
> not a user-facing concept.  Tree-sitter empowers major mode developers,
> but users shouldn't have to know or care about tree-sitter to benefit
> from it.  So a TS-specific configuration user option is not the best
> approach IMO.  (For the same reason, I'm not a big fan of the "powered
> by tree-sitter" that we have today in the docstring of various modes.)

I principle, I agree, and that’s what we’ve been doing: we expose tree-sitter as a mechanism for major modes to build on. However, the reality is, something like this just aren’t going to be transparent—users know it’s powered by tree-sitter, they’ll need to have tree-sitter knowledge for non-trivial customizations (eg, add a font-lock rule or indent rule), and the major mode is tied-in with tree-sitter, implementing the major mode with another parser library would require creating a new major mode.

Once you accept that these modes are tied with tree-sitter, a TS-specific configuration starts to seem pretty reasonable. Also, think of the advantage of using a single set of configuration: no need to read the manual and learn the variables etc for each individual major modes; as long as the mode uses tree-sitter features, you learn the configuration framework once and use it on every mode.

> Instead, when it comes to settings that apply across major modes, I
> think we should focus on knobs that make sense whether or not a mode is
> TS-based, and have the TS-based modes respect such universal settings.

Perhaps you mean something that configures things like font-lock, imenu, etc? (I can’t really think of a third example). That’s a bit ambitious, the return is unclear to me, and gods bless whoever wants to fight through emacs-devel for it :-) OTOH, a tree-sitter configuration variable is pretty concrete, I know people want it, and more or less noncontroversial (I hope).

Yuan

Re: Tree-sitter central configuration variable

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Thu, 28 Nov 2024 21:49:33 -0800
> Cc: Vincenzo Pupillo <v.pupillo@gmail.com>,
>  Emacs Devel <emacs-devel@gnu.org>,
>  Stefan Kangas <stefankangas@gmail.com>,
>  Stefan Monnier <monnier@iro.umontreal.ca>,
>  Dmitry Gutov <dmitry@gutov.dev>
> 
> Actually, I want to expand this to something that allows users to configure tree-sitter modes and toggle on/off tree-sitter features.
> 
> Since Emacs 29, I see many people ask about how to configure tree-sitter modes by setting some variable. It seems that people much prefer setting a variable than adding a major mode hook that calls some functions. Also, admittedly adding custom font-lock or indent rules aren’t very straightforward for users.
> 
> In Emacs 29, we went with the major-mode hook approach for customization since the major mode inheritance situation wasn’t yet clear, the hook approach provides most flexibility, and we don’t really know what we want. But at this point I think we can add another layer of convenience. As long as this convenience layer handles 90% of the use-cases and doesn’t add any confusion, it’ll be a net-gain.

We are discussing the solutions to these issues in this thread:

  https://lists.gnu.org/archive/html/emacs-devel/2024-11/msg00636.html

Feel free to chime in and state your opinions about the solutions
being proposed there.

> What do you guys think about something like this:
> 
> (setq treesit-global-configuration
>       '((c-ts-mode
>          ;; Set treesit-font-lock-level to 4
>          (font-lock-level . 4)
>          ;; Disable tree-sitter’s outline support
>          (outline . disable)
>          ;; Enable these features on top of the default ones.
>          (font-lock-enable . (function property variable))
>          ;; Disable these features.
>          (font-lock-disable . (definition))
>          ;; Add extra font-lock rules
>          (font-lock-extra-rules
>           ( :feature 'my-rules
>             :language 'c
>             ((some_query) @some-face)))
>          (simple-indent-extra-rules
>           (c
>            (matcher anchor offset))
>           (d
>            (matcher anchor offset)))
>          )))
> 
> This config will apply to c-ts-mode or its derived mode. Users can add extra font-lock and indent rules by setting this variable.

I'm not sure I understand what problem(s) this intends to solve?  Why
should we have a global variable that customizes several different
modes, if the specific customizations made by that variable depend on
the individual modes?

> One thing I don’t like is how it handles languages. In this POC language-specific settings are nested under the mode. I’m ok with mode-language hierarchy, but the nesting adds a lot of nesting levels to the variable. And the language nesting isn’t consistent, some settings have language nesting, some don’t.

We don't have a notion of "language", we only have major modes.  Emacs
30 makes it easier to specify settings common to modes that serve the
same "language" by introducing the notion of "extra parent" modes.
But that is only a small half-step; I don't think we have figured out
what should be the goal.

Re: Tree-sitter central configuration variable

[Yuan Fu]

> On Nov 29, 2024, at 12:04 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Thu, 28 Nov 2024 21:49:33 -0800
>> Cc: Vincenzo Pupillo <v.pupillo@gmail.com>,
>> Emacs Devel <emacs-devel@gnu.org>,
>> Stefan Kangas <stefankangas@gmail.com>,
>> Stefan Monnier <monnier@iro.umontreal.ca>,
>> Dmitry Gutov <dmitry@gutov.dev>
>> 
>> Actually, I want to expand this to something that allows users to configure tree-sitter modes and toggle on/off tree-sitter features.
>> 
>> Since Emacs 29, I see many people ask about how to configure tree-sitter modes by setting some variable. It seems that people much prefer setting a variable than adding a major mode hook that calls some functions. Also, admittedly adding custom font-lock or indent rules aren’t very straightforward for users.
>> 
>> In Emacs 29, we went with the major-mode hook approach for customization since the major mode inheritance situation wasn’t yet clear, the hook approach provides most flexibility, and we don’t really know what we want. But at this point I think we can add another layer of convenience. As long as this convenience layer handles 90% of the use-cases and doesn’t add any confusion, it’ll be a net-gain.
> 
> We are discussing the solutions to these issues in this thread:
> 
>  https://lists.gnu.org/archive/html/emacs-devel/2024-11/msg00636.html
> 
> Feel free to chime in and state your opinions about the solutions
> being proposed there.

That thread is about turning tree-sitter modes on and off. I think Dmitry’s proposal seems pretty good, and I don’t have much to add. What I’m proposing here is rather about configuring tree-sitter modes: turning features (outline, imenu, etc) on and off, customizing things (font-lock, indent, etc).

> 
>> What do you guys think about something like this:
>> 
>> (setq treesit-global-configuration
>>      '((c-ts-mode
>>         ;; Set treesit-font-lock-level to 4
>>         (font-lock-level . 4)
>>         ;; Disable tree-sitter’s outline support
>>         (outline . disable)
>>         ;; Enable these features on top of the default ones.
>>         (font-lock-enable . (function property variable))
>>         ;; Disable these features.
>>         (font-lock-disable . (definition))
>>         ;; Add extra font-lock rules
>>         (font-lock-extra-rules
>>          ( :feature 'my-rules
>>            :language 'c
>>            ((some_query) @some-face)))
>>         (simple-indent-extra-rules
>>          (c
>>           (matcher anchor offset))
>>          (d
>>           (matcher anchor offset)))
>>         )))
>> 
>> This config will apply to c-ts-mode or its derived mode. Users can add extra font-lock and indent rules by setting this variable.
> 
> I'm not sure I understand what problem(s) this intends to solve?  Why
> should we have a global variable that customizes several different
> modes, if the specific customizations made by that variable depend on
> the individual modes?

I’m trying to solve several things: first off, turning on/off some tree-sitter features. Originally someone wants to turn off tree-sitter mode’s outline support. Technically they don’t really need to turn of tree-sitter’s outline, they just need to make the minor mode they’re using override tree-sitter’s outline configuration completely. Nevertheless, wanting to turn off a tree-sitter feature seems a reasonable ask so I want to support it.

Secondly, many people wants to configure tree-sitter modes by setting a variable. Right now, if one needs to add/remove some font-lock features they need to do this:

(defun x-mode-setup ()
  (treesit-font-lock-recompute-features '(feature-a) '(feature-b)))
(add-hook 'x-ts-mode-hook #'x-mode-setup)

If they want to add some custom font-lock rules:

(defun x-mode-setup ()
  (setq treesit-font-lock-settings
        (append treesit-font-lock-settings
                (treesit-font-lock-rules
                 :feature 'custom-feature
                 :language 'lang
                 ((some_node) @face))))
  (treesit-font-lock-recompute-features))
(add-hook 'x-ts-mode-hook #'x-mode-setup)

That might be a bit convoluted. I’m hoping that I can simplify things by the proposed variable.

> 
>> One thing I don’t like is how it handles languages. In this POC language-specific settings are nested under the mode. I’m ok with mode-language hierarchy, but the nesting adds a lot of nesting levels to the variable. And the language nesting isn’t consistent, some settings have language nesting, some don’t.
> 
> We don't have a notion of "language", we only have major modes.  Emacs
> 30 makes it easier to specify settings common to modes that serve the
> same "language" by introducing the notion of "extra parent" modes.
> But that is only a small half-step; I don't think we have figured out
> what should be the goal.

We’re talking about different things. The language I was talking about are the languages in a multi-language modes. For a HTML mode, you’ll have HTML, CSS, and Javascript, and the we might want to configure the three languages differently in the HTML major mode. Eg, add a custom indent rule for CSS in the mode.

Yuan

Re: Tree-sitter central configuration variable

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Fri, 29 Nov 2024 01:07:37 -0800
> Cc: Vincenzo Pupillo <v.pupillo@gmail.com>,
>  Emacs Devel <emacs-devel@gnu.org>,
>  Stefan Kangas <stefankangas@gmail.com>,
>  Stefan Monnier <monnier@iro.umontreal.ca>,
>  dmitry@gutov.dev
> 
> >> (setq treesit-global-configuration
> >>      '((c-ts-mode
> >>         ;; Set treesit-font-lock-level to 4
> >>         (font-lock-level . 4)
> >>         ;; Disable tree-sitter’s outline support
> >>         (outline . disable)
> >>         ;; Enable these features on top of the default ones.
> >>         (font-lock-enable . (function property variable))
> >>         ;; Disable these features.
> >>         (font-lock-disable . (definition))
> >>         ;; Add extra font-lock rules
> >>         (font-lock-extra-rules
> >>          ( :feature 'my-rules
> >>            :language 'c
> >>            ((some_query) @some-face)))
> >>         (simple-indent-extra-rules
> >>          (c
> >>           (matcher anchor offset))
> >>          (d
> >>           (matcher anchor offset)))
> >>         )))
> >> 
> >> This config will apply to c-ts-mode or its derived mode. Users can add extra font-lock and indent rules by setting this variable.
> > 
> > I'm not sure I understand what problem(s) this intends to solve?  Why
> > should we have a global variable that customizes several different
> > modes, if the specific customizations made by that variable depend on
> > the individual modes?
> 
> I’m trying to solve several things: first off, turning on/off some tree-sitter features. Originally someone wants to turn off tree-sitter mode’s outline support. Technically they don’t really need to turn of tree-sitter’s outline, they just need to make the minor mode they’re using override tree-sitter’s outline configuration completely. Nevertheless, wanting to turn off a tree-sitter feature seems a reasonable ask so I want to support it.
> 
> Secondly, many people wants to configure tree-sitter modes by setting a variable. Right now, if one needs to add/remove some font-lock features they need to do this:
> 
> (defun x-mode-setup ()
>   (treesit-font-lock-recompute-features '(feature-a) '(feature-b)))
> (add-hook 'x-ts-mode-hook #'x-mode-setup)
> 
> If they want to add some custom font-lock rules:
> 
> (defun x-mode-setup ()
>   (setq treesit-font-lock-settings
>         (append treesit-font-lock-settings
>                 (treesit-font-lock-rules
>                  :feature 'custom-feature
>                  :language 'lang
>                  ((some_node) @face))))
>   (treesit-font-lock-recompute-features))
> (add-hook 'x-ts-mode-hook #'x-mode-setup)
> 
> That might be a bit convoluted. I’m hoping that I can simplify things by the proposed variable.

But we didn't support any such features with the "traditional" regexp-
and syntax-based fontifications, so why would tree-sitter based modes
be special in this regard?  Are you saying that tree-sitter based
modes somehow need to update rules for fontification, indentation, and
other stuff much more frequently than the "traditional" modes?  If so,
why?

Btw, I don't see the above hooks more convoluted than your
treesit-global-configuration variable.  They both need a good
understanding of tree-sitter grammars and of Lisp.

> > We don't have a notion of "language", we only have major modes.  Emacs
> > 30 makes it easier to specify settings common to modes that serve the
> > same "language" by introducing the notion of "extra parent" modes.
> > But that is only a small half-step; I don't think we have figured out
> > what should be the goal.
> 
> We’re talking about different things. The language I was talking about are the languages in a multi-language modes. For a HTML mode, you’ll have HTML, CSS, and Javascript, and the we might want to configure the three languages differently in the HTML major mode. Eg, add a custom indent rule for CSS in the mode.

How would the variable you propose help in doing that?  IOW, I guess I
don't see clearly enough the problem you want to solve in this case
and the solution you are proposing for that.  Please elaborate.

Re: Tree-sitter central configuration variable

[Stefan Monnier]

> Since Emacs 29, I see many people ask about how to configure
> tree-sitter modes by setting some variable.

This is not specific to tree-sitter.

> It seems that people much prefer setting a variable than adding
> a major mode hook that calls some functions.

But that boxes them into the subset of possibilities that have been
pre-imagined by those who designed the set of variables.

> What do you guys think about something like this:
>
> (setq treesit-global-configuration
>       '((c-ts-mode
>          ;; Set treesit-font-lock-level to 4
>          (font-lock-level . 4)
>          ;; Disable tree-sitter’s outline support
>          (outline . disable)
>          ;; Enable these features on top of the default ones.
>          (font-lock-enable . (function property variable))
>          ;; Disable these features.
>          (font-lock-disable . (definition))
>          ;; Add extra font-lock rules
>          (font-lock-extra-rules
>           ( :feature 'my-rules
>             :language 'c
>             ((some_query) @some-face)))
>          (simple-indent-extra-rules
>           (c
>            (matcher anchor offset))
>           (d
>            (matcher anchor offset)))
>          )))

Sounds to me like this is inventing a new programming language, just one
that's a lot more restrictive than ELisp.

Is it really better than ELisp which could look like:

    (add-hook 'c-ts-mode-hook #'my-c-ts-mode-preferences)
    (defun my-c-ts-mode-preferences ()
      (setq-local treesit-font-lock-level 4)
      (treesit-outline-mode -1)
      (treesit-font-lock-enable '(function property variable))
      (treesit-font-lock-disable '(definition))
      (treesit-font-lock-add-rules
       '( :feature 'my-rules
          :language 'c
          ((some_query) @some-face)))
      (treesit-indent-add-rules
       '((c
          (matcher anchor offset))
         (d
          (matcher anchor offset)))
      ))

Admittedly, the add-hook/defun dance could be improved, and that would
benefit more than just tree-sitter.  E.g. a new macro like

    (defmacro custom-set-hook (hook &rest body)
      (declare (indent 1) (debug (sexp def-body)))
      (let ((funname (intern (format "custom-set-hook--%s" hook))))
        `(progn (add-hook ',hook #',funnmame)
                (defun ,funname () ,@body))))

> One thing I don’t like is how it handles languages. In this POC
> language-specific settings are nested under the mode. I’m ok with
> mode-language hierarchy, but the nesting adds a lot of nesting levels
> to the variable.  And the language nesting isn’t consistent, some
> settings have language nesting, some don’t.

I don't really understand what you're referring to, but for
`treesit-font-lock-add-rules` would could try and auto-add the `:language`?


        Stefan

Re: Tree-sitter central configuration variable

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 3629 bytes --]

Chiming in with my minor voice. I prefer my mode configurations generally
self contained and not centralized as a single global configuration
variable would encourage. When I share my configurations with others, it
also suggests that if they adopt a global configuration variable on top of
what I share, it will introduce subtle changes in behavior that I may not
at first understand the source of when helping others. I definitely prefer
proper lisp over an opaque configuration variable.

-Stephane

On Fri, Nov 29, 2024 at 12:04 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > Since Emacs 29, I see many people ask about how to configure
> > tree-sitter modes by setting some variable.
>
> This is not specific to tree-sitter.
>
> > It seems that people much prefer setting a variable than adding
> > a major mode hook that calls some functions.
>
> But that boxes them into the subset of possibilities that have been
> pre-imagined by those who designed the set of variables.
>
> > What do you guys think about something like this:
> >
> > (setq treesit-global-configuration
> >       '((c-ts-mode
> >          ;; Set treesit-font-lock-level to 4
> >          (font-lock-level . 4)
> >          ;; Disable tree-sitter’s outline support
> >          (outline . disable)
> >          ;; Enable these features on top of the default ones.
> >          (font-lock-enable . (function property variable))
> >          ;; Disable these features.
> >          (font-lock-disable . (definition))
> >          ;; Add extra font-lock rules
> >          (font-lock-extra-rules
> >           ( :feature 'my-rules
> >             :language 'c
> >             ((some_query) @some-face)))
> >          (simple-indent-extra-rules
> >           (c
> >            (matcher anchor offset))
> >           (d
> >            (matcher anchor offset)))
> >          )))
>
> Sounds to me like this is inventing a new programming language, just one
> that's a lot more restrictive than ELisp.
>
> Is it really better than ELisp which could look like:
>
>     (add-hook 'c-ts-mode-hook #'my-c-ts-mode-preferences)
>     (defun my-c-ts-mode-preferences ()
>       (setq-local treesit-font-lock-level 4)
>       (treesit-outline-mode -1)
>       (treesit-font-lock-enable '(function property variable))
>       (treesit-font-lock-disable '(definition))
>       (treesit-font-lock-add-rules
>        '( :feature 'my-rules
>           :language 'c
>           ((some_query) @some-face)))
>       (treesit-indent-add-rules
>        '((c
>           (matcher anchor offset))
>          (d
>           (matcher anchor offset)))
>       ))
>
> Admittedly, the add-hook/defun dance could be improved, and that would
> benefit more than just tree-sitter.  E.g. a new macro like
>
>     (defmacro custom-set-hook (hook &rest body)
>       (declare (indent 1) (debug (sexp def-body)))
>       (let ((funname (intern (format "custom-set-hook--%s" hook))))
>         `(progn (add-hook ',hook #',funnmame)
>                 (defun ,funname () ,@body))))
>
> > One thing I don’t like is how it handles languages. In this POC
> > language-specific settings are nested under the mode. I’m ok with
> > mode-language hierarchy, but the nesting adds a lot of nesting levels
> > to the variable.  And the language nesting isn’t consistent, some
> > settings have language nesting, some don’t.
>
> I don't really understand what you're referring to, but for
> `treesit-font-lock-add-rules` would could try and auto-add the `:language`?
>
>
>         Stefan
>
>
>

[-- Attachment #2: Type: text/html, Size: 4649 bytes --]

Re: Tree-sitter central configuration variable

[Yuan Fu]

> On Nov 29, 2024, at 9:22 AM, Ship Mints <shipmints@gmail.com> wrote:
> 
> Chiming in with my minor voice. I prefer my mode configurations generally self contained and not centralized as a single global configuration variable would encourage. When I share my configurations with others, it also suggests that if they adopt a global configuration variable on top of what I share, it will introduce subtle changes in behavior that I may not at first understand the source of when helping others. I definitely prefer proper lisp over an opaque configuration variable.
> 
> -Stephane
> 
> On Fri, Nov 29, 2024 at 12:04 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > Since Emacs 29, I see many people ask about how to configure
> > tree-sitter modes by setting some variable.
> 
> This is not specific to tree-sitter.
> 
> > It seems that people much prefer setting a variable than adding
> > a major mode hook that calls some functions.
> 
> But that boxes them into the subset of possibilities that have been
> pre-imagined by those who designed the set of variables.
> 
> > What do you guys think about something like this:
> >
> > (setq treesit-global-configuration
> >       '((c-ts-mode
> >          ;; Set treesit-font-lock-level to 4
> >          (font-lock-level . 4)
> >          ;; Disable tree-sitter’s outline support
> >          (outline . disable)
> >          ;; Enable these features on top of the default ones.
> >          (font-lock-enable . (function property variable))
> >          ;; Disable these features.
> >          (font-lock-disable . (definition))
> >          ;; Add extra font-lock rules
> >          (font-lock-extra-rules
> >           ( :feature 'my-rules
> >             :language 'c
> >             ((some_query) @some-face)))
> >          (simple-indent-extra-rules
> >           (c
> >            (matcher anchor offset))
> >           (d
> >            (matcher anchor offset)))
> >          )))
> 
> Sounds to me like this is inventing a new programming language, just one
> that's a lot more restrictive than ELisp.
> 
> Is it really better than ELisp which could look like:
> 
>     (add-hook 'c-ts-mode-hook #'my-c-ts-mode-preferences)
>     (defun my-c-ts-mode-preferences ()
>       (setq-local treesit-font-lock-level 4)
>       (treesit-outline-mode -1)
>       (treesit-font-lock-enable '(function property variable))
>       (treesit-font-lock-disable '(definition))
>       (treesit-font-lock-add-rules
>        '( :feature 'my-rules
>           :language 'c
>           ((some_query) @some-face)))
>       (treesit-indent-add-rules
>        '((c
>           (matcher anchor offset))
>          (d
>           (matcher anchor offset)))
>       ))
> 
> Admittedly, the add-hook/defun dance could be improved, and that would
> benefit more than just tree-sitter.  E.g. a new macro like
> 
>     (defmacro custom-set-hook (hook &rest body)
>       (declare (indent 1) (debug (sexp def-body)))
>       (let ((funname (intern (format "custom-set-hook--%s" hook))))
>         `(progn (add-hook ',hook #',funnmame)
>                 (defun ,funname () ,@body))))
> 
> > One thing I don’t like is how it handles languages. In this POC
> > language-specific settings are nested under the mode. I’m ok with
> > mode-language hierarchy, but the nesting adds a lot of nesting levels
> > to the variable.  And the language nesting isn’t consistent, some
> > settings have language nesting, some don’t.
> 
> I don't really understand what you're referring to, but for
> `treesit-font-lock-add-rules` would could try and auto-add the `:language`?
> 
> 
>         Stefan
> 
> 

It seems that we’re getting a resounding no here, so I’m gonna drop this :) On a side note, I was planning to add a convenience function for adding font-lock rules, so users don’t need to do the append dance. Turns out I’ve already written one but then forgot about it :)

Maybe what we really need is a good guide/manual section on installing and configuring tree-sitter major modes. I wrote a detailed guide for writing tree-sitter major modes, but didn’t write one for installing and configuring tree-sitter major modes.

Yuan

Re: Tree-sitter central configuration variable

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Mon, 2 Dec 2024 22:40:56 -0800
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Eli Zaretskii <eliz@gnu.org>,
>  Vincenzo Pupillo <v.pupillo@gmail.com>, Emacs Devel <emacs-devel@gnu.org>,
>  Stefan Kangas <stefankangas@gmail.com>, Dmitry Gutov <dmitry@gutov.dev>
> 
> Maybe what we really need is a good guide/manual section on installing and configuring tree-sitter major modes. I wrote a detailed guide for writing tree-sitter major modes, but didn’t write one for installing and configuring tree-sitter major modes.

We might consider doing that once we figure out what is the best way
of installing and configuring these modes ;-)  For now, we are
discussing the different alternatives for doing so, and have not yet
reached an agreement, AFAIU, let alone implemented it.

Re: Define treesit-font-lock-level as buffer local

[Eli Zaretskii]

Ping!

> Date: Sat, 23 Nov 2024 14:20:28 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: v.pupillo@gmail.com, emacs-devel@gnu.org
> 
> > From: Yuan Fu <casouri@gmail.com>
> > Date: Sun, 10 Nov 2024 00:04:00 -0800
> > Cc: Vincenzo Pupillo <v.pupillo@gmail.com>,
> >  emacs-devel@gnu.org
> > 
> > 
> > 
> > > On Nov 9, 2024, at 12:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > > 
> > >> From: Yuan Fu <casouri@gmail.com>
> > >> Date: Sat, 9 Nov 2024 00:36:31 -0800
> > >> Cc: Eli Zaretskii <eliz@gnu.org>,
> > >> emacs-devel@gnu.org
> > >> 
> > >>>> Also, we could support a value of that variable which is a list, like
> > >>>> we do with font-lock-maximum-decoration.  This will be more useful
> > >>>> than making the variable buffer-local, since presumably your
> > >>>> preferences are per-mode, not per-buffer.  
> > >>> Yes, this would be a better solution.
> > >>> 
> > >>>> Yuan, WDYT?
> > >>>> 
> > >> 
> > >> Perhaps even per-language, for multi-language modes. I think it’s a valid use case. IMO, specify the level by language is better than mode. For single language modes, using the language is equivalent to using the mode; for multi-language modes, using the language allows more flexibility. 
> > > 
> > > Patches welcome.
> > 
> > I’ll prepare one soon.
> 
> Did you have time to work on this?
> 
> 

Re: Define treesit-font-lock-level as buffer local

[Yuan Fu]

> On Dec 7, 2024, at 4:04 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> Ping!
> 
>> Date: Sat, 23 Nov 2024 14:20:28 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: v.pupillo@gmail.com, emacs-devel@gnu.org
>> 
>>> From: Yuan Fu <casouri@gmail.com>
>>> Date: Sun, 10 Nov 2024 00:04:00 -0800
>>> Cc: Vincenzo Pupillo <v.pupillo@gmail.com>,
>>> emacs-devel@gnu.org
>>> 
>>> 
>>> 
>>>> On Nov 9, 2024, at 12:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>>> 
>>>>> From: Yuan Fu <casouri@gmail.com>
>>>>> Date: Sat, 9 Nov 2024 00:36:31 -0800
>>>>> Cc: Eli Zaretskii <eliz@gnu.org>,
>>>>> emacs-devel@gnu.org
>>>>> 
>>>>>>> Also, we could support a value of that variable which is a list, like
>>>>>>> we do with font-lock-maximum-decoration.  This will be more useful
>>>>>>> than making the variable buffer-local, since presumably your
>>>>>>> preferences are per-mode, not per-buffer.  
>>>>>> Yes, this would be a better solution.
>>>>>> 
>>>>>>> Yuan, WDYT?
>>>>>>> 
>>>>> 
>>>>> Perhaps even per-language, for multi-language modes. I think it’s a valid use case. IMO, specify the level by language is better than mode. For single language modes, using the language is equivalent to using the mode; for multi-language modes, using the language allows more flexibility.
>>>> 
>>>> Patches welcome.
>>> 
>>> I’ll prepare one soon.
>> 
>> Did you have time to work on this?

Ok, following the recent discussion, I implemented the original idea, which is an alist mapping major modes to font lock levels. The check is made with derived-mode-p, so c-mode should work for c-ts-mode too, but I still used c-ts-mode in the example in the docstring since I don’t want to complicate it. The change is pushed to master.

Yuan

Re: Define treesit-font-lock-level as buffer local

[Vincenzo Pupillo]

In data domenica 8 dicembre 2024 08:44:25 Ora standard dell’Europa centrale, 
Yuan Fu ha scritto:
> 
> Ok, following the recent discussion, I implemented the original idea, which
> is an alist mapping major modes to font lock levels. The check is made with
> derived-mode-p, so c-mode should work for c-ts-mode too, but I still used
> c-ts-mode in the example in the docstring since I don’t want to complicate
> it. The change is pushed to master.
> 
> Yuan

Thank you Yuan!

Vincenzo

Re: Define treesit-font-lock-level as buffer local

[Vincenzo Pupillo]

[-- Attachment #1: Type: text/plain, Size: 818 bytes --]

Ciao Yuan,
I found a bug in python-ts-mode, it does not handle treesit-font-lock-level 
even as a list. The attached patch fixes this bug.

Thank you.

Vincenzo

In data sabato 14 dicembre 2024 12:14:02 Ora standard dell’Europa centrale, 
Vincenzo Pupillo ha scritto:
> In data domenica 8 dicembre 2024 08:44:25 Ora standard dell’Europa centrale,
> Yuan Fu ha scritto:
> > Ok, following the recent discussion, I implemented the original idea,
> > which
> > is an alist mapping major modes to font lock levels. The check is made
> > with
> > derived-mode-p, so c-mode should work for c-ts-mode too, but I still used
> > c-ts-mode in the example in the docstring since I don’t want to complicate
> > it. The change is pushed to master.
> > 
> > Yuan
> 
> Thank you Yuan!
> 
> Vincenzo



[-- Attachment #2: 0001-python-ts-mode-now-handles-the-treesit-font-lock-lev.patch --]
[-- Type: text/x-patch, Size: 1456 bytes --]

From ec63146bfcb54c78af7fac3cb0791b94cb1a43ca Mon Sep 17 00:00:00 2001
From: Vincenzo Pupillo <vincenzo.pupillo@unimi.it>
Date: Tue, 17 Dec 2024 12:14:29 +0100
Subject: [PATCH] python-ts-mode now handles the treesit-font-lock-level
 correctly.

'treesit-font-lock-level' can be a number or a list, it should be
handled using 'treesit--compute-font-lock-level'.

* lisp/progmodes/python.el (python--treesit-fontify-string): Apply
'treesit--compute-font-lock-level' to 'treesit-font-lock-level'.
---
 lisp/progmodes/python.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index dca0936c826..de1c6f551c4 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -1108,7 +1108,8 @@ python--treesit-fontify-string
          (ignore-interpolation (not
                                 (seq-some
                                  (lambda (feats) (memq 'string-interpolation feats))
-                                 (seq-take treesit-font-lock-feature-list treesit-font-lock-level))))
+                                 (seq-take treesit-font-lock-feature-list
+                                           (treesit--compute-font-lock-level treesit-font-lock-level)))))
          ;; If interpolation is enabled, highlight only
          ;; string_start/string_content/string_end children.  Do not
          ;; touch interpolation node that can occur inside of the
-- 
2.47.1

Re: Define treesit-font-lock-level as buffer local

[Yuan Fu]

> On Dec 17, 2024, at 3:31 AM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> Ciao Yuan,
> I found a bug in python-ts-mode, it does not handle treesit-font-lock-level 
> even as a list. The attached patch fixes this bug.
> 
> Thank you.
> 
> Vincenzo

Thank you! I merged your patch.

Yuan