Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.

Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3358 bytes --]


guix-commits@gnu.org writes:

> This is an automated email from the git hooks/post-receive script.
>
> abcdw pushed a commit to branch master
> in repository guix.
>
> The following commit(s) were added to refs/heads/master by this push:
>      new c77a3d4554 gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.
> c77a3d4554 is described below
>
> commit c77a3d4554175ee5320ccc713aa21b1ba3e6b8a3
> Author: Andrew Tropin <andrew@trop.in>
> AuthorDate: Fri Feb 10 12:32:12 2023 +0400
>
>     gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.
>     
>     gnu/packages/emacs.scm (emacs)[native-search-paths]: Add a search-path for
>     tree-sitter grammars.
>     gnu/packages/aux-files/emacs/guix-emacs.el: Add directories from
>     TREE_SITTER_GRAMMAR_PATH to treesit-extra-load-path.
> ---
>  gnu/packages/aux-files/emacs/guix-emacs.el |  7 +++++++
>  gnu/packages/emacs.scm                     | 10 +++++++++-
>  2 files changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el b/gnu/packages/aux-files/emacs/guix-emacs.el
> index 56dbcb8d67..708093267d 100644
> --- a/gnu/packages/aux-files/emacs/guix-emacs.el
> +++ b/gnu/packages/aux-files/emacs/guix-emacs.el
> @@ -76,6 +76,13 @@ The files in the list do not have extensions (.el, .elc)."
>                     (when (file-directory-p pkg-dir)
>                       (package-load-descriptor pkg-dir)))))))))))
>  
> +;; If emacs built with tree-sitter, read the value of the environment variable
> +;; to make tree-sitter grammars available in emacs out-of-the-box.
> +(with-eval-after-load 'treesit
> +  (when-let ((grammar-path (getenv "TREE_SITTER_GRAMMAR_PATH")))
> +    (mapcar (lambda (x) (add-to-list 'treesit-extra-load-path x))
> +            (split-string grammar-path ":"))))
> +
>  (provide 'guix-emacs)
>  
>  ;;; guix-emacs.el ends here
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 4ce41deb88..b6a66bafac 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -367,7 +367,15 @@
>              (files '("lib/emacs/native-site-lisp")))
>             (search-path-specification
>              (variable "INFOPATH")
> -            (files '("share/info")))))
> +            (files '("share/info")))
> +           ;; tree-sitter support is not yet available in emacs 28, but this
> +           ;; search path won't harm and also will be beneficial for
> +           ;; emacs-next and other emacs-* packages, which have tree-sitter
> +           ;; support enabled.  Please, remove this comment, when emacs
> +           ;; package is updated to 29.
> +           (search-path-specification
> +            (variable "TREE_SITTER_GRAMMAR_PATH")
> +            (files '("lib/tree-sitter")))))
>  
>      (home-page "https://www.gnu.org/software/emacs/")
>      (synopsis "The extensible, customizable, self-documenting text editor")

The change to guix-emacs.el is pretty impactful here since that affects
all the emacs packages including emacs-minimal I think.

If I've understood your comment correctly, that does seem pretty
wasteful since those rebuilds because of the changed emacs-minimal won't
have any affect since it's version 28 and not 29, right? I think
applying this change to just emacs-next would have been possible.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 3934 bytes --]

On 2023-02-11 11:29, Christopher Baines wrote:

> guix-commits@gnu.org writes:
>
>> This is an automated email from the git hooks/post-receive script.
>>
>> abcdw pushed a commit to branch master
>> in repository guix.
>>
>> The following commit(s) were added to refs/heads/master by this push:
>>      new c77a3d4554 gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.
>> c77a3d4554 is described below
>>
>> commit c77a3d4554175ee5320ccc713aa21b1ba3e6b8a3
>> Author: Andrew Tropin <andrew@trop.in>
>> AuthorDate: Fri Feb 10 12:32:12 2023 +0400
>>
>>     gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.
>>     
>>     gnu/packages/emacs.scm (emacs)[native-search-paths]: Add a search-path for
>>     tree-sitter grammars.
>>     gnu/packages/aux-files/emacs/guix-emacs.el: Add directories from
>>     TREE_SITTER_GRAMMAR_PATH to treesit-extra-load-path.
>> ---
>>  gnu/packages/aux-files/emacs/guix-emacs.el |  7 +++++++
>>  gnu/packages/emacs.scm                     | 10 +++++++++-
>>  2 files changed, 16 insertions(+), 1 deletion(-)
>>
>> diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el b/gnu/packages/aux-files/emacs/guix-emacs.el
>> index 56dbcb8d67..708093267d 100644
>> --- a/gnu/packages/aux-files/emacs/guix-emacs.el
>> +++ b/gnu/packages/aux-files/emacs/guix-emacs.el
>> @@ -76,6 +76,13 @@ The files in the list do not have extensions (.el, .elc)."
>>                     (when (file-directory-p pkg-dir)
>>                       (package-load-descriptor pkg-dir)))))))))))
>>  
>> +;; If emacs built with tree-sitter, read the value of the environment variable
>> +;; to make tree-sitter grammars available in emacs out-of-the-box.
>> +(with-eval-after-load 'treesit
>> +  (when-let ((grammar-path (getenv "TREE_SITTER_GRAMMAR_PATH")))
>> +    (mapcar (lambda (x) (add-to-list 'treesit-extra-load-path x))
>> +            (split-string grammar-path ":"))))
>> +
>>  (provide 'guix-emacs)
>>  
>>  ;;; guix-emacs.el ends here
>> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
>> index 4ce41deb88..b6a66bafac 100644
>> --- a/gnu/packages/emacs.scm
>> +++ b/gnu/packages/emacs.scm
>> @@ -367,7 +367,15 @@
>>              (files '("lib/emacs/native-site-lisp")))
>>             (search-path-specification
>>              (variable "INFOPATH")
>> -            (files '("share/info")))))
>> +            (files '("share/info")))
>> +           ;; tree-sitter support is not yet available in emacs 28, but this
>> +           ;; search path won't harm and also will be beneficial for
>> +           ;; emacs-next and other emacs-* packages, which have tree-sitter
>> +           ;; support enabled.  Please, remove this comment, when emacs
>> +           ;; package is updated to 29.
>> +           (search-path-specification
>> +            (variable "TREE_SITTER_GRAMMAR_PATH")
>> +            (files '("lib/tree-sitter")))))
>>  
>>      (home-page "https://www.gnu.org/software/emacs/")
>>      (synopsis "The extensible, customizable, self-documenting text editor")
>
> The change to guix-emacs.el is pretty impactful here since that affects
> all the emacs packages including emacs-minimal I think.
>
> If I've understood your comment correctly, that does seem pretty
> wasteful since those rebuilds because of the changed emacs-minimal won't
> have any affect since it's version 28 and not 29, right? I think
> applying this change to just emacs-next would have been possible.

It could be applied to emacs-next only and we could move it to emacs
later when it updated to 29, but I checked guix refresh --list-dependent
emacs and it shown less than 100 packages so I decided to apply it
straight to emacs, but I forgot that emacs-minimal is inherited from
emacs and I guess you are right and it will affect all the emacs
packages.  Anything we would like to do about it right now?

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 4259 bytes --]

On 2023-02-11 16:35, Andrew Tropin wrote:

> On 2023-02-11 11:29, Christopher Baines wrote:
>
>> guix-commits@gnu.org writes:
>>
>>> This is an automated email from the git hooks/post-receive script.
>>>
>>> abcdw pushed a commit to branch master
>>> in repository guix.
>>>
>>> The following commit(s) were added to refs/heads/master by this push:
>>>      new c77a3d4554 gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.
>>> c77a3d4554 is described below
>>>
>>> commit c77a3d4554175ee5320ccc713aa21b1ba3e6b8a3
>>> Author: Andrew Tropin <andrew@trop.in>
>>> AuthorDate: Fri Feb 10 12:32:12 2023 +0400
>>>
>>>     gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.
>>>     
>>>     gnu/packages/emacs.scm (emacs)[native-search-paths]: Add a search-path for
>>>     tree-sitter grammars.
>>>     gnu/packages/aux-files/emacs/guix-emacs.el: Add directories from
>>>     TREE_SITTER_GRAMMAR_PATH to treesit-extra-load-path.
>>> ---
>>>  gnu/packages/aux-files/emacs/guix-emacs.el |  7 +++++++
>>>  gnu/packages/emacs.scm                     | 10 +++++++++-
>>>  2 files changed, 16 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el b/gnu/packages/aux-files/emacs/guix-emacs.el
>>> index 56dbcb8d67..708093267d 100644
>>> --- a/gnu/packages/aux-files/emacs/guix-emacs.el
>>> +++ b/gnu/packages/aux-files/emacs/guix-emacs.el
>>> @@ -76,6 +76,13 @@ The files in the list do not have extensions (.el, .elc)."
>>>                     (when (file-directory-p pkg-dir)
>>>                       (package-load-descriptor pkg-dir)))))))))))
>>>  
>>> +;; If emacs built with tree-sitter, read the value of the environment variable
>>> +;; to make tree-sitter grammars available in emacs out-of-the-box.
>>> +(with-eval-after-load 'treesit
>>> +  (when-let ((grammar-path (getenv "TREE_SITTER_GRAMMAR_PATH")))
>>> +    (mapcar (lambda (x) (add-to-list 'treesit-extra-load-path x))
>>> +            (split-string grammar-path ":"))))
>>> +
>>>  (provide 'guix-emacs)
>>>  
>>>  ;;; guix-emacs.el ends here
>>> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
>>> index 4ce41deb88..b6a66bafac 100644
>>> --- a/gnu/packages/emacs.scm
>>> +++ b/gnu/packages/emacs.scm
>>> @@ -367,7 +367,15 @@
>>>              (files '("lib/emacs/native-site-lisp")))
>>>             (search-path-specification
>>>              (variable "INFOPATH")
>>> -            (files '("share/info")))))
>>> +            (files '("share/info")))
>>> +           ;; tree-sitter support is not yet available in emacs 28, but this
>>> +           ;; search path won't harm and also will be beneficial for
>>> +           ;; emacs-next and other emacs-* packages, which have tree-sitter
>>> +           ;; support enabled.  Please, remove this comment, when emacs
>>> +           ;; package is updated to 29.
>>> +           (search-path-specification
>>> +            (variable "TREE_SITTER_GRAMMAR_PATH")
>>> +            (files '("lib/tree-sitter")))))
>>>  
>>>      (home-page "https://www.gnu.org/software/emacs/")
>>>      (synopsis "The extensible, customizable, self-documenting text editor")
>>
>> The change to guix-emacs.el is pretty impactful here since that affects
>> all the emacs packages including emacs-minimal I think.
>>
>> If I've understood your comment correctly, that does seem pretty
>> wasteful since those rebuilds because of the changed emacs-minimal won't
>> have any affect since it's version 28 and not 29, right? I think
>> applying this change to just emacs-next would have been possible.
>
> It could be applied to emacs-next only and we could move it to emacs
> later when it updated to 29, but I checked guix refresh --list-dependent
> emacs and it shown less than 100 packages so I decided to apply it
> straight to emacs, but I forgot that emacs-minimal is inherited from
> emacs and I guess you are right and it will affect all the emacs
> packages.  Anything we would like to do about it right now?

Let's keep it as it is and I'll store it in mind for the next time.

There is an idea to update guix refresh --list-dependent to handle the
case with inherited packages as well.  WDYT?

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.

[Ludovic Courtès]

Hi,

Andrew Tropin <andrew@trop.in> skribis:

> On 2023-02-11 16:35, Andrew Tropin wrote:

[...]

>>> The change to guix-emacs.el is pretty impactful here since that affects
>>> all the emacs packages including emacs-minimal I think.
>>>
>>> If I've understood your comment correctly, that does seem pretty
>>> wasteful since those rebuilds because of the changed emacs-minimal won't
>>> have any affect since it's version 28 and not 29, right? I think
>>> applying this change to just emacs-next would have been possible.
>>
>> It could be applied to emacs-next only and we could move it to emacs
>> later when it updated to 29, but I checked guix refresh --list-dependent
>> emacs and it shown less than 100 packages so I decided to apply it
>> straight to emacs, but I forgot that emacs-minimal is inherited from
>> emacs and I guess you are right and it will affect all the emacs
>> packages.  Anything we would like to do about it right now?
>
> Let's keep it as it is and I'll store it in mind for the next time.

Right.

> There is an idea to update guix refresh --list-dependent to handle the
> case with inherited packages as well.  WDYT?

Unfortunately, it’s not possible because inheritance info isn’t
available at run time.

However, peer review and <https://qa.guix.gnu.org> should be able to
catch it.  No big deal, but next time we’ll know.

Thanks,
Ludo’.

Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 1843 bytes --]

On 2023-02-12 01:14, Ludovic Courtès wrote:

> Hi,
>
> Andrew Tropin <andrew@trop.in> skribis:
>
>> On 2023-02-11 16:35, Andrew Tropin wrote:
>
> [...]
>
>>>> The change to guix-emacs.el is pretty impactful here since that affects
>>>> all the emacs packages including emacs-minimal I think.
>>>>
>>>> If I've understood your comment correctly, that does seem pretty
>>>> wasteful since those rebuilds because of the changed emacs-minimal won't
>>>> have any affect since it's version 28 and not 29, right? I think
>>>> applying this change to just emacs-next would have been possible.
>>>
>>> It could be applied to emacs-next only and we could move it to emacs
>>> later when it updated to 29, but I checked guix refresh --list-dependent
>>> emacs and it shown less than 100 packages so I decided to apply it
>>> straight to emacs, but I forgot that emacs-minimal is inherited from
>>> emacs and I guess you are right and it will affect all the emacs
>>> packages.  Anything we would like to do about it right now?
>>
>> Let's keep it as it is and I'll store it in mind for the next time.
>
> Right.
>
>> There is an idea to update guix refresh --list-dependent to handle the
>> case with inherited packages as well.  WDYT?
>
> Unfortunately, it’s not possible because inheritance info isn’t
> available at run time.
>
> However, peer review and <https://qa.guix.gnu.org> should be able to
> catch it.  No big deal, but next time we’ll know.

I need to figure out how to use qa service, it seems the patch
https://yhetil.org/87edqxri41.fsf@trop.in

is not recognized by
https://qa.guix.gnu.org/issue/49946

Probably it relies on subject for extracting apropriate patches for CI?

Will need to take a look at the source code, thank you for the idea. 

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

’inherit’ and list-dependent (was Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.)

[Simon Tournier]

Hi,

On dim., 12 févr. 2023 at 01:14, Ludovic Courtès <ludo@gnu.org> wrote:

>> There is an idea to update guix refresh --list-dependent to handle the
>> case with inherited packages as well.  WDYT?
>
> Unfortunately, it’s not possible because inheritance info isn’t
> available at run time.

Well, with the current implementation of ’inherit’, which is just
copy/paste at the record level, indeed it is not possible to detect some
relationship.

However, we could imagine to use ’package/inherit’ or another variant
instead of plain ’inherit’ for creating these inherited packages.  Doing
so, we could collect some information, e.g., in the field ’properties’,
which could be used then by --list-dependent.

Many of us are bitten by this.  I remember a recent update of Git which
also missed the dependency of git-minimal. :-)

For sure, QA helps a lot.  Somehow, braces and belt. ;-)


Cheers,
simon

Re: ’inherit’ and list-dependent (was Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.)

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 1550 bytes --]

Hello,

Simon Tournier <zimon.toutoune@gmail.com> ezt írta (időpont: 2023. febr.
14., K 13:25):

> Hi,
>
> On dim., 12 févr. 2023 at 01:14, Ludovic Courtès <ludo@gnu.org> wrote:
>
> >> There is an idea to update guix refresh --list-dependent to handle the
> >> case with inherited packages as well.  WDYT?
> >
> > Unfortunately, it’s not possible because inheritance info isn’t
> > available at run time.
>

My idea here was to rescan the source with a minimal scanner and build up
an inheritance aware dependency graph. We could then make this information
available in some way, but about that part I did not think about too much
yet.

Another thing that could possibly work is to enrich the package with
inheritance information, but I am not sure that this can be done in a
backwards compatible way. Wdyt?

>
> Well, with the current implementation of ’inherit’, which is just
> copy/paste at the record level, indeed it is not possible to detect some
> relationship.
>
> However, we could imagine to use ’package/inherit’ or another variant
> instead of plain ’inherit’ for creating these inherited packages.  Doing
> so, we could collect some information, e.g., in the field ’properties’,
> which could be used then by --list-dependent.
>
> Many of us are bitten by this.  I remember a recent update of Git which
> also missed the dependency of git-minimal. :-)
>
> For sure, QA helps a lot.  Somehow, braces and belt. ;-)
>
>
> Cheers,
> simon
>
Regards,
g_bor

>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2441 bytes --]

Re: ’inherit’ and list-dependent (was Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.)

[Ludovic Courtès]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> On dim., 12 févr. 2023 at 01:14, Ludovic Courtès <ludo@gnu.org> wrote:
>
>>> There is an idea to update guix refresh --list-dependent to handle the
>>> case with inherited packages as well.  WDYT?
>>
>> Unfortunately, it’s not possible because inheritance info isn’t
>> available at run time.
>
> Well, with the current implementation of ’inherit’, which is just
> copy/paste at the record level, indeed it is not possible to detect some
> relationship.
>
> However, we could imagine to use ’package/inherit’ or another variant
> instead of plain ’inherit’ for creating these inherited packages.  Doing
> so, we could collect some information, e.g., in the field ’properties’,
> which could be used then by --list-dependent.

In effect that means keeping back the chain of inherited objects, which
would lead to space leaks.

> Many of us are bitten by this.  I remember a recent update of Git which
> also missed the dependency of git-minimal. :-)

I agree it’d be nice to solve.  I can’t think of a good way to do that
though.

Thoughts?

Ludo’.

Re: ’inherit’ and list-dependent (was Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.)

[Simon Tournier]

Hi,

On Tue, 21 Feb 2023 at 23:55, Ludovic Courtès <ludo@gnu.org> wrote:

>> However, we could imagine to use ’package/inherit’ or another variant
>> instead of plain ’inherit’ for creating these inherited packages.  Doing
>> so, we could collect some information, e.g., in the field ’properties’,
>> which could be used then by --list-dependent.
>
> In effect that means keeping back the chain of inherited objects, which
> would lead to space leaks.

[...]

> I agree it’d be nice to solve.  I can’t think of a good way to do that
> though.

What do you mean by “space leaks”?

Well, we already have ways to create variants collecting information, as
’package-with-python2’ or ’package-with-ocamlX.0Y’, and
’package/inherit’ is somehow another one.

Why would it not be a good way?


Cheers,
simon

Re: ’inherit’ and list-dependent (was Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.)

[Ludovic Courtès]

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> On Tue, 21 Feb 2023 at 23:55, Ludovic Courtès <ludo@gnu.org> wrote:
>
>>> However, we could imagine to use ’package/inherit’ or another variant
>>> instead of plain ’inherit’ for creating these inherited packages.  Doing
>>> so, we could collect some information, e.g., in the field ’properties’,
>>> which could be used then by --list-dependent.
>>
>> In effect that means keeping back the chain of inherited objects, which
>> would lead to space leaks.
>
> [...]
>
>> I agree it’d be nice to solve.  I can’t think of a good way to do that
>> though.
>
> What do you mean by “space leaks”?

Unbounded memory usage: each copy of an object is linked back to its
“parent” (“previous generation”).

Ludo’.

Re: ’inherit’ and list-dependent (was Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.)

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

Hello,

Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2023. febr. 25., Szo
19:10):

> Simon Tournier <zimon.toutoune@gmail.com> skribis:
>
> > On Tue, 21 Feb 2023 at 23:55, Ludovic Courtès <ludo@gnu.org> wrote:
> >
> >>> However, we could imagine to use ’package/inherit’ or another variant
> >>> instead of plain ’inherit’ for creating these inherited packages.
> Doing
> >>> so, we could collect some information, e.g., in the field ’properties’,
> >>> which could be used then by --list-dependent.
> >>
> >> In effect that means keeping back the chain of inherited objects, which
> >> would lead to space leaks.
> >
> > [...]
> >
> >> I agree it’d be nice to solve.  I can’t think of a good way to do that
> >> though.
> >
> > What do you mean by “space leaks”?
>
> Unbounded memory usage: each copy of an object is linked back to its
> “parent” (“previous generation”).
>
How much heavier is this than the links to inputs?

Regards, g_bor

>
> Ludo’.
>
>

[-- Attachment #2: Type: text/html, Size: 1818 bytes --]

Re: ’inherit’ and list-dependent (was Re: branch master updated: gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.)

[Simon Tournier]

Hi,

On Sat, 25 Feb 2023 at 19:10, Ludovic Courtès <ludo@gnu.org> wrote:

>>> In effect that means keeping back the chain of inherited objects, which
>>> would lead to space leaks.

[...]

>> What do you mean by “space leaks”?
>
> Unbounded memory usage: each copy of an object is linked back to its
> “parent” (“previous generation”).

As Gábor, I miss some details. :-) And I do not understand what would be
these “space leak” since what I am suggesting is somehow done with all
Python 2 packages using package-with-python2
(package-with-explicit-python) for example.

Cheers,
simon