Use (eval-when-compile 'treesit) to save us from writing declare-function forms

Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

Some of my recent changes to c-ts-mode.el uses some more C treesit functions, and I (again) forgot to add their declare-function forms to c-ts-mode.el, and Andrea has to fix it for me. The main problem for me is that (I think) you don’t get warning for them unless you build a non-tree-sitter build of Emacs. So I’m prone to forgot about them. Also it’s a burden for anyone wanting to use tree-sitter functions.

treesit.el already has all the declare-function forms, how about we use (eval-when-compile 'treesit) in other files? I believe that should alleviate us from adding those declare-function forms in other files?

I want to ask because IIRC I proposed this a while back but there were objections, the details of which I forgot.

Yuan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Andrea Corallo]

Yuan Fu <casouri@gmail.com> writes:

> Some of my recent changes to c-ts-mode.el uses some more C treesit
> functions, and I (again) forgot to add their declare-function forms to
> c-ts-mode.el, and Andrea has to fix it for me. The main problem for me
> is that (I think) you don’t get warning for them unless you build a
> non-tree-sitter build of Emacs. So I’m prone to forgot about
> them. Also it’s a burden for anyone wanting to use tree-sitter
> functions.
>
> treesit.el already has all the declare-function forms, how about we
> use (eval-when-compile 'treesit) in other files? I believe that should
> alleviate us from adding those declare-function forms in other files?
>
> I want to ask because IIRC I proposed this a while back but there were objections, the details of which I forgot.
>
> Yuan

Hi Yuan,

I was not part of the discussion and I don't know what's the downside of
(eval-when-compile 'treesit), maybe another alternative would be to have
like a dedicated treesit-delcs.el to require where necessary?

  Andrea

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Mon, 2 Dec 2024 22:31:48 -0800
> 
> Some of my recent changes to c-ts-mode.el uses some more C treesit functions, and I (again) forgot to add their declare-function forms to c-ts-mode.el, and Andrea has to fix it for me. The main problem for me is that (I think) you don’t get warning for them unless you build a non-tree-sitter build of Emacs. So I’m prone to forgot about them. Also it’s a burden for anyone wanting to use tree-sitter functions.
> 
> treesit.el already has all the declare-function forms, how about we use (eval-when-compile 'treesit) in other files? I believe that should alleviate us from adding those declare-function forms in other files?

I don't understand" c-ts-mode.el already does

  (require 'treesit)

So what would eval-when-compile add to that?

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

> On Dec 3, 2024, at 5:12 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Mon, 2 Dec 2024 22:31:48 -0800
>> 
>> Some of my recent changes to c-ts-mode.el uses some more C treesit functions, and I (again) forgot to add their declare-function forms to c-ts-mode.el, and Andrea has to fix it for me. The main problem for me is that (I think) you don’t get warning for them unless you build a non-tree-sitter build of Emacs. So I’m prone to forgot about them. Also it’s a burden for anyone wanting to use tree-sitter functions.
>> 
>> treesit.el already has all the declare-function forms, how about we use (eval-when-compile 'treesit) in other files? I believe that should alleviate us from adding those declare-function forms in other files?
> 
> I don't understand" c-ts-mode.el already does
> 
>  (require 'treesit)
> 
> So what would eval-when-compile add to that?

When Emacs is built without tree-sitter, none of the treesit.c functions are available (except for treesit-ready-p). Now if this Emacs compiles c-ts-mode.el, it’ll signal undefined function error for all those functions. So we add declare-function forms for all the treesit.c functions used in c-ts-mode.el, and have to update the declare-function forms whenever we use some new treesit.c functions.

If we use eval-when-compile, we don’t need to write declare-function forms in c-ts-mode.el anymore, because treesit.el has declare-function forms for all treesit.c functions.

Yuan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

> On Dec 3, 2024, at 12:30 AM, Andrea Corallo <acorallo@gnu.org> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>> Some of my recent changes to c-ts-mode.el uses some more C treesit
>> functions, and I (again) forgot to add their declare-function forms to
>> c-ts-mode.el, and Andrea has to fix it for me. The main problem for me
>> is that (I think) you don’t get warning for them unless you build a
>> non-tree-sitter build of Emacs. So I’m prone to forgot about
>> them. Also it’s a burden for anyone wanting to use tree-sitter
>> functions.
>> 
>> treesit.el already has all the declare-function forms, how about we
>> use (eval-when-compile 'treesit) in other files? I believe that should
>> alleviate us from adding those declare-function forms in other files?
>> 
>> I want to ask because IIRC I proposed this a while back but there were objections, the details of which I forgot.
>> 
>> Yuan
> 
> Hi Yuan,
> 
> I was not part of the discussion and I don't know what's the downside of
> (eval-when-compile 'treesit), maybe another alternative would be to have
> like a dedicated treesit-delcs.el to require where necessary?
> 

That’ll be ok too. Though 99.99% the case, if some package uses tree-sitter, they’d need something in treesit.el too.

Yuan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 4 Dec 2024 10:19:42 -0800
> Cc: emacs-devel@gnu.org
> 
> 
> 
> > On Dec 3, 2024, at 5:12 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> > I don't understand" c-ts-mode.el already does
> > 
> >  (require 'treesit)
> > 
> > So what would eval-when-compile add to that?
> 
> When Emacs is built without tree-sitter, none of the treesit.c functions are available (except for treesit-ready-p). Now if this Emacs compiles c-ts-mode.el, it’ll signal undefined function error for all those functions. So we add declare-function forms for all the treesit.c functions used in c-ts-mode.el, and have to update the declare-function forms whenever we use some new treesit.c functions.
> 
> If we use eval-when-compile, we don’t need to write declare-function forms in c-ts-mode.el anymore, because treesit.el has declare-function forms for all treesit.c functions.

That's what I don't understand: how will eval-when-compile which loads
treesit.el be different from (require 'treesit) we already have in
c-ts-mode?  Don't they both load treesit.el?

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

> On Dec 4, 2024, at 10:57 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 4 Dec 2024 10:19:42 -0800
>> Cc: emacs-devel@gnu.org
>> 
>> 
>> 
>>> On Dec 3, 2024, at 5:12 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>> I don't understand" c-ts-mode.el already does
>>> 
>>> (require 'treesit)
>>> 
>>> So what would eval-when-compile add to that?
>> 
>> When Emacs is built without tree-sitter, none of the treesit.c functions are available (except for treesit-ready-p). Now if this Emacs compiles c-ts-mode.el, it’ll signal undefined function error for all those functions. So we add declare-function forms for all the treesit.c functions used in c-ts-mode.el, and have to update the declare-function forms whenever we use some new treesit.c functions.
>> 
>> If we use eval-when-compile, we don’t need to write declare-function forms in c-ts-mode.el anymore, because treesit.el has declare-function forms for all treesit.c functions.
> 
> That's what I don't understand: how will eval-when-compile which loads
> treesit.el be different from (require 'treesit) we already have in
> c-ts-mode?  Don't they both load treesit.el?

I maybe wrong here, but I thought if you don’t use eval-when-compile, simply byte-compiling c-ts-mode.el wouldn’t load treesit.el?

Yuan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 4 Dec 2024 22:25:33 -0800
> Cc: emacs-devel@gnu.org
> 
> >> If we use eval-when-compile, we don’t need to write declare-function forms in c-ts-mode.el anymore, because treesit.el has declare-function forms for all treesit.c functions.
> > 
> > That's what I don't understand: how will eval-when-compile which loads
> > treesit.el be different from (require 'treesit) we already have in
> > c-ts-mode?  Don't they both load treesit.el?
> 
> I maybe wrong here, but I thought if you don’t use eval-when-compile, simply byte-compiling c-ts-mode.el wouldn’t load treesit.el?

Yes, it would.  You can clearly see this in a debugger if you put a
breakpoint in Fload and then byte-compile c-ts-mode.el.

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

> On Dec 4, 2024, at 11:07 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 4 Dec 2024 22:25:33 -0800
>> Cc: emacs-devel@gnu.org
>> 
>>>> If we use eval-when-compile, we don’t need to write declare-function forms in c-ts-mode.el anymore, because treesit.el has declare-function forms for all treesit.c functions.
>>> 
>>> That's what I don't understand: how will eval-when-compile which loads
>>> treesit.el be different from (require 'treesit) we already have in
>>> c-ts-mode?  Don't they both load treesit.el?
>> 
>> I maybe wrong here, but I thought if you don’t use eval-when-compile, simply byte-compiling c-ts-mode.el wouldn’t load treesit.el?
> 
> Yes, it would.  You can clearly see this in a debugger if you put a
> breakpoint in Fload and then byte-compile c-ts-mode.el.

Does that mean declare-function forms only affect the file it’s in? Because if treesit.el has declare-function forms for all the treesit.c functions, and c-ts-mode.el requires treesit.el, why do we need to have declare-function forms in c-ts-mode.el?

Yuan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Fri, 6 Dec 2024 17:12:38 -0800
> Cc: emacs-devel@gnu.org
> 
> 
> 
> > On Dec 4, 2024, at 11:07 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri@gmail.com>
> >> Date: Wed, 4 Dec 2024 22:25:33 -0800
> >> Cc: emacs-devel@gnu.org
> >> 
> >>>> If we use eval-when-compile, we don’t need to write declare-function forms in c-ts-mode.el anymore, because treesit.el has declare-function forms for all treesit.c functions.
> >>> 
> >>> That's what I don't understand: how will eval-when-compile which loads
> >>> treesit.el be different from (require 'treesit) we already have in
> >>> c-ts-mode?  Don't they both load treesit.el?
> >> 
> >> I maybe wrong here, but I thought if you don’t use eval-when-compile, simply byte-compiling c-ts-mode.el wouldn’t load treesit.el?
> > 
> > Yes, it would.  You can clearly see this in a debugger if you put a
> > breakpoint in Fload and then byte-compile c-ts-mode.el.
> 
> Does that mean declare-function forms only affect the file it’s in? Because if treesit.el has declare-function forms for all the treesit.c functions, and c-ts-mode.el requires treesit.el, why do we need to have declare-function forms in c-ts-mode.el?

Seems like that.  Stefan, am I missing something here?

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Stefan Monnier]

>> Does that mean declare-function forms only affect the file it’s in?
>> Because if treesit.el has declare-function forms for all the treesit.c
>> functions, and c-ts-mode.el requires treesit.el, why do we need to have
>> declare-function forms in c-ts-mode.el?
> Seems like that.  Stefan, am I missing something here?

Yes, `declare-function` *should* follow the same principles as `(defvar
FOO)`, meaning that they are compiler directives not functions to
execute.  They affect warnings only in the current lexical scope, and
requiring a file full of `(defvar FOO)` and `declare-function` will have
no effect at all.

[ Side note regarding this *should*: it currently doesn't work quite as
  well as `defvar` because its effect is always file-wide, whereas it
  should be limited to the current scope.  ]


        Stefan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]



> On Dec 7, 2024, at 5:47 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>>> Does that mean declare-function forms only affect the file it’s in?
>>> Because if treesit.el has declare-function forms for all the treesit.c
>>> functions, and c-ts-mode.el requires treesit.el, why do we need to have
>>> declare-function forms in c-ts-mode.el?
>> Seems like that.  Stefan, am I missing something here?
> 
> Yes, `declare-function` *should* follow the same principles as `(defvar
> FOO)`, meaning that they are compiler directives not functions to
> execute.  They affect warnings only in the current lexical scope, and
> requiring a file full of `(defvar FOO)` and `declare-function` will have
> no effect at all.
> 
> [ Side note regarding this *should*: it currently doesn't work quite as
>  well as `defvar` because its effect is always file-wide, whereas it
>  should be limited to the current scope.  ]

Defining a macro that contains the declare-function forms, and calling it in eval-when-compile forms seem to work (see patch), WDYT? The name (treesit-declare-c-functions) could be better, any ideas?

Yuan


[-- Attachment #2: declare-function.patch --]
[-- Type: application/octet-stream, Size: 7491 bytes --]

From ba77886d5e250770251d8c1abec93d4fcf0e0268 Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Sat, 7 Dec 2024 22:13:07 -0800
Subject: [PATCH] declare-function POC

---
 lisp/progmodes/c-ts-mode.el |  17 +-----
 lisp/treesit.el             | 103 ++++++++++++++++++++----------------
 2 files changed, 60 insertions(+), 60 deletions(-)

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index dec9411b87c..4a1f616b048 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -73,21 +73,8 @@
 (require 'c-ts-common)
 (eval-when-compile (require 'rx))
 
-(declare-function treesit-parser-create "treesit.c")
-(declare-function treesit-parser-root-node "treesit.c")
-(declare-function treesit-parser-set-included-ranges "treesit.c")
-(declare-function treesit-node-parent "treesit.c")
-(declare-function treesit-node-start "treesit.c")
-(declare-function treesit-node-end "treesit.c")
-(declare-function treesit-node-child "treesit.c")
-(declare-function treesit-node-child-by-field-name "treesit.c")
-(declare-function treesit-node-type "treesit.c")
-(declare-function treesit-node-prev-sibling "treesit.c")
-(declare-function treesit-node-first-child-for-pos "treesit.c")
-(declare-function treesit-node-next-sibling "treesit.c")
-(declare-function treesit-node-eq "treesit.c")
-(declare-function treesit-node-match-p "treesit.c")
-(declare-function treesit-query-compile "treesit.c")
+(eval-when-compile
+  (treesit-declare-c-functions))
 
 ;;; Custom variables
 
diff --git a/lisp/treesit.el b/lisp/treesit.el
index db3a706f016..2e189394983 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -39,61 +39,74 @@
 
 ;;; Function declarations
 
-(declare-function treesit-language-available-p "treesit.c")
-(declare-function treesit-language-version "treesit.c")
+(defmacro treesit-declare-c-functions ()
+  "Declare C functions and variables defined in treesit.c.
 
-(declare-function treesit-parser-p "treesit.c")
-(declare-function treesit-node-p "treesit.c")
-(declare-function treesit-compiled-query-p "treesit.c")
-(declare-function treesit-query-p "treesit.c")
-(declare-function treesit-query-language "treesit.c")
+This macro is only needed for byte-compiling in Emacs not built with
+tree-sitter library.  It should be used with `eval-when-compile' like
+this:
 
-(declare-function treesit-node-parser "treesit.c")
+    (eval-when-compile
+      (treesit-declare-c-functions))"
+  '(progn
+     (declare-function treesit-language-available-p "treesit.c")
+     (declare-function treesit-language-version "treesit.c")
 
-(declare-function treesit-parser-create "treesit.c")
-(declare-function treesit-parser-delete "treesit.c")
-(declare-function treesit-parser-list "treesit.c")
-(declare-function treesit-parser-buffer "treesit.c")
-(declare-function treesit-parser-language "treesit.c")
-(declare-function treesit-parser-tag "treesit.c")
+     (declare-function treesit-parser-p "treesit.c")
+     (declare-function treesit-node-p "treesit.c")
+     (declare-function treesit-compiled-query-p "treesit.c")
+     (declare-function treesit-query-p "treesit.c")
+     (declare-function treesit-query-language "treesit.c")
 
-(declare-function treesit-parser-root-node "treesit.c")
+     (declare-function treesit-node-parser "treesit.c")
 
-(declare-function treesit-parser-set-included-ranges "treesit.c")
-(declare-function treesit-parser-included-ranges "treesit.c")
-(declare-function treesit-parser-changed-ranges "treesit.c")
-(declare-function treesit-parser-add-notifier "treesit.c")
+     (declare-function treesit-parser-create "treesit.c")
+     (declare-function treesit-parser-delete "treesit.c")
+     (declare-function treesit-parser-list "treesit.c")
+     (declare-function treesit-parser-buffer "treesit.c")
+     (declare-function treesit-parser-language "treesit.c")
+     (declare-function treesit-parser-tag "treesit.c")
 
-(declare-function treesit-node-type "treesit.c")
-(declare-function treesit-node-start "treesit.c")
-(declare-function treesit-node-end "treesit.c")
-(declare-function treesit-node-string "treesit.c")
-(declare-function treesit-node-parent "treesit.c")
-(declare-function treesit-node-child "treesit.c")
-(declare-function treesit-node-check "treesit.c")
-(declare-function treesit-node-field-name-for-child "treesit.c")
-(declare-function treesit-node-child-count "treesit.c")
-(declare-function treesit-node-child-by-field-name "treesit.c")
-(declare-function treesit-node-next-sibling "treesit.c")
-(declare-function treesit-node-prev-sibling "treesit.c")
-(declare-function treesit-node-first-child-for-pos "treesit.c")
-(declare-function treesit-node-descendant-for-range "treesit.c")
-(declare-function treesit-node-eq "treesit.c")
+     (declare-function treesit-parser-root-node "treesit.c")
 
-(declare-function treesit-pattern-expand "treesit.c")
-(declare-function treesit-query-expand "treesit.c")
-(declare-function treesit-query-compile "treesit.c")
-(declare-function treesit-query-capture "treesit.c")
+     (declare-function treesit-parser-set-included-ranges "treesit.c")
+     (declare-function treesit-parser-included-ranges "treesit.c")
+     (declare-function treesit-parser-changed-ranges "treesit.c")
+     (declare-function treesit-parser-add-notifier "treesit.c")
 
-(declare-function treesit-search-subtree "treesit.c")
-(declare-function treesit-search-forward "treesit.c")
-(declare-function treesit-induce-sparse-tree "treesit.c")
-(declare-function treesit-subtree-stat "treesit.c")
-(declare-function treesit-node-match-p "treesit.c")
+     (declare-function treesit-node-type "treesit.c")
+     (declare-function treesit-node-start "treesit.c")
+     (declare-function treesit-node-end "treesit.c")
+     (declare-function treesit-node-string "treesit.c")
+     (declare-function treesit-node-parent "treesit.c")
+     (declare-function treesit-node-child "treesit.c")
+     (declare-function treesit-node-check "treesit.c")
+     (declare-function treesit-node-field-name-for-child "treesit.c")
+     (declare-function treesit-node-child-count "treesit.c")
+     (declare-function treesit-node-child-by-field-name "treesit.c")
+     (declare-function treesit-node-next-sibling "treesit.c")
+     (declare-function treesit-node-prev-sibling "treesit.c")
+     (declare-function treesit-node-first-child-for-pos "treesit.c")
+     (declare-function treesit-node-descendant-for-range "treesit.c")
+     (declare-function treesit-node-eq "treesit.c")
 
-(declare-function treesit-available-p "treesit.c")
+     (declare-function treesit-pattern-expand "treesit.c")
+     (declare-function treesit-query-expand "treesit.c")
+     (declare-function treesit-query-compile "treesit.c")
+     (declare-function treesit-query-capture "treesit.c")
 
-(defvar treesit-thing-settings)
+     (declare-function treesit-search-subtree "treesit.c")
+     (declare-function treesit-search-forward "treesit.c")
+     (declare-function treesit-induce-sparse-tree "treesit.c")
+     (declare-function treesit-subtree-stat "treesit.c")
+     (declare-function treesit-node-match-p "treesit.c")
+
+     (declare-function treesit-available-p "treesit.c")
+
+     (defvar treesit-thing-settings)))
+
+(eval-when-compile
+  (treesit-declare-c-functions))
 
 ;;; Custom options
 
-- 
2.39.5 (Apple Git-151)


[-- Attachment #3: Type: text/plain, Size: 2 bytes --]

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Sat, 7 Dec 2024 22:39:21 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  emacs-devel@gnu.org
> 
> > On Dec 7, 2024, at 5:47 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > 
> >>> Does that mean declare-function forms only affect the file it’s in?
> >>> Because if treesit.el has declare-function forms for all the treesit.c
> >>> functions, and c-ts-mode.el requires treesit.el, why do we need to have
> >>> declare-function forms in c-ts-mode.el?
> >> Seems like that.  Stefan, am I missing something here?
> > 
> > Yes, `declare-function` *should* follow the same principles as `(defvar
> > FOO)`, meaning that they are compiler directives not functions to
> > execute.  They affect warnings only in the current lexical scope, and
> > requiring a file full of `(defvar FOO)` and `declare-function` will have
> > no effect at all.
> > 
> > [ Side note regarding this *should*: it currently doesn't work quite as
> >  well as `defvar` because its effect is always file-wide, whereas it
> >  should be limited to the current scope.  ]
> 
> Defining a macro that contains the declare-function forms, and calling it in eval-when-compile forms seem to work (see patch), WDYT? The name (treesit-declare-c-functions) could be better, any ideas?

Looks a bit ugly, but maybe this is the best we can do.

Andrea, WDYT?

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Stefan Monnier]

> +(eval-when-compile
> +  (treesit-declare-c-functions))

`treesit-declare-c-functions` is a macro, that expands to
`declare-function`s, i.e. to "no-ops".
Why wrap it within `eval-when-compile`?

If it works within an `eval-when-compile`, it's only by accident (just
like the fact that (eval-when-compile (defvar foo)) "works" is an
accident): code shouldn't rely on it.

It's not the evaluation of `defvar` or `declare-function` that is needed
(because such an evaluation should have no effect, as you can see if
you look at the definition of `declare-function`), but instead the
"calls" to `defvar` or `declare-function` need to be seen/processed by
the compiler.  By wrapping them within an `eval-when-compile` you risk
*hiding* them from the compiler.


        Stefan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Dmitry Gutov]

On 04/12/2024 20:19, Yuan Fu wrote:
> When Emacs is built without tree-sitter, none of the treesit.c functions are available (except for treesit-ready-p)

Have we considered having all of those functions defined but doing 
nothing in that case? Alias them to 'ignore' or something that raises an 
error.

I think then we won't have to maintain these 'declare-function' lists in 
every package that depends on treesit.c.

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

> On Dec 8, 2024, at 7:27 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> +(eval-when-compile
>> +  (treesit-declare-c-functions))
> 
> `treesit-declare-c-functions` is a macro, that expands to
> `declare-function`s, i.e. to "no-ops".
> Why wrap it within `eval-when-compile`?

I thought that might be TRT since they’re only needed for compiling the file.

> 
> If it works within an `eval-when-compile`, it's only by accident (just
> like the fact that (eval-when-compile (defvar foo)) "works" is an
> accident): code shouldn't rely on it.
> 
> It's not the evaluation of `defvar` or `declare-function` that is needed
> (because such an evaluation should have no effect, as you can see if
> you look at the definition of `declare-function`), but instead the
> "calls" to `defvar` or `declare-function` need to be seen/processed by
> the compiler.  By wrapping them within an `eval-when-compile` you risk
> *hiding* them from the compiler.

I thought eval-when-compile is more of a “include if compiling” macro, should’ve checked :)

We can also go with Dmitry’s idea, and just define all the functions in treesit.el if Emacs is not built with tree-sitter. We can detect that with treesit-available-p.

Yuan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Stefan Monnier]

> I thought eval-when-compile is more of a “include if compiling” macro, should’ve checked :)
>
> We can also go with Dmitry’s idea, and just define all the functions in
> treesit.el if Emacs is not built with tree-sitter. We can detect that with
> treesit-available-p.

I haven't really thought about it, so I don't have a strong opinion.
AFAICT your proposal should work fine even if you remove the
`eval-when-compile` wrappers to which I objected.  🙂


        Stefan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Dmitry Gutov]

On 09/12/2024 04:40, Stefan Monnier wrote:
>> I thought eval-when-compile is more of a “include if compiling” macro, should’ve checked 🙂
>>
>> We can also go with Dmitry’s idea, and just define all the functions in
>> treesit.el if Emacs is not built with tree-sitter. We can detect that with
>> treesit-available-p.
> I haven't really thought about it, so I don't have a strong opinion.
> AFAICT your proposal should work fine even if you remove the
> `eval-when-compile` wrappers to which I objected.  🙂

It seems to me that, having taken than route, we would recommend every

   (require 'treesit)

to be accompanied by

   (treesit-declare-c-functions)

...or the familiar declare-function list, of course.

And the same would also be true for transitive dependencies (so, both 
c-ts-common and c-ts-mode, even though otherwise the latter could drop 
the (require 'treesit) form).

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sat, 7 Dec 2024 22:39:21 -0800
>> Cc: Eli Zaretskii <eliz@gnu.org>,
>>  emacs-devel@gnu.org
>> 
>> > On Dec 7, 2024, at 5:47 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> > 
>> >>> Does that mean declare-function forms only affect the file it’s in?
>> >>> Because if treesit.el has declare-function forms for all the treesit.c
>> >>> functions, and c-ts-mode.el requires treesit.el, why do we need to have
>> >>> declare-function forms in c-ts-mode.el?
>> >> Seems like that.  Stefan, am I missing something here?
>> > 
>> > Yes, `declare-function` *should* follow the same principles as `(defvar
>> > FOO)`, meaning that they are compiler directives not functions to
>> > execute.  They affect warnings only in the current lexical scope, and
>> > requiring a file full of `(defvar FOO)` and `declare-function` will have
>> > no effect at all.
>> > 
>> > [ Side note regarding this *should*: it currently doesn't work quite as
>> >  well as `defvar` because its effect is always file-wide, whereas it
>> >  should be limited to the current scope.  ]
>> 
>> Defining a macro that contains the declare-function forms, and
>> calling it in eval-when-compile forms seem to work (see patch),
>> WDYT? The name (treesit-declare-c-functions) could be better, any
>> ideas?
>
> Looks a bit ugly, but maybe this is the best we can do.
>
> Andrea, WDYT?

If it works having the 'treesit-declare-c-functions' without the
'eval-when-compile' around would be IMO okay.

  Andrea

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Andrea Corallo]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 04/12/2024 20:19, Yuan Fu wrote:
>> When Emacs is built without tree-sitter, none of the treesit.c functions are available (except for treesit-ready-p)
>
> Have we considered having all of those functions defined but doing
> nothing in that case? Alias them to 'ignore' or something that raises
> an error.
>
> I think then we won't have to maintain these 'declare-function' lists
> in every package that depends on treesit.c.

The suggestion of having the C functions defined to nops when
HAVE_TREE_SITTER is not defined is cleaner in principle, but I think
there are two downsides:

- We don't tipically do it this way, so it would be an exception

- Also we should do it for all the functions in treesit.c, otherwise we
  don't solve the original problem of introducing warnings by mistake on
  non treesit builds.

  Andrea

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Dmitry Gutov]

On 12/12/2024 01:38, Andrea Corallo wrote:
> The suggestion of having the C functions defined to nops when
> HAVE_TREE_SITTER is not defined is cleaner in principle, but I think
> there are two downsides:
> 
> - We don't tipically do it this way, so it would be an exception

We also don't typically require a caller to have a 'declare-function' 
for every function they use from a package that they (require ...) already.

> - Also we should do it for all the functions in treesit.c, otherwise we
>    don't solve the original problem of introducing warnings by mistake on
>    non treesit builds.

Sure. All non-static ones anyway.

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

> On Dec 11, 2024, at 4:23 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
> On 12/12/2024 01:38, Andrea Corallo wrote:
>> The suggestion of having the C functions defined to nops when
>> HAVE_TREE_SITTER is not defined is cleaner in principle, but I think
>> there are two downsides:
>> - We don't tipically do it this way, so it would be an exception
> 
> We also don't typically require a caller to have a 'declare-function' for every function they use from a package that they (require ...) already.
> 
>> - Also we should do it for all the functions in treesit.c, otherwise we
>>   don't solve the original problem of introducing warnings by mistake on
>>   non treesit builds.
> 
> Sure. All non-static ones anyway.

I tried to define all the C functions and it’s not easy to do: doing it in C seems pretty tedious, we’d need DEFUN form and defsubr for each function (is there a easier way?). If done in lisp, we need to do it in a separate file and _load_ it before compiling treesit.el and any other package that uses tree-sitter. I’m not familiar with Emacs’s build process so I don’t know how and where to add such a file and make Emacs load it on startup.

Can someone enlighten me on how to define these functions in either C or lisp?

I think defining them as no-op for Emacs w/o tree-sitter is a good move, because otherwise if some user compiles an Emacs w/o tree-sitter and compiles a bunch of packages, and one of the package happens to be based on tree-sitter, they’ll see a bunch of warnings unless the package author added declare-function forms or the proposed macro. It’s better to have tree-sitter functions always defined and save everybody some hassle.

Yuan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: Yuan Fu <casouri@gmail.com>,  Eli Zaretskii <eliz@gnu.org>,
>   emacs-devel@gnu.org
> Date: Wed, 11 Dec 2024 18:38:32 -0500
> 
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
> > On 04/12/2024 20:19, Yuan Fu wrote:
> >> When Emacs is built without tree-sitter, none of the treesit.c functions are available (except for treesit-ready-p)
> >
> > Have we considered having all of those functions defined but doing
> > nothing in that case? Alias them to 'ignore' or something that raises
> > an error.
> >
> > I think then we won't have to maintain these 'declare-function' lists
> > in every package that depends on treesit.c.
> 
> The suggestion of having the C functions defined to nops when
> HAVE_TREE_SITTER is not defined is cleaner in principle, but I think
> there are two downsides:
> 
> - We don't tipically do it this way, so it would be an exception
> 
> - Also we should do it for all the functions in treesit.c, otherwise we
>   don't solve the original problem of introducing warnings by mistake on
>   non treesit builds.

Yes, I agree.

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Eli Zaretskii]

> Date: Thu, 12 Dec 2024 02:23:02 +0200
> Cc: Yuan Fu <casouri@gmail.com>, Eli Zaretskii <eliz@gnu.org>,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 12/12/2024 01:38, Andrea Corallo wrote:
> > The suggestion of having the C functions defined to nops when
> > HAVE_TREE_SITTER is not defined is cleaner in principle, but I think
> > there are two downsides:
> > 
> > - We don't tipically do it this way, so it would be an exception
> 
> We also don't typically require a caller to have a 'declare-function' 
> for every function they use from a package that they (require ...) already.

This is not the case here, since declare-function are needed for
functions defined in treesit.c, not in treesit.el.

> > - Also we should do it for all the functions in treesit.c, otherwise we
> >    don't solve the original problem of introducing warnings by mistake on
> >    non treesit builds.
> 
> Sure. All non-static ones anyway.

Only those defined with DEFUN.  The rest are not visible from Lisp,
whether static or not.

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 11 Dec 2024 22:05:56 -0800
> Cc: Andrea Corallo <acorallo@gnu.org>,
>  Eli Zaretskii <eliz@gnu.org>,
>  emacs-devel@gnu.org
> 
> 
> 
> > On Dec 11, 2024, at 4:23 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> > 
> > On 12/12/2024 01:38, Andrea Corallo wrote:
> >> The suggestion of having the C functions defined to nops when
> >> HAVE_TREE_SITTER is not defined is cleaner in principle, but I think
> >> there are two downsides:
> >> - We don't tipically do it this way, so it would be an exception
> > 
> > We also don't typically require a caller to have a 'declare-function' for every function they use from a package that they (require ...) already.
> > 
> >> - Also we should do it for all the functions in treesit.c, otherwise we
> >>   don't solve the original problem of introducing warnings by mistake on
> >>   non treesit builds.
> > 
> > Sure. All non-static ones anyway.
> 
> I tried to define all the C functions and it’s not easy to do: doing it in C seems pretty tedious, we’d need DEFUN form and defsubr for each function (is there a easier way?). If done in lisp, we need to do it in a separate file and _load_ it before compiling treesit.el and any other package that uses tree-sitter. I’m not familiar with Emacs’s build process so I don’t know how and where to add such a file and make Emacs load it on startup.
> 
> Can someone enlighten me on how to define these functions in either C or lisp?
> 
> I think defining them as no-op for Emacs w/o tree-sitter is a good move, because otherwise if some user compiles an Emacs w/o tree-sitter and compiles a bunch of packages, and one of the package happens to be based on tree-sitter, they’ll see a bunch of warnings unless the package author added declare-function forms or the proposed macro. It’s better to have tree-sitter functions always defined and save everybody some hassle.

FWIW, I'd rather we didn't go this way, for the reasons Andrea
explained (and I agreed).  Inventing some new technique or tricks just
for this one case, when we don't do anything like that in any other
similar case, doesn't sound like a good idea, because it would mean
one more subtle protocol to support in Emacs development.  It gets in
the way while reviewing patches, for example, because it is not easy
to remember when special conventions must be observed.  E.g., should
someone come up with a new DEFUN in treesit.c, someone should remember
that a no-op function by the same name should be defined at the same
time.  Or we'd need to #ifdef out the body of each DEFUN, which makes
the code harder to read.

Can we find some other way of making this easier, without defining
no-op functions in treesit.c?

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Yuan Fu]

> On Dec 11, 2024, at 10:43 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 11 Dec 2024 22:05:56 -0800
>> Cc: Andrea Corallo <acorallo@gnu.org>,
>> Eli Zaretskii <eliz@gnu.org>,
>> emacs-devel@gnu.org
>> 
>> 
>> 
>>> On Dec 11, 2024, at 4:23 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
>>> 
>>> On 12/12/2024 01:38, Andrea Corallo wrote:
>>>> The suggestion of having the C functions defined to nops when
>>>> HAVE_TREE_SITTER is not defined is cleaner in principle, but I think
>>>> there are two downsides:
>>>> - We don't tipically do it this way, so it would be an exception
>>> 
>>> We also don't typically require a caller to have a 'declare-function' for every function they use from a package that they (require ...) already.
>>> 
>>>> - Also we should do it for all the functions in treesit.c, otherwise we
>>>>  don't solve the original problem of introducing warnings by mistake on
>>>>  non treesit builds.
>>> 
>>> Sure. All non-static ones anyway.
>> 
>> I tried to define all the C functions and it’s not easy to do: doing it in C seems pretty tedious, we’d need DEFUN form and defsubr for each function (is there a easier way?). If done in lisp, we need to do it in a separate file and _load_ it before compiling treesit.el and any other package that uses tree-sitter. I’m not familiar with Emacs’s build process so I don’t know how and where to add such a file and make Emacs load it on startup.
>> 
>> Can someone enlighten me on how to define these functions in either C or lisp?
>> 
>> I think defining them as no-op for Emacs w/o tree-sitter is a good move, because otherwise if some user compiles an Emacs w/o tree-sitter and compiles a bunch of packages, and one of the package happens to be based on tree-sitter, they’ll see a bunch of warnings unless the package author added declare-function forms or the proposed macro. It’s better to have tree-sitter functions always defined and save everybody some hassle.
> 
> FWIW, I'd rather we didn't go this way, for the reasons Andrea
> explained (and I agreed).  Inventing some new technique or tricks just
> for this one case, when we don't do anything like that in any other
> similar case, doesn't sound like a good idea, because it would mean
> one more subtle protocol to support in Emacs development.  It gets in
> the way while reviewing patches, for example, because it is not easy
> to remember when special conventions must be observed.  E.g., should
> someone come up with a new DEFUN in treesit.c, someone should remember
> that a no-op function by the same name should be defined at the same
> time.  Or we'd need to #ifdef out the body of each DEFUN, which makes
> the code harder to read.
> 
> Can we find some other way of making this easier, without defining
> no-op functions in treesit.c?

Cool. I don’t want to waste more of everybody’s time on this, so I pushed the original patch (sans eval-when-compile) which I believe everybody is ok with.

Yuan

Re: Use (eval-when-compile 'treesit) to save us from writing declare-function forms

[Stefan Monnier]

>>> I thought eval-when-compile is more of a “include if compiling” macro, should’ve checked 🙂
>>>
>>> We can also go with Dmitry’s idea, and just define all the functions in
>>> treesit.el if Emacs is not built with tree-sitter. We can detect that with
>>> treesit-available-p.
>> I haven't really thought about it, so I don't have a strong opinion.
>> AFAICT your proposal should work fine even if you remove the
>> `eval-when-compile` wrappers to which I objected.  🙂
>
> It seems to me that, having taken than route, we would recommend every
>
>   (require 'treesit)
>
> to be accompanied by
>
>   (treesit-declare-c-functions)
>
> ...or the familiar declare-function list, of course.
>
> And the same would also be true for transitive dependencies (so, both
>  c-ts-common and c-ts-mode, even though otherwise the latter could drop the
> (require 'treesit) form).

Yup.  The only cleanish way to avoid having to do that (short of
a major improvement to the byte-compiler's handling of tracking when
functions are defined) is to arrange it to all those function are always
defined (and presumably signal errors when treesit is not actually
available).


        Stefan