all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Using the content of a dynamic variable in a macro
@ 2023-02-24  6:31 Damien Cassou
  2023-02-26 11:47 ` Michael Heerdegen
  0 siblings, 1 reply; 11+ messages in thread
From: Damien Cassou @ 2023-02-24  6:31 UTC (permalink / raw)
  To: help-gnu-emacs

Hi,

I'm wondering why the code below works but won't compile.

foo.el:
  (defvar foo-var '((message "hello world")))
  
  (defmacro foo-macro ()
    `(progn
       ,@foo-var))
  
  (defun foo-fun ()
    (foo-macro))


$ emacs --batch -l foo.el --eval '(foo-fun)'
hello world

$ emacs --batch --eval '(find-file "foo.el")' --eval '(emacs-lisp-byte-compile)'
In toplevel form:
foo.el:32:1: Error: Symbol’s value as variable is void: foo-var

Why isn't the compiler aware of the foo-var variable?

-- 
Damien Cassou

"Success is the ability to go from one failure to another without
losing enthusiasm." --Winston Churchill



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Using the content of a dynamic variable in a macro
@ 2023-02-25  8:34 Damien Cassou
  2023-02-25  9:38 ` Eli Zaretskii
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Damien Cassou @ 2023-02-25  8:34 UTC (permalink / raw)
  To: emacs-devel

I tried to send the mail below to help-gnu-emacs@gnu.org (without being
subscribed) but it never reached the archive:
https://lists.gnu.org/archive/html/help-gnu-emacs/2023-02/threads.html.

Hi,

I'm wondering why the code below works but won't compile.

foo.el:
  (defvar foo-var '((message "hello world")))
  
  (defmacro foo-macro ()
    `(progn
       ,@foo-var))
  
  (defun foo-fun ()
    (foo-macro))


$ emacs --batch -l foo.el --eval '(foo-fun)'
hello world

$ emacs --batch --eval '(find-file "foo.el")' --eval '(emacs-lisp-byte-compile)'
In toplevel form:
foo.el:32:1: Error: Symbol’s value as variable is void: foo-var

Why isn't the compiler aware of the foo-var variable?

-- 
Damien Cassou

"Success is the ability to go from one failure to another without
losing enthusiasm." --Winston Churchill



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-25  8:34 Using the content of a dynamic variable in a macro Damien Cassou
@ 2023-02-25  9:38 ` Eli Zaretskii
  2023-02-25  9:43   ` Damien Cassou
  2023-02-25  9:40 ` tomas
  2023-02-27 19:03 ` Basil Contovounesios
  2 siblings, 1 reply; 11+ messages in thread
From: Eli Zaretskii @ 2023-02-25  9:38 UTC (permalink / raw)
  To: Damien Cassou; +Cc: emacs-devel

> From: Damien Cassou <damien@cassou.me>
> Date: Sat, 25 Feb 2023 09:34:07 +0100
> 
> I'm wondering why the code below works but won't compile.
> 
> foo.el:
>   (defvar foo-var '((message "hello world")))
>   
>   (defmacro foo-macro ()
>     `(progn
>        ,@foo-var))
>   
>   (defun foo-fun ()
>     (foo-macro))
> 
> 
> $ emacs --batch -l foo.el --eval '(foo-fun)'
> hello world
> 
> $ emacs --batch --eval '(find-file "foo.el")' --eval '(emacs-lisp-byte-compile)'
> In toplevel form:
> foo.el:32:1: Error: Symbol’s value as variable is void: foo-var
> 
> Why isn't the compiler aware of the foo-var variable?

Because lexical-binding is turned on?



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-25  8:34 Using the content of a dynamic variable in a macro Damien Cassou
  2023-02-25  9:38 ` Eli Zaretskii
@ 2023-02-25  9:40 ` tomas
  2023-02-27 19:03 ` Basil Contovounesios
  2 siblings, 0 replies; 11+ messages in thread
From: tomas @ 2023-02-25  9:40 UTC (permalink / raw)
  To: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1392 bytes --]

On Sat, Feb 25, 2023 at 09:34:07AM +0100, Damien Cassou wrote:
> I tried to send the mail below to help-gnu-emacs@gnu.org (without being
> subscribed) but it never reached the archive:
> https://lists.gnu.org/archive/html/help-gnu-emacs/2023-02/threads.html.
> 
> Hi,
> 
> I'm wondering why the code below works but won't compile.
> 
> foo.el:
>   (defvar foo-var '((message "hello world")))
>   
>   (defmacro foo-macro ()
>     `(progn
>        ,@foo-var))
>   
>   (defun foo-fun ()
>     (foo-macro))
> 
> 
> $ emacs --batch -l foo.el --eval '(foo-fun)'
> hello world
> 
> $ emacs --batch --eval '(find-file "foo.el")' --eval '(emacs-lisp-byte-compile)'
> In toplevel form:
> foo.el:32:1: Error: Symbol’s value as variable is void: foo-var
> 
> Why isn't the compiler aware of the foo-var variable?

Because the (dynamic) variable comes into existence later,
at run time (at compile time the (defvar ...) hasn't yet
been executed).

You might try to wrap your defvar in an (eval-when-compile ...)
form. Other more knowledgeable folks around here might chime
in whether (or when) this may be a good idea or not.

But the main takeaway is that compile time (and thus macro
expansion time) and run time are kind of different "worlds".

(See also eval-and-compile, but in this case you'll be fine
with eval-when-compile, I think)

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-25  9:38 ` Eli Zaretskii
@ 2023-02-25  9:43   ` Damien Cassou
  2023-02-25 12:32     ` Lynn Winebarger
  0 siblings, 1 reply; 11+ messages in thread
From: Damien Cassou @ 2023-02-25  9:43 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:
>> Why isn't the compiler aware of the foo-var variable?
>
> Because lexical-binding is turned on?

yes it is but I miss the deep understanding to fully understand.

-- 
Damien Cassou

"Success is the ability to go from one failure to another without
losing enthusiasm." --Winston Churchill



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-25  9:43   ` Damien Cassou
@ 2023-02-25 12:32     ` Lynn Winebarger
  2023-02-27  3:24       ` Richard Stallman
  0 siblings, 1 reply; 11+ messages in thread
From: Lynn Winebarger @ 2023-02-25 12:32 UTC (permalink / raw)
  To: Damien Cassou; +Cc: Eli Zaretskii, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

On Sat, Feb 25, 2023, 5:19 AM Damien Cassou <damien@cassou.me> wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
> >> Why isn't the compiler aware of the foo-var variable?
> >
> > Because lexical-binding is turned on?
>
> yes it is but I miss the deep understanding to fully understand.
>

Lexical binding isn't the issue - the variable is global in scope.
The issue is that when you explicitly run the byte-compiler in batch mode,
the "defvar" expression is only compiled, not evaluated, while the defmacro
is evaluated, and the application of the macro function is evaluated during
compilation.
Hence you need "eval-when-compile" per Tomas's response.
The reason it works in interactive mode is that the defvar is evaluated,
not compiled, and the macro function is evaluated in the run-time
environment.

Lynn

[-- Attachment #2: Type: text/html, Size: 1389 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-24  6:31 Damien Cassou
@ 2023-02-26 11:47 ` Michael Heerdegen
  0 siblings, 0 replies; 11+ messages in thread
From: Michael Heerdegen @ 2023-02-26 11:47 UTC (permalink / raw)
  To: Damien Cassou; +Cc: help-gnu-emacs

Damien Cassou <damien@cassou.me> writes:

> I'm wondering why the code below works but won't compile.
>
> foo.el:
>   (defvar foo-var '((message "hello world")))
>   
>   (defmacro foo-macro ()
>     `(progn
>        ,@foo-var))
>   
>   (defun foo-fun ()
>     (foo-macro))
> [...]
> foo.el:32:1: Error: Symbol’s value as variable is void: foo-var
>
> Why isn't the compiler aware of the foo-var variable?

The compiler is aware of it, but it doesn't eval the definition.  It is
a feature that the compiler normally does not load any part of the
program when compiling, including variable definitions.  Note that in
the general case, evaluating a defvar form might require to load more
parts of a file, or even the complete file.

If you do need to eval something when compiling, use `eval-when-compile'
or `eval-and-compile' respectively.


Michael.



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-25 12:32     ` Lynn Winebarger
@ 2023-02-27  3:24       ` Richard Stallman
  2023-02-27 19:05         ` Basil Contovounesios
  0 siblings, 1 reply; 11+ messages in thread
From: Richard Stallman @ 2023-02-27  3:24 UTC (permalink / raw)
  To: Lynn Winebarger; +Cc: damien, eliz, emacs-devel

  > Lexical binding isn't the issue - the variable is global in scope.
  > The issue is that when you explicitly run the byte-compiler in batch mode,
  > the "defvar" expression is only compiled, not evaluated, while the defmacro
  > is evaluated, and the application of the macro function is evaluated during
  > compilation.

Should we change this to evaluate defvar in batch mode?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)





^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-25  8:34 Using the content of a dynamic variable in a macro Damien Cassou
  2023-02-25  9:38 ` Eli Zaretskii
  2023-02-25  9:40 ` tomas
@ 2023-02-27 19:03 ` Basil Contovounesios
  2 siblings, 0 replies; 11+ messages in thread
From: Basil Contovounesios @ 2023-02-27 19:03 UTC (permalink / raw)
  To: Damien Cassou; +Cc: emacs-devel

Damien Cassou [2023-02-25 09:34 +0100] wrote:

> I'm wondering why the code below works but won't compile.
>
> foo.el:
>   (defvar foo-var '((message "hello world")))
>   
>   (defmacro foo-macro ()
>     `(progn
>        ,@foo-var))
>   
>   (defun foo-fun ()
>     (foo-macro))
>
>
> $ emacs --batch -l foo.el --eval '(foo-fun)'
> hello world
>
> $ emacs --batch --eval '(find-file "foo.el")' --eval '(emacs-lisp-byte-compile)'
> In toplevel form:
> foo.el:32:1: Error: Symbol’s value as variable is void: foo-var
>
> Why isn't the compiler aware of the foo-var variable?

[ Lynn and Tomas have already answered this, but just to put it all
  together and cite the manual: ]

The reason is that the byte-compiler generally avoids evaluating the
source it is compiling, with the exception of forms like 'require' and
'eval-and-compile'.

But at the same time, the byte-compiler needs to expand all macro calls.

So in foo-fun it tries to expand the call to foo-macro, which inspects
the value of foo-var, but foo-var's initial value has not yet been
evaluated (by the byte-compiler, that is).

To load a definition into the byte-compiler, it needs to be placed in
either eval-when-compile (for things specific to the byte-compilation
process, such as loading macro definitions from another library),
eval-and-compile (for things that are needed at load time as well, such
as defining a missing function or modifying load-path), or a separate
library that is then 'require'd.

'require' is probably the least likely to confuse the reader, but is
often overkill.  In the example foo-var is only ever used during
macroexpansion, so eval-when-compile is okay - the variable will be
completely omitted from the .elc, but no load/run time code uses it
anyway.

But IME this is a rare use-case for global variables, which are more
often also present in the resulting .elc - so eval-and-compile may be
safer or more appropriate depending on the context.  It is also closer
to 'require'.

This stuff is briefly touched on under "(elisp) Compilation Functions":

--8<---------------cut here---------------start------------->8---
   Be careful when writing macro calls in files that you intend to
byte-compile.  Since macro calls are expanded when they are compiled,
the macros need to be loaded into Emacs or the byte compiler will not do
the right thing.  The usual way to handle this is with ‘require’ forms
which specify the files containing the needed macro definitions (see
Named Features).  Normally, the byte compiler does not evaluate the
code that it is compiling, but it handles ‘require’ forms specially, by
loading the specified libraries.  To avoid loading the macro definition
files when someone _runs_ the compiled program, write
‘eval-when-compile’ around the ‘require’ calls (see Eval During
Compile).  For more details, see Compiling Macros.
--8<---------------cut here---------------end--------------->8---

HTH,

-- 
Basil



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-27  3:24       ` Richard Stallman
@ 2023-02-27 19:05         ` Basil Contovounesios
  2023-02-28  2:44           ` Lynn Winebarger
  0 siblings, 1 reply; 11+ messages in thread
From: Basil Contovounesios @ 2023-02-27 19:05 UTC (permalink / raw)
  To: Richard Stallman; +Cc: Lynn Winebarger, damien, eliz, emacs-devel

Richard Stallman [2023-02-26 22:24 -0500] wrote:

>   > Lexical binding isn't the issue - the variable is global in scope.
>   > The issue is that when you explicitly run the byte-compiler in batch mode,
>   > the "defvar" expression is only compiled, not evaluated, while the defmacro
>   > is evaluated, and the application of the macro function is evaluated during
>   > compilation.
>
> Should we change this to evaluate defvar in batch mode?

The byte-compiler avoids evaluating the defvar also in non-batch mode.

So the question is how much of the code that it is compiling should the
byte-compiler evaluate.

We may need to reuse or possibly extend the notions of Lisp code safety
we have before the byte-compiler can start evaluating arbitrary
expressions.  I'm not sure it's worth the complexity.

-- 
Basil



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Using the content of a dynamic variable in a macro
  2023-02-27 19:05         ` Basil Contovounesios
@ 2023-02-28  2:44           ` Lynn Winebarger
  0 siblings, 0 replies; 11+ messages in thread
From: Lynn Winebarger @ 2023-02-28  2:44 UTC (permalink / raw)
  To: Basil Contovounesios
  Cc: Richard Stallman, Damien Cassou, Eli Zaretskii, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1726 bytes --]

On Mon, Feb 27, 2023, 2:05 PM Basil Contovounesios <contovob@tcd.ie> wrote:

> Richard Stallman [2023-02-26 22:24 -0500] wrote:
>
> >   > Lexical binding isn't the issue - the variable is global in scope.
> >   > The issue is that when you explicitly run the byte-compiler in batch
> mode,
> >   > the "defvar" expression is only compiled, not evaluated, while the
> defmacro
> >   > is evaluated, and the application of the macro function is evaluated
> during
> >   > compilation.
> >
> > Should we change this to evaluate defvar in batch mode?
>
> The byte-compiler avoids evaluating the defvar also in non-batch mode.
>
> So the question is how much of the code that it is compiling should the
> byte-compiler evaluate.
>
> We may need to reuse or possibly extend the notions of Lisp code safety
> we have before the byte-compiler can start evaluating arbitrary
> expressions.  I'm not sure it's worth the complexity.
>

I didn't word my response very precisely.  I mentioned batch mode because
the user was byte compiling in a separate batch process, so it seemed
clearer to say that the compile time and evaluation time environments must
be distinct in that case.
The difference I described above happens any time a file is compiled as a
unit and not evaluated.
One reason to not make defvar (and defun) take effect in the compile-time
environment is that newcomers to across may mistakenly believe that
changing the value of a variable should be able to dynamically change the
code produced by instances already loaded.  In other words, believing that
macro expansion can be dynamic, when in fact the result of the expansion is
a constant embedded in the compiled file (modulo optimization by the byte
compiler).

Lynn

[-- Attachment #2: Type: text/html, Size: 2484 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2023-02-28  2:44 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-02-25  8:34 Using the content of a dynamic variable in a macro Damien Cassou
2023-02-25  9:38 ` Eli Zaretskii
2023-02-25  9:43   ` Damien Cassou
2023-02-25 12:32     ` Lynn Winebarger
2023-02-27  3:24       ` Richard Stallman
2023-02-27 19:05         ` Basil Contovounesios
2023-02-28  2:44           ` Lynn Winebarger
2023-02-25  9:40 ` tomas
2023-02-27 19:03 ` Basil Contovounesios
  -- strict thread matches above, loose matches on Subject: below --
2023-02-24  6:31 Damien Cassou
2023-02-26 11:47 ` Michael Heerdegen

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.