Re: Expansion of macros at compile time in eval-and-compile bodies

[Stefan Monnier <monnier@iro.umontreal.ca>]

>   In order for compilation of macro calls to work, the macros must
>   already be defined in Lisp when the calls to them are compiled.  The
>   compiler has a special feature to help you do this: if a file being
>   compiled contains a ‘defmacro’ form, the macro is defined temporarily
>   for the rest of the compilation of that file.
>
> That would seem to imply that the error shouldn't happen, because the
> macro is defined in the same file before it is called.

The "temporarily" part is too vague to precisely describe what happens.
What really happens is that the compiler remembers the macros it has
compiled in the current file so that when it seen a call to them it can
expand it.  But it's not defined globally, so any macro-expansion that
is not performed directly by the byte-compiler doesn't actually see it.

Whether the provided example should or should not work depends on how we
think macro-expansion within `eval-and-compile` should behave.
Maybe Common-Lisp's `eval-when` specifies this more precisely, and that
could be a motivation to make the behavior of `eval-and-compile` more
precise as well (and that might also motivate changing its behavior in
corner cases like the above, tho not necessarily).

>   You can get a similar result by putting BODY in a separate file and
>   referring to that file with ‘require’.

Clearly, if you put BODY into a separate file and `require` it, BODY
won't be macro-expanded directly by the compiler that compiles the
`require` (it should ideally be macro-expanded in a separate compilation
phase before compiling the file that `require`s).

> Does that mean that the body of an eval-and-compile form is evaluated in
> a different environment than the one defined by the file containing the
> eval-and-compile form?

Pretty much.  tho I don't think any code relies on that detail.

> If so, putting the macro definition in an
> eval-and-compile form wouldn't seem to imply that a macro defined in it
> would be available in subsequent eval-and-compile forms.

Yes, it does, because the two forms contained within those two
`eval-and-compile` will be executed in sequence within the same Emacs
session, so the first will globally define the macro, which will thus be
defined when macro-expanding the second.

> Maybe I'm just missing something obvious, but I wonder if the manual is
> missing an explanation of this finer point.

A lof of such finer points are purposefully not documented because the
exact behavior will depend on circumstances (we want to leave some
freedom to the implementation).

IOW, the user of those macros should avoid relying on those
finer points.

>> Changing topic I have no idea why this instead go through.  To me looks
>> like a bug, isn't?
>>
>> #+BEGIN_SRC elisp
>> ;;; bufler.el  -*- lexical-binding: t; -*-
>>
>> (defmacro bufler-defauto-group ())
>>
>>  (eval-and-compile
>>    (bufler-defauto-group))
>> #+BEGIN_END

I think it's a side-effect of `byte-compile-recurse-toplevel`, which
tries to implement the behavior described in CLHS section
3.2.3.1, "Processing of Top Level Forms".

Maybe this could be considered as a bug, but we could also just say that
it's one of those finer points on which one shouldn't rely.


        Stefan