A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

Hey Emacs-devel,

We recently ran into a problem in Flycheck that confused us.  Evaluating the following twice yields different results on the first evaluation (nil) and on the subsequent ones (t):

;; nil on the first run; t on subsequent ones
(progn
  (defmacro m (f)
    `(function ,f))
  (functionp (m (lambda ()))))

Here it is with more info added:

;; "NOK #'(lambda nil)" on the first run; "OK (lambda nil)" on subsequent runs.
(progn
  (defmacro m8 (f)
    `(function ,f))
  (let ((out (m8 (lambda ()))))
    (message "%s %S"
             (if (functionp out) "OK" "NOK")
             out)))

This seems to stem from the way macro-evaluation works: running macroexpand-last-sexp before the first evaluation yields this:

(progn
  (defalias 'm (cons 'macro #'(lambda (f) (list 'function f))))
  (functionp (m #'(lambda nil))))

… but on subsequent evaluations it yields that:

(progn
  (defalias 'm (cons 'macro #'(lambda (f) (list 'function f))))
  (functionp #'(lambda nil)))

Is this expected?  Did we miss something in the manual about this?
Background info: https://github.com/flycheck/flycheck/issues/1398

Thanks!
Clément.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

> We recently ran into a problem in Flycheck that confused us.
> Evaluating the following twice yields different results on the first
> evaluation (nil) and on the subsequent ones (t):
>
> ;; nil on the first run; t on subsequent ones
> (progn
>   (defmacro m (f)
>     `(function ,f))
>   (functionp (m (lambda ()))))

Depends what you mean by "evaluating".  At top-level macroexpansion is
supposed to be careful to handle the above correctly (i.e. to evaluate
the defmacro) before performing the macro-expansion of the (functionp ...).
But for that we had to add ad-hoc code both to the byte-compiler and to
the lread.c code that does the eager-macroexpansion of non-compiled files.

So, how do you "evaluate" the above code in order to see this problem?


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-17 16:58, Stefan Monnier wrote:
>> We recently ran into a problem in Flycheck that confused us.
>> Evaluating the following twice yields different results on the first
>> evaluation (nil) and on the subsequent ones (t):
>>
>> ;; nil on the first run; t on subsequent ones
>> (progn
>>   (defmacro m (f)
>>     `(function ,f))
>>   (functionp (m (lambda ()))))
> 
> Depends what you mean by "evaluating".  At top-level macroexpansion is
> supposed to be careful to handle the above correctly (i.e. to evaluate
> the defmacro) before performing the macro-expansion of the (functionp ...).
> But for that we had to add ad-hoc code both to the byte-compiler and to
> the lread.c code that does the eager-macroexpansion of non-compiled files.
> 
> So, how do you "evaluate" the above code in order to see this problem?

Using C-M-x, or just by putting it in my .emacs.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

>>> We recently ran into a problem in Flycheck that confused us.
>>> Evaluating the following twice yields different results on the first
>>> evaluation (nil) and on the subsequent ones (t):
>>>
>>> ;; nil on the first run; t on subsequent ones
>>> (progn
>>>   (defmacro m (f)
>>>     `(function ,f))
>>>   (functionp (m (lambda ()))))
>> 
>> Depends what you mean by "evaluating".  At top-level macroexpansion is
>> supposed to be careful to handle the above correctly (i.e. to evaluate
>> the defmacro) before performing the macro-expansion of the (functionp ...).
>> But for that we had to add ad-hoc code both to the byte-compiler and to
>> the lread.c code that does the eager-macroexpansion of non-compiled files.
>> 
>> So, how do you "evaluate" the above code in order to see this problem?
>
> Using C-M-x, or just by putting it in my .emacs.

I guess we could tweak C-M-x to try and make it work as well, but as for
"just putting it in my .emacs" it should already work (and does work for
me).

The code to handle it when loading a file is in
readevalloop_eager_expand_eval (in src/lread.c), and the code to handle
it for the byte-compiler is in byte-compile-recurse-toplevel (in
bytecomp.el).

Hmm... looking further I see that it "should" already work.
There's just a macroexpand-all that gets in the way.
So the patch below (which I pushed to master) makes it work (at least
for the non-edebug case).


        Stefan


diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index e6e55a37a7..935e55c5d7 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -1132,7 +1132,9 @@ elisp--eval-last-sexp
         (eval-expression-get-print-arguments eval-last-sexp-arg-internal)))
     ;; Setup the lexical environment if lexical-binding is enabled.
     (elisp--eval-last-sexp-print-value
-     (eval (eval-sexp-add-defvars (elisp--preceding-sexp)) lexical-binding)
+     (eval (macroexpand-all
+            (eval-sexp-add-defvars (elisp--preceding-sexp)))
+           lexical-binding)
      (if insert-value (current-buffer) t) no-truncate char-print-limit)))
 
 (defun elisp--eval-last-sexp-print-value
@@ -1165,7 +1167,6 @@ elisp--eval-last-sexp-fake-value
 (defun eval-sexp-add-defvars (exp &optional pos)
   "Prepend EXP with all the `defvar's that precede it in the buffer.
 POS specifies the starting position where EXP was found and defaults to point."
-  (setq exp (macroexpand-all exp))      ;Eager macro-expansion.
   (if (not lexical-binding)
       exp
     (save-excursion

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-18 13:02, Stefan Monnier wrote:
>> Using C-M-x, or just by putting it in my .emacs.
> 
> I guess we could tweak C-M-x to try and make it work as well, but as for
> "just putting it in my .emacs" it should already work (and does work for
> me).
> 
> The code to handle it when loading a file is in
> readevalloop_eager_expand_eval (in src/lread.c), and the code to handle
> it for the byte-compiler is in byte-compile-recurse-toplevel (in
> bytecomp.el).
> 
> Hmm... looking further I see that it "should" already work.
> There's just a macroexpand-all that gets in the way.
> So the patch below (which I pushed to master) makes it work (at least
> for the non-edebug case).

Thanks, that works!  Unfortunately, it looks like I oversimplified the problem, and the the more complete example below is still misbehaved.  Sorry for the oversimplification.

Here's a repro closer to the problem we ran into in Flycheck:

* In a new file test-macro.el, write this:
  
  (defmacro mmm (f)
    `(function ,f))
  (provide 'test-macro)

* In a new file test.el, write this

  (with-eval-after-load 'test-macro
    (let ((out (mmm (lambda ()))))
      (message "with-eval-after-load: %S" out)))

  (load (expand-file-name "test-macro.el"))

* In test.el, run M-x eval-buffer; I get the following output:

  Loading /tmp/test-macro.el (source)...
  with-eval-after-load: #'(lambda nil)
  Loading /tmp/test-macro.el (source)...done

* In test.el, run M-x eval-buffer again; I get the output shown below.  The printed value changes from #'(lambda nil) to (lambda nil), and the with-eval-after-load message is printed twice during loading, for some reason:

  with-eval-after-load: (lambda nil)
  Loading /tmp/test-macro.el (source)...
  with-eval-after-load: #'(lambda nil)
  with-eval-after-load: (lambda nil)
  Loading /tmp/test-macro.el (source)...done

I get these results on the latest master.

Thanks for your help!
Clément.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

> * In a new file test-macro.el, write this:
>   
>   (defmacro mmm (f)
>     `(function ,f))
>   (provide 'test-macro)
>
> * In a new file test.el, write this
>
>   (with-eval-after-load 'test-macro
>     (let ((out (mmm (lambda ()))))
>       (message "with-eval-after-load: %S" out)))
>
>   (load (expand-file-name "test-macro.el"))

This is not supposed to work: your test.el uses `mmm` before it gets
defined by the subsequent `load`.

It's true that Emacs doesn't guarantee that it will always fail to work,
but ... you get what you deserve.


        Stefan


PS: BTW, moving the `load` earlier may make it work more often but it
    still wouldn't make it right.  You'd additionally need to replace
    the `load` by `require` or to wrap it inside a `eval-when-compile`.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-25 22:18, Stefan Monnier wrote:
>> * In a new file test-macro.el, write this:
>>   
>>   (defmacro mmm (f)
>>     `(function ,f))
>>   (provide 'test-macro)
>>
>> * In a new file test.el, write this
>>
>>   (with-eval-after-load 'test-macro
>>     (let ((out (mmm (lambda ()))))
>>       (message "with-eval-after-load: %S" out)))
>>
>>   (load (expand-file-name "test-macro.el"))
> 
> This is not supposed to work: your test.el uses `mmm` before it gets
> defined by the subsequent `load`.

Uh, really? But the body of with-eval-after-load is (supposed to be) evaluated after the FILE argument is loaded, not before, right?  `mmm' is only used in `with-eval-after-load' in the code above.  

The manual says the following about with-eval-after-load:

     This macro arranges to evaluate BODY at the end of loading the file
     LIBRARY, each time LIBRARY is loaded.  If LIBRARY is already
     loaded, it evaluates BODY right away.

Am I misunderstanding something?  In what sense does the code above use `mmm' before it gets defined?  And, does that mean that with-eval-after-load can't be used to run a macro after the package defining it has been loaded?

I guess what's confusing to me is that calling with-eval-after-load seems to be diffrent from directly adding a form to after-load-alist, as shown by the fact that the following snippet works fine:

  (add-to-list
   'after-load-alist
   (list 'test-macro
         (lambda ()
           (eval '(let ((out (mmm (lambda ()))))
                    (message "after-load-alist: %S" out))))))

  (load (expand-file-name "test-macro.el"))

Why does this behave differently? (and isn't that difference a bug in with-eval-after-load?)

> It's true that Emacs doesn't guarantee that it will always fail to work,
> but ... you get what you deserve.

:/ That seems a bit uncalled for, but maybe I'm just misinterpreting a joke?  "what [I] deserve", in the present case, is multiple hours of my time spent debugging an issue raised by a well-meaning Flycheck user, on top of multiple hours of the other maintainer's time… I'm not sure in what sense either of us deserved that.  Sorry if I'm misunderstanding your comment.

> PS: BTW, moving the `load` earlier may make it work more often but it
>     still wouldn't make it right.  You'd additionally need to replace
>     the `load` by `require` or to wrap it inside a `eval-when-compile`.

Can you explain why?  with-eval-after-load doesn't mention any of these things.

Thanks for your help,
Clément.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

>> This is not supposed to work: your test.el uses `mmm` before it gets
>> defined by the subsequent `load`.
> Uh, really? But the body of with-eval-after-load is (supposed to be)
> evaluated after the FILE argument is loaded, not before, right?  `mmm'
> is only used in `with-eval-after-load' in the code above.

The time of evaluation doesn't matter.  Think of the case where we
*compile* the file.


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-26 08:11, Stefan Monnier wrote:
>>> This is not supposed to work: your test.el uses `mmm` before it gets
>>> defined by the subsequent `load`.
>> Uh, really? But the body of with-eval-after-load is (supposed to be)
>> evaluated after the FILE argument is loaded, not before, right?  `mmm'
>> is only used in `with-eval-after-load' in the code above.
> 
> The time of evaluation doesn't matter.  Think of the case where we
> *compile* the file.

Why is that relevant here, given that I'm *not* compiling the file?
(FWIW, I do indeed expect with-eval-after-load to protect its argument from compilation, yes.  Is it documented not to?)

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

> Why is that relevant here, given that I'm *not* compiling the file?

If the semantics depends on interpretation-vs-compilation, then you're
on your own.  Macroexpansion happens "some time" between the moment the
machine leans that a given sexp is to be interpreted as code and the
moment that code is executed.

> (FWIW, I do indeed expect with-eval-after-load to protect its argument from
> compilation, yes.

Why?

> Is it documented not to?)

Only `eval` does, basically.


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

>> Is it documented not to?)
> Only `eval` does, basically.

Actually, I think the criteria is rather: is it `quote`d?


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-26 11:38, Stefan Monnier wrote:
>>> Is it documented not to?)
>> Only `eval` does, basically.
> 
> Actually, I think the criteria is rather: is it `quote`d?

Is that actually a usable criterion?  Any macro can quote or unquote its argument, right?  For example, the problem that I described about with-eval-after-load also happens with eval-after-load, yet the argument to that is quoted.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

> Is that actually a usable criterion?  Any macro can quote or unquote its
> argument, right?

Everything can happen, indeed.  But usually a macro which takes a code
argument and puts it within a quote is perceived as a problem: e.g. it
tends to very quickly annoy its users because they can't refer to
lexically-scoped arguments any more.  Admittedly, for
with-eval-after-load this is not very problematic because it's unusual
to have with-eval-after-load elsewhere than at top-level, so making code
like:

    (let ((x 1))
      (with-eval-after-load FOO
        (message "%S" x)))

work correctly is not nearly as important as it is for many other macros.

Code looks a lot like data, but for reasons of lexical scoping,
*variables* are quite different from symbols, yet we don't have any data
to represent *variables* (so we use symbols instead).

> For example, the problem that I described about with-eval-after-load
> also happens with eval-after-load, yet the argument to that is quoted.

[ I was wondering when someone would notice.  ]

Indeed, when I introduced `with-eval-after-load` and I also changed
`eval-after-load` by adding a compiler macro which turns

     (eval-after-load FOO 'BAR)
into
     (eval-after-load FOO `(,(lambda () BAR)))

so that BAR can be properly eagerly macro-expanded, so the
byte-compiler can look at BAR and emit warnings about it, and also so
that BAR will be interpreted using lexical-binding if the containing
file uses lexical-binding.

Here's the kind of thing I had in mind (can't think of any concrete
example for it, tho, the motivation came from such cases in defadvice
and interactive forms instead, which had the same problem is keeping
code quoted thus preventing compilation and eager macroexpansion):

    (eval-when-compile (require 'cl))
    (eval-after-load FOO '(flet ...))

There are a few other places where we "undo" a quote, as in

    (mapcar '(lambda ...) ...)

These were all done because it seemed to be beneficial more often than
it is harmful.  I must admit that I wasn't 100% sure that "unquoting"
its arg was "more often beneficial", tho the fact that I haven't heard
anyone even mention this until now seems to argue that even tho it's
maybe not "more often beneficial" at least not often harmful.


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-26 12:31, Stefan Monnier wrote:
>> Is that actually a usable criterion?  Any macro can quote or unquote its
>> argument, right?
> 
> Everything can happen, indeed.  But usually a macro which takes a code
> argument and puts it within a quote is perceived as a problem: e.g. it
> tends to very quickly annoy its users because they can't refer to
> lexically-scoped arguments any more.  Admittedly, for
> with-eval-after-load this is not very problematic because it's unusual
> to have with-eval-after-load elsewhere than at top-level, so making code
> like:
> 
>     (let ((x 1))
>       (with-eval-after-load FOO
>         (message "%S" x)))
> 
> work correctly is not nearly as important as it is for many other macros.

Thanks, this is a good point, which I hadn't considered.
It's not obvious to me, however, which one is more important: make something like 

  (with-eval-after-load 'flycheck
    (flycheck-define-checker …))

work, or ensuring that with-eval-after-load can be used to capture lexical variables.

One of the issues to consider is that the usual fix of (require)-ing — or (require-when-compile)-ing — the package that defines the macro isn't satisfactory in this case, since with-eval-after-load is used to run code *after* the package is loaded.  And, since eval-when-compile now behaves the same way, there doesn't seem to be a user-friendly way at the moment to delay the execution a bit of code that includes macros until these macros are available.

Btw, autoloading these macros wouldn't fix this, right?

>> For example, the problem that I described about with-eval-after-load
>> also happens with eval-after-load, yet the argument to that is quoted.
> 
> [ I was wondering when someone would notice.  ]
> 
> Indeed, when I introduced `with-eval-after-load` and I also changed
> `eval-after-load` by adding a compiler macro which turns
> 
>      (eval-after-load FOO 'BAR)
> into
>      (eval-after-load FOO `(,(lambda () BAR)))
> 
> so that BAR can be properly eagerly macro-expanded, so the
> byte-compiler can look at BAR and emit warnings about it, 

That makes sense, but it also has high rates of false positives; writing the following warns on foo-x:

  (eval-after-load 'foo
    '(setq foo-x 1))

… regardless f whether `foo' defines `foo-x'.

> Here's the kind of thing I had in mind (can't think of any concrete
> example for it, tho, the motivation came from such cases in defadvice
> and interactive forms instead, which had the same problem is keeping
> code quoted thus preventing compilation and eager macroexpansion):
> 
>     (eval-when-compile (require 'cl))
>     (eval-after-load FOO '(flet ...))

I see. Tricky.

> There are a few other places where we "undo" a quote, as in
> 
>     (mapcar '(lambda ...) ...)
> 
> These were all done because it seemed to be beneficial more often than
> it is harmful.  I must admit that I wasn't 100% sure that "unquoting"
> its arg was "more often beneficial", tho the fact that I haven't heard
> anyone even mention this until now seems to argue that even tho it's
> maybe not "more often beneficial" at least not often harmful.

I think we've had that particular issue for a while in Flycheck — it's just that we didn't understand it until fairly recently.  It pops up when people put stuff in their use-package configuration, in particular.

Clément.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

> One of the issues to consider is that the usual fix of (require)-ing —
> or (require-when-compile)-ing — the package that defines the macro
> isn't satisfactory in this case, since with-eval-after-load is used to
> run code *after* the package is loaded.  And, since eval-when-compile
> now behaves the same way, there doesn't seem to be a user-friendly way
> at the moment to delay the execution a bit of code that includes
> macros until these macros are available.

We could add a new macro, which does kind of the "reverse" of
`eval-when-compile`:

    (defmacro dont-compile-before-eval (&rest body)
      "Wrap BODY to hide it from macroexpander and compiler.
    BODY will not be compiled nor macroexpanded until the code is
    actually evaluated.
    Note: BODY cannot refer to surrounding lexically scoped variables."
      `(eval '(progn . ,body) ',lexical-binding))

to make it slightly more "user-friendly".

I have several times been tempted to tweak the byte-compiler and
macroexpander to automatically do this kind of wrapping when faced with
a call of the form (FOO ...) where FOO is not known at all
(i.e. instead of presuming that FOO will be a function), but the "Note:"
above would make such an auto-rewrite only safe when there's no
surrounding lexically-scoped vars (i.e. at top-level or when
lexical-binding is nil).

> Btw, autoloading these macros wouldn't fix this, right?

Yes and no: it would make it behave semantically correctly, but it would
cause flycheck to be loaded eagerly, just as with "require-when-compile".

> I think we've had that particular issue for a while in Flycheck — it's just
> that we didn't understand it until fairly recently.  It pops up when people
> put stuff in their use-package configuration, in particular.

Similar problems show up when package FOO wants to provide support for
interacting with package BAR but doesn't want to force BAR to be
a dependency: if the support for BAR involves the use of macros provided
by BAR, then you need extra gymnastics to prevent the byte-compiler from
mis-compiling the files when BAR is not (yet) installed.

Another related situation is when package FOO wants to use a feature
(and associated macro) only available in Emacs>NN.MM. Using the usual

    (when (fboundp 'blabla)
      (blabla ...))

is sufficient except when the file is compiled with Emacs<NN.MM and then
used on Emacs>NN.MM since the (blabla ...) was miscompiled.
AUCTeX uses a macro `TeX--if-macro-fboundp` to encapsulate its
solution to the problem.

The cleanest solution is to avoid macros ;-)


        Stefan "who loves macros"

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-26 11:31, Stefan Monnier wrote:
>> Why is that relevant here, given that I'm *not* compiling the file?

> If the semantics depends on interpretation-vs-compilation, then you're
> on your own.  Macroexpansion happens "some time" between the moment the
> machine leans that a given sexp is to be interpreted as code and the
> moment that code is executed.

Stefan, I think we're miscommunicating.  Sorry if I'm being unclear or unhelpful.  I'm trying to understand what the proper fix is to a bug that was reported by a Flycheck user.  You're telling me that I "get what I deserved" and that I'm "on my own" :/

My user's problem is the following: Flycheck defines a macro that lets you define a new checker.  One of our users wrote something like this:

(with-eval-after-load 'flycheck
  (flycheck-define-checker
    …))

This doesn't work.  I claim that it should, or that it should be documented to not work.  Do you disagree?

>> (FWIW, I do indeed expect with-eval-after-load to protect its argument from
>> compilation, yes.
> Why?

Because of the documentation says that this macro arranges for evaluation to be delayed.  It's OK if the contents are compiled, but only if that doesn't change the semantics.  Otherwise, either the docs of with-eval-after-load are wrong, or with-eval-after-load has a bug, AFAICT.

Sorry if I'm missing something obvious.
Clément.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

> My user's problem is the following: Flycheck defines a macro that lets
> you define a new checker.  One of our users wrote something like this:
>
> (with-eval-after-load 'flycheck
>   (flycheck-define-checker
>     …))
>
> This doesn't work.  I claim that it should, or that it should be
> documented to not work.  Do you disagree?

I think it is or should be documented, indeed, but not in
`with-eval-after-load` since this applies to everything else as well.

If you need to use a macro that's in file `foo`, then AFAIK you have
2 options:

- (require 'foo), either directly or transitively.
- hide your macro call behind a quote so it can't be compiled before
  `foo` is loaded (regardless of whether you do compile or not
  irrelevant: the right way to think about it is "what happens when it
  gets compiled").

> Because of the documentation says that this macro arranges for evaluation to
> be delayed.

But macro expansion can take place long before evaluation.
E.g. `defun` also delays evaluation of its body.  But

    (defun foo ()
      ... (mmm blabla) ...)

    (defmacro mmm (...) ...)

will suffer from the same problem (tho at least in this particular
case, the byte-compiler will notice and emit a warning, IIRC).


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-26 12:13, Stefan Monnier wrote:
>> My user's problem is the following: Flycheck defines a macro that lets
>> you define a new checker.  One of our users wrote something like this:
>>
>> (with-eval-after-load 'flycheck
>>   (flycheck-define-checker
>>     …))
>>
>> This doesn't work.  I claim that it should, or that it should be
>> documented to not work.  Do you disagree?
> 
> I think it is or should be documented, indeed, but not in
> `with-eval-after-load` since this applies to everything else as well.

Got it.

> If you need to use a macro that's in file `foo`, then AFAIK you have
> 2 options:
> 
> - (require 'foo), either directly or transitively.
> - hide your macro call behind a quote so it can't be compiled before
>   `foo` is loaded (regardless of whether you do compile or not
>   irrelevant: the right way to think about it is "what happens when it
>   gets compiled").

Thanks, that makes sense.

My claim is that there could (should?) be a third option: hide the macro call in a with-eval-after-load block, and that your second point should cover eval-after-load: I find it especially confusing that eval-after-load bypasses the quote.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-26 12:32, Clément Pit-Claudel wrote:
> I find it especially confusing that eval-after-load bypasses the quote.

Looks like it wasn't always like that; eval-after-load's behavior was changed in 2013:

commit de0503df97a507a523a192e877a8d5c7439c4846
Author: Stefan Monnier <monnier@iro.umontreal.ca>
Date:   Thu Jun 13 18:24:52 2013 -0400

    * lisp/subr.el (with-eval-after-load): New macro.
    (eval-after-load): Allow form to be a function.
    take advantage of lexical-binding.
    (do-after-load-evaluation): Use dolist and adjust to new format.
    * lisp/simple.el (bad-packages-alist): Use dolist and with-eval-after-load.
    * doc/lispref/loading.texi (Hooks for Loading): Document with-eval-after-load
    instead of eval-after-load.  Don't document after-load-alist.
    * src/lread.c (syms_of_lread):
    * src/fns.c (Fprovide): Adjust to new format of after-load-alist.

That code added a (declare (compiler-macro …)) form to eval-after-load, which says ';; Quote with lambda so the compiler can look inside'.

Was the change in behavior intended? Without that 'declare' form, eval-after-load doesn't look into its argument and makes it easy to delay the interpretation of a snippet until a file is loaded.

Clément.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Radon Rosborough]

> I think it is or should be documented

The process by which macroexpansion, byte-compilation, and evaluation
all interact is highly nontrivial and not intuitive to the
lay-programmer. I don't know of any other language than Elisp with a
comparably complex interpreter process. Is there any central location
where all this stuff is documented (i.e. not "if you look at these
five different sections of the manual and put them together...")? I
have encountered problems similar to the ones described in this
thread, and think I mostly understand them, but it is all inference
and guesswork.

Best,
Rаdon

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

> (with-eval-after-load 'flycheck
>   (flycheck-define-checker
>     …))

I was about to suggest that you might ask the Flycheck maintainer to
provide a function that can be used instead of the macro
flycheck-define-checker, but I see that it's pretty much already the
case: just use flycheck-define-command-checker.

Another option, is to compile that file and then you can do

    (eval-when-compile (require 'flycheck))
    (with-eval-after-load 'flycheck
      (flycheck-define-checker
        …))

it also works if you don't byte-compile, except that the
with-eval-after-load becomes ineffective.


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Another option, is to compile that file and then you can do
>
>     (eval-when-compile (require 'flycheck))
>     (with-eval-after-load 'flycheck
>       (flycheck-define-checker
>         …))
>
> it also works if you don't byte-compile, except that the
> with-eval-after-load becomes ineffective.

So that's not really an option.

I fear that this problem, although there are acceptable workarounds,
could be a regularly recurring issue.  It's not an unnatural expectation
that code in `eval-when-compile' is allowed to use all of the stuff
defined in the FILE.  People use this in their init files, not all will
know about the underlying mechanisms, so this is not really optimal.

Would it make sense to introduce an optional argument for
`eval-after-load' that, when non-nil, arranges that the FORM is
untouched until the FILE is loaded, and then compiled and loaded?  

A disadvantage would be that compiling the init file would not warn any
more about problems in the specified FORM.


Michael.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

>> it also works if you don't byte-compile, except that the
>> with-eval-after-load becomes ineffective.
> So that's not really an option.

Depends: if we automatically byte-compile all files then it shouldn't be
a real problem.


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-27 16:35, Stefan Monnier wrote:
> I was about to suggest that you might ask the Flycheck maintainer to
> provide a function that can be used instead of the macro
> flycheck-define-checker, but I see that it's pretty much already the
> case: just use flycheck-define-command-checker.

Oh, the Flycheck maintainers is already well aware of the issue: there are two of them, and one of them is me :)
That function is semi-internal: we don't want people to have to use it.

> Another option, is to compile that file and then you can do
> 
>     (eval-when-compile (require 'flycheck))
>     (with-eval-after-load 'flycheck
>       (flycheck-define-checker
>         …))
> 
> it also works if you don't byte-compile, except that the
> with-eval-after-load becomes ineffective.

What do you mean by ineffective? It would just run immediately because the form above loads Flycheck when interpreted, right?

Clément.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

>> I was about to suggest that you might ask the Flycheck maintainer to
>> provide a function that can be used instead of the macro
>> flycheck-define-checker, but I see that it's pretty much already the
>> case: just use flycheck-define-command-checker.
> Oh, the Flycheck maintainers is already well aware of the issue: there are
> two of them, and one of them is me :)
> That function is semi-internal: we don't want people to have to use it.

Then I guess my suggestion is to make it non-internal.

>> it also works if you don't byte-compile, except that the
>> with-eval-after-load becomes ineffective.
> What do you mean by ineffective? It would just run immediately because the
> form above loads Flycheck when interpreted, right?

Yes.


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

>          (lambda ()
>            (eval '(let ((out (mmm (lambda ()))))
>                     (message "after-load-alist: %S" out))))))

The use of `eval` means that you *hide* the call to `mmm` until that
point, so in that case indeed the macro can't be expanded earlier.


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-26 08:13, Stefan Monnier wrote:
>>          (lambda ()
>>            (eval '(let ((out (mmm (lambda ()))))
>>                     (message "after-load-alist: %S" out))))))
> 
> The use of `eval` means that you *hide* the call to `mmm` until that
> point, so in that case indeed the macro can't be expanded earlier.

Then why doesn't with-eval-after-load do that?  Is it just a bug?

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

> Then why doesn't with-eval-after-load do that?

What would be the benefit?


        Stefan

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Michael Heerdegen]

Hello Clément,

Are we sure `mmm' always does what you expect?  Note that it gets called
with different arguments in these two cases.  You have

(mmm (lambda () nil)) ==> (lambda nil nil)

With the `with-eval-after-load' called before `mmm' is defined, I get an
entry like this in `after-load-alist':

| (test-macro
|   #[0 #1="byte code..."
|       [(lambda nil
|          (let
|              ((out
|                (mmm
|                 #'(lambda nil nil))))
|            (message "with-eval-after-load: %S" out)))
|       ...]
|       11])

The lambda macro has been expanded, `mmm' not (because it had not been
known).  So `mmm' get called with the sharp-quoted lambda form, and that
returns something different:

(mmm #'(lambda nil nil)) ==> #'(lambda nil nil)

Maybe you can change `mmm' so that it handles both of these cases as you
want?  Or use a function like (defun mmm2 (f) `#',f)?


Michael.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Clément Pit-Claudel]

On 2018-02-26 11:37, Michael Heerdegen wrote:
> Hello Clément,

Hey Michael,

> Are we sure `mmm' always does what you expect?

Hard to say. I'm trying to gauge how much of what I observe is a bug, and how much is expected :)

> Note that it gets called
> with different arguments in these two cases.  

I think that's a problem, yes.  But is it the expected behavior for it to be called with different arguments?

> You have
> 
> (mmm (lambda () nil)) ==> (lambda nil nil)
> 
> With the `with-eval-after-load' called before `mmm' is defined, I get an
> entry like this in `after-load-alist':
> 
> | (test-macro
> |   #[0 #1="byte code..."
> |       [(lambda nil
> |          (let
> |              ((out
> |                (mmm
> |                 #'(lambda nil nil))))
> |            (message "with-eval-after-load: %S" out)))
> |       ...]
> |       11])

Right, I think that's broken.

> Maybe you can change `mmm' so that it handles both of these cases as you
> want?

We're considering this for Flycheck, yup :)  I suggested a similar solution in https://github.com/flycheck/flycheck/issues/1398#issuecomment-365660601

Or use a function like (defun mmm2 (f) `#',f)?

I don't think that works for our particular use case, because we need to handle unquoted functions.

Mostly though, I'd like to understand where the issue comes from (my current understanding is that it's a miscompilation), and whether it can be fixed.

(The original problem popped up a few years ago from a user of use-package, but back then we didn't track it down; this time I'd like to fix it once and for all)

Thanks for your help!
Clément.

Re: A combination of defmacro, functionp, and quoted lambdas yields different results on consecutive evaluations

[Stefan Monnier]

>> | (test-macro
>> |   #[0 #1="byte code..."
>> |       [(lambda nil
>> |          (let
>> |              ((out
>> |                (mmm
>> |                 #'(lambda nil nil))))
>> |            (message "with-eval-after-load: %S" out)))
>> |       ...]
>> |       11])
>
> Right, I think that's broken.

Yes, at that time it's already too late.

>> Maybe you can change `mmm' so that it handles both of these cases as you
>> want?
> We're considering this for Flycheck, yup :)

That won't help the case where the code was compiled.  You really need
to think in terms of compilation: people generally don't have a clear
idea of when macroexpansion takes place for interpreted code, whereas
it's clearer in the case the code is compiled.

> Mostly though, I'd like to understand where the issue comes from (my current
> understanding is that it's a miscompilation), and whether it can be fixed.

That's right, it gets miscompiled because the compiler isn't told about
the `mmm` macro beforehand so it presumes that it's an unknown function.


        Stefan