Why (eval-when-compile (require 'foo)) does not bind functions during compilation?

Why (eval-when-compile (require 'foo)) does not bind functions during compilation?

[Marcin Borkowski]

As in the subject.  The manual says that it does define macros from the
library “foo”, but not functions.  Why is that so?  I would think that
the above form actually evaluates the (require 'foo) part during
compilation, so in particular it should evaluate all the defuns there –
but both experiments and manual confirm that I’m wrong.  Where is my
mental model incorrect?

TIA,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Why (eval-when-compile (require 'foo)) does not bind functions during compilation?

[Pascal J. Bourguignon]

Marcin Borkowski <mbork@mbork.pl> writes:

> As in the subject.  The manual says that it does define macros from the
> library “foo”, but not functions.  Why is that so?  I would think that
> the above form actually evaluates the (require 'foo) part during
> compilation, so in particular it should evaluate all the defuns there –
> but both experiments and manual confirm that I’m wrong.  Where is my
> mental model incorrect?

Your mental model should be that compilation is reading data and
producing data, and there's no reason for that process to modify the
compiler.

A compiler is a machine like that:

            +--------------------------------+
            |  source      --->      object  |
            | language              language |
            +---------+            +---------+
                      |            |
                      |            |
                      |            |
                      |  machine   |
                      |  language  |
                      +------------+
                      +------------+
                       \ machine  /
                        \language/
                         \      /  <-- machine
                          \    /
                           \  /
                            \/


It takes a program written in the source language:

            +-----------+
            | program 1 |
            |           |
            |           |
            |           |
            |  source   |
            | language  |
            +-----------+
            +--------------------------------+
            |  source      --->      object  |
            | language              language |
            +---------+            +---------+
                      |            |
                      |            |
                      |            |
                      |  machine   |
                      |  language  |
                      +------------+
                      +------------+
                       \ machine  /
                        \language/
                         \      /  <-- machine
                          \    /
                           \  /
                            \/


And produce an equivalent program in object language:

            +-----------+        +-----------+
            | program 1 |        | program 1 |
            |           |        |           |
            |           |        |           |
            |           |  -->   |           |
            |  source   |        |  object   |
            | language  |        | language  |
            +-----------+        +-----------+
            +--------------------------------+
            |  source      --->      object  |
            | language              language |
            +---------+            +---------+
                      |            |
                      |            |
                      |            |
                      |  machine   |
                      |  language  |
                      +------------+
                      +------------+
                       \ machine  /
                        \language/
                         \      /  <-- machine
                          \    /
                           \  /
                            \/



There's no reason to incorporate this program in object language into
the compiler itself.  For one thing, it's possible the object language
and the machine language to be different, so there would be no use of
merging them.  For another, you could be compiling a function of a new
version of the compiler, and if you merged it with the running compiler,
you could easily break it.

 +------------------------------+     +------------------------------+
 |  source     --->     object  | ->  |  source     --->     object  |
 | language            language |     | language            language |
 +---------+          +---------+     +---------+          +---------+
           |          |                         |          |          
           |          |                         |          |          
           |          |                         |          |          
           |  source  |                         |  object  |          
           | language |                         | language |          
           +----------+                         +----------+          
           +-----------------------------------------------+
           |  source             --->              object  |
           | language                             language |
           +----------------+            +-----------------+
                            |            |
                            |            |
                            |            |
                            |  machine   |
                            |  language  |
                            +------------+
                            +------------+
                             \ machine  /
                              \language/
                               \      /  <-- machine
                                \    /
                                 \  /
                                  \/


Therefore it's a good feature that compiling a function doesn't install
it in the environment used to compile it.

Instead, you should use load to instal compiled functions into the
run-time environment, after compiling:

    (load (byte-compile-file "source.el"))
    (load "source.elc")



Of course, the case of macros is different since they are explicitely
and well delimited compiler hooks.  The compiler has to take them into
account, to be able to further process the source.  And those macros may
need to use to compute their expansion some functions defined in that
source, therefore you should either load those functions before using
the macros, in a separate file, compiling and loading each file in
turn, or using eval-when-compile to define those functions.

Which is exactly the opposite of what you say it does. I don't know what
documentation you're reading, but it's wrong.


----(example.el)----------------------------------------------------------------

(eval-when-compile
 (defun gen-form (x)
   `(progn
      (insert ,(format "%S" x) "\n")
      (insert ,(format "%S" x) "\n")
      ',x)))

(defmacro m (x)
  (gen-form x))

(defun f ()
  (m "life?")
  (m 42))

--------------------------------------------------------------------------------

(byte-compile-file "~/Desktop/example.el")

(gen-form '(+ 30 3))
--> (progn (insert "(+ 30 3)" "
") (insert "(+ 30 3)" "
") (quote (+ 30 3)))

;; The object defined in eval-when-compile are incorporated into the
;; compiler environment.

(f) --> Debugger entered--Lisp error: (void-function f)

;; The others are not (unless they're macros).  The compiler still notes
;; that f will be fbound though.

(load  "~/Desktop/example.elc")

(f)
inserts:
    "life?"
    "life?"
    42
    42
--> 42

-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: Why (eval-when-compile (require 'foo)) does not bind functions during compilation?

[Barry Margolin]

In article <874mjhdyu8.fsf@kuiper.lan.informatimago.com>,
 "Pascal J. Bourguignon" <pjb@informatimago.com> wrote:

> Your mental model should be that compilation is reading data and
> producing data, and there's no reason for that process to modify the
> compiler.

That basic model applies to most languages, but Lisp is a little 
different. The very existence of eval-when-compile means that the 
compiler itself is programmable. Lisp allows reflection and 
self-reference like this.

And once you allow that, the OP's question becomes very meaningful.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Why (eval-when-compile (require 'foo)) does not bind functions during compilation?

[Michael Heerdegen]

Marcin Borkowski <mbork@mbork.pl> writes:

> As in the subject.  The manual says that it does define macros from
> the library “foo”, but not functions.  Why is that so?  I would think
> that the above form actually evaluates the (require 'foo) part during
> compilation, so in particular it should evaluate all the defuns there
> – but both experiments and manual confirm that I’m wrong.

Please anybody with more insight into this correct me if I'm wrong, but
I must object.

Where does the manual say that function definitions are not loaded?  In
the code of the byte compiler I don't see that require is handled
specially for `eval-when-compile'.  AFAICT the `require' in

    (eval-when-compile (require 'foo))

is executed normally while compiling, and that will load also functions
defined in foo into Emacs.  Not doing that would even be problematic,
since functions defined in foo may be needed to expand macros defined in
foo that are used in the file to be compiled.  Loading only foo's macros
might not be enough to be able to compile the file.

So, no, AFAIK, the `require' in `eval-when-compile' is evaluated as you
expected.  Dunno what tests you performed.  I made one test and saw also
functions had been defined.

Of course it is a different thing when you restart Emacs and load the
compiled file.  Then, "foo" will not be loaded, so the functions defined
in "foo" are not loaded.  The macros aren't either, because they had
been expanded when compiling.  That's I think what you meant the manual
described: if you only make use of macros of "foo" in a library, use 
(with-eval-after-load (require 'foo)) in the file to avoid to load foo
when loading the compiled file because it is just not necessary then.


Michael.