Recursive byte-compilation

Recursive byte-compilation

[Paul Pogonyshev]

[-- Attachment #1: Type: text/plain, Size: 2670 bytes --]

In a certain tool I'm having a situation where I need to install a package
while byte-compiling a file. Since installing a package also involves
byte-compilation, it results in byte-compilation invoked when inside
another byte-compilation. It seems to work _mostly_ fine, but still there
are quirks.

Here is a short program that I created to emulate this scenario:

    (eval-when-compile
      (let ((default-directory (expand-file-name "/tmp/myprivatelib/")))
        (make-directory default-directory t)
        (with-temp-file "thelib.el"
          (insert "(defun im-a-library-function ()\n"
                  "  (trigger-a-warning-here-1))\n"
                  "(provide 'thelib)\n"))
        (byte-compile-file "thelib.el")))

    (defun blabla ()
      (trigger-a-warning-here-2))

To test, run from the command line (having saved it as `main.el'):

    $ emacs --batch --eval "(print (byte-compile-file \"main.el\"))"

The "program" is deliberately written to issue byte-compilation warnings.
First quirk is that the path for outer file (i.e. `main.el') is written
fully, because it is apparently found from `/tmp/myprivatelib':

    In end of data:
    thelib.el:4:1: Warning: the function ‘trigger-a-warning-here-1’ is not
known
       to be defined.

    In end of data:
    ~/test/nested-compilation/main.el:12:1: Warning: the function
       ‘trigger-a-warning-here-2’ is not known to be defined.

    t

If, for a test, you remove the `(byte-compile-file "thelib.el")' form, file
path becomes just `main.el', as expected. In other words, *presence of
inner byte-compilation can affect output of outer*, which doesn't look
correct to me.

Another, more important problem is that the outer byte-compilation will
fail if inner one fails (just e.g. remove a paren from "(provide
'thelib)\n" string):

    In toplevel form:
    thelib.el:3:1: Error: End of file during parsing

    In end of data:
    ~/test/nested-compilation/main.el:12:1: Warning: the function
       ‘trigger-a-warning-here-2’ is not known to be defined.

    nil

It feels wrong, because byte-compilation of `main.el' itself _was
successful_. What did fail was a different and only tangentially related
byte-compilation of `thelib.el'.

Is there a way to completely "insulate" the outer invocation of
`byte-compile-...' from the inner? I.e. maybe my simple program above (and
the real tool) can be just expanded to avoid such problems, while keeping
recursive byte-compilation behavior?

If no, would Emacs be interested in resolving such an issue or would it be
deemed too special and unimportant?

Paul

[-- Attachment #2: Type: text/html, Size: 3623 bytes --]

Re: Recursive byte-compilation

[Stefan Monnier]

> In a certain tool I'm having a situation where I need to install
> a package while byte-compiling a file.  Since installing a package
> also involves byte-compilation, it results in byte-compilation invoked
> when inside another byte-compilation.  It seems to work _mostly_ fine,
> but still there are quirks.

These look clearly like bugs, so better treat them as such.


        Stefan