bug#15602: Compiling several files in the same session [2.0.9]

[ludo@gnu.org (Ludovic Courtès)]

[-- Attachment #1: Type: text/plain, Size: 854 bytes --]

ludo@gnu.org (Ludovic Courtès) skribis:

> In the last case (one, three, two), the compiler:
>
>   1. compiles ‘one.scm’, which creates module (one) in the global name
>      space with just ‘expansion-time’ in its exported bindings;
>
>   2. when compiling ‘three.scm’, it loads ‘two.scm’; since (two) uses
>      (one), it does ‘(resolve-module '(one))’, and since (one) already
>      exists it is used;
>
>      however, the (one) we have comes from step 1, and lacks the
>      ‘run-time’ binding, hence the unbound variable failure.
>
> I think the right thing would be to use a separate module hierarchy in
> the dynamic extent of ‘compile-file’, somehow, such that all module side
> effects are isolated.

In Guix (the ‘guix pull’ command, which compiles all of Guix), I ended
up with this:


[-- Attachment #2: Type: text/x-scheme, Size: 1446 bytes --]

 (define* (compile-file* file #:key output-file (opts '()))
   ;; Like 'compile-file', but remove any (guix …) and (gnu …) modules
   ;; created during the process as an ugly workaround for
   ;; <http://bugs.gnu.org/15602> (FIXME).  This ensures correctness,
   ;; but is overly conservative and very slow.

   (define (module-directory+file module)
     ;; Return the directory for MODULE, like the 'dir-hint' in
     ;; boot-9.scm.
     (match (module-name module)
       ((beginning ... last)
        (values (string-concatenate
                 (map (lambda (elt)
                        (string-append (symbol->string elt)
                                       file-name-separator-string))
                      beginning))
                (symbol->string last)))))

   (define (clear-module-tree! root)
     ;; Delete all the modules under ROOT.
     (hash-for-each (lambda (name module)
                      (module-remove! root name)
                      (let-values (((dir name)
                                    (module-directory+file module)))
                        (set-autoloaded! dir name #f))
                      (clear-module-tree! module))
                    (module-submodules root))
     (hash-clear! (module-submodules root)))

   (compile-file file #:output-file output-file #:opts opts)

   (for-each (compose clear-module-tree! resolve-module)
             '((guix) (gnu))))

[-- Attachment #3: Type: text/plain, Size: 516 bytes --]


> Of course the above can be worked around by running ‘compile-file’ in a
> child process, but forking alone is more expensive than ‘compile-file’,
> so that’s not really a solution when there are many files.

As it turns out, the hack above is just as slow as forking: what takes
time is not forking, but reloading the same modules over and over again.

So we should have a way to keep modules that have been fully evaluated,
and to discard modules that have not.

Ideas welcome.

Ludo’.