bug#32160: auto compile fails to recompile when included source files change

bug#32160: auto compile fails to recompile when included source files change

[Arun Isaac]

Guile's auto compile fails to recompile when included source files
change.

For example, I have two source files -- foo.scm and bar.scm. foo.scm
includes bar.scm using `include'. When I run foo.scm for the first time,
it is auto compiled and run correctly. But, if I modify bar.scm and run
foo.scm again, foo.scm is not rebuilt and the old version is run.

Here is a concrete example of what I mean.

$ cat foo.scm
(include "bar.scm")

$ cat bar.scm
(display "old")
(newline)

$ guile -s foo.scm
;;; note: source file /tmp/test/foo.scm
;;;       newer than compiled /home/arun/.cache/guile/ccache/2.2-LE-8-3.A/tmp/test/foo.scm.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /tmp/test/foo.scm
;;; compiled /home/arun/.cache/guile/ccache/2.2-LE-8-3.A/tmp/test/foo.scm.go
old

$ sed -i 's/old/new/' bar.scm

$ cat bar.scm
(display "new")
(newline)

$ guile -s foo.scm
old

In the last step, guile still printed "old" without recompiling. It
should have recompiled and printed "new".

bug#32160: auto compile fails to recompile when included source files change

[Mark H Weaver]

Hi Arun,

Arun Isaac <arunisaac@systemreboot.net> writes:

> Guile's auto compile fails to recompile when included source files
> change.
>
> For example, I have two source files -- foo.scm and bar.scm. foo.scm
> includes bar.scm using `include'. When I run foo.scm for the first time,
> it is auto compiled and run correctly. But, if I modify bar.scm and run
> foo.scm again, foo.scm is not rebuilt and the old version is run.

Yes, this is a known limitation in Guile's auto-compilation support.
Unfortunately, we don't have any system in place to track these
dependencies between modules.  Years ago, I spent some time thinking
about how to do it, but it's nontrivial given the unstructured and
imperative model by which modules load themselves and their
dependencies.  The .go file format would also need to be extended to
embed the dependency information.

There's also the question of how to fingerprint the dependencies.  If
timestamps are used, it would defeat deterministic builds, but if hashes
are used, it might dramatically slow down module loading, especially
given that it would not be sufficient to check the immediate
dependencies: the full transitive closure of the module dependency graph
would need to be traversed.

And then there's the unfortunate fact that although circular module
dependencies are (sensibly) prohibited by standard Scheme and most other
languages, in Guile they are not prohibited, and although they cause a
great many headaches, they sort-of-work if you are lucky, and Guix now
depends heavily on them.  Last I checked, Guix has a large number of
package modules (well over 50, and possibly more than 100 at this point)
that form a strongly connected component in the module dependency graph.

I'm sorry that I don't have a better answer for you.  If you'd like to
investigate further, I'd be glad to give you pointers and advice, but
it's not a project for the faint-hearted :-/

      Regards,
        Mark

bug#32160: auto compile fails to recompile when included source files change

[Mark H Weaver]

My response to Arun Isaac <arunisaac@systemreboot.net> was rejected
by his mail server:

  SMTP error from remote mail server after RCPT TO:<arunisaac@systemreboot.net>:
  550 Sender Policy Framework (SPF) verification failed

I do not have DNS records supporting SPF for my domain netris.org, and
apparently his mail server is configured to reject all mail that is not
authenticated by SPF.  So, I'm unable to send him email.  Oh well.  This
is the first time I've run into this problem.

If someone could let him know about this issue, I would be grateful.

      Mark

bug#32160: auto compile fails to recompile when included source files change

[Mark H Weaver]

Mark H Weaver <mhw@netris.org> writes:

> My response to Arun Isaac <arunisaac@systemreboot.net> was rejected
> by his mail server:
[...]
> If someone could let him know about this issue, I would be grateful.

tomas@tuxteam.de kindly forwarded my messages to Arun, so there's no
need for anyone else to do it now.  Thanks, Tomas!

      Mark

bug#32160: auto compile fails to recompile when included source files change

[Arun Isaac]

>> If you'd like to investigate further, I'd be glad to give you
>> pointers and advice, but it's not a project for the faint-hearted :-/

I am faint-hearted and do not wish to pursue this further. :-P I just
thought it might be useful to report. Should I close this bug report, or
leave it open for someone to fix in the future?

bug#32160: auto compile fails to recompile when included source files change

[Mark H Weaver]

Hi Arun,

Arun Isaac <arunisaac@systemreboot.net> writes:

>>> If you'd like to investigate further, I'd be glad to give you
>>> pointers and advice, but it's not a project for the faint-hearted :-/
>
> I am faint-hearted and do not wish to pursue this further. :-P I just
> thought it might be useful to report. Should I close this bug report, or
> leave it open for someone to fix in the future?

I'd prefer to leave it open, because it is indeed an important issue
that would be good to fix eventually.

      Thanks!
        Mark