Ludovic Courtès schreef op za 02-10-2021 om 16:12 [+0200]:
[reordered]
> > So I think the cache should also check if these dependencies have been modified.
> > To keep track of the dependencies, something like the ‘compile-all,compile:
> > Keep track of dependencies of compiled modules.’ patch from
> > <https://issues.guix.gnu.org/50384> could be used, though the 'include', 'load',
> > 'include-from-path' and maybe 'use-modules' (if something like "guix shell -Lextra-modules-directory ..."
> > is done) macros would need to be replaced.
> 
> Problem is that any attempt to keep track of dependencies is always an
> approximation because macros can do anything—I can have my own macro
> that reads files at expansion time.  So I’m inclined to not even try.

I expect most people use 'include', 'load', 'include-from-path' or 'load-from-path'
instead of writing their own macro reading files at expansion time, and if they do
write their own macro, I expect it would be implemented in terms of the former anyway,
so I expect replacing these macros and 'use-modules' to be sufficient, especially
if the flag proposed below is available as ‘escape hatch’ for when the dependency
tracking is insufficient.

This is not merely an expansion-time problem, files loaded at 'load' or 'eval' time
are important as well.  E.g., consider the case where the manifest is something like

  (list
    (package
      (inherit stuff)
      (source (local-file "stuff" #:recursive? #t)))
    other-packages ...)

then all files inside 'stuff' are important as well.  I don't see how dependencies
could be tracked here without an excessive amount of 'stat' calls, maybe guix should
ignore these dependencies (possibly with a warning, and a reference to the manual
documenting which dependencies are tracked and which are not?). 

Ludovic Courtès schreef op za 02-10-2021 om 16:12 [+0200]:
> I’m not sure how to address it.  We’d need a cache key that’s more
> precise than inode/mtime, yet cheap to compute.

The mtime of the file and the mtime of every dependency (ignoring modules from
Guile, Guix and channels to reduce the number of calls to 'stat'). 

> Perhaps we need to live with that limitation, document it, and provide a
> flag to force a rebuild?

A documented flag to always consider the cache stale seems good, though I think
at least the dependencies made with the common macros and procedures 'include',
'load', 'include-from-path', 'load-from-path', 'use-modules' and non-recursive
'local-file' could be tracked, though this could be left as a TODO for later
I suppose.

Greetings,
Maxime.