Generating compiled scm (.go) files as part of LilyPond build

[Ian Hulin <ian@hulin.org.uk>]

Firstly, apologies if there is anywhere obvious I've failed to look such
as guile and/or guile mailing list histories.

LilyPond has a large number of .scm files it used and loads into its own
'lily' module using an initialization scheme script lily.scm.  Lily.scm
is the one that does the heavy lifting loading the .scm files.

Our long-term aim, when we are able to move to using Guile V2.0 as an
infrastructure, is to byte-compile as many of these as possible during
the Lily build using something like.
$ guile-tool compile <scm file>
   --output-file=<compiled-scm-dir><.go file>

(In the following paras, <lilypond-root> is the run-time base directory
being used by LilyPond.)
At the moment, during Lily initialization we prefix the guile path
%load-path with <lilypond-root>/scm as this is where we keep all the
LilyPond-specific scm files. This works fine when interpreting
everything using V1.8.7.  It also works reasonably well with 1.9/2.0 if
we use AUTOCOMPILE, as then guile seems to keep a private cache of
.scm.go files in somewhere like:
/home/<username>/.cache/guile/ccache/2.0-R-LE-4/
  home/<username>//lilypond/scm.

When we ask to load the <lilypond-root>/scm/<file>,scm using
primitive-load-path, Guile does all its file date modification magic,
against the <file>.scm.go file in the cache to decide if it needs to
generate newer compiled file.

OK, here's the question:  if we decide not to rely on AUTOCOMPILE, and
we are able generate our own .go files in, say, <lilypond-root>/out/scm,
how do we get guile to use the compiled version
<lilypond-root>/out/scm/<file>.go in preference to a possibly
non-existent .scm.go file in the cache?  There are some hints of
configure/type variables LOAD_PATH and COMPILED_LOAD_PATH in NEWS, and I
hoped that there might me a %compiled-load-path run-time equivalent to
COMPILED_LOAD_PATH just like %load-path corresponds to LOAD_PATH, but it
doesn't exist as at V1.9.13.  If there was, we could possibly prefix
<lilypond-root>/out/scm to the %compiled-load-path in the same way we
fiddle with %load-path for the interpreter.  Of course, I'm open to any
more elegant solutions.

Thanks in advance for any help and/or suggestions,

Cheers,

Ian Hulin