Re: Generating compiled scm (.go) files as part of LilyPond build

[Ian Hulin <ian@hulin.org.uk>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue 19 Jul 2011 15:28:41 BST, Andy Wingo wrote:
> Hi Ian,
> 
> On Tue 19 Jul 2011 15:18, Ian Hulin <ian@hulin.org.uk> writes:
> 
>> It may boil down to a matter of taste, but I find double and
>> triple extensions on a filename intrinsically nasty.  I've normally
>> come across them before on Windows systems where a filename such as
>> thing.htm.exe usually means there's malware or a trojan or suchlike
>> on your system. See also comments below.
> 
> Consider it an opaque key into a cache.  We could change in the
> future to default to the SHA1 of the whole path.  That does have
> some advantages regarding flattening the directory hierarchy,
> resulting in fewer stat operations.
> 
Sorry, but how does this benefit LilyPond as a user of the Guile API?
I'm used to viewing file extensions as an indication of the format of
the file content. In our simple view scheme sources are *.scm, and
compiled scheme files are *.go.

Guile documentation implies something similar (Chap 4 Environment
Variable sections on GUILE_AUTO_COMPILE and GUILE_LOAD_COMPILED_PATH)

"If a compiled (‘.go’) file corresponding to a ‘.scm’ file is not found
or is not newer than the ‘.scm’ file, the ‘.scm’ file will be compiled
on the fly, and the
resulting ‘.go’ file stored away. An advisory note will be printed on
the console."

"GUILE_LOAD_COMPILED_PATH
This variable may be used to augment the path that is searched for
compiled Scheme files (‘.go’ files) when loading"

Neither of these sections leads the reader to expect an actual file name
generated to be
;;; compiled
/long/path/name/testing.scm.go
                       ^^^^^^^
>> ian@nanny-ogg:~/src/Guile/guile-2.0.2$ meta/guile -L $PWD
> 
> This will set XDG_CACHE_HOME=${top_builddir}/cache.
> 
The Guile Reference Manual Chapter 4 p 38 specifically says this
defaults to $HOME/cache

>> The problem is knowing where the cache is. For Lilypond, we need to
>> have a common root directory off of which we can hang the compiled
>> files in scm/out.
> 
> Why do you care?  (Honest question.)
> 
(Honest answer)
Because if Lilypond is run up with the Guile default of --auto-compile
it will check for and generate compiled files in a different place to
where we want to have produced them at build-time.  Also once for a
project like LilyPond these files would need to be in a shared directory
on the file system. Relying on the default compiled file spec would have
a set of cached files somewhere on each user's cache in their home
directory.


> To me there are two cases:
> 
> 1) You compile the files yourself.  You either ensure that the .go 
> files end up in the right places, or adjust 
> GUILE_LOAD_COMPILED_PATH.
> 
But GUILE_LOAD_COMPILED_PATH setting won't prevent the vanilla
compiled-file-name from checking in the wrong place for at compile time.
OK we can brew our own expectation of where the file will be, and
provide our own ly:load and intelligence to pick up the compiled file,
but we have to do this in conjunction with a
scm_putenv("GUILE_AUTO_COMPILE=0") call in our initialization code.

> 2) You rely on autocompilation.  In this case there is no guarantee 
> about where the files go, besides residing in the 
> XDG_CACHE_DIR/guile.
> 
Looks like this is a no-no.  We can't control what the files are called,
or if they'd be recognized once they've been moved to their location
after /make install/ for LilyPond. Also there's the issue of embedded
Scheme statements in Lily source files (which could contain their own
(load "xxxx") statements).

It's a real shame because it looks like auto-compile is a feature
designed to make life easy for Guile users (including layered projects
like LilyPond). Unfortunately its default settings prevent us from using
it. :-(.

I've also got a question re the (include) procedure which Ludovic
mentioned in a post on guile-users.  I'll start a different thread for
that one.

Thanks for the help so far,

Cheers,
Ian

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOJg5cAAoJEBqidDirZqASMj0H/2jn/jiNG+MtT+YEc762baxL
8pMC4ijNMwWEnMwzuWm1LHhHiJqG5roxvinPoe5bjeC8RBNSMOW7SKGkJ+ao5pTK
vYfBm55WqdpAvAVQA9OaDkL6J33TEa6z1ioNYxj2yFOHHsj0HHVnhUM29RDNYgmw
9duK9lKtv37AGT0W9aj6bGx8FRGFUOFgRUnHMCtn7usE1DboXQgjaY3IQ46XYs6n
XGfuFz1JBXqhKEjjK7Fbu5M44A0eD2yWGJ01K3eVQqB1OmIrz7zy7wU495Wqd4im
/eop+BCJRnpRAdHIR0y/LgrrlPSpD+mulBk9BOuoAglXM6N8spaQQY/mj8MkrKU=
=igTg
-----END PGP SIGNATURE-----