Bootstrapping GHC

Bootstrapping GHC

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

I’ve been thinking about GHC recently.  (A quick reminder: there are
binary versions [1], but we can’t use them because some binaries point
to ‘/bin/sh’.)  I believe the following should work for x86_64 and i686:

1. We fetch the source version [2] (manually), adjust ‘/bin/sh’, and
   build everything using a non-Guix GHC.

2. Then we copy the needed binaries somewhere.

3. After that we write a build recipe.  It should fetch the binary
   version of GHC [1], fix ‘/bin/sh’, and replace some binaries with the
   binaries from the second step.

Questions:

1. We’d have to host the binaries (step 2) somewhere and repeat the
   process whenever the hash of ‘/bin/sh’ changes.  How often will this
   happen?

2. Does Guix allow to fetch files from multiple places?  Is it possible
   to use ‘url-fetch’ plus a local file?  Could you show an example?

[1] http://www.haskell.org/ghc/dist/7.0.1/ghc-7.0.1-i386-unknown-linux.tar.bz2
[2] http://www.haskell.org/ghc/dist/7.0.1/ghc-7.0.1-src.tar.bz2

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Bootstrapping GHC

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

> I’ve been thinking about GHC recently.  (A quick reminder: there are
> binary versions [1], but we can’t use them because some binaries point
> to ‘/bin/sh’.)  I believe the following should work for x86_64 and i686:

I don’t know GHC, but apparently it can be bootstrapped from
intermediate generated C files:

  http://www.haskell.org/ghc/docs/6.4.1/html/building/sec-porting-ghc.html

Wouldn’t that be better than fiddling with binaries out-of-band?

> Questions:
>
> 1. We’d have to host the binaries (step 2) somewhere and repeat the
>    process whenever the hash of ‘/bin/sh’ changes.  How often will this
>    happen?

The hash of /bin/sh?  Where did you see /bin/sh?  :-)

> 2. Does Guix allow to fetch files from multiple places?  Is it possible
>    to use ‘url-fetch’ plus a local file?  Could you show an example?

The ‘uri’ argument can be a list:

  (origin
    (method url-fetch)
    (uri (list "http://..." "file://..."))
    (sha256 ...))

However, ‘url-fetch’ doesn’t support file://.

The other option would be to allow ‘source’ to be a list; that’s easily
implemented, just not done yet.

HTH,
Ludo’.

Re: Bootstrapping GHC

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 1850 bytes --]

> I don’t know GHC, but apparently it can be bootstrapped from
> intermediate generated C files:

>   http://www.haskell.org/ghc/docs/6.4.1/html/building/sec-porting-ghc.html

> Wouldn’t that be better than fiddling with binaries out-of-band?

The recent versions of the compiler don’t support this.  So you have to
use the old version.  I’ve spent four hours on this and didn’t even get
to the intermediate C files (that’s the first step).  The compiler
depends on the old libraries, so you have to downgrade.  That’s the main
difficulty.

Also, you can’t build GHC 7.6.3 using GHC 6.4.1.  Users would have to
build several compilers any time they want the latest version.  That may
take more than six hours (or even longer) on an average machine.

So I’d prefer to avoid this if it’s possible.

>> 1. We’d have to host the binaries (step 2) somewhere and repeat the
>>    process whenever the hash of ‘/bin/sh’ changes.  How often will this
>>    happen?

> The hash of /bin/sh?  Where did you see /bin/sh?  :-)

I meant this one:

scheme@(guile-user)> ,use (guix build utils)
scheme@(guile-user)> (which "sh")
$1 = "/nix/var/nix/profiles/default/guix-profile/bin/sh"
scheme@(guile-user)> (readlink $1)
$2 = "/nix/store/fzcdfwyyin5dr7finlaq2kph396nrlli-bash-4.2/bin/sh"

How often may the hash change?  In other words, what does Bash use to
“source” the hash function?  Will it happen whenever you merge
‘core-updates’?

> The ‘uri’ argument can be a list:

>   (origin
>     (method url-fetch)
>     (uri (list "http://..." "file://..."))
>     (sha256 ...))

> However, ‘url-fetch’ doesn’t support file://.

> The other option would be to allow ‘source’ to be a list; that’s easily
> implemented, just not done yet.

Thanks, I’ll look into it.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Bootstrapping GHC

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

>> I don’t know GHC, but apparently it can be bootstrapped from
>> intermediate generated C files:
>
>>   http://www.haskell.org/ghc/docs/6.4.1/html/building/sec-porting-ghc.html
>
>> Wouldn’t that be better than fiddling with binaries out-of-band?
>
> The recent versions of the compiler don’t support this.  So you have to
> use the old version.  I’ve spent four hours on this and didn’t even get
> to the intermediate C files (that’s the first step).  The compiler
> depends on the old libraries, so you have to downgrade.  That’s the main
> difficulty.
>
> Also, you can’t build GHC 7.6.3 using GHC 6.4.1.  Users would have to
> build several compilers any time they want the latest version.  That may
> take more than six hours (or even longer) on an average machine.
>
> So I’d prefer to avoid this if it’s possible.

Blech.  I find it worrisome if GHC doesn’t have a better bootstrapping
story.  How do they port it then?

>>> 1. We’d have to host the binaries (step 2) somewhere and repeat the
>>>    process whenever the hash of ‘/bin/sh’ changes.  How often will this
>>>    happen?
>
>> The hash of /bin/sh?  Where did you see /bin/sh?  :-)
>
> I meant this one:
>
> scheme@(guile-user)> ,use (guix build utils)
> scheme@(guile-user)> (which "sh")
> $1 = "/nix/var/nix/profiles/default/guix-profile/bin/sh"
> scheme@(guile-user)> (readlink $1)
> $2 = "/nix/store/fzcdfwyyin5dr7finlaq2kph396nrlli-bash-4.2/bin/sh"
>
> How often may the hash change?  In other words, what does Bash use to
> “source” the hash function?  Will it happen whenever you merge
> ‘core-updates’?

Yes.  (The hash in the store file name is the hash of all the inputs
that led to that directory (info "(guix) Introduction").)

>> The ‘uri’ argument can be a list:
>
>>   (origin
>>     (method url-fetch)
>>     (uri (list "http://..." "file://..."))
>>     (sha256 ...))
>
>> However, ‘url-fetch’ doesn’t support file://.
>
>> The other option would be to allow ‘source’ to be a list; that’s easily
>> implemented, just not done yet.
>
> Thanks, I’ll look into it.

Great, thanks.

Ludo’.