For a slimmer GHC

For a slimmer GHC

[Ludovic Courtès]

Hello!

GHC is insanely large, 1.2G for its closure, most of which is itself:

--8<---------------cut here---------------start------------->8---
store item                                                       total    self
/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2            1209.7   860.5  71.1%
/gnu/store/k6r37137lfpg3l3igi50c7lj2za7kqly-ld-wrapper-0           153.7     0.0   0.0%
/gnu/store/czs63sm4l0s4a56ab38dqvkx19yzylbq-perl-5.16.1            141.2    49.2   4.1%
/gnu/store/hddjjpkfvwaf1j1q3qwpvby0rid3k8by-gcc-4.9.3              138.1    77.3   6.4%
/gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc         120.2    42.6   3.5%
/gnu/store/fmxxkrpwajcnb9cyncgh4f4z6ybknl1g-guile-2.0.11           109.0    16.1   1.3%
/gnu/store/y5psndwpbbkjrf856x757psb708y62dn-binutils-2.25.1         82.5    44.6   3.7%
/gnu/store/mnwjrkbfzkb5ifhqf8hssf3cxfvg11l6-coreutils-8.24          77.8    13.8   1.1%
/gnu/store/fzp98iyq7a2i4d4siw0ab0y0wa95vv8k-readline-6.3            68.6     1.2   0.1%

[...]

$ du -ms /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/
871	/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/
$ du -ms /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/
871	/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/
--8<---------------cut here---------------end--------------->8---

The main problem is that, for each module, we have three variants:

--8<---------------cut here---------------start------------->8---
$ du -ms    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/*
3	/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/Control
1	/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/Data
5	/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/libHStransformers-0.4.2.0-3eG64VdP2vzGjP6wJiCp5X.a
2	/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/libHStransformers-0.4.2.0-3eG64VdP2vzGjP6wJiCp5X-ghc7.10.2.so
7	/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/libHStransformers-0.4.2.0-3eG64VdP2vzGjP6wJiCp5X_p.a
--8<---------------cut here---------------end--------------->8---

What about removing all the .a?  Would that be OK?

On that topic I found
<https://ghc.haskell.org/trac/ghc/wiki/DynamicByDefault> but it’s not
clear to me whether this is relevant here.  At worst we can add a phase
that removes these files.

A secondary issue is documentation: There’s a “doc” output, but
‘ghc:out’ depends on ‘ghc:doc’:

--8<---------------cut here---------------start------------->8---
$ guix gc --references /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/ | grep doc
/gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
$ du -ms /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
47	/gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
$ grep -r r539jrq7jk9vkmm1255i5jqs7skn4fag    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2
/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/process-1.2.3.0-f0287ac288afc0705be775d1adda59ee.conf:haddock-interfaces: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/process-1.2.3.0/process.haddock
/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/process-1.2.3.0-f0287ac288afc0705be775d1adda59ee.conf:haddock-html: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/process-1.2.3.0
/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/Cabal-1.22.4.0-43c3ae30d75ac742e521d26b63721876.conf:haddock-interfaces: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/Cabal-1.22.4.0/Cabal.haddock
/gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/Cabal-1.22.4.0-43c3ae30d75ac742e521d26b63721876.conf:haddock-html: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/Cabal-1.22.4.0
--8<---------------cut here---------------end--------------->8---

Any idea if we could avoid references to the “doc” output in these
*.conf files?  For instance, if there’s a variable like, say,
‘HADDOCK_PATH’, we can certainly remove the hardcoded references

Federico, Paul, Eric, … thoughts?  :-)

Thanks,
Ludo’.

For a slimmer GHC

[Federico Beffa]

ludo@gnu.org (Ludovic Courtès) writes:

> Hello!
>
> GHC is insanely large, 1.2G for its closure, most of which is itself:
>
> store item                                                       total    self
> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2            1209.7   860.5  71.1%
> /gnu/store/k6r37137lfpg3l3igi50c7lj2za7kqly-ld-wrapper-0           153.7     0.0   0.0%
> /gnu/store/czs63sm4l0s4a56ab38dqvkx19yzylbq-perl-5.16.1            141.2    49.2   4.1%
> /gnu/store/hddjjpkfvwaf1j1q3qwpvby0rid3k8by-gcc-4.9.3              138.1    77.3   6.4%
> /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc         120.2    42.6   3.5%
> /gnu/store/fmxxkrpwajcnb9cyncgh4f4z6ybknl1g-guile-2.0.11           109.0    16.1   1.3%
> /gnu/store/y5psndwpbbkjrf856x757psb708y62dn-binutils-2.25.1         82.5    44.6   3.7%
> /gnu/store/mnwjrkbfzkb5ifhqf8hssf3cxfvg11l6-coreutils-8.24          77.8    13.8   1.1%
> /gnu/store/fzp98iyq7a2i4d4siw0ab0y0wa95vv8k-readline-6.3            68.6     1.2   0.1%
>
> [...]
>
> $ du -ms /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/
> 871    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/
> $ du -ms /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/
> 871    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/
>
> The main problem is that, for each module, we have three variants:
>
> $ du -ms    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/*
> 3    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/Control
> 1    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/Data
> 5    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/libHStransformers-0.4.2.0-3eG64VdP2vzGjP6wJiCp5X.a
> 2    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/libHStransformers-0.4.2.0-3eG64VdP2vzGjP6wJiCp5X-ghc7.10.2.so
> 7    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/trans_3eG64VdP2vzGjP6wJiCp5X/libHStransformers-0.4.2.0-3eG64VdP2vzGjP6wJiCp5X_p.a
>
> What about removing all the .a?  Would that be OK?
>
> On that topic I found
> <https://ghc.haskell.org/trac/ghc/wiki/DynamicByDefault> but it’s not
> clear to me whether this is relevant here.  At worst we can add a phase
> that removes these files.

I do not know much about this topic, but I think that the discussion at
the link you provided is relevant, as probably is the one at

https://ghc.haskell.org/trac/ghc/wiki/DynamicLinkingDebate

My interpretation of the above discussions is that it is in principle OK
to remove statically linked libraries, but: (i) you loose the
possibility to create statically linked executables (the default; the
discussion here is about Haskell libraries, not system libraries), (ii)
dynamically linked executables are slower than statically linked ones
and (iii) it may create some build systems compatibility problems.

Personally I would keep them as the above discussions give me the
impression that dynamic linking is not very mature in GHC. In any case,
if people feel strongly about removing static libraries, then I would at
least add an option to the build-system to easily re-enable their
creation.

>
> A secondary issue is documentation: There’s a “doc” output, but
> ‘ghc:out’ depends on ‘ghc:doc’:
>
> $ guix gc --references /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/ | grep doc
> /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
> $ du -ms /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
> 47    /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
> $ grep -r r539jrq7jk9vkmm1255i5jqs7skn4fag    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2
> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/process-1.2.3.0-f0287ac288afc0705be775d1adda59ee.conf:haddock-interfaces: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/process-1.2.3.0/process.haddock
> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/process-1.2.3.0-f0287ac288afc0705be775d1adda59ee.conf:haddock-html: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/process-1.2.3.0
> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/Cabal-1.22.4.0-43c3ae30d75ac742e521d26b63721876.conf:haddock-interfaces: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/Cabal-1.22.4.0/Cabal.haddock
> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/Cabal-1.22.4.0-43c3ae30d75ac742e521d26b63721876.conf:haddock-html: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/Cabal-1.22.4.0
>
> Any idea if we could avoid references to the “doc” output in these
> *.conf files?  For instance, if there’s a variable like, say,
> ‘HADDOCK_PATH’, we can certainly remove the hardcoded references

As far as I understand, these full paths entries in .conf files are used
to generate links in the HTML documentation. As one example, when you
generate the documentation for the 'ghc-mtl' package, the generated
documentation includes links, e.g., to the documentation of the
'identity' function which resides in another package. I believe that the
location of the latter is retrieved from the .conf files.

Browsing the Haddock documentation I do not see environment variables
https://www.haskell.org/haddock/doc/html/invoking.html

Regards,
Fede

Re: For a slimmer GHC

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

>> What about removing all the .a?  Would that be OK?
>>
>> On that topic I found
>> <https://ghc.haskell.org/trac/ghc/wiki/DynamicByDefault> but it’s not
>> clear to me whether this is relevant here.  At worst we can add a phase
>> that removes these files.
>
> I do not know much about this topic, but I think that the discussion at
> the link you provided is relevant, as probably is the one at
>
> https://ghc.haskell.org/trac/ghc/wiki/DynamicLinkingDebate
>
> My interpretation of the above discussions is that it is in principle OK
> to remove statically linked libraries, but: (i) you loose the
> possibility to create statically linked executables (the default; the
> discussion here is about Haskell libraries, not system libraries), (ii)
> dynamically linked executables are slower than statically linked ones
> and (iii) it may create some build systems compatibility problems.

(i) and (ii) apply to all the packages in Guix, not just Haskell
packages: most of the time we provide only shared libraries, and PIC
code includes a slight “performance penalty” (and possibly large memory
savings if several processes use the library.)

I’m not sure what (iii) means though?

> Personally I would keep them as the above discussions give me the
> impression that dynamic linking is not very mature in GHC. In any case,
> if people feel strongly about removing static libraries, then I would at
> least add an option to the build-system to easily re-enable their
> creation.

I think we could even simply move them to a different output, for those
who really need them.

>> A secondary issue is documentation: There’s a “doc” output, but
>> ‘ghc:out’ depends on ‘ghc:doc’:
>>
>> $ guix gc --references /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/ | grep doc
>> /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
>> $ du -ms /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
>> 47    /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc
>> $ grep -r r539jrq7jk9vkmm1255i5jqs7skn4fag    /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2
>> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/process-1.2.3.0-f0287ac288afc0705be775d1adda59ee.conf:haddock-interfaces: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/process-1.2.3.0/process.haddock
>> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/process-1.2.3.0-f0287ac288afc0705be775d1adda59ee.conf:haddock-html: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/process-1.2.3.0
>> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/Cabal-1.22.4.0-43c3ae30d75ac742e521d26b63721876.conf:haddock-interfaces: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/Cabal-1.22.4.0/Cabal.haddock
>> /gnu/store/1iwl222h2qw80fyr578sdjdki0pbcjm0-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d/Cabal-1.22.4.0-43c3ae30d75ac742e521d26b63721876.conf:haddock-html: /gnu/store/r539jrq7jk9vkmm1255i5jqs7skn4fag-ghc-7.10.2-doc/share/doc/ghc/html/libraries/Cabal-1.22.4.0
>>
>> Any idea if we could avoid references to the “doc” output in these
>> *.conf files?  For instance, if there’s a variable like, say,
>> ‘HADDOCK_PATH’, we can certainly remove the hardcoded references
>
> As far as I understand, these full paths entries in .conf files are used
> to generate links in the HTML documentation. As one example, when you
> generate the documentation for the 'ghc-mtl' package, the generated
> documentation includes links, e.g., to the documentation of the
> 'identity' function which resides in another package. I believe that the
> location of the latter is retrieved from the .conf files.
>
> Browsing the Haddock documentation I do not see environment variables
> https://www.haskell.org/haddock/doc/html/invoking.html

OK, too bad.

Thanks for your feedback!

Ludo’.

Re: For a slimmer GHC

[Federico Beffa]

On Mon, Nov 2, 2015 at 3:11 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Federico Beffa <beffa@ieee.org> skribis:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>>> What about removing all the .a?  Would that be OK?
>>>
>>> On that topic I found
>>> <https://ghc.haskell.org/trac/ghc/wiki/DynamicByDefault> but it’s not
>>> clear to me whether this is relevant here.  At worst we can add a phase
>>> that removes these files.
>>
>> I do not know much about this topic, but I think that the discussion at
>> the link you provided is relevant, as probably is the one at
>>
>> https://ghc.haskell.org/trac/ghc/wiki/DynamicLinkingDebate
>>
>> My interpretation of the above discussions is that it is in principle OK
>> to remove statically linked libraries, but: (i) you loose the
>> possibility to create statically linked executables (the default; the
>> discussion here is about Haskell libraries, not system libraries), (ii)
>> dynamically linked executables are slower than statically linked ones
>> and (iii) it may create some build systems compatibility problems.
>
> (i) and (ii) apply to all the packages in Guix, not just Haskell
> packages: most of the time we provide only shared libraries, and PIC
> code includes a slight “performance penalty” (and possibly large memory
> savings if several processes use the library.)
>
> I’m not sure what (iii) means though?

I'm not sure either :-) It's one of the reasons listed in the
discussion at the above link. I'm just passing on the information.

>
>> Personally I would keep them as the above discussions give me the
>> impression that dynamic linking is not very mature in GHC. In any case,
>> if people feel strongly about removing static libraries, then I would at
>> least add an option to the build-system to easily re-enable their
>> creation.
>
> I think we could even simply move them to a different output, for those
> who really need them.

That would be ideal, but I'm not sure you can simply move them. The
location of a library must be listed in the binary library cache. I do
not know if you can have more entries for a single library. I guess
that would have to be investigated experimentally.

Regards,
Fede