Re: New build system: copy-build-system

[Julien Lepiller <julien@lepiller.eu>]

Le 27 janvier 2020 11:17:10 GMT-05:00, zimoun <zimon.toutoune@gmail.com> a écrit :
>On Mon, 27 Jan 2020 at 16:51, Pierre Neidhardt <mail@ambrevar.xyz>
>wrote:
>>
>> zimoun <zimon.toutoune@gmail.com> writes:
>>
>> > And for example, the 'copy-build-system' could:
>> >
>> >  - fetch the data from an archive, such http://data.astrometry.net
>or
>> > IPFS or Zenodo or <name-it>
>> >  - fetch the resulting <hash> of /gnu/store/<hash>-name-version
>from susbtitutes
>>
>> Build systems don't fetch data.
>
>I know (isolated blabla). :-)
>
>"The package using the 'copy-build-system'" is a better wording? ;-)
>
>I do not know exactly how a derivation is computed and what is used to
>compute the hash used in the store. Ok, first I have to do my
>homework. :-)
>But one does not want to hash several tens of GB.
>
>Whatever, I just have the feeling that what you are proposing (with
>small tweaks) could improve the situation for "large" data set. I do
>not know...
>
>Cheers,
>simon
>
>ps:
>Well, maybe we should work in the same office... it will be easier and
>faster than exchanging all these emails and I know yours has an
>awesome view :-D

From my own experiments, computing the hash of a source is pretty fast, even on a large archive (a few seconds for a 5.5 GB file, using guix hash the-file). However, guix download the-file will fail when you don't have (roughly) 3 times the file's size in free memory. So a large source would be whatever your RAM is / 3 or so.

We should improve that, but that's another issue :)

Also, the file size has nothing to do with derivation computation, which only relies on the declared sha256 of the source (known statically).