From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julien Lepiller Subject: Re: New build system: copy-build-system Date: Mon, 27 Jan 2020 13:18:20 -0500 Message-ID: References: <87sgk2dkuv.fsf@ambrevar.xyz> <87pnf5arh1.fsf@ambrevar.xyz> <87k15dapxk.fsf@ambrevar.xyz> <875zgwc2zd.fsf@ambrevar.xyz> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:33082) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iw8yC-0003z1-5W for guix-devel@gnu.org; Mon, 27 Jan 2020 13:18:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iw8yA-0003zI-Qt for guix-devel@gnu.org; Mon, 27 Jan 2020 13:18:43 -0500 In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane-mx.org@gnu.org Sender: "Guix-devel" To: zimoun , Pierre Neidhardt Cc: Guix Devel Le 27 janvier 2020 11:17:10 GMT-05:00, zimoun a =C3=A9crit : >On Mon, 27 Jan 2020 at 16:51, Pierre Neidhardt >wrote: >> >> zimoun writes: >> >> > And for example, the 'copy-build-system' could: >> > >> > - fetch the data from an archive, such http://data=2Eastrometry=2Ene= t >or >> > IPFS or Zenodo or >> > - fetch the resulting of /gnu/store/-name-version >from susbtitutes >> >> Build systems don't fetch data=2E > >I know (isolated blabla)=2E :-) > >"The package using the 'copy-build-system'" is a better wording? ;-) > >I do not know exactly how a derivation is computed and what is used to >compute the hash used in the store=2E Ok, first I have to do my >homework=2E :-) >But one does not want to hash several tens of GB=2E > >Whatever, I just have the feeling that what you are proposing (with >small tweaks) could improve the situation for "large" data set=2E I do >not know=2E=2E=2E > >Cheers, >simon > >ps: >Well, maybe we should work in the same office=2E=2E=2E it will be easier = and >faster than exchanging all these emails and I know yours has an >awesome view :-D From=20my own experiments, computing the hash of a source is pretty fast, ev= en on a large archive (a few seconds for a 5=2E5 GB file, using guix hash t= he-file)=2E However, guix download the-file will fail when you don't have (= roughly) 3 times the file's size in free memory=2E So a large source would = be whatever your RAM is / 3 or so=2E We should improve that, but that's another issue :) Also, the file size has nothing to do with derivation computation, which o= nly relies on the declared sha256 of the source (known statically)=2E