From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andreas Enge Subject: Re: updating list of substitutes Date: Tue, 13 Oct 2015 14:52:58 +0200 Message-ID: <20151013125258.GA5574@debian> References: <87a8y1q49z.fsf@gnu.org> <20150421084028.GB16564@thebird.nl> <20150422114635.GA24566@thebird.nl> <20151011074654.GA8058@thebird.nl> <87wputz1uj.fsf@gnu.org> <20151011212714.GA9401@thebird.nl> <87d1wk8y7e.fsf@netris.org> <20151012060607.GA11012@thebird.nl> <871td0auiw.fsf@gnu.org> <20151013121140.GA18001@thebird.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45905) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zlz4j-0001ua-1c for guix-devel@gnu.org; Tue, 13 Oct 2015 08:53:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zlz4f-0002Hy-LW for guix-devel@gnu.org; Tue, 13 Oct 2015 08:53:04 -0400 Content-Disposition: inline In-Reply-To: <20151013121140.GA18001@thebird.nl> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Pjotr Prins Cc: guix-devel@gnu.org On Tue, Oct 13, 2015 at 02:11:40PM +0200, Pjotr Prins wrote: > I just checked the substitute cache in /var/guix/substitute/cache and > these files are small (3K only!). So, all that pain to fetch a 3K file > :). I thought it was network related, but it is generating this cache > file that is expensive. Well, there is one file per package. Assuming that one would limit the cache to only one architecture, that would still end up with 2500*3KB=7,5MB, a somewhat larger file. Still, my impression (also from using substitutes) is that _access_ to hydra is slow; once the download starts, even larger files arrive quite quickly (since nginx caching is enabled). So it may be much more efficient to download one bigger file once every three hours, than a small file for more or less each transaction. But this may be complicated to realise; what should the file be? Of course I am mainly thinking of one branch and its current list of packages - but which one? And nothing prevents people from remaining on some old git commit, and hydra will happily serve any package that is still available in its store. So the implementation would need to change from a reply to a simple request if some packages are available, to some logic on the hydra side guessing the list of packages a client might be interested in. So maybe just switching to a more powerful and reactive hydra machine would be the better way to solve this problem! Andreas PS: And we should stop "updating list of substitutes from 'http://hydra.gnu.org'" upon the command "guix package -r upower". Which list is requested during this call?