From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark H Weaver Subject: Re: updating list of substitutes Date: Mon, 12 Oct 2015 12:31:21 -0400 Message-ID: <87d1wkaw12.fsf@netris.org> References: <20150421064525.GA15795@thebird.nl> <87a8y1q49z.fsf@gnu.org> <20150421084028.GB16564@thebird.nl> <20150422114635.GA24566@thebird.nl> <20151011074654.GA8058@thebird.nl> <87wputz1uj.fsf@gnu.org> <20151011212714.GA9401@thebird.nl> <87d1wk8y7e.fsf@netris.org> <20151012060607.GA11012@thebird.nl> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:55387) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zlg0a-00086M-Gg for guix-devel@gnu.org; Mon, 12 Oct 2015 12:31:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zlg0X-0006Er-Ol for guix-devel@gnu.org; Mon, 12 Oct 2015 12:31:32 -0400 Received: from world.peace.net ([50.252.239.5]:60790) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zlg0X-0006Ee-ME for guix-devel@gnu.org; Mon, 12 Oct 2015 12:31:29 -0400 In-Reply-To: <20151012060607.GA11012@thebird.nl> (Pjotr Prins's message of "Mon, 12 Oct 2015 08:06:07 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Pjotr Prins Cc: guix-devel@gnu.org Pjotr Prins writes: > On Mon, Oct 12, 2015 at 01:15:01AM -0400, Mark H Weaver wrote: >> The phrase "the substitute list" suggests a single, complete list of all >> available substitutes, but there is no such list. Instead, quoting >> Ludovic above: >> >> "When building a package FOO, Guix looks for substitutes for FOO >> and its prerequisites (those not already available locally.) It >> maintains in /var/guix/substitute/cache a cache of those lookups." >> >> So, if you build package BAR immediately after building FOO, a different >> set of substitutes is queried, and typically that involves more lookups >> (unless FOO runtime-depends on BAR). >> >> Does that make sense? > > Right. So, why don't we have one list for every build? That would save > connecting to the one single server every time and be less fragile. I'm not sure how to make this work well in practice. One issue is that the set of available substitutes changes very frequently. Take a look at the "Finished at" field of to get an idea. When Hydra is busy, the set of available substitutes changes every few minutes, and sometimes several times each minute. How would we deal with that? There are other difficult problems as well, e.g. that the set of available substitutes includes builds for multiple branches of our git repo, multiple architectures, and multiple "evaluations" corresponding to different commits on any given branch of our git repo. In summary, the full set of available substitutes is typically quite large and changes frequently, so this approach would entail a lot of wasted network bandwidth (and load on hydra) to maintain the complete list of substitutes on every client machine, although only a small fraction of these would be of interest to any given user. Does that make sense? Mark