From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Subject: bug#22990: Grafts leads to inefficient substitute info retrieval Date: Sun, 08 Jan 2017 22:44:41 +0100 Message-ID: <87k2a5rzue.fsf@gnu.org> References: <8737rxx8gk.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45130) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cQLGz-0000tG-ST for bug-guix@gnu.org; Sun, 08 Jan 2017 16:45:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cQLGv-00084f-VF for bug-guix@gnu.org; Sun, 08 Jan 2017 16:45:05 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:60010) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cQLGv-00084Y-SQ for bug-guix@gnu.org; Sun, 08 Jan 2017 16:45:01 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1cQLGv-0005R7-Jo for bug-guix@gnu.org; Sun, 08 Jan 2017 16:45:01 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <8737rxx8gk.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Fri, 11 Mar 2016 17:52:59 +0100") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: 22990@debbugs.gnu.org ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > Concretely, what this means is this: > > $ guix package -u > substitute: updating list of substitutes from 'http://mirror.guixsd.org= '... 100.0% > substitute: updating list of substitutes from 'http://mirror.guixsd.org= '... 100.0% > substitute: updating list of substitutes from 'http://mirror.guixsd.org= '... 100.0% > substitute: updating list of substitutes from 'http://mirror.guixsd.org= '... 100.0% > substitute: updating list of substitutes from 'http://mirror.guixsd.org= '... 100.0% > substitute: updating list of substitutes from 'http://mirror.guixsd.org= '... 100.0% > substitute: updating list of substitutes from 'http://mirror.guixsd.org= '... 100.0% > [=E2=80=A6] > The following files would be downloaded: > > Each of the initial =E2=80=9Cupdating list=E2=80=9D message corresponds t= o an HTTP > request for a single narinfo file, which can take around 1 second. > > Instead, the ideal thing would be to fetch the narinfo files for all the > relevant packages at once; that way, we=E2=80=99d spawn =E2=80=98guix sub= stitute=E2=80=99 only > once, and it would benefit from HTTP pipelining (one round-trip instead > of N.) > > To achieve this, I=E2=80=99m thinking of extending gexp code such that ge= xp > compilers can return a list of applicable grafts. The =E2=80=98package= =E2=80=99 > compiler would do #:graft? #f and instead let =E2=80=98gexp->derivation= =E2=80=99 call > =E2=80=98graft-derivation=E2=80=99. The =E2=80=98wip-gexp-grafts=E2=80=99 branch does that. Namely, it=E2=80= =99s possible to know what grafts would apply to a gexp derivation build, so that one can first build ungrafted, and then apply the grafts to the results. So for a profile, we=E2=80=99d first build the profile as is, and only then would we graft it. Right now the tip of this branch is a hack such that =E2=80=98guix package= =E2=80=99: 1. Builds the original (ungrafted) derivation of the profile; 2. Manually calls =E2=80=98graft-derivation=E2=80=99 on that, passing it = the list of applicable grafts. Conceptually it=E2=80=99s what we want to do, but the drawback is that the caller (here =E2=80=98guix package=E2=80=99) goes through a lot of hops to = get the list of grafts and to apply it. I think we should instead have a way to annotate a derivation with a list of grafts, as well as a procedure to build that derivations in two phases (first the original derivation, then the grafts). To be continued=E2=80=A6 Ludo=E2=80=99.