unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
* bug#22990: Grafts leads to inefficient substitute info retrieval
@ 2016-03-11 16:52 Ludovic Courtès
  2016-03-12  9:23 ` Alex Kost
                   ` (3 more replies)
  0 siblings, 4 replies; 12+ messages in thread
From: Ludovic Courtès @ 2016-03-11 16:52 UTC (permalink / raw)
  To: 22990

As of right now (v0.9.0-2007-g66a30a3), ‘graft-derivation’ works either by:

  1. Fetching substitute info about the things being built so that it
     can determine its references, which in turns allows it to determine
     whether they need to be grafted.

  2. Building stuff, as a last resort, so that it can determine its
     references.

Case #1 is hopefully going to be the most common.

The problem with #1 is that when building a profile, we do one
‘package-derivation’ call for each package in the profile, which
translates in one ‘graft-derivation’ call for each relevant package¹,
which translates into one ‘references/substitutes’ call for each.

Concretely, what this means is this:

  $ guix package -u
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'... 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'... 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'... 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'... 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'... 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'... 100.0%
  substitute: updating list of substitutes from 'http://mirror.guixsd.org'... 100.0%
  […]
  The following files would be downloaded:

Each of the initial “updating list” message corresponds to an HTTP
request for a single narinfo file, which can take around 1 second.

Instead, the ideal thing would be to fetch the narinfo files for all the
relevant packages at once; that way, we’d spawn ‘guix substitute’ only
once, and it would benefit from HTTP pipelining (one round-trip instead
of N.)

To achieve this, I’m thinking of extending gexp code such that gexp
compilers can return a list of applicable grafts.  The ‘package’
compiler would do #:graft? #f and instead let ‘gexp->derivation’ call
‘graft-derivation’.

I’ll give it a try and report back.

Ludo’.

¹ A package is “relevant” if ‘package-grafts’ returns a non-empty list.

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2020-03-30  2:36 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-03-11 16:52 bug#22990: Grafts leads to inefficient substitute info retrieval Ludovic Courtès
2016-03-12  9:23 ` Alex Kost
2016-03-13 12:11   ` Ludovic Courtès
2016-03-15 18:49     ` Mark H Weaver
2016-03-15 21:50       ` Ludovic Courtès
2016-03-15  8:24 ` Ludovic Courtès
2017-01-08 21:44 ` Ludovic Courtès
2017-01-09 22:55   ` Ludovic Courtès
2017-01-11  9:54     ` Ludovic Courtès
2017-01-11 10:51       ` David Craven
2017-01-11 21:19         ` Ludovic Courtès
2020-03-29 13:41 ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).