From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Bavier Subject: [PATCH] guix: build: Add transitive source building. Date: Wed, 28 Jan 2015 14:02:13 -0600 Message-ID: <87r3uesmgq.fsf@member.fsf.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:33006) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YGYoh-0007JM-Jp for guix-devel@gnu.org; Wed, 28 Jan 2015 15:02:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YGYoc-0001S3-Ff for guix-devel@gnu.org; Wed, 28 Jan 2015 15:02:23 -0500 Received: from mail-ie0-x235.google.com ([2607:f8b0:4001:c03::235]:64213) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YGYoc-0001RU-8B for guix-devel@gnu.org; Wed, 28 Jan 2015 15:02:18 -0500 Received: by mail-ie0-f181.google.com with SMTP id rp18so24286436iec.12 for ; Wed, 28 Jan 2015 12:02:17 -0800 (PST) Received: from cooper.gmail.com (chippewa-nat.cray.com. [136.162.34.1]) by mx.google.com with ESMTPSA id o143sm2999988ioo.14.2015.01.28.12.02.15 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Jan 2015 12:02:16 -0800 (PST) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: guix-devel@gnu.org --=-=-= Hello Guix, This patch is intended mostly for discussion. It was brought up again on IRC yesterday the idea of prefetching package source into the store in order to be able to hack on guix offline. A naive approach such as running `guix build -S` on each entry of `guix package --list-available` isn't enough because of build-system-implicit inputs and package origin inputs, as well as non-public bootstrapping source. This patch changes the semantics of `guix build --source` so that it takes an optional argument, one of 'package', 'all' (default), or 'transitive'. `guix build --source=package foo` will build the source derivations for foo's source origin, which is the current behavior. `guix build --source=all foo` will build all of foo's origin's, including any which might be in its inputs (e.g. texlive's texmf-extra-src input). `guix build --source=transitive foo` will build the source origins for all of foo's transitive inputs. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-guix-build-Add-transitive-source-building.patch >From 81d161e418753ce2d136ea901cc28b11cee26314 Mon Sep 17 00:00:00 2001 From: Eric Bavier Date: Wed, 28 Jan 2015 13:33:28 -0600 Subject: [PATCH] guix: build: Add transitive source building. * guix/scripts/build.scm [--source]: Add optional arguments 'package', 'all', and 'transitive'. (package-source*, package-direct-source, package-transitive-source): New procedures. (options->derivations)[--source]: Use them. --- guix/scripts/build.scm | 78 ++++++++++++++++++++++++++++++++++++------------ 1 file changed, 59 insertions(+), 19 deletions(-) diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm index 07ced30..8a422fe 100644 --- a/guix/scripts/build.scm +++ b/guix/scripts/build.scm @@ -262,10 +262,21 @@ Build the given PACKAGE-OR-DERIVATION and return their output paths.\n")) (option '(#\V "version") #f #f (lambda args (show-version-and-exit "guix build"))) - - (option '(#\S "source") #f #f + (option '(#\S "source") #f #t (lambda (opt name arg result) - (alist-cons 'source? #t result))) + (match arg + ("package" + (alist-cons 'source package-source* result)) + ("all" + (alist-cons 'source package-direct-source result)) + ("transitive" + (alist-cons 'source package-transitive-source result)) + (#f + (alist-cons 'source package-source* result)) + (else + (leave (_ "invalid argument: '~a' option argument: ~a, ~ +must be one of 'package', 'all', or 'transitive'~%") + name arg))))) (option '(#\s "system") #t #f (lambda (opt name arg result) (alist-cons 'system arg @@ -299,6 +310,32 @@ Build the given PACKAGE-OR-DERIVATION and return their output paths.\n")) %standard-build-options)) +(define (package-source* package) + "List package-source but returns its results as a list" + (list (package-source package))) + +(define (package-direct-source package) + "Return all source origins associated with PACKAGE; including origins in +PACKAGE's inputs." + `(,@(or (and=> (package-source package) list) '()) + ,@(filter-map (match-lambda + ((_ (? origin? orig) _ ...) + orig) + (_ #f)) + (package-direct-inputs package)))) + +(define (package-transitive-source package) + "Return PACKAGE's direct sources, and its input sources, recursively." + (delete-duplicates + (concatenate (filter-map (match-lambda + ((_ (? origin? orig) _ ...) + (list orig)) + ((_ (? package? p) _ ...) + (package-direct-source p)) + (_ #f)) + (bag-transitive-inputs + (package->bag package)))))) + (define (options->derivations store opts) "Given OPTS, the result of 'args-fold', return a list of derivations to build." @@ -308,28 +345,31 @@ build." (triplet (cut package-cross-derivation <> <> triplet <>)))) - (define src? (assoc-ref opts 'source?)) + (define src (assoc-ref opts 'source)) (define sys (assoc-ref opts 'system)) (define graft? (assoc-ref opts 'graft?)) (parameterize ((%graft? graft?)) (let ((opts (options/with-source store (options/resolve-packages store opts)))) - (filter-map (match-lambda - (('argument . (? package? p)) - (if src? - (let ((s (package-source p))) - (package-source-derivation store s)) - (package->derivation store p sys))) - (('argument . (? derivation? drv)) - drv) - (('argument . (? derivation-path? drv)) - (call-with-input-file drv read-derivation)) - (('argument . (? store-path?)) - ;; Nothing to do; maybe for --log-file. - #f) - (_ #f)) - opts)))) + (concatenate + (filter-map (match-lambda + (('argument . (? package? p)) + (match src + (#f + (list (package->derivation store p sys))) + (proc + (map (cut package-source-derivation store <>) + (proc p))))) + (('argument . (? derivation? drv)) + (list drv)) + (('argument . (? derivation-path? drv)) + (list (call-with-input-file drv read-derivation))) + (('argument . (? store-path?)) + ;; Nothing to do; maybe for --log-file. + #f) + (_ #f)) + opts))))) (define (options/resolve-packages store opts) "Return OPTS with package specification strings replaced by actual -- 1.7.9.5 --=-=-= Obviously all this would need to be documented before actually apply such a patch. I've tested this patch by building e.g. gmsh (which I did not already have in the store):: $ guix build --source=transitive gmsh [...] $ guix build gmsh The build succeeded after building the glu, glproto, fltk, and xf86vidmodeproto inputs from source. I haven't yet tried making a change that would require rebootstrapping while offline. I think the idea is that this should work. What I'd like to get some feedback on: 1. Adding an optional argument to `guix build --source` currently breaks `guix build -S foo`, since the option parsing thinks "foo" is the argument for -S. I'm not sure how this could be reconciled. 2. There might be clearer user/developer-facing names for the arguments. 3. Am I understanding the use-case? Thanks, -- Eric Bavier --=-=-=--