From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id qPUDLIR3a1/GFAAA0tVLHw (envelope-from ) for ; Wed, 23 Sep 2020 16:27:48 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id KBENKIR3a18GCQAA1q6Kng (envelope-from ) for ; Wed, 23 Sep 2020 16:27:48 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 41C289404D3 for ; Wed, 23 Sep 2020 16:27:48 +0000 (UTC) Received: from localhost ([::1]:33304 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7cR-00067h-0T for larch@yhetil.org; Wed, 23 Sep 2020 12:27:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51618) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kL7Yp-00022E-6V for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:54312) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kL7Yo-0000Nu-SF for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kL7Yo-0001Yt-PP for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43578] [PATCH 1/4] packages: 'package-mapping' can recurse on implicit inputs. References: <20200923161253.2378-1-ludo@gnu.org> In-Reply-To: <20200923161253.2378-1-ludo@gnu.org> Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 23 Sep 2020 16:24:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43578 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43578@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 43578-submit@debbugs.gnu.org id=B43578.16008782135912 (code B ref 43578); Wed, 23 Sep 2020 16:24:02 +0000 Received: (at 43578) by debbugs.gnu.org; 23 Sep 2020 16:23:33 +0000 Received: from localhost ([127.0.0.1]:37616 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YK-0001XD-Lt for submit@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:33 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44652) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YI-0001Wq-Qs for 43578@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47175) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7YD-0000Is-Cx; Wed, 23 Sep 2020 12:23:25 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52578 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kL7YB-0002gP-NW; Wed, 23 Sep 2020 12:23:24 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Wed, 23 Sep 2020 18:23:15 +0200 Message-Id: <20200923162318.2800-1-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: 3.99 X-TUID: 56QrJDimaLIR * guix/packages.scm (build-system-with-package-mapping): New procedure. (package-mapping): Add #:deep? and honor it. * tests/packages.scm ("package-mapping"): Compare the direct inputs of the bag of P0 and that of P1. ("package-mapping, deep"): New test. --- doc/guix.texi | 5 ++-- guix/packages.scm | 65 +++++++++++++++++++++++++++++++++++----------- tests/packages.scm | 36 ++++++++++++++++++++++++- 3 files changed, 88 insertions(+), 18 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index f7e2204b53..4595008c4f 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6294,10 +6294,11 @@ A more generic procedure to rewrite a package dependency graph is @code{package-mapping}: it supports arbitrary changes to nodes in the graph. -@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] +@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] [#:deep? #f] Return a procedure that, given a package, applies @var{proc} to all the packages depended on and returns the resulting package. The procedure stops recursion -when @var{cut?} returns true for a given package. +when @var{cut?} returns true for a given package. When @var{deep?} is true, @var{proc} is +applied to implicit inputs as well. @end deffn @menu diff --git a/guix/packages.scm b/guix/packages.scm index 6598bd3149..171fd048ef 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -968,10 +968,31 @@ packages they depend on, recursively." (vhash-consq package #t visited) (fold set-insert closure dependencies)))))))) -(define* (package-mapping proc #:optional (cut? (const #f))) +(define (build-system-with-package-mapping bs rewrite) + "Return a variant of BS, a build system, that rewrites a bag's inputs by +passing them through REWRITE, a procedure that takes an input tuplet and +returns a \"rewritten\" input tuplet." + (define lower + (build-system-lower bs)) + + (define (lower* . args) + (let ((lowered (apply lower args))) + (bag + (inherit lowered) + (build-inputs (map rewrite (bag-build-inputs lowered))) + (host-inputs (map rewrite (bag-host-inputs lowered))) + (target-inputs (map rewrite (bag-target-inputs lowered)))))) + + (build-system + (inherit bs) + (lower lower*))) + +(define* (package-mapping proc #:optional (cut? (const #f)) + #:key deep?) "Return a procedure that, given a package, applies PROC to all the packages depended on and returns the resulting package. The procedure stops recursion -when CUT? returns true for a given package." +when CUT? returns true for a given package. When DEEP? is true, PROC is +applied to implicit inputs as well." (define (rewrite input) (match input ((label (? package? package) outputs ...) @@ -980,21 +1001,35 @@ when CUT? returns true for a given package." (_ input))) + (define mapping-property + ;; Property indicating whether the package has already been processed. + (gensym " package-mapping-done")) + (define replace (mlambdaq (p) - ;; Return a variant of P with PROC applied to P and its explicit - ;; dependencies, recursively. Memoize the transformations. Failing to - ;; do that, we would build a huge object graph with lots of duplicates, - ;; which in turns prevents us from benefiting from memoization in - ;; 'package-derivation'. - (let ((p (proc p))) - (package - (inherit p) - (location (package-location p)) - (inputs (map rewrite (package-inputs p))) - (native-inputs (map rewrite (package-native-inputs p))) - (propagated-inputs (map rewrite (package-propagated-inputs p))) - (replacement (and=> (package-replacement p) proc)))))) + ;; If P is the result of a previous call, return it. + (if (assq-ref (package-properties p) mapping-property) + p + + ;; Return a variant of P with PROC applied to P and its explicit + ;; dependencies, recursively. Memoize the transformations. Failing + ;; to do that, we would build a huge object graph with lots of + ;; duplicates, which in turns prevents us from benefiting from + ;; memoization in 'package-derivation'. + (let ((p (proc p))) + (package + (inherit p) + (location (package-location p)) + (build-system (if deep? + (build-system-with-package-mapping + (package-build-system p) rewrite) + (package-build-system p))) + (inputs (map rewrite (package-inputs p))) + (native-inputs (map rewrite (package-native-inputs p))) + (propagated-inputs (map rewrite (package-propagated-inputs p))) + (replacement (and=> (package-replacement p) proc)) + (properties `((,mapping-property . #t) + ,@(package-properties p)))))))) replace) diff --git a/tests/packages.scm b/tests/packages.scm index cbd0503733..f33332a461 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1172,15 +1172,24 @@ (let* ((dep (dummy-package "chbouib" (native-inputs `(("x" ,grep))))) (p0 (dummy-package "example" + (source 77) (inputs `(("foo" ,coreutils) ("bar" ,grep) ("baz" ,dep))))) (transform (lambda (p) (package (inherit p) (source 42)))) (rewrite (package-mapping transform)) - (p1 (rewrite p0))) + (p1 (rewrite p0)) + (bag0 (package->bag p0)) + (bag1 (package->bag p1))) (and (eq? p1 (rewrite p0)) (eqv? 42 (package-source p1)) + + ;; Implicit inputs should be left unchanged (skip "source", "foo", + ;; "bar", and "baz" in this comparison). + (equal? (drop (bag-direct-inputs bag0) 4) + (drop (bag-direct-inputs bag1) 4)) + (match (package-inputs p1) ((("foo" dep1) ("bar" dep2) ("baz" dep3)) (and (eq? dep1 (rewrite coreutils)) ;memoization @@ -1194,6 +1203,31 @@ (and (eq? dep (rewrite grep)) (package-source dep)))))))))) +(test-equal "package-mapping, deep" + '(42) + (let* ((p0 (dummy-package "example" + (inputs `(("foo" ,coreutils) + ("bar" ,grep))))) + (transform (lambda (p) + (package (inherit p) (source 42)))) + (rewrite (package-mapping transform #:deep? #t)) + (p1 (rewrite p0)) + (bag (package->bag p1))) + (and (eq? p1 (rewrite p0)) + (match (bag-direct-inputs bag) + ((("source" 42) ("foo" dep1) ("bar" dep2) rest ..1) + (and (eq? dep1 (rewrite coreutils)) ;memoization + (eq? dep2 (rewrite grep)) + (= 42 (package-source dep1)) + (= 42 (package-source dep2)) + + ;; Check that implicit inputs of P0 also got rewritten. + (delete-duplicates + (map (match-lambda + ((_ package . _) + (package-source package))) + rest)))))))) + (test-assert "package-input-rewriting" (let* ((dep (dummy-package "chbouib" (native-inputs `(("x" ,grep))))) -- 2.28.0