From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id gOAHJHki/mNSQQEAbAwnHQ (envelope-from ) for ; Tue, 28 Feb 2023 16:49:13 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id yBb9I3ki/mNwJAEA9RJhRA (envelope-from ) for ; Tue, 28 Feb 2023 16:49:13 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4AD86860E for ; Tue, 28 Feb 2023 16:49:13 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pX2DM-00004r-B1; Tue, 28 Feb 2023 10:48:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pX2D7-00081H-5x for guix-patches@gnu.org; Tue, 28 Feb 2023 10:48:14 -0500 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pX2Cw-0004by-Mn for guix-patches@gnu.org; Tue, 28 Feb 2023 10:48:11 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pX2Cw-0003rW-IC for guix-patches@gnu.org; Tue, 28 Feb 2023 10:48:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#61363] [PATCH v2 3/3] self: Apply grafts to the outputs of the guix derivation. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 28 Feb 2023 15:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61363 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 61363@debbugs.gnu.org Received: via spool by 61363-submit@debbugs.gnu.org id=B61363.167759923514649 (code B ref 61363); Tue, 28 Feb 2023 15:48:02 +0000 Received: (at 61363) by debbugs.gnu.org; 28 Feb 2023 15:47:15 +0000 Received: from localhost ([127.0.0.1]:51796 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pX2CA-0003oC-Cx for submit@debbugs.gnu.org; Tue, 28 Feb 2023 10:47:14 -0500 Received: from mira.cbaines.net ([212.71.252.8]:42292) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pX2C1-0003nR-PG for 61363@debbugs.gnu.org; Tue, 28 Feb 2023 10:47:06 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699]) by mira.cbaines.net (Postfix) with ESMTPSA id 6EAB916C22 for <61363@debbugs.gnu.org>; Tue, 28 Feb 2023 15:47:04 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 11115824 for <61363@debbugs.gnu.org>; Tue, 28 Feb 2023 15:47:03 +0000 (UTC) From: Christopher Baines Date: Tue, 28 Feb 2023 15:47:03 +0000 Message-Id: <20230228154703.3952-3-mail@cbaines.net> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230228154703.3952-1-mail@cbaines.net> References: <20230228154703.3952-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1677599353; a=rsa-sha256; cv=none; b=WOom3uzmHKsXu8xZxhTkieswVTlkJW2YftyIgD4yCK5trZGjUyaPMwSNu+ZjyOc+2G8GlM T1hZMKvnaXq8njuO4ISz2YvGKWMS0ErQ2/3/KOkrKN2AXSv8lbVEWYfl7SPIxiuVDiJtrA tvSR8jEoWjhlAAZHW//hd9hi+vsakCWCG2LYtS2ABeFpEWKby66qaK4uvR3/FdodFXBsnC Nx+wjlcr6Bnpc3dVBmSo5JHfoMsTVrzcpG2/8r8rST+2EQTm7lzfOBH2n+q5IxQhteDt1u cSdVJP3WqOJz5EPD4H1jXi84V8vJbq68xZ0Luw95EKrIvW+6Z/DclzwbiQLS2Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1677599353; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=frLe/dC6iVlvEjezr+y1lppv0nMxEAKRR0FXmVCj2FA=; b=S7YDvlkhdyrF7oWbDvUDoIz0DpgSWU96o/njOFWGR+uPx1POTP91F1AtzObZVFdM5mIOPk RundKs6r1aZS6da7w6s7RXI9otl9sETWGlJAJUKtbvP7v7ZcGqd4Meg6PG1Eh7wV6ItjUs c1svVJQTISSKdkc47fXtgEGWc4GJN60JwGI+Ym/zqRhUOrZrV0+4x2S73AkyAYz9d26oQQ AKZRZxbFCZeBtUl8LiiCqoah5cnv6sXk+Vi2MIW0rrTDDm7WvjoKR86zA55heTWDCL+rPV iC+rB83q/8GEh8dBXt8L8c2+bVSOt6QvB8yZ/b2FvYAUG1DP7p49XEsb72deOQ== X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -1.69 X-Spam-Score: -1.69 X-Migadu-Queue-Id: 4AD86860E Authentication-Results: aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none X-TUID: ekkopO8BDs8F Rather than having grafts apply to the derivation itself. This moves grafting here to work like grafting for packages, where you can think of the grafted outputs as a transformed variant of the ungrafted outputs. I'm looking at this as it'll allow the Guix Data Service to compute the derivations without grafts, and for these to be useful for substitutes regardless of whether users are using grafts. * guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword argument, to control grafting when computing the guix derivation. * build-aux/build-self.scm (build-program): Call guix-derivation with #:graft? (%graft?) to make the compute-guix-derivation script use or not use grafts as desired. --- build-aux/build-self.scm | 4 +- guix/self.scm | 101 +++++++++++++++++++++++++++++++-------- 2 files changed, 84 insertions(+), 21 deletions(-) diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm index 02822a2ee8..6d0037f20c 100644 --- a/build-aux/build-self.scm +++ b/build-aux/build-self.scm @@ -353,7 +353,9 @@ (define fake-git #:channel-metadata '#$channel-metadata #:pull-version - #$pull-version) + #$pull-version + #:graft? + #$(%graft?)) #:system system)) derivation-file-name)))))) #:module-path (list source)))) diff --git a/guix/self.scm b/guix/self.scm index c5de3ab8fc..8842275ff8 100644 --- a/guix/self.scm +++ b/guix/self.scm @@ -22,6 +22,7 @@ (define-module (guix self) #:use-module (guix i18n) #:use-module (guix modules) #:use-module (guix gexp) + #:use-module (guix grafts) #:use-module (guix store) #:use-module (guix monads) #:use-module (guix discovery) @@ -32,6 +33,7 @@ (define-module (guix self) #:use-module ((guix build utils) #:select (find-files)) #:use-module (srfi srfi-1) #:use-module (srfi srfi-9) + #:use-module (srfi srfi-9 gnu) #:use-module (srfi srfi-35) #:use-module (ice-9 match) #:export (make-config.scm @@ -244,6 +246,50 @@ (define* (file-append* item file #:key (recursive? #t)) ;; which isn't great. (file-append item "/" file)))) +(define graft-derivation* + (store-lift graft-derivation)) + +(define package-grafts* + (store-lift package-grafts)) + +;; Apply grafts explicitly +(define-immutable-record-type + (%explicit-grafting obj packages) + explicit-grafting? + (obj explicit-grafting-obj) ;obj + (packages explicit-grafting-packages)) ;list of s + +(define (write-explicit-grafting rec port) + (match rec + (($ obj packages) + (format port "#" obj packages)))) + +(define (explicit-grafting obj packages) + (%explicit-grafting obj packages)) + +(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting ) + system target) + (match explicit-grafting + (($ obj packages) + (mlet* %store-monad ((drv (without-grafting + (lower-object obj system #:target target))) + (grafts + (mapm %store-monad + (lambda (pkg) + (package-grafts* pkg system #:target target)) + packages))) + (match (delete-duplicates + (concatenate grafts)) + (() + (return drv)) + (grafts + (mlet %store-monad ((guile (package->derivation + (guile-for-grafts) + system #:graft? #f))) + (graft-derivation* drv grafts + #:system system + #:guile guile)))))))) + (define* (locale-data source domain #:optional (directory domain)) "Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to @@ -754,7 +800,8 @@ (define* (compiled-guix source #:key (gzip (specification->package "gzip")) (bzip2 (specification->package "bzip2")) (xz (specification->package "xz")) - (guix (specification->package "guix"))) + (guix (specification->package "guix")) + (graft? #t)) "Return a file-like object that contains a compiled Guix." (define guile-avahi (specification->package "guile-avahi")) @@ -1024,25 +1071,34 @@ (define (built-modules node-subset) guile-lzma dependencies) #:guile guile-for-build - #:guile-version guile-version))) - (whole-package name modules dependencies - #:command command - #:guile guile-for-build - - ;; Include 'guix-daemon'. XXX: Here we inject an - ;; older snapshot of guix-daemon, but that's a good - ;; enough approximation for now. - #:daemon (specification->package "guix-daemon") - - #:info (info-manual source) - #:miscellany (miscellaneous-files source) - #:guile-version guile-version))) + #:guile-version guile-version)) + (obj + (whole-package name modules dependencies + #:command command + #:guile guile-for-build + + ;; Include 'guix-daemon'. XXX: Here we inject + ;; an older snapshot of guix-daemon, but + ;; that's a good enough approximation for now. + #:daemon (specification->package "guix-daemon") + + #:info (info-manual source) + #:miscellany (miscellaneous-files source) + #:guile-version guile-version))) + (if graft? + (explicit-grafting obj + (map (compose force cdr) %packages)) + obj))) ((= 0 pull-version) ;; Legacy 'guix pull': return the .scm and .go files as one ;; directory. - (built-modules (lambda (node) - (list (node-source node) - (node-compiled node))))) + (let ((obj (built-modules (lambda (node) + (list (node-source node) + (node-compiled node)))))) + (if graft? + (explicit-grafting obj + (map (compose force cdr) %packages)) + obj))) (else ;; Unsupported 'guix pull' version. #f))) @@ -1272,7 +1328,8 @@ (define (process-directory directory files output) (define* (guix-derivation source version #:optional (guile-version (effective-version)) #:key (pull-version 0) - channel-metadata) + channel-metadata + (graft? #t)) "Return, as a monadic value, the derivation to build the Guix from SOURCE for GUILE-VERSION. Use VERSION as the version string. Use CHANNEL-METADATA as the channel metadata sexp to include in (guix config). @@ -1309,7 +1366,11 @@ (define guile #:pull-version pull-version #:guile-version (if (>= pull-version 1) "3.0" guile-version) - #:guile-for-build guile))) + #:guile-for-build guile + #:graft? graft?))) (if guix - (lower-object guix) + (if graft? + (lower-object guix) + (without-grafting + (lower-object guix))) (return #f))))) -- 2.39.1