From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id qJfYGbSMhmazPgEAe85BDQ:P1 (envelope-from ) for ; Thu, 04 Jul 2024 11:51:16 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id qJfYGbSMhmazPgEAe85BDQ (envelope-from ) for ; Thu, 04 Jul 2024 13:51:16 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1720093876; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=MhNRcglWAIey1W72BnQfrgVSkCcKmvdMmOE3o0IPdLY=; b=FFKFtcE0mCg2C0pEgAkVIe+N42F4oqhP2N+S5bfc3mmhmthxfw4VFjWmRd47tBg7x1jWcM LG1mxjiulNTQyYnkFZ9c8TuqCUyy1pnvHdnNPgZP0i4KFtiw+q7QZdaNMwE8c5R1g+/RBU Ww09EmuRIc0S13Ges8Gc2ND4uPP0tHT93fCHS+v5g46P1whUrGKEgpoycHjR/wtxDEf96w laj9WGHqYBB7YMpuo4GAhPbBgBohCP8D3p55bGrN4KQvDMkBsFAn7CIC2ff8WbNGggXr1D 3TuVAP2CZOv6ui2LYdDih5pUnIZw5OJD7BHoR06D5xsfS3G2zeHTbX75ylAorw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1720093876; a=rsa-sha256; cv=none; b=bLVoQ9yWyyi7DeW+y0e55ihWOk4kUyhygNTiGW9Apcr4JwZbx6HQsIXk2ka2xTpJjqvN2a 8AjHdok1Xj4er4ACwgVSdiocIPLC5iM2c58yJXwQt/yWBjzn/f+lNNwcgHPEoUMq3wY67N 8DcLewf6U2I5WtRzhgc4nIz7EAOV5WQCOLDHAPpyZC815+Vf19Z7vCEelcE2Y3z72sPBJq 0Q2qDQ89E9V9XNMile9qcsvAz59GRzsvX1Gu/qvzbiGdrVxH+zkBR/NFJohfc2GtyTMkAp pFCAfliVkc+yfg7bnlUKCEGc9XmK4EYugYuR+S+ToVP7nC5Qvoh7wVeO1J4G7w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E727163BD7 for ; Thu, 4 Jul 2024 13:51:15 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sPKzW-0004J5-43; Thu, 04 Jul 2024 07:51:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sPKzS-0004H8-M9 for guix-patches@gnu.org; Thu, 04 Jul 2024 07:51:07 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sPKzR-0004tJ-9K; Thu, 04 Jul 2024 07:51:06 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sPKzO-0006ql-Ez; Thu, 04 Jul 2024 07:51:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#71038] [PATCH v3 2/2] guix: channels: Enable specifiying available builtin builders. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Thu, 04 Jul 2024 11:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 71038 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 71038@debbugs.gnu.org Cc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 71038-submit@debbugs.gnu.org id=B71038.172009382826278 (code B ref 71038); Thu, 04 Jul 2024 11:51:02 +0000 Received: (at 71038) by debbugs.gnu.org; 4 Jul 2024 11:50:28 +0000 Received: from localhost ([127.0.0.1]:41684 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sPKyq-0006pl-0i for submit@debbugs.gnu.org; Thu, 04 Jul 2024 07:50:28 -0400 Received: from mira.cbaines.net ([212.71.252.8]:43900) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sPKye-0006pT-Bg for 71038@debbugs.gnu.org; Thu, 04 Jul 2024 07:50:26 -0400 Received: from localhost (nat-inria-guest-32-gw-01-bso.bordeaux.inria.fr [194.199.1.32]) by mira.cbaines.net (Postfix) with ESMTPSA id EB72927BBE9 for <71038@debbugs.gnu.org>; Thu, 4 Jul 2024 12:50:11 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id e6d59174 for <71038@debbugs.gnu.org>; Thu, 4 Jul 2024 11:50:10 +0000 (UTC) From: Christopher Baines Date: Thu, 4 Jul 2024 13:50:09 +0200 Message-ID: <0d2d50291d3c0c7be10c8a535bf64dec49a4f1eb.1720093809.git.mail@cbaines.net> X-Mailer: git-send-email 2.45.2 In-Reply-To: <14fc15bdb8b5fd615fb93e025e86ccf70cf81814.1720093809.git.mail@cbaines.net> References: <14fc15bdb8b5fd615fb93e025e86ccf70cf81814.1720093809.git.mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Queue-Id: E727163BD7 X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -1.89 X-Spam-Score: -1.89 X-TUID: HhyX6Z+yM8oL When computing channel instance derivations. This is useful when you want to generate compatible derivations that can be run with a daemon that potentially doesn't support builtin builders that the daemon you're using to generate the derivations has. I'm looking at this in particular because I want to use this in the data service, since it provides substitutes for derivations, and since these can be built on other machines, it's useful to control which builtin builders they depend on. Fixes: . * build-aux/build-self.scm (build-program): Accept #:built-in-builders and pass along to port->connection or open-connection as approriate. (build): Accept and pass on #:built-in-builders. * guix/channels.scm (build-from-source, build-channel-instance, channel-instance-derivations, channel-instances->manifest, channel-instances->derivation): Accept and pass on #:built-in-builders. Change-Id: I315c990de66c6f7dca25a859165a5568abe385ea --- build-aux/build-self.scm | 29 +++++++++++------ guix/channels.scm | 67 ++++++++++++++++++++++++++++++---------- 2 files changed, 70 insertions(+), 26 deletions(-) diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm index 02822a2ee8..d9299b9af2 100644 --- a/build-aux/build-self.scm +++ b/build-aux/build-self.scm @@ -241,8 +241,12 @@ (define guile-gcrypt (define* (build-program source version #:optional (guile-version (effective-version)) - #:key (pull-version 0) (channel-metadata #f)) - "Return a program that computes the derivation to build Guix from SOURCE." + #:key (pull-version 0) (channel-metadata #f) + built-in-builders) + "Return a program that computes the derivation to build Guix from SOURCE. +If BUILT-IN-BUILDERS is provided, it should be a list of +strings and this will be used instead of the builtin builders provided by the +build daemon, from within the generated build program." (define select? ;; Select every module but (guix config) and non-Guix modules. ;; Also exclude (guix channels): it is autoloaded by (guix describe), but @@ -331,11 +335,16 @@ (define* (build-program source version ;; case, attempt to open a new connection. (let* ((proto (string->number protocol-version)) (store (if (integer? proto) - (port->connection (duplicate-port - (current-input-port) - "w+0") - #:version proto) - (open-connection))) + (port->connection + (duplicate-port + (current-input-port) + "w+0") + #:version proto + #:built-in-builders + '#$built-in-builders) + (open-connection + #:built-in-builders + '#$built-in-builders))) (sock (socket AF_UNIX SOCK_STREAM 0))) ;; Connect to BUILD-OUTPUT and send it the raw ;; build output. @@ -406,7 +415,7 @@ (define* (build source (guile-version (if (> pull-version 0) "3.0" (effective-version))) - + built-in-builders #:allow-other-keys #:rest rest) "Return a derivation that unpacks SOURCE into STORE and compiles Scheme @@ -415,7 +424,9 @@ (define* (build source ;; SOURCE. (mlet %store-monad ((build (build-program source version guile-version #:channel-metadata channel-metadata - #:pull-version pull-version)) + #:pull-version pull-version + #:built-in-builders + built-in-builders)) (system (if system (return system) (current-system))) (home -> (getenv "HOME")) diff --git a/guix/channels.scm b/guix/channels.scm index 0d7bc541cc..34f63eb833 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -704,11 +704,15 @@ (define (with-trivial-build-handler mvalue) store)))) (define* (build-from-source instance - #:key core verbose? (dependencies '()) system) + #:key core verbose? (dependencies '()) system + built-in-builders) "Return a derivation to build Guix from INSTANCE, using the self-build script contained therein. When CORE is true, build package modules under SOURCE using CORE, an instance of Guix. By default, build for the current -system, or SYSTEM if specified." +system, or SYSTEM if specified. If BUILT-IN-BUILDERS is +provided, it should be a list of strings and this will be used instead of the +builtin builders provided by the build daemon for store connections used +during this process." (define name (symbol->string (channel-name (channel-instance-channel instance)))) @@ -750,20 +754,28 @@ (define* (build-from-source instance #:verbose? verbose? #:version commit #:system system #:channel-metadata (channel-instance->sexp instance) - #:pull-version %pull-version)))) + #:pull-version %pull-version + #:built-in-builders + built-in-builders)))) ;; Build a set of modules that extend Guix using the standard method. (standard-module-derivation name source core dependencies))) (define* (build-channel-instance instance system - #:optional core (dependencies '())) + #:optional core (dependencies '()) + #:key built-in-builders) "Return, as a monadic value, the derivation for INSTANCE, a channel instance, for SYSTEM. DEPENDENCIES is a list of extensions providing Guile -modules that INSTANCE depends on." +modules that INSTANCE depends on. If BUILT-IN-BUILDERS is +provided, it should be a list of strings and this will be used instead of the +builtin builders provided by the build daemon for store connections used +during this process." (build-from-source instance #:core core #:dependencies dependencies - #:system system)) + #:system system + #:built-in-builders + built-in-builders)) (define (resolve-dependencies instances) "Return a procedure that, given one of the elements of INSTANCES, returns @@ -793,9 +805,13 @@ (define (resolve-dependencies instances) (lambda (instance) (vhash-foldq* cons '() instance edges))) -(define* (channel-instance-derivations instances #:key system) +(define* (channel-instance-derivations instances #:key system + built-in-builders) "Return the list of derivations to build INSTANCES, in the same order as -INSTANCES. Build for the current system by default, or SYSTEM if specified." +INSTANCES. Build for the current system by default, or SYSTEM if specified. +If BUILT-IN-BUILDERS is provided, it should be a list of +strings and this will be used instead of the builtin builders provided by the +build daemon for store connections used during this process." (define core-instance ;; The 'guix' channel is treated specially: it's an implicit dependency of ;; all the other channels. @@ -809,11 +825,15 @@ (define* (channel-instance-derivations instances #:key system) (define (instance->derivation instance) (mlet %store-monad ((system (if system (return system) (current-system)))) (mcached (if (eq? instance core-instance) - (build-channel-instance instance system) + (build-channel-instance instance system + #:built-in-builders + built-in-builders) (mlet %store-monad ((core (instance->derivation core-instance)) (deps (mapm %store-monad instance->derivation (edges instance)))) - (build-channel-instance instance system core deps))) + (build-channel-instance instance system core deps + #:built-in-builders + built-in-builders))) instance system))) @@ -915,10 +935,13 @@ (define (channel-instance->sexp instance) intro)))))) '())))) -(define* (channel-instances->manifest instances #:key system) +(define* (channel-instances->manifest instances #:key system + built-in-builders) "Return a profile manifest with entries for all of INSTANCES, a list of channel instances. By default, build for the current system, or SYSTEM if -specified." +specified. If BUILT-IN-BUILDERS is provided, it should be a +list of strings and this will be used instead of the builtin builders provided +by the build daemon for store connections used during this process." (define (instance->entry instance drv) (let ((commit (channel-instance-commit instance)) (channel (channel-instance-channel instance))) @@ -934,8 +957,11 @@ (define* (channel-instances->manifest instances #:key system) (properties `((source ,(channel-instance->sexp instance))))))) - (mlet* %store-monad ((derivations (channel-instance-derivations instances - #:system system)) + (mlet* %store-monad ((derivations (channel-instance-derivations + instances + #:system system + #:built-in-builders + built-in-builders)) (entries -> (map instance->entry instances derivations))) (return (manifest entries)))) @@ -990,10 +1016,17 @@ (define %channel-profile-hooks ;; The default channel profile hooks. (cons package-cache-file %default-profile-hooks)) -(define (channel-instances->derivation instances) +(define* (channel-instances->derivation instances + #:key built-in-builders) "Return the derivation of the profile containing INSTANCES, a list of -channel instances." - (mlet %store-monad ((manifest (channel-instances->manifest instances))) +channel instances. If BUILT-IN-BUILDERS is provided, it +should be a list of strings and this will be used instead of the builtin +builders provided by the build daemon for store connections used during this +process." + (mlet %store-monad ((manifest (channel-instances->manifest + instances + #:built-in-builders + built-in-builders))) ;; Emit a profile in format version so that, if INSTANCES denotes an old ;; Guix, it can still read that profile, for instance for the purposes of ;; 'guix describe'. -- 2.45.2