From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id EFP+DEN+62Lm+QAAbAwnHQ (envelope-from ) for ; Thu, 04 Aug 2022 10:07:31 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id 48ntC0N+62JWBAAAG6o9tA (envelope-from ) for ; Thu, 04 Aug 2022 10:07:31 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1161627C3A for ; Thu, 4 Aug 2022 10:07:29 +0200 (CEST) Received: from localhost ([::1]:52794 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oJVtA-0006ym-G5 for larch@yhetil.org; Thu, 04 Aug 2022 04:07:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44090) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oJVjT-0008Jq-M9 for gwl-devel@gnu.org; Thu, 04 Aug 2022 03:57:29 -0400 Received: from mailrelay.tugraz.at ([129.27.2.202]:55656) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oJVjQ-00081M-SI for gwl-devel@gnu.org; Thu, 04 Aug 2022 03:57:27 -0400 Received: from lprikler-laptop.ist.intra (gw.ist.tugraz.at [129.27.202.101]) by mailrelay.tugraz.at (Postfix) with ESMTPSA id 4Lz1KQ6lZsz3wkV; Thu, 4 Aug 2022 09:57:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tugraz.at; s=mailrelay; t=1659599835; bh=0FC8Igm2qE7kZXFD7iSwylQiwto05uGvqIHJiq0f8tE=; h=From:Date:Subject:to:cc; b=Pu/fDXiVp95TEiED4Apfq1G57FFJgP0x5+bG9G7RBVKCGgWFjPhPOH/hLCNc1RSUd ZkUx8JJmAq7KYSwtkdImflKmRyGqRBYNHUNaWxLhXO6R2Tj2pvmQujP009HAY7wSUO s7BohKh56FpTa7pRrnuxmwo7y4ftIWHJuECV+dis= X-Mailbox-Line: From 744653384c21b6429e9ab6de1073e32c31942257 Mon Sep 17 00: 00:00 2001 From: Liliana Marie Prikler Date: Thu, 4 Aug 2022 09:43:17 +0200 Subject: [PATCH] processes: Store packages as manifest. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 to: gwl-devel@gnu.org cc: Olivier Dion Content-Transfer-Encoding: 8bit Message-ID: <581bd6f4ad0b5258152b2ae9804f6cd866876984.camel@ist.tugraz.at> X-TUG-Backscatter-control: waObeELIUl4ypBWmcn/8wQ X-Scanned-By: MIMEDefang 2.74 on 129.27.10.117 Received-SPF: none client-ip=129.27.2.202; envelope-from=liliana.prikler@ist.tugraz.at; helo=mailrelay.tugraz.at X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: gwl-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gwl-devel-bounces+larch=yhetil.org@gnu.org Sender: "gwl-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1659600450; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=0FC8Igm2qE7kZXFD7iSwylQiwto05uGvqIHJiq0f8tE=; b=kbgwSzW+tIxnIswROZmInsyRz+KPYmbv8tUHQ2uGTWUZVcF9KRokZA/+vEatNW5LtAAazu rnmFYmcCdxRCDsTg6Oogamps8bn9VFoSnxgIgHmByyTuu3PG4tJwPmwpTFrrcjM3ksQ7LE GhbVCDt4N1CjOIZ580JCuU9WH04A0DhBeaoTajcd2AatzdoCm4uGt269R3NuNPLnoM29T8 oKIFaRN/9rP7rSeK4+b6xxhPLYLEBbhLLR3eruW3IZ103wNR/3eGlfrE3BEIWugqBrdIkG Q+1WPLtG7ZZYLOiVcjyidU2xafh+cEJBgDHVA2e0uQbg0Mxdps4YPtG05/Zj1w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1659600450; a=rsa-sha256; cv=none; b=tYEiKl1z8G9dK8V6eINlgJW5TIJ/5qiF5M5UkmTgMFQKxkB/nFsvz5bvDfF1Nhy/qqQ2Nr QS9Bh+Nq19FQCzIPvLIVTs3ss27VKtMQEwSSK9G7ZS1mCG7VUswi0BlEnF/IYJnMR/up7s lemsS/xVYTq+9QHDSDn969BVWhS0Mhy0cdw0uPGcDt481vwdsrObIFA0eJBmj4O4ILi7pj ElpXMwgxpds+IK65+uX3mW7MzRXCXmO594aXgZDrQwvSrTcHImG20+sjGt5cL1CxRMuVwZ g4LiWoDHOlPwb5FLcSovrnCByXeCypllVmrQ6/nI7G08QRgZw7BSZUexF1MtbA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=tugraz.at header.s=mailrelay header.b="Pu/fDXiV"; dmarc=pass (policy=none) header.from=tugraz.at; spf=pass (aspmx1.migadu.com: domain of "gwl-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="gwl-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.91 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=tugraz.at header.s=mailrelay header.b="Pu/fDXiV"; dmarc=pass (policy=none) header.from=tugraz.at; spf=pass (aspmx1.migadu.com: domain of "gwl-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="gwl-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 1161627C3A X-Spam-Score: -4.91 X-Migadu-Scanner: scn0.migadu.com X-TUID: djvIDxPqtVcw This makes handling of package outputs nondestructive. It also fixes an unrelated issue in the ordering of process packages – the implicit bash-minimal is now ordered *last*. * gwl/packages.scm (lookup-package): Return multiple values. (package-output): Deleted variable. * gwl/workflows/utils.scm (activate-workflow-environment!): Adjust accordingly. * gwl/processes.scm ()[packages]: Use manifest as init-form. Use manifest? as validator. Return a manifest in transformer. * gwl/processes.scm (process->script): Adjust accordingly. * gwl/workflows/graph.scm (workflow-dot-prettify-node): Likewise. --- gwl/packages.scm | 6 +--- gwl/processes.scm | 64 ++++++++++++++++++++++------------------- gwl/workflows/graph.scm | 3 +- gwl/workflows/utils.scm | 2 +- 4 files changed, 39 insertions(+), 36 deletions(-) diff --git a/gwl/packages.scm b/gwl/packages.scm index 6a598ba..37107f6 100644 --- a/gwl/packages.scm +++ b/gwl/packages.scm @@ -46,7 +46,6 @@ lookup-package valid-package? package-name - package-output bash-minimal build-time-guix @@ -86,8 +85,7 @@ (_ (raise (condition (&gwl-package-error (package-spec specification)))))))) - (set! (package-output package) output) - package)) + (values package output))) (define (valid-package? val) (or (package? val) @@ -110,8 +108,6 @@ the version. By default, DELIMITER is \"@\"." ((? inferior-package? pkg) (inferior-package-full-name pkg)))) -(define package-output (make-object-property)) - (define bash-minimal (mlambda () (lookup-package "bash-minimal"))) diff --git a/gwl/processes.scm b/gwl/processes.scm index 2452d1f..07b376a 100644 --- a/gwl/processes.scm +++ b/gwl/processes.scm @@ -24,8 +24,10 @@ #:use-module ((guix profiles) #:select (profile + manifest manifest? manifest-search-paths - packages->manifest)) + packages->manifest + concatenate-manifests)) #:use-module ((guix search-paths) #:select (search-path-specification->sexp)) @@ -38,6 +40,7 @@ #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) #:use-module (srfi srfi-35) + #:use-module (srfi srfi-71) #:use-module (ice-9 threads) #:use-module (ice-9 rdelim) #:export (make-process @@ -184,27 +187,33 @@ (packages #:accessor process-packages #:init-keyword #:packages - #:init-value '() + #:init-form (manifest '()) #:implicit-list? #t - #:validator (lambda (value) - (every valid-package? value)) + #:validator manifest? #:transformer ;; TODO: the instance name is not be available at this point, so we ;; can't report the process name here. We should move the ;; transformers and validators to a point after initialization. - (lambda (instance value) - (map (match-lambda - ((and (? string?) spec) - (lookup-package spec)) - ((and (? valid-package?) pkg) - pkg) - (x - (raise - (condition - (&gwl-type-error - (expected-type (list "" "" "")) - (actual-value x)))))) - value))) + (match-lambda* + ((_ (? manifest? value)) value) + ((_ packages) + (packages->manifest + (map + (match-lambda + ((? string? spec) + (let ((pkg out (lookup-package spec))) + (list pkg out))) + ((? valid-package? pkg) + pkg) + (((? valid-package? pkg) (? string? out)) + (list pkg out)) + (x + (raise + (condition + (&gwl-type-error + (expected-type (list "" "" "")) + (actual-value x)))))) + packages))))) (inputs #:accessor process-raw-inputs #:init-keyword #:inputs @@ -686,12 +695,11 @@ tags if WITH-TAGS? is #FALSE or missing." "Return a lowerable object for the script that will execute the PROCESS." (let* ((name (process-full-name process)) - (packages (cons (bash-minimal) - (process-packages process))) - (manifest (packages->manifest (map - (lambda (pkg) - (list pkg (package-output pkg))) - packages))) + (manifest (concatenate-manifests + ;; Put process packages before bash-minimal, so that + ;; they're not shadowed. + (list (process-packages process) + (packages->manifest (list (bash-minimal)))))) (profile (profile (content manifest))) (search-paths (delete-duplicates (map search-path-specification->sexp @@ -700,12 +708,10 @@ PROCESS." (exp (with-imported-modules (source-module-closure (script-modules)) #~(begin - #$@(if (null? packages) '() - `((use-modules (guix search-paths)) - (set-search-paths (map sexp->search-path-specification - ',search-paths) - (cons ,profile - ',packages)))) + (use-modules (guix search-paths)) + (set-search-paths (map sexp->search-path-specification + '#$search-paths) + (list #$profile)) #$(if out `(setenv "out" ,out) "") (setenv "_GWL_PROFILE" #$profile) (use-modules (ice-9 match)) diff --git a/gwl/workflows/graph.scm b/gwl/workflows/graph.scm index ea3fec9..7ea2fca 100644 --- a/gwl/workflows/graph.scm +++ b/gwl/workflows/graph.scm @@ -17,6 +17,7 @@ (define-module (gwl workflows graph) #:use-module (ice-9 format) #:use-module (ice-9 match) + #:use-module (guix profiles) #:use-module (gwl packages) #:use-module (gwl processes) #:use-module (gwl workflows) @@ -46,7 +47,7 @@ label=<~a
\ (match (process-packages process) (() "") (inputs (format #f "
Uses: ~{~a~^, ~}." - (map package-name inputs))))))) + (map manifest-entry-name inputs))))))) (define (workflow-restriction->dot process . restrictions) "Write the dependency relationships of a restriction in dot format." diff --git a/gwl/workflows/utils.scm b/gwl/workflows/utils.scm index 666d5f0..22e6ced 100644 --- a/gwl/workflows/utils.scm +++ b/gwl/workflows/utils.scm @@ -180,7 +180,7 @@ modify the load path of the current process." ((package-names (required-packages file-name)) (_assert (not (null? package-names))) (manifest (packages->manifest - (map lookup-package package-names))) + (map (compose list lookup-package) package-names))) (profile (profile (content manifest))) (profile-directory (parameterize ((%guile-for-build (default-guile-derivation))) -- 2.37.1