From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 1JJzF1YIW1+LRgAA0tVLHw (envelope-from ) for ; Fri, 11 Sep 2020 05:17:10 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id +CK5ElYIW1+MMAAAB5/wlQ (envelope-from ) for ; Fri, 11 Sep 2020 05:17:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C092C94013C for ; Fri, 11 Sep 2020 05:17:09 +0000 (UTC) Received: from localhost ([::1]:50464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGbQq-0005qc-M6 for larch@yhetil.org; Fri, 11 Sep 2020 01:17:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kGbQk-0005qH-Lb for guix-patches@gnu.org; Fri, 11 Sep 2020 01:17:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:58721) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kGbQk-0001nS-Cs for guix-patches@gnu.org; Fri, 11 Sep 2020 01:17:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kGbQk-0005nd-6Y for guix-patches@gnu.org; Fri, 11 Sep 2020 01:17:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43193] [PATCH] guix: Add --with-dependency-source option Resent-From: Jesse Gibbons Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 11 Sep 2020 05:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43193 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43193@debbugs.gnu.org Received: via spool by 43193-submit@debbugs.gnu.org id=B43193.159980141822281 (code B ref 43193); Fri, 11 Sep 2020 05:17:02 +0000 Received: (at 43193) by debbugs.gnu.org; 11 Sep 2020 05:16:58 +0000 Received: from localhost ([127.0.0.1]:42034 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGbQf-0005nI-NV for submit@debbugs.gnu.org; Fri, 11 Sep 2020 01:16:58 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:34171) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGbQe-0005n5-57 for 43193@debbugs.gnu.org; Fri, 11 Sep 2020 01:16:56 -0400 Received: by mail-pf1-f194.google.com with SMTP id v196so6504898pfc.1 for <43193@debbugs.gnu.org>; Thu, 10 Sep 2020 22:16:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=xLILgtYB5U7mNFbrseGmfu+0IeQ96pscWWvGULjwTmY=; b=QC7FXqzPEnw4KtRzxkypKv3Cpg9zCpsEbvGIUbSUsmYLoPJtNLWUJwuIyYxGmF+Xa9 PQX4CuqDy5/CCfhtMWYogLDO6S01cgShmefXTiCjYM5c1dlpqkMRoa7raZLdGjbvH0JT vvtyqmnauoMeCdOhHjkZoaBOBxJ8xXbpXVt0KwvYnaOuIfXVnRvboIsrpd5z37pbVqO/ CmNqaKR0OGWuYQ3XCmpfT5a/XHkKT/n2B6EC6gpM0NdTvgmRH+O9xPd+unyfAjWR/Ona HainYhZpfIXrYRv3aRHlVMMlEs4P4BRhtlFZ3/HVHpuBWOOPkRLK3Y34pKULx5eKzka6 Mwbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=xLILgtYB5U7mNFbrseGmfu+0IeQ96pscWWvGULjwTmY=; b=anekytNYn8w1FJZ3buQVJx9Ej5RQzjTnlDA5CWsPU72EpQ6uMm04oKmmUnoniGxFgx hQo/gtebT3zM/xtf44RnrOsLxQMHrdqs8jKWqa02dMTrqFf3sjkzA7r6WQ+87TghysZY AmU+9cTL7YC59Fzie1yneEb4JfJj2gwZmNr3Ov+2wobc51bUH/XN/paKgqswwD7TuKkI /KyZddz+W2/qQRkgRcChCZ+76LzMSLPYkoMFFvaeAer6nkeqfXehHzTacOiVBrQ5jtfD /TOwsmEy42gFM1giginplw9iQj6splTQ5kVpr8MtVybvL+VbcWHGu+nu9t5uqv0Kt8bN 7d0A== X-Gm-Message-State: AOAM5303hobN0WGy0Du9sLAc9ldR4sl+NT8lJEW2bZ0uuulxII2AuvwH ijCO3kgb+gB9SnEluTHR9nqKBCmdfd60Vg== X-Google-Smtp-Source: ABdhPJyWhXEVx17lXTvEKre8I6i84OfkIP7+cEfBixXIkYYxkGFgNHeEWpwamUu6DFplZ3AphWVNwQ== X-Received: by 2002:a63:fc18:: with SMTP id j24mr445875pgi.452.1599801409763; Thu, 10 Sep 2020 22:16:49 -0700 (PDT) Received: from [192.168.1.25] ([38.141.58.134]) by smtp.gmail.com with ESMTPSA id ca6sm642804pjb.53.2020.09.10.22.16.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 10 Sep 2020 22:16:48 -0700 (PDT) References: <9c3a00ba-19bf-a8e9-8803-d150e91f1b6e@gmail.com> <87y2lige5l.fsf@gnu.org> From: Jesse Gibbons Message-ID: <7f52c72f-c280-b585-d1ad-ca012f804910@gmail.com> Date: Thu, 10 Sep 2020 23:16:28 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Icedove/68.12.0 MIME-Version: 1.0 In-Reply-To: <87y2lige5l.fsf@gnu.org> Content-Type: multipart/mixed; boundary="------------0008A2252BA00AF6095D44A9" Content-Language: en-US X-Spam-Score: -3.3 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -4.3 (----) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=QC7FXqzP; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: efbBP+61aSCV This is a multi-part message in MIME format. --------------0008A2252BA00AF6095D44A9 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 9/10/20 3:43 AM, Ludovic Courtès wrote: > Hi Jesse, > > Jesse Gibbons skribis: > >>     The attached patch adds support for the --with-dependency-source >> common build option. It can be used to specify the sources of >> dependencies of specified packages. With this step, we can close >> #42155. This is also a step in closing #43061. > Excellent! > >> Note that I suggested making a new >> --with-dependency-source=package=source build option in response to >> #42155 and nobody responded with an objection. So I am sending this >> patch with the assumption that there are no objections to this new >> build option, and that if a member of the community wants to >> completely replace the behavior of --with-source with the behavior of >> the new option, that person can do the work required to not break >> --with-source=source. > OK. Like I wrote at , I wouldn’t > mind simply calling this new option ‘--with-source’. What we’d lose by > doing so is the warning you get if you do > ‘--with-source=does-not-exist=whatever’, saying the option had no > effect, but that’s about it. The new ‘--with-source’ behavior > (recursive) would be consistent with the other options, which, to me, is > more important. I agree that '--with-source' is a better name for the option, but I don't want to lose that particular functionality. I worked a little more to alter "--with-source" while still preserving the simple '--with-source=source' option, because once it's committed to master, it will be difficult to turn back and get the ideal implementation. The result is a bit dirty and should be refactored and cleaned, but at least it works. Attached is the updated patch. > > WDYT? > >> >From 91a89277067fd454ad77edb3a09ed06382f3694c Mon Sep 17 00:00:00 2001 >> From: Jesse Gibbons >> Date: Thu, 3 Sep 2020 17:45:08 -0600 >> Subject: [PATCH v1 1/1] guix: Add --with-dependency-source option >> >> * guix/scripts/build.scm: (transform-package-inputs/source): new >> function >> (evaluate-source-replacement-specs): new function >> (%transformations): add with-dependency-source option >> (%transformation-options): add with-dependency-source-option >> (show-transformation-options-help): document --with-dependency-source > [...] > >> +(define (evaluate-source-replacement-specs specs proc) >> + "Parse SPECS, a list of strings like \"guile=/path/to/source\", and return a >> +list of package pairs, where (PROC PACKAGE URL) returns the replacement package. >> +Raise an error if an element of SPECS uses invalid syntax, or if a package it >> +refers to could not be found." >> + (map (lambda (spec) >> + (match (string-tokenize spec %not-equal) >> + ((spec url) >> + (define (replace old) >> + (proc old url)) >> + (cons spec replace)) >> + (x >> + (leave (G_ "invalid replacement specification: ~s~%") spec)))) > ^ > Add “source” here. It’s always a good idea to not have the exact same > error message in several places. :-) Fixed. > Could you: > > 1. adjust doc/guix.texi accordingly? That is, if we rename this new > option to ‘--with-source’, simply add a note stating that it’s > recursive. I included this in the attached patch. > 2. add a test to tests/guix-build.sh? There are already --with-source > tests in other files. You can mimic them, essentially to make sure > the option has an effect. > 3. optionally add an entry as a separate commit to > etc/news.scm. I can do that for you if you want. > Do you still think the tests should be updated and this change should be announced in the news file? I'm willing to do these. > Thanks! > > Ludo’. -Jesse --------------0008A2252BA00AF6095D44A9 Content-Type: text/x-patch; charset=UTF-8; name="0001-guix-Make-with-source-option-recursive.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-guix-Make-with-source-option-recursive.patch" >From 2786da1e7011c59f08fc150dfa284f35bc0ed093 Mon Sep 17 00:00:00 2001 From: Jesse Gibbons Date: Thu, 3 Sep 2020 17:45:08 -0600 Subject: [PATCH 1/1] guix: Make --with-source option recursive * guix/scripts/build.scm: (transform-package-inputs/source): new function (evaluate-source-replacement-specs): new function (%transformations): change with-source to use evaluate-source-replacement-specs *doc/guix.texi (Package Transformation Options): Document it. --- doc/guix.texi | 3 ++- guix/scripts/build.scm | 39 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 40 insertions(+), 2 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 1d6782e6fa..4036861c23 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -9129,7 +9129,8 @@ without having to type in the definitions of package variants @itemx --with-source=@var{package}=@var{source} @itemx --with-source=@var{package}@@@var{version}=@var{source} Use @var{source} as the source of @var{package}, and @var{version} as -its version number. +its version number. This replacement is applied recursively on all +dependencies only if PACKAGE is specified. @var{source} must be a file name or a URL, as for @command{guix download} (@pxref{Invoking guix download}). diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm index 6286a43c02..095457b174 100644 --- a/guix/scripts/build.scm +++ b/guix/scripts/build.scm @@ -280,6 +280,28 @@ current 'gnutls' package, after which version 3.5.4 is grafted onto them." (rewrite obj) obj)))) +(define (transform-package-inputs/source replacement-specs) + "Return a procedure that, when passed a package, replaces its direct +dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of +strings like \"guile=/path/to/source\" or +\"guile=https://www.example.com/guile-source.tar.gz\" meaning that, any +dependency on a package called \"guile\" must be replaced with a dependency on a +\"guile\" built with the source at the specified location." + (match (string-tokenize (car replacement-specs) %not-equal) + ((spec url) + (lambda (store obj) + (let* ((replacements (evaluate-source-replacement-specs replacement-specs + (lambda (old url) + (package-with-source store old url)))) + (rewrite (package-input-rewriting/spec replacements)) + (rewrite* (lambda (obj) + (rewrite obj)))) + (if (package? obj) + (rewrite* obj) + obj)))) + ((url) + (transform-package-source replacement-specs)))) + (define %not-equal (char-set-complement (char-set #\=))) @@ -314,6 +336,21 @@ syntax, or if a package it refers to could not be found." (leave (G_ "invalid replacement specification: ~s~%") spec)))) specs)) +(define (evaluate-source-replacement-specs specs proc) + "Parse SPECS, a list of strings like \"guile=/path/to/source\", and return a +list of package pairs, where (PROC PACKAGE URL) returns the replacement package. +Raise an error if an element of SPECS uses invalid syntax, or if a package it +refers to could not be found." + (map (lambda (spec) + (match (string-tokenize spec %not-equal) + ((spec url) + (define (replace old) + (proc old url)) + (cons spec replace)) + (x + (leave (G_ "invalid source replacement specification: ~s~%") spec)))) + specs)) + (define (transform-package-source-branch replacement-specs) "Return a procedure that, when passed a package, replaces its direct dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of @@ -398,7 +435,7 @@ a checkout of the Git repository at the given URL." ;; key used in the option alist, and the cdr is the transformation ;; procedure; it is called with two arguments: the store, and a list of ;; things to build. - `((with-source . ,transform-package-source) + `((with-source . ,transform-package-inputs/source) (with-input . ,transform-package-inputs) (with-graft . ,transform-package-inputs/graft) (with-branch . ,transform-package-source-branch) -- 2.28.0 --------------0008A2252BA00AF6095D44A9--