From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qOQbJU+80GE6bQEAgWs5BA (envelope-from ) for ; Sat, 01 Jan 2022 21:40:47 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id EElSIk+80GGXDgEA9RJhRA (envelope-from ) for ; Sat, 01 Jan 2022 21:40:47 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BEC5222912 for ; Sat, 1 Jan 2022 21:40:46 +0100 (CET) Received: from localhost ([::1]:57202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n3lBF-0007b0-OV for larch@yhetil.org; Sat, 01 Jan 2022 15:40:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45624) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3lAa-0007ZL-CR for guix-patches@gnu.org; Sat, 01 Jan 2022 15:40:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:47927) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n3lAa-00043P-3C for guix-patches@gnu.org; Sat, 01 Jan 2022 15:40:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1n3lAa-0003Cg-1p for guix-patches@gnu.org; Sat, 01 Jan 2022 15:40:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#50072] [PATCH v2 4/4] upstream: Support updating 'git-fetch' origins. Resent-From: Maxime Devos Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 01 Jan 2022 20:40:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50072 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50072@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Sarah Morgensen , Maxime Devos Received: via spool by 50072-submit@debbugs.gnu.org id=B50072.164106959812271 (code B ref 50072); Sat, 01 Jan 2022 20:40:04 +0000 Received: (at 50072) by debbugs.gnu.org; 1 Jan 2022 20:39:58 +0000 Received: from localhost ([127.0.0.1]:59469 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n3lAT-0003Bl-Je for submit@debbugs.gnu.org; Sat, 01 Jan 2022 15:39:58 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:42664) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n3lAL-0003Ai-36 for 50072@debbugs.gnu.org; Sat, 01 Jan 2022 15:39:54 -0500 Received: from localhost.localdomain ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by michel.telenet-ops.be with bizsmtp id dYfm2600C4UW6Th06YfnkP; Sat, 01 Jan 2022 21:39:47 +0100 From: Maxime Devos Date: Sat, 1 Jan 2022 20:39:40 +0000 Message-Id: <20220101203940.149517-5-maximedevos@telenet.be> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220101203940.149517-1-maximedevos@telenet.be> References: <20220101203940.149517-1-maximedevos@telenet.be> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r21; t=1641069587; bh=piqveXyl/U9vmoG/ROFsFuzs3/uReMmo25hpTwa0WIE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=iJ6TdfGoj/8b9ldPTWKRa8IrdozeIqYAF4wlnuGQ2YrD2ZVaknDmdbSMTGCB12VjE J7dYlxfbqumiz4rdmnPHs/DdhLOA0/hzHy+AHuABCKwyvSh8JA681UVTCBj9husu/v j6lfZ+VyGpE3Mj25hNbJlOyw9GNzRXpU5BMS9wF1CDuKM8lt9re0sLgqWEq2IfuHeB qOkgH7O5UclyKCVnhDWtE6Ts8cb7feboMajDPWy+lxdfqyVBRPg+jfJZH8z7a5HrSY BPlKrPQRtbH1L7RZiU5LN2L5QRgS+yTxE66ApqgQGOwP7YzWjvN7TneWAal4NpsrL1 pSZSf7kM0DNVg== X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1641069647; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=piqveXyl/U9vmoG/ROFsFuzs3/uReMmo25hpTwa0WIE=; b=u5Y1LbtPXDPMwUmujO1CMrSBCv5zZVQ2w+t+eae1NdONicO4nr4PrhWh3r3QAgqiLQDop0 wdPgWBoKIdA4igej7jNcgXyfTwzKo6DT89Kus8k0Pt0dEhe4x5uGfDjLMfaN58gJlzxsLK CbLSyHCVF6tze95mWfIijtXTvNWz+pzsGbDe3AQKLK8bOR3sMLRUU/o98ZAmUR7Ic0G72Q UG1oQspEAeaUU6e+L4tbpg9AMkfTzVlPN+OaD1VHc917sxomu8I89uMK4YN2Bcn4QiAoh5 uMX4AcgGqrcmdzseSTQt+A+rvwD+JfBbFtqR23yeKZOSUcrgWCxb1RkLKG3yig== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1641069647; a=rsa-sha256; cv=none; b=DEQUyuzyiEmsfBnqdWOot81kWTDBZXburTb3d7k1P9UZXXALS8f8qP7T9lR21MXU6+CpXI hQk+6haGvEYR0t4yYOeyZrUjcy8M5gsl3fOOQ7/39b2/gxCElzYaLiT9VihiDK9Mqi0d/K 2QVs1WSh9aI2PnAG33gyadafu/3dIUfzKA7YPodIhAQVFxsdpXZhIReYvDWX2SBNCRP1eS 5qCmvYSYPJliVfqfP069Wd/aaQ3zrEnSnIPWQYRhokhJoGRmCH/IC3JFaOiLx5t+ijZnYQ 0Ww5DyGtguOQSwXG0LCTPiRFb4L64zIFROA180azkI/57MABUuf2xnzOlmdnqQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=telenet.be header.s=r21 header.b=iJ6TdfGo; dmarc=fail reason="SPF not aligned (relaxed)" header.from=telenet.be (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.08 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=telenet.be header.s=r21 header.b=iJ6TdfGo; dmarc=fail reason="SPF not aligned (relaxed)" header.from=telenet.be (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: BEC5222912 X-Spam-Score: -1.08 X-Migadu-Scanner: scn1.migadu.com X-TUID: sTNiL86n5z5F From: Sarah Morgensen Updaters need to be modified to return 'git-reference' objects. This patch modifies the 'generic-git' and 'minetest' updater, but others might need to be modified as well. * guix/upstream.scm (package-update/git-fetch): New procedure. ()[urls]: Document it can be a 'git-reference'. (%method-updates): Add 'git-fetch' mapping. (update-package-source): Support 'git-reference' sources. (upstream-source-compiler): Bail out gracefully if the source is a git origin. * guix/import/git.scm (latest-git-tag-version): Always return two values and document that the tag is returned as well. (latest-git-release)[urls]: Use the 'git-reference' instead of the repository URL. * guix/import/minetest.scm (latest-minetest-release)[urls]: Don't wrap the 'git-reference' in a list. * tests/minetest.scm (upstream-source->sexp): Adjust to new convention. Co-authored-by: Maxime Devos --- guix/import/git.scm | 22 +++++++++------ guix/import/minetest.scm | 6 ++-- guix/upstream.scm | 60 ++++++++++++++++++++++++++++++++++++---- tests/minetest.scm | 7 ++--- 4 files changed, 74 insertions(+), 21 deletions(-) diff --git a/guix/import/git.scm b/guix/import/git.scm index 1eb219f3fe..4cf404677c 100644 --- a/guix/import/git.scm +++ b/guix/import/git.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2021 Xinglu Chen ;;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2022 Maxime Devos ;;; ;;; This file is part of GNU Guix. ;;; @@ -34,6 +35,7 @@ #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) #:use-module (srfi srfi-35) + #:use-module (srfi srfi-71) #:export (%generic-git-updater ;; For tests. @@ -172,21 +174,21 @@ repository at URL." (values version tag))))))) (define (latest-git-tag-version package) - "Given a PACKAGE, return the latest version of it, or #f if the latest version -could not be determined." + "Given a PACKAGE, return the latest version of it and the corresponding git +tag, or #false and #false if the latest version could not be determined." (guard (c ((or (git-no-tags-error? c) (git-no-valid-tags-error? c)) (warning (or (package-field-location package 'source) (package-location package)) (G_ "~a for ~a~%") (condition-message c) (package-name package)) - #f) + (values #f #f)) ((eq? (exception-kind c) 'git-error) (warning (or (package-field-location package 'source) (package-location package)) (G_ "failed to fetch Git repository for ~a~%") (package-name package)) - #f)) + (values #f #f))) (let* ((source (package-source package)) (url (git-reference-url (origin-uri source))) (property (cute assq-ref (package-properties package) <>))) @@ -208,14 +210,16 @@ could not be determined." "Return an for the latest release of PACKAGE." (let* ((name (package-name package)) (old-version (package-version package)) - (url (git-reference-url (origin-uri (package-source package)))) - (new-version (latest-git-tag-version package))) - - (and new-version + (old-reference (origin-uri (package-source package))) + (new-version new-version-tag (latest-git-tag-version package))) + (and new-version new-version-tag (upstream-source (package name) (version new-version) - (urls (list url)))))) + (urls (git-reference + (url (git-reference-url old-reference)) + (commit new-version-tag) + (recursive? (git-reference-recursive? old-reference)))))))) (define %generic-git-updater (upstream-updater diff --git a/guix/import/minetest.scm b/guix/import/minetest.scm index 44671d8480..9df13e45ae 100644 --- a/guix/import/minetest.scm +++ b/guix/import/minetest.scm @@ -503,9 +503,9 @@ or #false if the latest release couldn't be determined." (upstream-source (package (package:package-name pkg)) (version (release-version release)) - (urls (list (download:git-reference - (url (package-repository contentdb-package)) - (commit (release-commit release)))))))) + (urls (download:git-reference + (url (package-repository contentdb-package)) + (commit (release-commit release))))))) (define %minetest-updater (upstream-updater diff --git a/guix/upstream.scm b/guix/upstream.scm index 632e9ebc4f..0df2e78d30 100644 --- a/guix/upstream.scm +++ b/guix/upstream.scm @@ -2,6 +2,8 @@ ;;; Copyright © 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021 Ludovic Courtès ;;; Copyright © 2015 Alex Kost ;;; Copyright © 2019 Ricardo Wurmus +;;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2021, 2022 Maxime Devos ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,12 +26,14 @@ #:use-module (guix discovery) #:use-module ((guix download) #:select (download-to-store url-fetch)) + #:use-module (guix git-download) #:use-module (guix gnupg) #:use-module (guix packages) #:use-module (guix diagnostics) #:use-module (guix ui) #:use-module (guix base32) #:use-module (guix gexp) + #:use-module (guix git) #:use-module (guix store) #:use-module ((guix derivations) #:select (built-derivations derivation->output-path)) #:autoload (gcrypt hash) (port-sha256) @@ -93,7 +97,7 @@ upstream-source? (package upstream-source-package) ;string (version upstream-source-version) ;string - (urls upstream-source-urls) ;list of strings + (urls upstream-source-urls) ;list of strings|git-reference (signature-urls upstream-source-signature-urls ;#f | list of strings (default #f)) (input-changes upstream-source-input-changes @@ -361,8 +365,12 @@ values: 'interactive' (default), 'always', and 'never'." system target) "Download SOURCE from its first URL and lower it as a fixed-output derivation that would fetch it." - (mlet* %store-monad ((url -> (first (upstream-source-urls source))) - (signature + (define url + (match (upstream-source-urls source) + ((first . _) first) + (_ (raise (formatted-message + (G_ "git origins are unsupported by --with-latest")))))) + (mlet* %store-monad ((signature -> (and=> (upstream-source-signature-urls source) first)) (tarball ((store-lift download-tarball) url signature))) @@ -430,9 +438,35 @@ SOURCE, an ." #:key-download key-download))) (values version tarball source)))))) +(define (guess-version-transform commit from-version) + "Return a one-argument proc that transforms FROM-VERSION to COMMIT, or #f +if no such transformation could be determined." + ;; Just handle prefixes for now, since that's the most common. + (if (string-suffix? from-version commit) + (let* ((version-length (string-length from-version)) + (commit-prefix (string-drop-right commit version-length))) + (lambda (version) + (string-append commit-prefix version))) + #f)) + +(define* (package-update/git-fetch store package source #:key key-download) + "Return the version, checkout, and SOURCE, to update PACKAGE to +SOURCE, an ." + ;; TODO: it would be nice to authenticate commits, e.g. with + ;; "guix git authenticate" or a list of permitted signing keys. + (define ref (upstream-source-urls source)) ; a + (values (upstream-source-version source) + (latest-repository-commit + store + (git-reference-url ref) + #:ref `(tag-or-commit . ,(git-reference-commit ref)) + #:recursive? (git-reference-recursive? ref)) + source)) + (define %method-updates ;; Mapping of origin methods to source update procedures. - `((,url-fetch . ,package-update/url-fetch))) + `((,url-fetch . ,package-update/url-fetch) + (,git-fetch . ,package-update/git-fetch))) (define* (package-update store package #:optional (updaters (force %updaters)) @@ -492,9 +526,22 @@ new version string if an update was made, and #f otherwise." (origin-hash (package-source package)))) (old-url (match (origin-uri (package-source package)) ((? string? url) url) + ((? git-reference? ref) + (git-reference-url ref)) (_ #f))) (new-url (match (upstream-source-urls source) - ((first _ ...) first))) + ((first _ ...) first) + ((? git-reference? ref) + (git-reference-url ref)) + (_ #f))) + (old-commit (match (origin-uri (package-source package)) + ((? git-reference? ref) + (git-reference-commit ref)) + (_ #f))) + (new-commit (match (upstream-source-urls source) + ((? git-reference? ref) + (git-reference-commit ref)) + (_ #f))) (file (and=> (location-file loc) (cut search-path %load-path <>)))) (if file @@ -508,6 +555,9 @@ new version string if an update was made, and #f otherwise." 'filename file)) (replacements `((,old-version . ,version) (,old-hash . ,hash) + ,@(if (and old-commit new-commit) + `((,old-commit . ,new-commit)) + '()) ,@(if (and old-url new-url) `((,(dirname old-url) . ,(dirname new-url))) diff --git a/tests/minetest.scm b/tests/minetest.scm index 77b9aa928f..cbb9e83889 100644 --- a/tests/minetest.scm +++ b/tests/minetest.scm @@ -387,10 +387,9 @@ during a dynamic extent where that package is available on ContentDB." ;; Update detection (define (upstream-source->sexp upstream-source) - (define urls (upstream-source-urls upstream-source)) - (unless (= 1 (length urls)) - (error "only a single URL is expected")) - (define url (first urls)) + (define url (upstream-source-urls upstream-source)) + (unless (git-reference? url) + (error "a is expected")) `(,(upstream-source-package upstream-source) ,(upstream-source-version upstream-source) ,(git-reference-url url) -- 2.30.2