From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id yKnpA54j/2TwXAAAauVa8A:P1 (envelope-from ) for ; Mon, 11 Sep 2023 16:26:38 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id yKnpA54j/2TwXAAAauVa8A (envelope-from ) for ; Mon, 11 Sep 2023 16:26:38 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CAEF6481AC for ; Mon, 11 Sep 2023 16:26:37 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=oHz5EoLW; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1694442397; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=ShZItNlvYjaRrNo4JPlFc63xoz7be61ooWEzJHPU0v4=; b=c0VOH/w1pOGrK0JsEucROrAk+dUPmMdidKZ22UgIj9S4JiXGhuvTPR5ZunEsCMOe0nBUl6 nZYq3V+8Vh3SboWYRsOZjiRVjmTonNHfwboFNTB9eBvF8PymUqYGavYqHEZLxcA2ZcSE7Q 55JDcibMoj0SJQQEwqKtOjhzRg+vlZ4gJwke0q94a02N5yshEaZJlvau3MfAeunZKYfdt2 BiB0O8dFPOIrWFEI4nfROv08zocoThQyQcJ63LIYZ9pJRB/2e1J3AAAD70SljWNxYPsR1J pinKxWOr5fx5/H1USYEK03LPsWe2nrYodIaEQB18iAbcsI2T4avg3GCY0rZEPQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=oHz5EoLW; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Seal: i=1; s=key1; d=yhetil.org; t=1694442397; a=rsa-sha256; cv=none; b=j3+1vTW40CiWI1q/7MjFCJm8ugNJHfXwak81uYx8lmCaXssUDo8tes75JbhhFIzE4Z5U/D y/q4pRqdumTRh9CtMX3Sa/it/i/TQor1971SRFTDZgSjB5KyXUSJkTKcfinwd3cmw+BXSs j0scl0WTyY0LM3/Fg5CvG7hp39JRzhOVeobnxHZ9C2X9tVxsZMFcfgLfFcZRdKsbo376mF 10eT8cAV+iXoIjX6rc93+b/dkAcebPeilZ7Uw/Kcizx4LqYVtqy7Eme3vRMhUHnvKDlSre Gt8DVT0lHBj/nWEbetPEvVwAhBKkUssiE5tUqZn+yUR4B8qVjF5bBnmCrAV8Rw== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhrc-0003sJ-KD; Mon, 11 Sep 2023 10:26:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrX-0003qZ-GG for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhrW-00035s-0k; Mon, 11 Sep 2023 10:26:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhrX-0006hj-VX; Mon, 11 Sep 2023 10:26:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 7/8] git-download: Use =?UTF-8?Q?=E2=80=9Cbuiltin:git-download=E2=80=9D?= when available. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:26:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444236025717 (code B ref 65866); Mon, 11 Sep 2023 14:26:03 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:00 +0000 Received: from localhost ([127.0.0.1]:54280 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrT-0006ge-QM for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:26:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55022) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrO-0006fJ-G1 for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrF-000344-3n; Mon, 11 Sep 2023 10:25:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=ShZItNlvYjaRrNo4JPlFc63xoz7be61ooWEzJHPU0v4=; b=oHz5EoLWBmpAliEnOeLs QaqCICjXjhsBGU2R5y13a00M9MZTquQrf2G/MwdumOXgCo6WVwW6H+LNQoeStmf8/3jxuZM1EVI+B ZdJvDOI8CYZ1LzKNLec4T3mcScJkNSrl22/p9/mx7BiQR3SyiTx8hw/WAI8TcPO6AMpYy08ykLz0T vgXZ6oWBJfcjOrlj6M+eDr3chUCZDHUqY/Ul0sdGh9QbSNJOltn3GyKWQbVcSgODlB65odEHlClzu dumDECqVnrin3wRDY1pYuSB+n1Eq/dR0d/uhhi/2gt7cW7UXNIpsgkAY+FfR5dag4wQA6r3/mhod3 VKh1/qCMbTKJYQ==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:25 +0200 Message-ID: <2cd5b127be6d64e640e569f262cef3bbb89f58a6.1694441831.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx1.migadu.com X-Migadu-Spam-Score: -1.42 X-Spam-Score: -1.42 X-Migadu-Queue-Id: CAEF6481AC X-TUID: gyRDW41DGtbt Fixes . Longer-term this will remove Git from the derivation graph when its sole use is to perform a checkout for a fixed-output derivation, thereby breaking dependency cycles that can arise in these situations. * guix/git-download.scm (git-fetch): Rename to… (git-fetch/in-band): … this. Deal with GIT or GUILE being #f. (git-fetch/built-in, built-in-builders*, git-fetch): New procedures. * tests/builders.scm ("git-fetch, file URI"): New test. --- guix/git-download.scm | 68 +++++++++++++++++++++++++++++++++++++------ tests/builders.scm | 29 +++++++++++++++++- 2 files changed, 87 insertions(+), 10 deletions(-) diff --git a/guix/git-download.scm b/guix/git-download.scm index f1f19397c6..505dff0a89 100644 --- a/guix/git-download.scm +++ b/guix/git-download.scm @@ -27,6 +27,7 @@ (define-module (guix git-download) #:use-module (guix records) #:use-module (guix packages) #:use-module (guix modules) + #:use-module ((guix derivations) #:select (raw-derivation)) #:autoload (guix build-system gnu) (standard-packages) #:autoload (guix download) (%download-fallback-test) #:autoload (git bindings) (libgit2-init!) @@ -78,15 +79,19 @@ (define (git-package) (let ((distro (resolve-interface '(gnu packages version-control)))) (module-ref distro 'git-minimal))) -(define* (git-fetch ref hash-algo hash - #:optional name - #:key (system (%current-system)) (guile (default-guile)) - (git (git-package))) - "Return a fixed-output derivation that fetches REF, a -object. The output is expected to have recursive hash HASH of type -HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." +(define* (git-fetch/in-band ref hash-algo hash + #:optional name + #:key (system (%current-system)) + (guile (default-guile)) + (git (git-package))) + "Return a fixed-output derivation that performs a Git checkout of REF, using +GIT and GUILE (thus, said derivation depends on GIT and GUILE). + +This method is deprecated in favor of the \"builtin:git-download\" builder. +It will be removed when versions of guix-daemon implementing +\"builtin:git-download\" will be sufficiently widespread." (define inputs - `(("git" ,git) + `(("git" ,(or git (git-package))) ;; When doing 'git clone --recursive', we need sed, grep, etc. to be ;; available so that 'git submodule' works. @@ -154,7 +159,8 @@ (define* (git-fetch ref hash-algo hash #:recursive? recursive? #:git-command "git"))))) - (mlet %store-monad ((guile (package->derivation guile system))) + (mlet %store-monad ((guile (package->derivation (or guile (default-guile)) + system))) (gexp->derivation (or name "git-checkout") build ;; Use environment variables and a fixed script name so @@ -181,6 +187,50 @@ (define* (git-fetch ref hash-algo hash #:recursive? #t #:guile-for-build guile))) +(define* (git-fetch/built-in ref hash-algo hash + #:optional name + #:key (system (%current-system))) + "Return a fixed-output derivation without any dependency that performs a Git +checkout of REF, using the \"builtin:git-download\" derivation builder." + (raw-derivation (or name "git-checkout") "builtin:git-download" '() + #:system system + #:hash-algo hash-algo + #:hash hash + #:recursive? #t + #:env-vars + `(("url" . ,(object->string + (match (%download-fallback-test) + ('content-addressed-mirrors + "https://example.org/does-not-exist") + (_ + (git-reference-url ref))))) + ("commit" . ,(git-reference-commit ref)) + ("recursive?" . ,(object->string + (git-reference-recursive? ref)))) + #:leaked-env-vars '("http_proxy" "https_proxy" + "LC_ALL" "LC_MESSAGES" "LANG" + "COLUMNS") + #:local-build? #t)) + +(define built-in-builders* + (store-lift built-in-builders)) + +(define* (git-fetch ref hash-algo hash + #:optional name + #:key (system (%current-system)) + guile git) + "Return a fixed-output derivation that fetches REF, a +object. The output is expected to have recursive hash HASH of type +HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." + (mlet %store-monad ((builtins (built-in-builders*))) + (if (member "git-download" builtins) + (git-fetch/built-in ref hash-algo hash name + #:system system) + (git-fetch/in-band ref hash-algo hash name + #:system system + #:guile guile + #:git git)))) + (define (git-version version revision commit) "Return the version string for packages using git-download." ;; git-version is almost exclusively executed while modules are being loaded. diff --git a/tests/builders.scm b/tests/builders.scm index 0b5577c7a3..619caa5f31 100644 --- a/tests/builders.scm +++ b/tests/builders.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2018, 2019, 2021 Ludovic Courtès +;;; Copyright © 2012-2015, 2018-2019, 2021, 2023 Ludovic Courtès ;;; Copyright © 2021 Lars-Dominik Braun ;;; ;;; This file is part of GNU Guix. @@ -20,6 +20,7 @@ (define-module (tests builders) #:use-module (guix download) + #:use-module (guix git-download) #:use-module (guix build-system) #:use-module (guix build-system gnu) #:use-module (guix build gnu-build-system) @@ -31,9 +32,12 @@ (define-module (tests builders) #:use-module (guix base32) #:use-module (guix derivations) #:use-module (gcrypt hash) + #:use-module ((guix hash) #:select (file-hash*)) #:use-module (guix tests) + #:use-module (guix tests git) #:use-module (guix packages) #:use-module (gnu packages bootstrap) + #:use-module ((ice-9 ftw) #:select (scandir)) #:use-module (ice-9 match) #:use-module (ice-9 textual-ports) #:use-module (srfi srfi-1) @@ -84,6 +88,29 @@ (define url-fetch* (and (file-exists? out) (valid-path? %store out)))) +(test-equal "git-fetch, file URI" + '("." ".." "a.txt" "b.scm") + (let ((nonce (random-text))) + (with-temporary-git-repository directory + `((add "a.txt" ,nonce) + (add "b.scm" "#t") + (commit "Commit.") + (tag "v1.0.0" "The tag.")) + (run-with-store %store + (mlet* %store-monad ((hash + -> (file-hash* directory + #:algorithm (hash-algorithm sha256) + #:recursive? #t)) + (drv (git-fetch + (git-reference + (url (string-append "file://" directory)) + (commit "v1.0.0")) + 'sha256 hash + "git-fetch-test"))) + (mbegin %store-monad + (built-derivations (list drv)) + (return (scandir (derivation->output-path drv))))))))) + (test-assert "gnu-build-system" (build-system? gnu-build-system)) -- 2.41.0