From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id MAC7LXJaMV9fTgAA0tVLHw (envelope-from ) for ; Mon, 10 Aug 2020 14:32:18 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id oJ2TKXJaMV/fXQAAB5/wlQ (envelope-from ) for ; Mon, 10 Aug 2020 14:32:18 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 23FA29402A2 for ; Mon, 10 Aug 2020 14:32:17 +0000 (UTC) Received: from localhost ([::1]:39680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k58qU-0005Re-J4 for larch@yhetil.org; Mon, 10 Aug 2020 10:32:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35748) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k58qI-0005QB-UP for guix-patches@gnu.org; Mon, 10 Aug 2020 10:32:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:53765) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1k58qI-00009b-Kw for guix-patches@gnu.org; Mon, 10 Aug 2020 10:32:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1k58qI-0005mb-G8 for guix-patches@gnu.org; Mon, 10 Aug 2020 10:32:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42800] [PATCH] Add (guix git-repo-download). Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 10 Aug 2020 14:32:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 42800 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42800@debbugs.gnu.org Cc: Danny Milosavljevic X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.159706988620609 (code B ref -1); Mon, 10 Aug 2020 14:32:02 +0000 Received: (at submit) by debbugs.gnu.org; 10 Aug 2020 14:31:26 +0000 Received: from localhost ([127.0.0.1]:37078 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k58pd-0005IS-DN for submit@debbugs.gnu.org; Mon, 10 Aug 2020 10:31:26 -0400 Received: from lists.gnu.org ([209.51.188.17]:40128) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k58pa-0005FP-91 for submit@debbugs.gnu.org; Mon, 10 Aug 2020 10:31:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35564) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k58pX-0004bE-VF for guix-patches@gnu.org; Mon, 10 Aug 2020 10:31:17 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:41332) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k58pS-00006H-I6 for guix-patches@gnu.org; Mon, 10 Aug 2020 10:31:14 -0400 Received: from dayas.lan (80-110-127-146.cgn.dynamic.surfer.at [80.110.127.146]) by dd26836.kasserver.com (Postfix) with ESMTPSA id 6117E33624FE; Mon, 10 Aug 2020 16:31:07 +0200 (CEST) From: Danny Milosavljevic Date: Mon, 10 Aug 2020 13:39:31 +0200 Message-Id: <20200810113931.10003-1-dannym@scratchpost.org> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Tags: patch Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Received-SPF: none client-ip=85.13.145.193; envelope-from=dannym@scratchpost.org; helo=dd26836.kasserver.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/10 10:31:07 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -0.01 X-TUID: T6ix7YvB+0/w * guix/build/git-repo.scm: New file. * guix/git-repo-download.scm: New file. * Makefile.am (MODULES): Add them. --- Makefile.am | 2 + guix/build/git-repo.scm | 74 +++++++++++++++++ guix/git-repo-download.scm | 158 +++++++++++++++++++++++++++++++++++++ 3 files changed, 234 insertions(+) create mode 100644 guix/build/git-repo.scm create mode 100644 guix/git-repo-download.scm diff --git a/Makefile.am b/Makefile.am index 1e2c26f5ac..9c27113673 100644 --- a/Makefile.am +++ b/Makefile.am @@ -82,6 +82,7 @@ MODULES = \ guix/discovery.scm \ guix/bzr-download.scm \ guix/git-download.scm \ + guix/git-repo-download.scm \ guix/hg-download.scm \ guix/swh.scm \ guix/monads.scm \ @@ -176,6 +177,7 @@ MODULES = \ guix/build/bzr.scm \ guix/build/copy-build-system.scm \ guix/build/git.scm \ + guix/build/git-repo.scm \ guix/build/hg.scm \ guix/build/glib-or-gtk-build-system.scm \ guix/build/gnu-bootstrap.scm \ diff --git a/guix/build/git-repo.scm b/guix/build/git-repo.scm new file mode 100644 index 0000000000..571a022224 --- /dev/null +++ b/guix/build/git-repo.scm @@ -0,0 +1,74 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014, 2016, 2019 Ludovic Courtès +;;; Copyright © 2020 Danny Milosavljevic +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix build git-repo) + #:use-module (guix build utils) + #:use-module (srfi srfi-34) + #:use-module (ice-9 format) + #:export (git-repo-fetch)) + +;;; Commentary: +;;; +;;; This is the build-side support code of (guix git-repo-download). It +;;; allows a Git-repo repository to be cloned and checked out at a specific +;;; revision. +;;; +;;; Code: + +(define* (git-repo-fetch manifest-url manifest-revision directory + #:key (git-repo-command "git-repo")) + "Fetch packages according to the manifest at MANIFEST-URL with +MANIFEST-REVISION. MANIFEST-REVISION must be either a revision +or a branch. Return #t on success, #f otherwise." + + ;; Disable TLS certificate verification. The hash of the checkout is known + ;; in advance anyway. + (setenv "GIT_SSL_NO_VERIFY" "true") + + (mkdir-p directory) + + (guard (c ((invoke-error? c) + (format (current-error-port) + "git-repo-fetch: '~a~{ ~a~}' failed with exit code ~a~%" + (invoke-error-program c) + (invoke-error-arguments c) + (or (invoke-error-exit-status c) ;XXX: not quite accurate + (invoke-error-stop-signal c) + (invoke-error-term-signal c))) + (delete-file-recursively directory) + #f)) + (with-directory-excursion directory + (invoke git-repo-command "init" "-u" manifest-url "-b" manifest-revision + "--depth=1") + (invoke git-repo-command "sync" "-c" "--fail-fast" "-v" "-j" "3") + + ;; Delete vendor/**/.git, system/**/.git, toolchain/**/.git, + ;; .repo/**/.git etc since they contain timestamps. + (for-each delete-file-recursively + (find-files "." "^\\.git$" #:directories? #t)) + + ;; Delete git state directories since they contain timestamps. + (for-each delete-file-recursively + (find-files ".repo" "^.*\\.git$" #:directories? #t)) + + ;; This file contains timestamps. + (delete-file ".repo/.repo_fetchtimes.json") + #t))) + +;;; git-repo.scm ends here diff --git a/guix/git-repo-download.scm b/guix/git-repo-download.scm new file mode 100644 index 0000000000..27f7f1fa8d --- /dev/null +++ b/guix/git-repo-download.scm @@ -0,0 +1,158 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès +;;; Copyright © 2017 Mathieu Lirzin +;;; Copyright © 2017 Christopher Baines +;;; Copyright © 2020 Jakub Kądziołka +;;; Copyright © 2020 Danny Milosavljevic +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix git-repo-download) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix packages) + #:use-module (guix modules) + #:autoload (guix build-system gnu) (standard-packages) + #:use-module (git) ; FIXME Remove + #:use-module (ice-9 match) + #:use-module (ice-9 vlist) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) + #:export (git-repo-reference + git-repo-reference? + git-repo-reference-mainfest-url + git-repo-reference-revision + + git-repo-fetch + git-repo-version + git-repo-file-name)) + +;;; Commentary: +;;; +;;; An method that fetches a specific commit from a git-repo +;;; repository. +;;; The repository's manifest (URL and revision) can be specified with a +;; object. +;;; +;;; Code: + +(define-record-type* + git-repo-reference make-git-repo-reference + git-repo-reference? + (manifest-url git-repo-reference-manifest-url) + (manifest-revision git-repo-reference-manifest-revision)) + +(define (git-repo-package) + "Return the default git-repo package." + (let ((distro (resolve-interface '(gnu packages android)))) + (module-ref distro 'git-repo))) + +(define* (git-repo-fetch ref hash-algo hash + #:optional name + #:key (system (%current-system)) (guile (default-guile)) + (git-repo (git-repo-package))) + "Return a fixed-output derivation that fetches REF, a +object. The output is expected to have recursive hash HASH of type +HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." + ;; TODO: Remove. + (define inputs + (standard-packages)) + + (define zlib + (module-ref (resolve-interface '(gnu packages compression)) 'zlib)) + + (define guile-json + (module-ref (resolve-interface '(gnu packages guile)) 'guile-json-3)) + + (define gnutls + (module-ref (resolve-interface '(gnu packages tls)) 'gnutls)) + + (define config.scm + (scheme-file "config.scm" + #~(begin + (define-module (guix config) + #:export (%libz)) + + (define %libz + #+(file-append zlib "/lib/libz"))))) + + (define modules + (cons `((guix config) => ,config.scm) + (delete '(guix config) + (source-module-closure '((guix build git-repo) + (guix build utils) + (guix build download-nar)))))) + + (define build + (with-imported-modules modules + (with-extensions (list guile-json gnutls) ;for (guix swh) + #~(begin + (use-modules (guix build git-repo) + (guix build utils) + (guix build download-nar) + (ice-9 match)) + + ;; The 'git submodule' commands expects Coreutils, sed, + ;; grep, etc. to be in $PATH. + (set-path-environment-variable "PATH" '("bin") + (match '#+inputs + (((names dirs outputs ...) ...) + dirs))) + + (setvbuf (current-output-port) 'line) + (setvbuf (current-error-port) 'line) + + (or (git-repo-fetch (getenv "git-repo manifest-url") + (getenv "git-repo manifest-revision") + #$output + #:git-repo-command + (string-append #+git-repo "/bin/repo")) + (download-nar #$output)))))) + + (mlet %store-monad ((guile (package->derivation guile system))) + (gexp->derivation (or name "git-repo-checkout") build + + ;; Use environment variables and a fixed script name so + ;; there's only one script in store for all the + ;; downloads. + #:script-name "git-repo-download" + #:env-vars + `(("git-repo manifest-url" . + ,(git-repo-reference-manifest-url ref)) + ("git-repo manifest-revision" . + ,(git-repo-reference-manifest-revision ref))) + #:leaked-env-vars '("http_proxy" "https_proxy" + "LC_ALL" "LC_MESSAGES" "LANG" + "COLUMNS") + #:system system + #:local-build? #t ;don't offload repo cloning + #:hash-algo hash-algo + #:hash hash + #:recursive? #t + #:guile-for-build guile))) + +(define (git-repo-version version revision) + "Return the version string for packages using git-repo-download." + (string-append version "-" (string-join (string-split revision #\/) "_"))) + +(define (git-repo-file-name name version) + "Return the file-name for packages using git-repo-download." + (string-append name "-" version "-checkout")) + +