From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id wM66C+txNF+pbwAA0tVLHw (envelope-from ) for ; Wed, 12 Aug 2020 22:49:15 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id gCQKButxNF8UNAAA1q6Kng (envelope-from ) for ; Wed, 12 Aug 2020 22:49:15 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3FD6B9407C5 for ; Wed, 12 Aug 2020 22:49:14 +0000 (UTC) Received: from localhost ([::1]:37374 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k5zYV-0007cV-JJ for larch@yhetil.org; Wed, 12 Aug 2020 18:49:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41246) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k5zYM-0007Zd-6y for guix-patches@gnu.org; Wed, 12 Aug 2020 18:49:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:34328) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1k5zYL-0006cm-T3 for guix-patches@gnu.org; Wed, 12 Aug 2020 18:49:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1k5zYL-0003pc-Rv for guix-patches@gnu.org; Wed, 12 Aug 2020 18:49:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42800] [PATCH v2] Add (guix android-repo-download). Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 12 Aug 2020 22:49:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42800 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42800@debbugs.gnu.org Cc: Danny Milosavljevic Received: via spool by 42800-submit@debbugs.gnu.org id=B42800.159727249814670 (code B ref 42800); Wed, 12 Aug 2020 22:49:01 +0000 Received: (at 42800) by debbugs.gnu.org; 12 Aug 2020 22:48:18 +0000 Received: from localhost ([127.0.0.1]:45874 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k5zXa-0003oU-BU for submit@debbugs.gnu.org; Wed, 12 Aug 2020 18:48:18 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:46098) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k5zXX-0003oI-Oc for 42800@debbugs.gnu.org; Wed, 12 Aug 2020 18:48:12 -0400 Received: from dayas.lan (80-110-127-146.cgn.dynamic.surfer.at [80.110.127.146]) by dd26836.kasserver.com (Postfix) with ESMTPSA id D92F73368AF6; Thu, 13 Aug 2020 00:48:09 +0200 (CEST) From: Danny Milosavljevic Date: Thu, 13 Aug 2020 00:12:08 +0200 Message-Id: <20200812221208.31262-1-dannym@scratchpost.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200810153842.2nii3vcihuul6zkp@gravity> References: <20200810153842.2nii3vcihuul6zkp@gravity> MIME-Version: 1.0 Tags: patch Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.7 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -0.01 X-TUID: DH1vjoPBXki6 * guix/build/android-repo.scm: New file. * guix/android-repo-download.scm: New file. * Makefile.am (MODULES): Add them. --- Makefile.am | 2 + guix/android-repo-download.scm | 160 +++++++++++++++++++++++++++++++++ guix/build/android-repo.scm | 74 +++++++++++++++ 3 files changed, 236 insertions(+) create mode 100644 guix/android-repo-download.scm create mode 100644 guix/build/android-repo.scm diff --git a/Makefile.am b/Makefile.am index ee86571dcc..249cd24135 100644 --- a/Makefile.am +++ b/Makefile.am @@ -80,6 +80,7 @@ MODULES = \ guix/modules.scm \ guix/download.scm \ guix/discovery.scm \ + guix/android-repo-download.scm \ guix/bzr-download.scm \ guix/git-download.scm \ guix/hg-download.scm \ @@ -172,6 +173,7 @@ MODULES = \ guix/build/minify-build-system.scm \ guix/build/font-build-system.scm \ guix/build/go-build-system.scm \ + guix/build/android-repo.scm \ guix/build/asdf-build-system.scm \ guix/build/bzr.scm \ guix/build/copy-build-system.scm \ diff --git a/guix/android-repo-download.scm b/guix/android-repo-download.scm new file mode 100644 index 0000000000..722e885ec9 --- /dev/null +++ b/guix/android-repo-download.scm @@ -0,0 +1,160 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès +;;; Copyright © 2017 Mathieu Lirzin +;;; Copyright © 2017 Christopher Baines +;;; Copyright © 2020 Jakub Kądziołka +;;; Copyright © 2020 Danny Milosavljevic +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix android-repo-download) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix packages) + #:use-module (guix modules) + #:autoload (guix build-system gnu) (standard-packages) + #:use-module (git) ; FIXME Remove + #:use-module (ice-9 match) + #:use-module (ice-9 vlist) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) + #:export (android-repo-reference + android-repo-reference? + android-repo-reference-mainfest-url + android-repo-reference-revision + + android-repo-fetch + android-repo-version + android-repo-file-name)) + +;;; Commentary: +;;; +;;; An method that fetches a specific commit from an Android Repo +;;; repository. +;;; The repository's manifest (URL and revision) can be specified with a +;; object. +;;; +;;; Code: + +(define-record-type* + android-repo-reference make-android-repo-reference + android-repo-reference? + (manifest-url android-repo-reference-manifest-url) + (manifest-revision android-repo-reference-manifest-revision)) + +(define (git-repo-package) + "Return the default git-repo package." + (let ((distro (resolve-interface '(gnu packages android)))) + (module-ref distro 'git-repo))) + +(define* (android-repo-fetch ref hash-algo hash + #:optional name + #:key (system (%current-system)) + (guile (default-guile)) + (git-repo (git-repo-package))) + "Return a fixed-output derivation that fetches REF, an + object. The output is expected to have recursive +hash HASH of type HASH-ALGO (a symbol). Use NAME as the file name, or a +generic name if #f." + ;; TODO: Remove. + (define inputs + (standard-packages)) + + (define zlib + (module-ref (resolve-interface '(gnu packages compression)) 'zlib)) + + (define guile-json + (module-ref (resolve-interface '(gnu packages guile)) 'guile-json-3)) + + (define gnutls + (module-ref (resolve-interface '(gnu packages tls)) 'gnutls)) + + (define config.scm + (scheme-file "config.scm" + #~(begin + (define-module (guix config) + #:export (%libz)) + + (define %libz + #+(file-append zlib "/lib/libz"))))) + + (define modules + (cons `((guix config) => ,config.scm) + (delete '(guix config) + (source-module-closure '((guix build android-repo) + (guix build utils) + (guix build download-nar)))))) + + (define build + (with-imported-modules modules + (with-extensions (list guile-json gnutls) ;for (guix swh) + #~(begin + (use-modules (guix build android-repo) + (guix build utils) + (guix build download-nar) + (ice-9 match)) + + ;; The 'git submodule' commands expects Coreutils, sed, + ;; grep, etc. to be in $PATH. + (set-path-environment-variable "PATH" '("bin") + (match '#+inputs + (((names dirs outputs ...) ...) + dirs))) + + (setvbuf (current-output-port) 'line) + (setvbuf (current-error-port) 'line) + + (or (android-repo-fetch (getenv "android-repo manifest-url") + (getenv "android-repo manifest-revision") + #$output + #:git-repo-command + (string-append #+git-repo "/bin/repo")) + (download-nar #$output)))))) + + (mlet %store-monad ((guile (package->derivation guile system))) + (gexp->derivation (or name "android-repo-checkout") build + + ;; Use environment variables and a fixed script name so + ;; there's only one script in store for all the + ;; downloads. + #:script-name "android-repo-download" + #:env-vars + `(("android-repo manifest-url" . + ,(android-repo-reference-manifest-url ref)) + ("android-repo manifest-revision" . + ,(android-repo-reference-manifest-revision ref))) + #:leaked-env-vars '("http_proxy" "https_proxy" + "LC_ALL" "LC_MESSAGES" "LANG" + "COLUMNS") + #:system system + #:local-build? #t ;don't offload repo cloning + #:hash-algo hash-algo + #:hash hash + #:recursive? #t + #:guile-for-build guile))) + +(define (android-repo-version version revision) + "Return the version string for packages using android-repo-download." + (string-append version "-" (string-join (string-split revision #\/) "_"))) + +(define (android-repo-file-name name version) + "Return the file-name for packages using android-repo-download." + (string-append name "-" version "-checkout")) + + diff --git a/guix/build/android-repo.scm b/guix/build/android-repo.scm new file mode 100644 index 0000000000..cba50270ea --- /dev/null +++ b/guix/build/android-repo.scm @@ -0,0 +1,74 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014, 2016, 2019 Ludovic Courtès +;;; Copyright © 2020 Danny Milosavljevic +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix build android-repo) + #:use-module (guix build utils) + #:use-module (srfi srfi-34) + #:use-module (ice-9 format) + #:export (android-repo-fetch)) + +;;; Commentary: +;;; +;;; This is the build-side support code of (guix android-repo-download). +;;; It allows a multirepository managed by the git-repo tool to be cloned and +;;; checked out at a specific revision. +;;; +;;; Code: + +(define* (android-repo-fetch manifest-url manifest-revision directory + #:key (git-repo-command "git-repo")) + "Fetch packages according to the manifest at MANIFEST-URL with +MANIFEST-REVISION. MANIFEST-REVISION must be either a revision +or a branch. Return #t on success, #f otherwise." + + ;; Disable TLS certificate verification. The hash of the checkout is known + ;; in advance anyway. + (setenv "GIT_SSL_NO_VERIFY" "true") + + (mkdir-p directory) + + (guard (c ((invoke-error? c) + (format (current-error-port) + "android-repo-fetch: '~a~{ ~a~}' failed with exit code ~a~%" + (invoke-error-program c) + (invoke-error-arguments c) + (or (invoke-error-exit-status c) ;XXX: not quite accurate + (invoke-error-stop-signal c) + (invoke-error-term-signal c))) + (delete-file-recursively directory) + #f)) + (with-directory-excursion directory + (invoke git-repo-command "init" "-u" manifest-url "-b" manifest-revision + "--depth=1") + (invoke git-repo-command "sync" "-c" "--fail-fast" "-v" "-j" "3") + + ;; Delete vendor/**/.git, system/**/.git, toolchain/**/.git, + ;; .repo/**/.git etc since they contain timestamps. + (for-each delete-file-recursively + (find-files "." "^\\.git$" #:directories? #t)) + + ;; Delete git state directories since they contain timestamps. + (for-each delete-file-recursively + (find-files ".repo" "^.*\\.git$" #:directories? #t)) + + ;; This file contains timestamps. + (delete-file ".repo/.repo_fetchtimes.json") + #t))) + +;;; android-repo.scm ends here