From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id oN5oHWZtO2H/GQAAgWs5BA (envelope-from ) for ; Fri, 10 Sep 2021 16:36:22 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id xosBGWZtO2HoDAAAbx9fmQ (envelope-from ) for ; Fri, 10 Sep 2021 14:36:22 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9A4FB9C4E for ; Fri, 10 Sep 2021 16:36:21 +0200 (CEST) Received: from localhost ([::1]:52354 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mOhdc-00087c-JH for larch@yhetil.org; Fri, 10 Sep 2021 10:36:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43008) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mOhdK-00083Z-N9 for bug-guix@gnu.org; Fri, 10 Sep 2021 10:36:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:55625) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mOhdK-0007ee-Fk for bug-guix@gnu.org; Fri, 10 Sep 2021 10:36:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mOhdK-0000ew-Al for bug-guix@gnu.org; Fri, 10 Sep 2021 10:36:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#44187: [PATCH 1/3] swh: Support downloads of bare Git repositories. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 10 Sep 2021 14:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 44187 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 44187@debbugs.gnu.org Received: via spool by 44187-submit@debbugs.gnu.org id=B44187.16312845132459 (code B ref 44187); Fri, 10 Sep 2021 14:36:02 +0000 Received: (at 44187) by debbugs.gnu.org; 10 Sep 2021 14:35:13 +0000 Received: from localhost ([127.0.0.1]:38933 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOhcW-0000dS-IN for submit@debbugs.gnu.org; Fri, 10 Sep 2021 10:35:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53760) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOhcQ-0000cV-9d for 44187@debbugs.gnu.org; Fri, 10 Sep 2021 10:35:06 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47278) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mOhcK-0006gR-95; Fri, 10 Sep 2021 10:35:00 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=47222 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mOhcJ-0006Ir-KZ; Fri, 10 Sep 2021 10:35:00 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Fri, 10 Sep 2021 16:34:13 +0200 Message-Id: <20210910143415.14783-2-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210910143415.14783-1-ludo@gnu.org> References: <87pn0dk61v.fsf@gnu.org> <20210910143415.14783-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1631284581; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=4K06+/zlwjWwyNngVkHoF5SBWyPYLcZcAAP8636EJmw=; b=B8w5WhW+1zUzIGureLChAgwwtpFqtqtCZ0kgd83OI9SI9o7yVUo6wX9dgeMNEvkthMGVv0 oVFlcSdYZ4rZGAIMBDIE30oJBsYHSIVwe8UsZ9KYVr4+WR8kJ36w11GxKyYpNkpDfpNyoS iNOdC7qdexCZ3MuHuI6HNq8e7BU8V6t3LWByyhuYarsWkRtZm1Xlm+oReKMbi5NsMJPU9k Rtt4sDEHZU5+bKBx4N1wslMfXYo8eQOdkgos5oQ372sP01rv2KYGAvVzPz0uizSEl+VazJ PWnWvBjC3uQBuJGtDeQs0/2b1Pb/qOB1YxT+noWcmpr8cxYNvYaxDHJwtADJWQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1631284581; a=rsa-sha256; cv=none; b=tFNXWliimeSCf6yx/3ROccRCF83b/8uzk3s7vsaAo3Ky1GPhO6krbddTDXiRw9huLCSrbk PndUu1ie7URdsScq1pfxlGFGvLBln6E8LXNlHV1yFeucJkT2nrHCC32vOt51438uy25yrO jqq3is1BcWfj/wFuZRmsh1l0OOgrjNO9kKjgWebsIpH9JhT68SYubjsqFahbc+Iz4eaA5O AhW8KgW1t3m2DgYoEGhUB3JiRdDKSUl2wNjIxYXMmaYctURSujU6GbeidgfeLNbMVIQH8V o1BXeKJyaHNpOzdTqL2j/RZ70AFll33/Dzs26zHXG8jV6OxBCq354XHWjv+ZrQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -0.41 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 9A4FB9C4E X-Spam-Score: -0.41 X-Migadu-Scanner: scn0.migadu.com X-TUID: /YlR7HJiwOyE From: Ludovic Courtès * guix/swh.scm (swh-download-archive): New procedure. (swh-download-directory): Rewrite in terms of 'swh-download-archive'. (swh-download): Add #:archive-type and honor it. Use 'swh-download-archive' instead of 'swh-download-directory'. --- guix/swh.scm | 52 ++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 40 insertions(+), 12 deletions(-) diff --git a/guix/swh.scm b/guix/swh.scm index 3d5d2a410a..707551a799 100644 --- a/guix/swh.scm +++ b/guix/swh.scm @@ -645,20 +645,29 @@ delete it when leaving the dynamic extent of this call." (lambda () (false-if-exception (delete-file-recursively tmp-dir)))))) -(define* (swh-download-directory id output - #:key (log-port (current-error-port))) - "Download from Software Heritage the directory with the given ID, and -unpack it to OUTPUT. Return #t on success and #f on failure" +(define* (swh-download-archive swhid output + #:key + (archive-type 'flat) + (log-port (current-error-port))) + "Download from Software Heritage the directory or revision with the given +SWID, in the ARCHIVE-TYPE format (one of 'flat or 'git-bare), and unpack it to +OUTPUT. Return #t on success and #f on failure." (call-with-temporary-directory (lambda (directory) - (match (vault-fetch id 'directory #:log-port log-port) + (match (vault-fetch swhid + #:archive-type archive-type + #:log-port log-port) (#f (format log-port - "SWH: directory ~a could not be fetched from the vault~%" - id) + "SWH: object ~a could not be fetched from the vault~%" + swhid) #f) ((? port? input) - (let ((tar (open-pipe* OPEN_WRITE "tar" "-C" directory "-xzvf" "-"))) + (let ((tar (open-pipe* OPEN_WRITE "tar" "-C" directory + (match archive-type + ('flat "-xzvf") ;gzipped + ('git-bare "-xvf")) ;uncompressed + "-"))) (dump-port input tar) (close-port input) (let ((status (close-pipe tar))) @@ -672,6 +681,14 @@ unpack it to OUTPUT. Return #t on success and #f on failure" #:log (%make-void-port "w")) #t)))))))) +(define* (swh-download-directory id output + #:key (log-port (current-error-port))) + "Download from Software Heritage the directory with the given ID, and +unpack it to OUTPUT. Return #t on success and #f on failure." + (swh-download-archive (string-append "swh:1:dir:" id) output + #:archive-type 'flat + #:log-port log-port)) + (define (commit-id? reference) "Return true if REFERENCE is likely a commit ID, false otherwise---e.g., if it is a tag name. This is based on a simple heuristic so use with care!" @@ -679,8 +696,11 @@ it is a tag name. This is based on a simple heuristic so use with care!" (string-every char-set:hex-digit reference))) (define* (swh-download url reference output - #:key (log-port (current-error-port))) - "Download from Software Heritage a checkout of the Git tag or commit + #:key + (archive-type 'flat) + (log-port (current-error-port))) + "Download from Software Heritage a checkout (if ARCHIVE-TYPE is 'flat) or a +full Git repository (if ARCHIVE-TYPE is 'git-bare) of the Git tag or commit REFERENCE originating from URL, and unpack it in OUTPUT. Return #t on success and #f on failure. @@ -694,7 +714,15 @@ wait until it becomes available, which could take several minutes." (format log-port "SWH: found revision ~a with directory at '~a'~%" (revision-id revision) (swh-url (revision-directory-url revision))) - (swh-download-directory (revision-directory revision) output - #:log-port log-port)) + (swh-download-archive (match archive-type + ('flat + (string-append + "swh:1:dir:" (revision-directory revision))) + ('git-bare + (string-append + "swh:1:rev:" (revision-id revision)))) + output + #:archive-type archive-type + #:log-port log-port)) (#f #f))) -- 2.33.0