From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50382) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gOmBX-00068V-S8 for guix-patches@gnu.org; Mon, 19 Nov 2018 11:14:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gOmBW-0001zE-MZ for guix-patches@gnu.org; Mon, 19 Nov 2018 11:14:03 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:56850) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gOmBW-0001yH-1F for guix-patches@gnu.org; Mon, 19 Nov 2018 11:14:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gOmBV-0005VZ-Ok for guix-patches@gnu.org; Mon, 19 Nov 2018 11:14:01 -0500 Subject: [bug#33432] [PATCH 0/2] Download Git checkouts from Software Heritage as a last resort Resent-Message-ID: Received: from eggs.gnu.org ([2001:4830:134:3::10]:50296) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gOmBE-0005ym-Dv for guix-patches@gnu.org; Mon, 19 Nov 2018 11:13:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gOmBD-0001hG-IC for guix-patches@gnu.org; Mon, 19 Nov 2018 11:13:44 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 19 Nov 2018 17:13:25 +0100 Message-Id: <20181119161325.7801-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: 33432@debbugs.gnu.org Hello Guix! This patch series adds the Software Heritage (SWH) client library initially discussed at: https://lists.gnu.org/archive/html/guix-devel/2018-11/msg00285.html Furthermore, it uses it in (guix git-download) to download code from SWH when it is unavailable upstream and on our servers. This bit relies on the “vault” API of SWH, which allows you to fetch a checkout as a tarball. Not all revisions are readily available as tarballs, understandably, so the vault API has a mechanism that allows you to request the “cooking” of a specific checkout. Cooking is asynchronous and can take some time. https://docs.softwareheritage.org/devel/swh-vault/api.html When downloading over SWH, the ‘swh-download’ procedure first resolves the tag (if it’s a tag), then tries to download the corresponding tarball from the vault. If the vault doesn’t have it yet, it sends a cooking request and waits for it to complete by periodically checking the cooking status. In the future, we should provide a “lister” and “loader” so that SWH can regularly obtain a list of Guix packages with their source URL and commit/tag: https://forge.softwareheritage.org/T1352 The SWH team is also considering pre-cooking all VCS tags such that every time we refer to a tag, we can be sure its contents are already available in the vault: https://forge.softwareheritage.org/T1350 Feedback welcome! Ludo’. Ludovic Courtès (2): Add (guix swh). git-download: Download from Software Heritage as a last resort. Makefile.am | 1 + guix/git-download.scm | 64 +++-- guix/swh.scm | 551 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 596 insertions(+), 20 deletions(-) create mode 100644 guix/swh.scm -- 2.19.1