From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 2CdmIXbp4mEsSQEAgWs5BA (envelope-from ) for ; Sat, 15 Jan 2022 16:34:14 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id eBErHnbp4mE94gAAauVa8A (envelope-from ) for ; Sat, 15 Jan 2022 16:34:14 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DC6502BFF1 for ; Sat, 15 Jan 2022 16:34:13 +0100 (CET) Received: from localhost ([::1]:51526 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n8l4H-0007bH-1d for larch@yhetil.org; Sat, 15 Jan 2022 10:34:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:53994) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n8l47-0007b9-9M for guix-patches@gnu.org; Sat, 15 Jan 2022 10:34:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:48627) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n8l47-000409-00 for guix-patches@gnu.org; Sat, 15 Jan 2022 10:34:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1n8l46-00059o-Ht for guix-patches@gnu.org; Sat, 15 Jan 2022 10:34:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52828] [PATCH v2] swh: Do not rely on $PATH for tar and gzip. Resent-From: Timothy Sample Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 15 Jan 2022 15:34:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52828 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 52828@debbugs.gnu.org Received: via spool by 52828-submit@debbugs.gnu.org id=B52828.164226081219782 (code B ref 52828); Sat, 15 Jan 2022 15:34:02 +0000 Received: (at 52828) by debbugs.gnu.org; 15 Jan 2022 15:33:32 +0000 Received: from localhost ([127.0.0.1]:41530 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n8l3V-00058s-En for submit@debbugs.gnu.org; Sat, 15 Jan 2022 10:33:32 -0500 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:34419) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n8l3Q-00058Z-MC for 52828@debbugs.gnu.org; Sat, 15 Jan 2022 10:33:24 -0500 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 9165E5C00CD; Sat, 15 Jan 2022 10:33:15 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Sat, 15 Jan 2022 10:33:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=e95IXO CkHa+rxlUG96Bi/CYgeIdUiLEGzysQqCYdVrA=; b=OKg46fBBvmk2kugvmslFk9 gCAB0Dv0A2jeFNpdAKPA2T8h6zdBcOTBf/B5gwWQcLbi662qIVKTkt4nytqmTcrT s3LNwP45n1y29mpnVD1SDWoBvxcKVuuYCEvTlXM+o7pH0pvC7SrmA3rMlIzBrEtV HhvFt5GwMYDqb4gKrSXhs947nY8TKznjQ4U+ZamrgVQSDkF+qsM39nwDTYFW/VWB aplqh/pB0aK9V61/VIsUYN0SAXvjKvD+3+JcfXqY5hGbJ8Y30usawuaX8fFqhs3Q kz+YqP2MO1LuJ2NFw7d2Lssni3vQO4YhduFdK4vNoGtXQzFSJF9ATUjpxEd+wBHg == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrtdejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufhffjgfkfgggtgesmhdtreertderjeenucfhrhhomhepvfhimhhothhh hicuufgrmhhplhgvuceoshgrmhhplhgvthesnhhghihrohdrtghomheqnecuggftrfgrth htvghrnhepvdetjeevkefggfefgfevteehgffgledugfetudfggeeltdfhfeeuleejffeu ueffnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepsh grmhhplhgvthesnhhghihrohdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 15 Jan 2022 10:33:15 -0500 (EST) From: Timothy Sample References: <877dbpuc2n.fsf@ngyro.com> <87mtk9kiol.fsf@gnu.org> Date: Sat, 15 Jan 2022 10:33:14 -0500 In-Reply-To: <87mtk9kiol.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Wed, 05 Jan 2022 21:54:02 +0100") Message-ID: <8735lpxbd1.fsf_-_@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1642260854; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=e95IXOCkHa+rxlUG96Bi/CYgeIdUiLEGzysQqCYdVrA=; b=H7Fc0CCeoMMPyNUJQJVurjOos7MpZRHuJHHB6KdYstfd1PcEi3lFoc+9ooHqHDyskOJvUp icUOHB1b0rE4T16eIMFuTIMtlbVHWw+d9eO1hhFs4reQonxOD0ivcEKVXhW63PoyYC4P9o LiyKL9zI4xW1MUrtQFwgE24OpEr6J5vo2D3M1N5ItbntR/ag4CaGKzSJhNsU1JTxXOrPUf w+HdyhfCio9FVPihsqnwuRUTgQA2lhqoX8KVl65pGA0YeY6FcAc14Mc/oqMcngD3fIRYyr gxD1rAbVAtf6LMmWrqNcUzOXphFXIC1zSpSN08wkACBJsjF+T45lX704QE42Dw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1642260854; a=rsa-sha256; cv=none; b=YCO8Hh+gsrIml/PBU+seZTbLu0wu5kmcBAAMHTjJHAy6acaDHC5QNoAJEQ2Nwr6b4GuKYZ +Tl1PKorymixiQOdK37JL/KYZFBJVu0PeszzfV55GJEJrX0op9DxQGNGH37jUNYgFX/hg/ qcV5kvWoQjrrB8MLOmQRDi5pR026Gq/B0VNN7YgvYgG5aeGiRmysxyBBz7C+3u71ql9Cw+ C5jjVbcbZTlFnImhPSqKlC2rwxskINg/IgdjPmpUYFCI+n/T9RV8EvZu9O1RanRTxTeyAF WlIkUyrBtWoYbaT0kOcC2qzUZGaY96dhbTiviyWdDmbbQityPALI/6gwXHvTsA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=OKg46fBB; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.23 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=OKg46fBB; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: DC6502BFF1 X-Spam-Score: -1.23 X-Migadu-Scanner: scn1.migadu.com X-TUID: 4kWQwd7hWTbY --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludovic, Ludovic Court=C3=A8s writes: > What about uses of guix-daemon on other distros? Perhaps we assume tar > and gzip are already on PATH? That=E2=80=99s my guess. And if those other weird distros don=E2=80=99t ha= ve =E2=80=98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 in =E2=80=9C/usr/bin=E2=80=9D, I=E2=80=99m sure they= =E2=80=99re plenty accustomed to liberally patching their packages. ;) > I=E2=80=99d feel more comfortable with a solution at the package level. = In the > =E2=80=98guix=E2=80=99 package, or perhaps in a Makefile.am, we could har= d-code absolute > file names of tar and gzip in (guix scripts perform-download) and set > PATH from there. We=E2=80=99d need to do the same in (guix self) though. > > Alternatively, we could change (guix swh) to use Guile-Zlib and the tar > implementation of Gash-Utils or that of Disarchive. > > WDYT? I=E2=80=99ve attached a new patch that mixes those two suggestions but gets= the first one a little wrong. It uses the absolute path for =E2=80=98tar=E2=80= =99, but uses Guile-zlib for decompression. I honestly don=E2=80=99t have a strong opinion about when and where to set =E2=80=98$PATH=E2=80=99 vs. using a configured, absolute path. My original= patch assumed that it=E2=80=99s the user=E2=80=99s job to make sure that =E2=80= =98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 are available to Guix at runtime. This patch assumes that that linkage happens at configure time. The main benefit I could see to setting =E2=80=98$PATH=E2=80=99 in =E2=80=98(guix scripts perform-download)=E2=80= =99 is that we could add our =E2=80=98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 as a suffix. This makes i= t work while allowing users to choose whatever =E2=80=98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 they pr= efer. The downside is that whenever we use =E2=80=98(guix swh)=E2=80=99 have to remember to make sure = that =E2=80=98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 are available. Basically, I can to change this to do the setup in =E2=80=98perform-downloa= d=E2=80=99, but I really want to understand why. Thanks! --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=v2-0001-swh-Do-not-rely-on-PATH-for-tar-and-gzip.patch Content-Transfer-Encoding: quoted-printable >From 010666376890b6adf6c14253b1e2651b5c2861e8 Mon Sep 17 00:00:00 2001 From: Timothy Sample Date: Fri, 14 Jan 2022 18:03:10 -0500 Subject: [PATCH v2] swh: Do not rely on $PATH for tar and gzip. Fixes . * configure.ac: Find the path of the tar utility. * guix/config.scm.in (%tar): New variable. * guix/self.scm (specification->package): Add "tar". (make-config.scm): Add a 'tar' keyword parameter and use it to set the '%tar' variable. (compiled-guix): Add a 'tar' keyword parameter, and pass it to 'make-config.scm'; add 'guile-zlib' as an extension for "guix-core". * guix/swh.scm (swh-download-archive): Use Guile-zlib to decompress "flat" archives, and use an absolute path when invoking 'tar'. --- configure.ac | 4 ++++ guix/config.scm.in | 7 ++++++- guix/self.scm | 18 ++++++++++++++---- guix/swh.scm | 13 ++++++++----- 4 files changed, 32 insertions(+), 10 deletions(-) diff --git a/configure.ac b/configure.ac index 341cff8fbd..78f599b2f4 100644 --- a/configure.ac +++ b/configure.ac @@ -198,6 +198,10 @@ AC_SUBST([GZIP]) AC_SUBST([BZIP2]) AC_SUBST([XZ]) =20 +dnl The '(guix swh)' module uses 'tar'. +AC_PATH_PROG([TAR], [tar]) +AC_SUBST([TAR]) + LIBGCRYPT_LIBDIR=3D"no" LIBGCRYPT_PREFIX=3D"no" =20 diff --git a/guix/config.scm.in b/guix/config.scm.in index d582d91d74..7b400a9ff8 100644 --- a/guix/config.scm.in +++ b/guix/config.scm.in @@ -37,7 +37,9 @@ (define-module (guix config) %system %gzip %bzip2 - %xz)) + %xz + + %tar)) =20 ;;; Commentary: ;;; @@ -118,4 +120,7 @@ (define %bzip2 (define %xz "@XZ@") =20 +(define %tar + "@TAR@") + ;;; config.scm ends here diff --git a/guix/self.scm b/guix/self.scm index 943bb0b498..3944f1a98d 100644 --- a/guix/self.scm +++ b/guix/self.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2017, 2018, 2019, 2020, 2021 Ludovic Court=C3=A8s ;;; Copyright =C2=A9 2020 Martin Becze +;;; Copyright =C2=A9 2022 Timothy Sample ;;; ;;; This file is part of GNU Guix. ;;; @@ -68,6 +69,7 @@ (define specification->package ("gzip" (ref '(gnu packages compression) 'gzip)) ("bzip2" (ref '(gnu packages compression) 'bzip2)) ("xz" (ref '(gnu packages compression) 'xz)) + ("tar" (ref '(gnu packages base) 'tar)) ("po4a" (ref '(gnu packages gettext) 'po4a)) ("gettext" (ref '(gnu packages gettext) 'gettext-minimal)) ("gcc-toolchain" (ref '(gnu packages commencement) 'gcc-toolchain)) @@ -749,6 +751,7 @@ (define* (compiled-guix source #:key (gzip (specification->package "gzip")) (bzip2 (specification->package "bzip2")) (xz (specification->package "xz")) + (tar (specification->package "tar")) (guix (specification->package "guix"))) "Return a file-like object that contains a compiled Guix." (define guile-avahi @@ -832,7 +835,9 @@ (define* (compiled-guix source #:key ,(local-file "../guix/store/schema.sql"))) =20 #:extensions (list guile-gcrypt - guile-json) ;for (guix swh) + ;; The following are for (guix swh) + guile-json + guile-zlib) #:guile-for-build guile-for-build)) =20 (define *extra-modules* @@ -964,6 +969,7 @@ (define* (compiled-guix source #:key =3D> ,(make-config.scm #:gzip gzip #:bzip2 bzip2 #:xz xz + #:tar tar #:package-name %guix-package-name #:package-version @@ -1071,7 +1077,7 @@ (define %default-config-variables (%storedir . "/gnu/store") (%sysconfdir . "/etc"))) =20 -(define* (make-config.scm #:key gzip xz bzip2 +(define* (make-config.scm #:key gzip xz bzip2 tar (package-name "GNU Guix") (package-version "0") (channel-metadata #f) @@ -1097,7 +1103,8 @@ (define* (make-config.scm #:key gzip xz bzip2 %config-directory %gzip %bzip2 - %xz)) + %xz + %tar)) =20 (define %system #$(%current-system)) @@ -1142,7 +1149,10 @@ (define* (make-config.scm #:key gzip xz bzip2 (define %bzip2 #+(and bzip2 (file-append bzip2 "/bin/bzip2"))) (define %xz - #+(and xz (file-append xz "/bin/xz")))) + #+(and xz (file-append xz "/bin/xz"))) + + (define %tar + #+(and tar (file-append tar "/bin/tar")))) =20 ;; Guile 2.0 *requires* the 'define-module' to be at the ;; top-level or the 'toplevel-ref' in the resulting .go fil= e are diff --git a/guix/swh.scm b/guix/swh.scm index c7c1c873a2..93bb023192 100644 --- a/guix/swh.scm +++ b/guix/swh.scm @@ -3,6 +3,7 @@ ;;; Copyright =C2=A9 2020 Jakub K=C4=85dzio=C5=82ka ;;; Copyright =C2=A9 2021 Xinglu Chen ;;; Copyright =C2=A9 2021 Simon Tournier +;;; Copyright =C2=A9 2022 Timothy Sample ;;; ;;; This file is part of GNU Guix. ;;; @@ -23,6 +24,7 @@ (define-module (guix swh) #:use-module (guix base16) #:use-module (guix build utils) #:use-module ((guix build syscalls) #:select (mkdtemp!)) + #:use-module (guix config) #:use-module (web uri) #:use-module (web client) #:use-module (web response) @@ -35,6 +37,7 @@ (define-module (guix swh) #:use-module (ice-9 regex) #:use-module (ice-9 popen) #:use-module ((ice-9 ftw) #:select (scandir)) + #:use-module (zlib) #:export (%swh-base-url %verify-swh-certificate? %allow-request? @@ -674,11 +677,11 @@ (define* (swh-download-archive swhid output swhid) #f) ((? port? input) - (let ((tar (open-pipe* OPEN_WRITE "tar" "-C" directory - (match archive-type - ('flat "-xzvf") ;gzipped - ('git-bare "-xvf")) ;uncompressed - "-"))) + (let ((input (match archive-type + ;; "flat" archives are compressed. + ('flat (make-zlib-input-port input #:format 'gzip)) + ('git-bare input))) + (tar (open-pipe* OPEN_WRITE %tar "-C" directory "-xvf" "-"))) (dump-port input tar) (close-port input) (let ((status (close-pipe tar))) --=20 2.34.0 --=-=-=--