From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 6O5uN3q2216vKQAA0tVLHw (envelope-from ) for ; Sat, 06 Jun 2020 15:30:02 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id YA5AM3q2217DNQAAB5/wlQ (envelope-from ) for ; Sat, 06 Jun 2020 15:30:02 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8522694060B for ; Sat, 6 Jun 2020 15:30:02 +0000 (UTC) Received: from localhost ([::1]:59592 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jhall-0005Wm-Ae for larch@yhetil.org; Sat, 06 Jun 2020 11:30:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48224) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jhald-0005WP-D2 for guix-devel@gnu.org; Sat, 06 Jun 2020 11:29:53 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:43844) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jhalc-0002un-16; Sat, 06 Jun 2020 11:29:53 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 49fNjl2Zc1z1rqBD; Sat, 6 Jun 2020 17:29:47 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 49fNjl2F2Dz1r579; Sat, 6 Jun 2020 17:29:47 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id YrDQ_q5boKuk; Sat, 6 Jun 2020 17:29:46 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-52-71.dynamic.mnet-online.de [188.174.52.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Sat, 6 Jun 2020 17:29:46 +0200 (CEST) Received: from thisbe.goebel-consult.de (hermia.goebel-consult.de [192.168.110.7]) by hermia.goebel-consult.de (Postfix) with ESMTP id 063D660203; Sat, 6 Jun 2020 17:30:41 +0200 (CEST) Subject: Re: Downloader for "wrapped" tarbar? To: Marius Bakke , Guix-devel , zimoun References: <87y2p5uufv.fsf@gnu.org> From: Hartmut Goebel Organization: crazy-compilers.com Message-ID: <8716b84c-ba1e-2cce-3e14-c289f63a7d2a@crazy-compilers.com> Date: Sat, 6 Jun 2020 17:29:43 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: <87y2p5uufv.fsf@gnu.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="KVvmG3eWoGGVUg8A17DDTnsRqSTyZeLzQ" Received-SPF: none client-ip=212.18.0.10; envelope-from=h.goebel@crazy-compilers.com; helo=mail-out.m-online.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/06 11:29:47 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -2.11 X-TUID: i+Fmc7Hwk0X/ This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --KVvmG3eWoGGVUg8A17DDTnsRqSTyZeLzQ Content-Type: multipart/mixed; boundary="a3IETeSeyaYKQMJUP7t0CQai38mj9Vu9L"; protected-headers="v1" From: Hartmut Goebel To: Marius Bakke , Guix-devel , zimoun Message-ID: <8716b84c-ba1e-2cce-3e14-c289f63a7d2a@crazy-compilers.com> Subject: Re: Downloader for "wrapped" tarbar? References: <87y2p5uufv.fsf@gnu.org> In-Reply-To: <87y2p5uufv.fsf@gnu.org> --a3IETeSeyaYKQMJUP7t0CQai38mj9Vu9L Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: en-US Am 02.06.20 um 21:41 schrieb Marius Bakke: > It would be ideal to have an origin method that could extract the > "inner" tarball, i.e. contents.tar.gz for hex.pm and data.tar.gz in the= > case of RubyGems. As zimoun mentioned, a good place to start is look a= t > how other origin methods are implemented such as url-fetch/tarbomb, etc= =2E I started implementing into this direction and would like your advice on the design. I found two options: 1. When implementing some "url-fetch/wrapped" (name tdb), *two* items will be kept in the store: the "outer" and the "inner" tarball. This is since "url-fetch" and siblings use the built-in downloader, which AFAIK always puts the downloaded files into the store. In this case we need to check the hash of the "outer" tarball, as the built-in downloader requires a hash to be passed and to match. But then we can not check the hash of the "outer" tarball. How would this work with substitutes and download-nar? 2. When implementing some "wrapped-fetch" (name tdb), modeled like "git-fetch", there is no easy way for the user to verify the hash, as this is taken from the "inner" tarball. How does this work with substitutes, download-nar and SWH? --=20 Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | --a3IETeSeyaYKQMJUP7t0CQai38mj9Vu9L-- --KVvmG3eWoGGVUg8A17DDTnsRqSTyZeLzQ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE1K2LnBZ7dXxPCOh3e3UoEb93O2UFAl7btmcACgkQe3UoEb93 O2U7Ew/+JDHtGpAbdLXI7iTspEwRVGvD2Y/Ipbd12FKjAg+LxzWGoS5Azvz0KLDh O5d7ukcOldELpRpd7ITZ6sFDJ+vk2uroUPHhS7I2NWv4df1CRFwtIWbCCmg7XiMS FPAEN/xH4MtVtUbh3SxC/CBOeLxS8zktX91bDowzLSMcTjo3jShOm8hGxOWnqyD1 YNhGvn5ewP5Qorl6rJjLmS7V7Sr+E83wT94MPKq+kWnTr7o8Seo85OGNE8icvZcb ygMNKzW7h/zafPt/aQQrPRvQT647h5eF+DybrB8rZ1RJfNn4ugiYrmv60fUN6qdz irROUU5SusEewneb8ZQIm5bOgXZDw3p1UUuVhSHBawZJitT/AKJZcq2Wesrjwgw2 06bLxNwvpASJVC7qB7JOzLU59m0S+9ATnVfO32v8FLq0hhP5S/JejylTbZ9EDnrq ztFGWoSAKlxHPbEBL1PCLw1/yw8IdMaJx/yWGIrpR0jLkw+gymnVpFpoS0PouEFm TiInNy8XeT7Wqcqz1n7YtqKNLL95InFo8KQ/CtWAPWXkQwPxgrtpJq/pBco7P7Bk sgKw6FvGiZwfyY5mCyv6xwrWRVnSJTQEBnWHVjzeHPF+UVa2ByHUGsw2uKi2qKDy 8cW2rTAmcsmDV9f51wfs0Q+4mVCtYlNj/edBpUuZjx3G738M9tI= =nPQP -----END PGP SIGNATURE----- --KVvmG3eWoGGVUg8A17DDTnsRqSTyZeLzQ--