From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id GtHIEVv3FWJ+RQEAgWs5BA (envelope-from ) for ; Wed, 23 Feb 2022 09:59:07 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id IEQBDlv3FWL3uAAA9RJhRA (envelope-from ) for ; Wed, 23 Feb 2022 09:59:07 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E850D4414D for ; Wed, 23 Feb 2022 09:59:06 +0100 (CET) Received: from localhost ([::1]:52634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nMnUH-0007kC-ME for larch@yhetil.org; Wed, 23 Feb 2022 03:59:05 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45524) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nMnTO-0007gi-U6 for guix-devel@gnu.org; Wed, 23 Feb 2022 03:58:12 -0500 Received: from mout.kundenserver.de ([212.227.126.187]:47357) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nMnTK-0000Sp-Ps for guix-devel@gnu.org; Wed, 23 Feb 2022 03:58:10 -0500 Received: from hermia.goebel-consult.de ([79.211.183.108]) by mrelayeu.kundenserver.de (mreue009 [212.227.15.167]) with ESMTPSA (Nemesis) id 1MIMXE-1nQ2bG28PM-00ER35 for ; Wed, 23 Feb 2022 09:58:02 +0100 Received: from [192.168.110.2] (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id 6CDD35F790 for ; Wed, 23 Feb 2022 09:58:03 +0100 (CET) Message-ID: Date: Wed, 23 Feb 2022 09:57:59 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.6.1 Content-Language: en-US From: Hartmut Goebel To: guix-devel Organization: crazy-compilers.com Subject: (Re-) Designing extractong-downaloder Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K1:HZdHKhy5aBZzTMzAzrNCjRgqlCMbpcIltkQvlLUAesRECcPCJzE FVaeDvmtyJpVwG/u/7JaESFOcbBPhTmSiRgpMuviaHD3w+5tnGFeRAasbDPSv1ac5VV+SLK JJ93hmuCXgL+t2miM5f7WMgFBr45ZOwT4NZqofZ/vb1S22Y8Md912BRlWT0js27lsjGI4NV 2YaUBQ4aA5T24OTdEeLew== X-UI-Out-Filterresults: notjunk:1;V03:K0:QqdWI+Ja7Fc=:ZFuPyDErIndnMsbhjjGtKv uXkvPfiGl7K6m7xXBVkHY5mfP5Pso9M6gXXsrt5fCJ6ZcAtSvGdBaRI4EM7QZxlfneRtrGEiH ZTTudMLY9qRdd0i1THQnkhJtbd6TKs4+9zY+n4g8U/gU2nfPKlxrSS0JSqK/fG7xdw++UI3pt MKzFHuSJ7bLNnzJaNhFXVnV9hSbVIwdH8ZSr67R6fVcZQQAdUq4DtNe3Hth1yzjitbHJF6w0m /PnbjrNqAkgwu3fICJmKZqs7aDCwYiDECDxtrsKDEg3KNRA01Nl5eiN5ly0CSSEnwQeti3ecz hn55aaPaIPF+zVM1J+sdClsI8O3unBPAcK4EsIJWuJMdyN4Yf4/GF3DU5bqAH5ml66T5bC+yZ iwevu34WyW6kFEaR5V/YVOLOWLgxSHROsAd1SSvCmLVXOiF3r4yVprk3IZR1Ffz3fJXWlcBwe 6oUDT+wgHjkfdIkiAQDts0ctsB2JXSpqlp36eh2RP0VAMyYBlbMgNLbAZuTiTzj0I+I921sVs 8jxqAKB1x+Q1/B+NZgGC9B168JwCe8lKpHWz4JXVblEegRdlPeOmq/Hr0Fow/w8zakXVHHwwJ xKNiKXAQWOyisfpt8fMLAkJiH0xWVo3lc1Yx4Odar3fne8YZLlYl9ekwlAqwND4yhwnajM7tv joo3FiN90evgRm+9J2PMQPnx6cCp8ypgXzIa/x8IgqcGUks8SAoSA6oal2cWV0kQNFJ5s8ZbC UbjTEzebFxbu48Qg Received-SPF: none client-ip=212.227.126.187; envelope-from=h.goebel@crazy-compilers.com; helo=mout.kundenserver.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1645606747; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=EjnoNI0HwYCPFuXuuaGJMgZ3Jt8newKIDmqILFDH1Lc=; b=p+UxDypJDlu+r1yI+Muwul8HuRokrrffW0fKYvqnjvu3R2jeZhq5NSA+F2Xy+l5+p/5DGz 4W0iwTu/paC9diXYyB4LvxSr+lytyLJR4yhu1A/GK4JwHt2YbUAPSO5j8yuqkWVELzAtaC k5EG0bAYcEML+NzFqg246NYeOYLfecKnIOpnGz5vYIobKtFdfvBi0Sz3YvvyQ4IUKIY0/M +UdBG2wtnToYRiQ7YOX+dxUuestwKcJ9CnvmbsXTj4XIGJl97U5rrNgeZbYctE+1cgs49O ria4+3D4GfaG76rAtrUonrTELg5BOPkPwIjO3J/CmmOKof2+nDQ5iDmt+CzWKw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1645606747; a=rsa-sha256; cv=none; b=ittiCbGKpESt/a3u+bI5xh2imXBEr5w33z0K1VDSNCtc0rHOqFnTqNbSKMWEiOUOBLa9di NDRwwcxSsqb+ICDpiHg4p9Aido43tNlrO5kxu9Aar+QY3ZtW5jrf6ZNvGcq9SeOkQuKAVs 5pGgDgoMGgAV+/12kiq6iwuXqwCgr3QvpR5k2OzXyuhAgT2IwMu2KUtoCTpGd0/odFNCyB aDDzroeg6+WiLhIMLrva5Auaf+9fSFAzm+VLl/EAHi632h3wtyck84MSMYes5P6fs9iPgO 9d2MjU/UmAYXAldwZXm9KeCBxQD3JNyByR/3yg1Pn3Or5OJcH/pPi1zeyggQhA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.93 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: E850D4414D X-Spam-Score: -4.93 X-Migadu-Scanner: scn0.migadu.com X-TUID: 1FdRLnOQ+gI5 Hi, TL;DR: What do you think about the idea of an „extracting dowloader“? I'm about pick up work on „extracting downloader“ and the rebar build system (for erlang), see for a first try, In the aforementioned issue some points came up regarding the basic design of the patch. Thus before starting to write code, I'd like to agree on a basic design. The basic idea behind „extracting downloader“ is as follows: Packages provided by hex.pm (the distribution repository for erlang and elixir packages) are tar-archives containing some meta-data files and the actual source (contents.tar.gz), see example below, So the ideas was to only store the contents.tar.gz (instead of requiring an additional unpacking step). In some earlier discussion someone mentioned, this could be interesting for ruby gems, too. Storing only the archive would allow to have the archive's hash as the "source"-hash and allow for easy validation of the hash. Anyhow, much of the complexity of the current implementation (see issue 51061) is caused by this idea, since the code needs to postbone hashing to after the download. Also In some earlier discussion Ludo (afair) brought up the point whether e.g. swh would be able provide a source-package if hased this way. What do you think about the idea of an „extracting dowloader“? Example for a package from hex.pm: $ wget https://repo.hex.pm/tarballs/getopt-1.0.2.tar … $ tar tvf getopt-1.0.2.tar -rw-r--r-- 0/0               1 2000-01-01 01:00 VERSION -rw-r--r-- 0/0              64 2000-01-01 01:00 CHECKSUM -rw-r--r-- 0/0             451 2000-01-01 01:00 metadata.config -rw-r--r-- 0/0           14513 2000-01-01 01:00 contents.tar.gz -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |