From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id aPF+NjwXFmLOsAAAgWs5BA (envelope-from ) for ; Wed, 23 Feb 2022 12:15:08 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id WCkTNDwXFmKcegEA9RJhRA (envelope-from ) for ; Wed, 23 Feb 2022 12:15:08 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D9140DB15 for ; Wed, 23 Feb 2022 12:14:49 +0100 (CET) Received: from localhost ([::1]:43444 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nMpbd-0005iH-0E for larch@yhetil.org; Wed, 23 Feb 2022 06:14:49 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54338) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nMpak-0005eh-Gr for guix-devel@gnu.org; Wed, 23 Feb 2022 06:13:55 -0500 Received: from mout02.posteo.de ([185.67.36.66]:33439) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nMpah-0007az-Ov for guix-devel@gnu.org; Wed, 23 Feb 2022 06:13:54 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 0AFAA240101 for ; Wed, 23 Feb 2022 12:13:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1645614828; bh=Frq8esxNxY4CkVYC+upUAkyHN6pRyq5jXfPXJhRjmGg=; h=From:To:Cc:Subject:Date:From; b=TeXFTEXNJvSc7EVyfd0wHsBiioUvs48nmDg2zy/8sitUc06c+P0N5te1cH2DkVHA9 ahUuY2oN4ll8SmvB3gvI6KQHl+0OkeMP7vnJLHCSfB0+X+yfIGmYYRKwRsLIySgLq9 +N/VKq1ec/N+rBwbwzBGQzN7LsEIV/wWGFsxdgzTAxEInbYXA7//ljo7YJDNsL/DGI BFfateiZEDs/Kp32NvMSMmd4+otSB8baseuvvNtMseLreVytm0lXxNyu+6c9OGdPnX cA8sVJj6sj/z+16JnLUTuPW9GLx9yBc0TI8AXN+aZXAkjs9YQvSfJ0jCZrffTTEzln mlJsr1Yriqwww== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4K3YLz2wBhz9rxR; Wed, 23 Feb 2022 12:13:47 +0100 (CET) References: From: pukkamustard To: Hartmut Goebel Subject: Re: (Re-) Designing extractong-downaloder Date: Wed, 23 Feb 2022 10:52:40 +0000 In-reply-to: Message-ID: <86y221vnrp.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.67.36.66; envelope-from=pukkamustard@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1645614890; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=Frq8esxNxY4CkVYC+upUAkyHN6pRyq5jXfPXJhRjmGg=; b=llorTp6H4WPrWe0m1YJFG3deNwVvu2S84XtVLJewbzHXHk/uXOYjN0/j5TgqfPWmySSfu+ zpPp8yY+vtZ9TnU71F3jtYDdzi4JxLSTlIVYd9/mncaspMIjwHTJKtN8iMO/NtCANsaUe4 az90xZmJ1MbfSgCr3oUd7RL+I2yFcVO9GeZRIArrjlSn/BfDtiL11FjWREjkGtivNFprts bmzwW0tV2JRUnfe9wTuEtkX9emAwWh+55tzY9izdqtUyj6iUqV7kAMQPS03bIhf/36i4IH 8BZ3e1hSc1kRPbbj9AHCxNdhXgWBC2KNRjHuY51IUiXZTT2YG9Fk7DkUcNuRVw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1645614890; a=rsa-sha256; cv=none; b=QA6FyUjngPC1MAEmql+lpIFWGLtpk8yHXQhxuVorpcR0Td4h+v53HeVeYw26lraQMzV2f4 EgFFK2gGtBFa3irup7gm42aAxoFVmA4aaGw4htKj4bjoSJ8z5NxGHiLYAd0PHFlIJd1CvK ikDLbVlozMnH0qkb3/TtxfqZSUHs3A4QHm5rzlGZV8FwrjObeAmR6JX5Azvy7/+plQhg26 cG74zEN3DSx2kS+hPEBw0ZN2Y3a1yo2T4Qf+LwPwtR+WGw+ObO+YHNWvW3vg4a6I4WS0yX ngOcdEsYGMdo1tXYhTqude/YJY3vblBlnAve6Qh5ZaUD8/yrDd/wwB2E3fZSjg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=TeXFTEXN; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -9.83 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=TeXFTEXN; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: D9140DB15 X-Spam-Score: -9.83 X-Migadu-Scanner: scn0.migadu.com X-TUID: e0O1tGtmnmaj Hi Hartmut, Hartmut Goebel writes: > I'm about pick up work on =E2=80=9Eextracting downloader=E2=80=9C and the= rebar build > system (for erlang), I'm very much looking forward to this! > The basic idea behind =E2=80=9Eextracting downloader=E2=80=9C is as follo= ws: Packages > provided by hex.pm (the distribution repository for erlang and elixir > packages) are tar-archives containing some meta-data files and the > actual source (contents.tar.gz), see example below, So the ideas was > to only store the contents.tar.gz (instead of requiring an additional > unpacking step). Why use the source from hex.pm at all? Would it be possible to just fetch the hex.pm archive when importing a package, read the metadata.config file and then try and use upstream source (e.g. GitHub)? The hex.pm metadata.config file does not seem to exactly specify the upstream source. We would need some heuristics to figure this out. But maybe we could find a heuristic that works well enough? This would solve the double-archive problem. For packages where the heuristics fails we fallback and use the source as provided from hex.pm (unextracted) and use an additional build phase to do the double extraction? If this only affects a few packages then storing the source double-archived does not seem so bad. Thanks, pukkamustard