From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id yPm9FhyIF1+5dAAA0tVLHw (envelope-from ) for ; Wed, 22 Jul 2020 00:28:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id EDKhEhyIF190NAAAbx9fmQ (envelope-from ) for ; Wed, 22 Jul 2020 00:28:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BAF1C9403AA for ; Wed, 22 Jul 2020 00:28:11 +0000 (UTC) Received: from localhost ([::1]:38260 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jy2cD-00087O-LZ for larch@yhetil.org; Tue, 21 Jul 2020 20:28:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60466) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jy2c6-000872-NM for bug-guix@gnu.org; Tue, 21 Jul 2020 20:28:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:57437) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jy2c6-00063s-E7 for bug-guix@gnu.org; Tue, 21 Jul 2020 20:28:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jy2c6-00063o-BR for bug-guix@gnu.org; Tue, 21 Jul 2020 20:28:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#42162: Recovering source tarballs Resent-From: zimoun Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 22 Jul 2020 00:28:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42162 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 42162-submit@debbugs.gnu.org id=B42162.159537767623281 (code B ref 42162); Wed, 22 Jul 2020 00:28:02 +0000 Received: (at 42162) by debbugs.gnu.org; 22 Jul 2020 00:27:56 +0000 Received: from localhost ([127.0.0.1]:40750 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jy2bt-00063L-BZ for submit@debbugs.gnu.org; Tue, 21 Jul 2020 20:27:56 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:54287) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jy2br-000636-Gv for 42162@debbugs.gnu.org; Tue, 21 Jul 2020 20:27:48 -0400 Received: by mail-wm1-f65.google.com with SMTP id o8so289597wmh.4 for <42162@debbugs.gnu.org>; Tue, 21 Jul 2020 17:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=5QDd6uSCj/FVK2K/8I8yUUUC999qgy7poMJ7qfKgHLs=; b=sKSAyKWyF4pXMj+3o3AdG31lCcGxQeYGu6mJBum2KaZH+xvdUvdAxbNlwtyTv0qs6B Dr48xconj0PQWoKZYxzhc86q2idMtLeP0D0ZXziHpinjIIYbcuLm4ySYYz8egDsFcJ9/ L41CfBjRy/jYddVCdvkZR8XPqYBeaJCObV15qXuY4JkVaBVgW36W1x9XWQwSxijQNdNw szWKG7i7BXP3m5vMTmh0Hzkgqy6nk5JBXhdt17ccnWJdLDknUM1bGKUQ9HARuHl9H/iS IyMqqJYMoCmh2HkovOnzAUZzGU64O3DPvwLqWdI+C1xT5BZ8LCZS852bZXuFYm1pDAwU D61w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=5QDd6uSCj/FVK2K/8I8yUUUC999qgy7poMJ7qfKgHLs=; b=tl68hJgbGyWhYNZc8IDQ0Eh1HH5Xtf3Sy1GrBvufPObt2SrSirE+nqMTzOTHJMqJHl er12q1/gPTC71IuLqTsrK0ioeyfvXATSnCLsntqsHaDpIshhnka+9iBEM/niTexjUmH5 77OUrkV6liSTWQW9t/rh+6gQWjui8SQD+mMsZpRgUEaHFtif7HScXp0qmoETtfe92+6v 3Hj16qSnGMWnhpc7iEYLvavvxys3h1q2bfQnwvCdO+RuJAcCqWODe6d2kxy0gFB/1uei P0sMIaV888t++RWHf165zHIM8NMJw7wSqCC8aEDBCIkakgaE5P1EGYitS1Cf+V/JRKU7 rGGg== X-Gm-Message-State: AOAM5314fkf3ImDfl212ILTgxb/tUa3dWuPNdhLYLz0Ln6yz5iTLmRNx B2ow+8FxCW7Cz7TA0O1I/HM= X-Google-Smtp-Source: ABdhPJwyUfs50SFuLVSeEItPz1RVrh0T40zvaVI2HlPmzwKYQx1DG1IlQ0itgbI7kMACEXz/xI0hrg== X-Received: by 2002:a1c:9914:: with SMTP id b20mr6040325wme.15.1595377661599; Tue, 21 Jul 2020 17:27:41 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id n3sm29546791wre.29.2020.07.21.17.27.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2020 17:27:40 -0700 (PDT) From: zimoun In-Reply-To: <87k0ywlg1z.fsf@gnu.org> References: <87mu4iv0gc.fsf@inria.fr> <86h7uq8fmk.fsf@gmail.com> <87d05etero.fsf@gnu.org> <87r1tit5j6.fsf_-_@gnu.org> <87365mzil1.fsf@gnu.org> <87k0ywlg1z.fsf@gnu.org> Date: Wed, 22 Jul 2020 02:27:39 +0200 Message-ID: <86o8o81jic.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 42162@debbugs.gnu.org, Maurice =?UTF-8?Q?Br=C3=A9mond?= Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=sKSAyKWy; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: E8ivOF4mX2Eh Hi! On Tue, 21 Jul 2020 at 23:22, Ludovic Court=C3=A8s wrote: >>> >> =E2=80=A2 If we no longer deal with tarballs but upstreams keep si= gning >>> >> tarballs (not raw directory hashes), how can we authenticate our >>> >> code after the fact? >>> > >>> > Does Guix automatically authenticate code using signed tarballs? >>> >>> Not automatically; packagers are supposed to authenticate code when they >>> add a package (=E2=80=98guix refresh -u=E2=80=99 does that automaticall= y). >> >> So I miss the point of having this authentication information in the >> future where upstream has disappeared. > > What I meant above, is that often, what we have is things like detached > signatures of raw tarballs, or documents referring to a tarball hash: > > https://sympa.inria.fr/sympa/arc/swh-devel/2016-07/msg00009.html I still miss why it matters to store detached signature of raw tarballs. The authentication is done now (at package time and/or inclusion in the lookup table proposal). I miss why we would have to re-authenticate again later. IMHO, having a lookup table that returns the signatures from a tarball hash or an archive of all the OpenGPG keys ever published is another topic. >>> But today, we store tarball hashes, not directory hashes. >> >> We store what "guix hash" returns. ;-) >> So it is easy to migrate from tarball hashes to whatever else. :-) > > True, but that other thing, as it stands, would be a nar hash (like for > =E2=80=98git-fetch=E2=80=99), not a Git-tree hash (what SWH uses). Ok, now I am totally convinced that a lookup table is The Right Thing=E2=84= =A2. :-) >> I mean, it is "(sha256 (base32" and it is easy to have also >> "(sha256-tree (base32" or something like that. > > Right, but that first and foremost requires daemon support. > > It=E2=80=99s doable, but migration would have to take a long time, since = this is > touching core parts of the =E2=80=9Cprotocol=E2=80=9D. Doable but not necessary tractable. :-) >> I have not done yet the clear back-to-envelop computations. Roughly, >> there are ~23 commits on average per day updating packages, so say 70% >> of them are url-fetch, it is ~16 new tarballs per day, on average. >> How the model using a Git-repo will scale? Because, naively the >> output of "disassemble-archive" in full text (pretty-print format) for >> the hello-2.10.tar is 120KB and so 16*365*120K =3D ~700Mb per year >> without considering all the Git internals. Obviously, it depends on >> the number of files and I do not know if hello is a representative >> example. > > Interesting, thanks for making that calculation! We could make the > format more compact if needed. Compressing should help. Considering 14000 packages, based on this 120KB estimation, it leads to: 0.7*14k*120K=3D ~1.2GB for the Git-repo of the current Guix. Cheers, simon