From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 4AMQAkYBK19tKwAA0tVLHw (envelope-from ) for ; Wed, 05 Aug 2020 18:58:14 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id GJ6DOUUBK1+MFAAA1q6Kng (envelope-from ) for ; Wed, 05 Aug 2020 18:58:13 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4C7BE94051F for ; Wed, 5 Aug 2020 18:58:13 +0000 (UTC) Received: from localhost ([::1]:47788 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k3Oc6-0006Ty-UE for larch@yhetil.org; Wed, 05 Aug 2020 14:58:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57714) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k3Oby-0006TR-46 for bug-guix@gnu.org; Wed, 05 Aug 2020 14:58:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:40710) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1k3Obx-0002jz-RK for bug-guix@gnu.org; Wed, 05 Aug 2020 14:58:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1k3Obx-0006Zj-Ou for bug-guix@gnu.org; Wed, 05 Aug 2020 14:58:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#42162: Recovering source tarballs Resent-From: Timothy Sample Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 05 Aug 2020 18:58:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42162 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 42162-submit@debbugs.gnu.org id=B42162.159665386225250 (code B ref 42162); Wed, 05 Aug 2020 18:58:01 +0000 Received: (at 42162) by debbugs.gnu.org; 5 Aug 2020 18:57:42 +0000 Received: from localhost ([127.0.0.1]:52256 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k3Obd-0006ZC-Uw for submit@debbugs.gnu.org; Wed, 05 Aug 2020 14:57:42 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:37231) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k3Obb-0006Yy-EN for 42162@debbugs.gnu.org; Wed, 05 Aug 2020 14:57:40 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 8951CB14; Wed, 5 Aug 2020 14:57:33 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 05 Aug 2020 14:57:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=nd+7KPFLKTX6zY0yRN0xhTsAIdfOgFgJup8VS6osX Fs=; b=X425oK7UPAkzbxkpgDtVnkyCRVEfrJ3fDdzPERjLEapQ4ePSWoKXtpeyT ftSwlIaqOHn1UcuJHBebyN1Hkz2pb1idMJHFBonFcUos3kT/sq+poqtYLRoPhojt L9ujhzXc+TvfJrV3GOz5/bglgJdPN1zAsAiwUXadu9Iw6zORwRtatz82e2AQZk5v KigmjyhaY6c6i02TGh6osmn9ydeWjYhsUD8ijoFhyJEthelogIuwlHGzcOfkB6dK K9lCq3x8emAf8i9de6XwfKEoGZ1A415uD1fXUvUeciMvd/I19aVuKYYhZur16LGV on53/w5Z2aJ3uhfQiF7XCPZ1zNohA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrjeekgddufeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufhfffgjkfgfgggtgfesthhqredttderjeenucfhrhhomhepvfhimhho thhhhicuufgrmhhplhgvuceoshgrmhhplhgvthesnhhghihrohdrtghomheqnecuggftrf grthhtvghrnhepfefhgeegvefhteejheehffeutddthffhtefhgefhgfdvleekffekhfeu veevleffnecuffhomhgrihhnpehgnhhurdhorhhgpdgvgigrmhhplhgvrdhorhhgnecukf hppeejgedrudduiedrudekiedrgeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepshgrmhhplhgvthesnhhghihrohdrtghomh X-ME-Proxy: Received: from mrblack (74-116-186-44.qc.dsl.ebox.net [74.116.186.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 27710306005F; Wed, 5 Aug 2020 14:57:32 -0400 (EDT) From: Timothy Sample References: <87mu4iv0gc.fsf@inria.fr> <86h7uq8fmk.fsf@gmail.com> <87d05etero.fsf@gnu.org> <87r1tit5j6.fsf_-_@gnu.org> <875za4ykej.fsf@ngyro.com> <87bljvu4p4.fsf@gnu.org> <87d047u0l3.fsf@ngyro.com> <87wo2dnhgb.fsf@gnu.org> Date: Wed, 05 Aug 2020 14:57:31 -0400 In-Reply-To: <87wo2dnhgb.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Wed, 05 Aug 2020 19:14:12 +0200") Message-ID: <874kpgudic.fsf@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.7 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 42162@debbugs.gnu.org, Maurice =?UTF-8?Q?Br=C3=A9mond?= Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=messagingengine.com header.s=fm3 header.b=X425oK7U; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -0.01 X-TUID: TDyTJy7drLmv Hey, Ludovic Court=C3=A8s writes: > Note that we have . Last I checked, > SWH was ingesting it in its =E2=80=9Cqualification=E2=80=9D instance, so = it should be > ingesting it for good real soon if it=E2=80=99s not doing it already. Oh fantastic! I was going to volunteer to do it, so that=E2=80=99s one thi= ng off my list. > One can easily write a procedure that takes a tarball and returns a > that builds its database entry. So at each commit, we=E2= =80=99d > just rebuild things that have changed. That makes more sense. I will give this a shot soon. > If we expose the database over HTTP (like over cgit), we can arrange so > that (guix download) simply GETs db.example.org/sha256/xyz. No need to > fetch the whole database. > > It might be more reasonable to have a real database and a real service > around it, I=E2=80=99m sure Chris Baines would agree ;-), but we can choo= se URLs > that could easily be implemented by a =E2=80=9Creal=E2=80=9D service inst= ead of cgit in > the future. I got it working over cgit shortly after sending my last message. :) So far, I am very much on team =E2=80=9Cgood enough for now=E2=80=9D. > Timothy Sample skribis: > >> I was imagining an escape hatch beyond this, where one could look up a >> provenance record from when Disarchive ingested and verified a source >> code archive. The provenance record would tell you which version of >> Guix was used when saving the archive, so you could try your luck with >> using =E2=80=9Cguix time-machine=E2=80=9D to reproduce Disarchive=E2=80= =99s original >> computation. If we perform database migrations, you would need to >> travel back in time in the database, too. The idea is that you could >> work around breakages in Disarchive automatically using the Power of >> Guix=E2=84=A2. Just a stray thought, really. > > Seems to me it Shouldn=E2=80=99t Be Necessary? :-) > > I mean, as long as the format is extensible and =E2=80=9Cfuture-proof=E2= =80=9D, we=E2=80=99ll > always be able to rebuild tarballs and then re-disassemble them if we > need to compute new hashes or whatever. If Disarchive relies on external compressors, there=E2=80=99s an outside ch= ance that those compressors could change under our feet. In that case, one would want to be able to track down exactly which version of XZ was used when Disarchive verified that it could reassemble a given source archive. Maybe I=E2=80=99m being paranoid, but if the database entries are being computed by the CI infrastructure it would be pretty easy to note the Guix commit just in case. > I was thinking that it might be best to not use Guix for computations. > For example, have =E2=80=9Cdisarchive save=E2=80=9D not build derivations= and instead do > everything =E2=80=9Chere and now=E2=80=9D. That would make it easier for= others to > adopt. Wait, looking at the Git history, it looks like you already > addressed that point, neat. :-) Since my last message I managed to remove Guix as dependency completely. Right now it loads =E2=80=98(guix swh)=E2=80=99 opportunistically, but I mi= ght just copy the code in. Directory references now support multiple =E2=80=9Caddresses= =E2=80=9D so that you could have Nix-style, SWH-style, IPFS-style, etc. Hopefully my next message will have a WIP patch enabling Guix to use Disarchive! -- Tim