From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id IDxuAyGEtV98DAAA0tVLHw (envelope-from ) for ; Wed, 18 Nov 2020 20:29:21 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id dSC9OiCEtV/VHQAAbx9fmQ (envelope-from ) for ; Wed, 18 Nov 2020 20:29:20 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 524909401BF for ; Wed, 18 Nov 2020 20:29:20 +0000 (UTC) Received: from localhost ([::1]:36290 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kfU4t-0007si-83 for larch@yhetil.org; Wed, 18 Nov 2020 15:29:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kfU4d-0007sE-22 for guix-patches@gnu.org; Wed, 18 Nov 2020 15:29:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:53278) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kfU4c-0002cf-ES for guix-patches@gnu.org; Wed, 18 Nov 2020 15:29:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kfU4c-0002AL-Ba for guix-patches@gnu.org; Wed, 18 Nov 2020 15:29:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#44199] [PATCH 0/1] An origin method for GNUnet FS URI's Resent-From: Maxime Devos Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 18 Nov 2020 20:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 44199 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: zimoun , 44199@debbugs.gnu.org Received: via spool by 44199-submit@debbugs.gnu.org id=B44199.16057313038270 (code B ref 44199); Wed, 18 Nov 2020 20:29:02 +0000 Received: (at 44199) by debbugs.gnu.org; 18 Nov 2020 20:28:23 +0000 Received: from localhost ([127.0.0.1]:36591 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kfU3z-00029K-GG for submit@debbugs.gnu.org; Wed, 18 Nov 2020 15:28:23 -0500 Received: from rhcavuit03.kulnet.kuleuven.be ([134.58.240.136]:33126) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kfU3w-000299-DP for 44199@debbugs.gnu.org; Wed, 18 Nov 2020 15:28:22 -0500 X-KULeuven-Envelope-From: maxime.devos@student.kuleuven.be X-KULeuven-Scanned: Found to be clean X-KULeuven-ID: DB2AB120003.AC990 X-KULeuven-Information: Katholieke Universiteit Leuven Received: from icts-p-smtps-1.cc.kuleuven.be (icts-p-smtps-1e.kulnet.kuleuven.be [134.58.240.33]) by rhcavuit03.kulnet.kuleuven.be (Postfix) with ESMTP id DB2AB120003 for <44199@debbugs.gnu.org>; Wed, 18 Nov 2020 21:28:13 +0100 (CET) Received: from butterfly.local (178-119-10-153.access.telenet.be [178.119.10.153]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by icts-p-smtps-1.cc.kuleuven.be (Postfix) with ESMTPSA id A6FD940B2; Wed, 18 Nov 2020 21:28:13 +0100 (CET) Message-ID: X-Kuleuven: This mail passed the K.U.Leuven mailcluster From: Maxime Devos Date: Wed, 18 Nov 2020 21:28:05 +0100 In-Reply-To: <86361arvdh.fsf@gmail.com> References: <5c72bcb9c86934deda97d952eb5cd459e615b313.camel@student.kuleuven.be> <86blgn4wk5.fsf@gmail.com> <86361arvdh.fsf@gmail.com> Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-h8aFIE3Vr7uJXLQeEsxq" User-Agent: Evolution 3.34.2 MIME-Version: 1.0 X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -3.11 X-TUID: s8kNrvqe5k9j --=-h8aFIE3Vr7uJXLQeEsxq Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, (btw it's Maxim*e*, not Maxim. The =E2=80=98e=E2=80=99 isn't pronounced but= it's still written.) I'll try to address the various issues in separate e-mails. zimoun schreef op ma 16-11-2020 om 01:35 [+0100 > [snip] > From my understanding, this is a show stopper. It has to be solved first going further, IMHO. It is not possible to write manually the URI for all the packages. And as perhaps you read with the project =E2=80=99disassemble=E2=80=99, it is not straightforward. I agree! I see three straightforward answers to this. a) Fancy Write a GNUnet service using the DHT to map the hashes used in origin specifications (*) to URI's for the FS system. To let the local contribution to the DHT survive peer restarts, maintain a database (e.g. SQlite) of (Guix hash -> GNUnet hash) (^), that is expanded with each successful source (or binary) substitution or build. (Alternatively, as the DHT isn't anonymous, place hash -> GNUnet hash references into some well-known name space. Then hash lookup + FS should automatically be anonymous when desired.) Possible issues: time out behaviour, the DHT is not anonymous. Annoyance: probably requires extending the build daemon. Perhaps try regular downloads (e.g. via HTTP/S, ftp, ...) in parallel with the GNUnet download after a configurable delay? Perhaps use a well-known GNUnet FS namespace instead of the DHT for anonymous downloads? (*) Also usable for package outputs, if the hash of the output is used and not the hash of the outputs=20 (^) In case the database is full, delete some old entries b) Simple, slow introduction (no additional GNUnet services required) Extend (origin ...) with an optional gnunet-hash field. Adjust =E2=80=98guix download=E2=80=99, =E2=80=98guix refresh=E2=80=99 and = =E2=80=98guix import=E2=80=99 to emit the gnunet-hash (%) field. Plumb this field to the guix daemon somehow. Same approach is possible for IPFS. As packages are updated and new packages are defined, given sufficient time, there will be more packages with a gnunet-hash field than not. (%) Computing the gnunet-hash of a directory doesn't require a full-fledged GNUnet installation. My scheme-gnunet repository is not very far from the point where it can convert file trees + libextractor metadata into bytevectors, without depending on C gnunet. A TODO: different zlib's would produce different bytevectors --> different GNUnet hash --> perhaps always use a single version. A TODO (for nix archives on GNUnet): define EXTRACTOR_METATYPE_EXECUTABLE (or mimetype: application/x-executable). Perhaps use mimetype: x-inode/symlink (or something like that) as well? Repository URL: https://notabug.org/mdevos/scheme-gnunet c) Not scalable, but may reduce network traffic to ci.guix.gnu.org & co Like in a) keep a database of known (Guix hash -> GNUnet FS URI). Perhaps make this available through a web interface or git repository ... wait, this sounds familiar ... this seems to fit well into the =E2=80=98disarchive=E2=80=99 project! Greetings, Maxime --=-h8aFIE3Vr7uJXLQeEsxq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iJcEABYIAD8WIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCX7WD1SEcbWF4aW1lLmRl dm9zQHN0dWRlbnQua3VsZXV2ZW4uYmUACgkQSePuIhkXJe7bqAEAoDmT3o4g7bA2 KlNfK/0CvAylKeBL6s6A4G5/RSSuWHMBAJmOGs1KyOeYo3vigBmp8w4bwy5TnFj0 7/Q7srOIKCkF =qgHe -----END PGP SIGNATURE----- --=-h8aFIE3Vr7uJXLQeEsxq--