From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id gIECJAu3IGS0IQAASxT56A (envelope-from ) for ; Sun, 26 Mar 2023 23:20:11 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id IL1UIwu3IGQdrAAAG6o9tA (envelope-from ) for ; Sun, 26 Mar 2023 23:20:11 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 36BE144CBC for ; Sun, 26 Mar 2023 23:20:11 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pgXm9-0008VS-4g; Sun, 26 Mar 2023 17:19:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pgXm6-0008V7-Od for guix-devel@gnu.org; Sun, 26 Mar 2023 17:19:38 -0400 Received: from mail-4323.proton.ch ([185.70.43.23]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pgXm3-0003G4-6e for guix-devel@gnu.org; Sun, 26 Mar 2023 17:19:38 -0400 Date: Sun, 26 Mar 2023 21:19:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lendvai.name; s=protonmail2; t=1679865570; x=1680124770; bh=THUkCEWV9SfTMrpPDZVn/cxWYvhxCrp2QGWqkGYSft8=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=oCLidVgbEI/VOZo+OixkvZZbNSsbrayHhvTrSc8a3laNuY4LY5r3HSi/dAxVNHB3P 6m8uD4qhuS7HmOBzEyTYEXMvvHj+bqpJlS3cTv0sFsIYeBJloKQ2kODeYe/koMg4EP Vd3koXmgViFx4dwCJB1yV2wALfQKz0/Gkghzl8K7S91SVS3VeU/CF2qD8RoQ8R51s7 8vAlftst5lDGyIQYbf4q/zuh3u+B8M7RpYjqO+iPUeRBIpUgBlyBg7NKxbumHSIlFS xLvT0z/EB3+WcUqYrmJHYb4FIFKwAssfYEFtnIdB58EmhTNoqn7S8+BOTvdbcIZVXe Yv23d42kjloZA== To: Vijaya Anand From: Attila Lendvai Cc: pukkamustard , guix-devel@gnu.org Subject: Re: [GSoC 23] distributed substitutes, cost of storage Message-ID: In-Reply-To: References: Feedback-ID: 28384833:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.70.43.23; envelope-from=attila@lendvai.name; helo=mail-4323.proton.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1679865611; a=rsa-sha256; cv=none; b=GgBdaOjcH5HT1ARnHlHWbxEUImE3fLW2s0D2zQp2crns88sFtfj7PxcqUzlcg6CBSnbyWx kccyOHOVbvm00mm/csFZMo7NNzjOxLtPECmnBFXnPyMqr+0JzbF67ZNxEaaEirJcZO72Mi tGmkeJOt+zMWuguKoDJHOfb1lW1tjqcXoKu0hr7Pq3tjmwd5yMMijXBjeEzBJMAwDCVBpv R+WHwMAJXFYpVbAYwv/oWIneus+ygLUet7HksYrcoGHs3TPy7B3djcFtZO6gMD1/Ek6W0C FM+uslWbKFKuV8WwPuR7kAPDkm1V1eworQ4O81WnvJvS+GuExuXZVi6lg3do4A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=lendvai.name header.s=protonmail2 header.b=oCLidVgb; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1679865611; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=THUkCEWV9SfTMrpPDZVn/cxWYvhxCrp2QGWqkGYSft8=; b=tJ87sq0UuSdzBiz9ZqiN58LvBytNt+OEbDvTmDHEG8a5Gl157igu49lc/6U8hMddcyWWHm /7MD87/8cKACTKDNYwzXpvke0HHAA7822wuSxysqE1YzkJhI1Uc6BvuIFiBWHk2BTkTYc0 qaHsmHrslT8dYSv+ke/+sc5HMKd3cpY66nMmLTmtuZsmiL4iku1pyCGit4jC+COm78PPWm CnfFK4EgWwHF7sgrXRGrUM2JEeqT6TrwMrz8gZKN5rH0ExUOr2AjuHNTwjmmYcNwSQa2GL PgFHwh/iY2H46TGfrnZcG7dIBheJ5QRtsx2EPfXvknlWDmz9BXQRTMG0U4kgzg== Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=lendvai.name header.s=protonmail2 header.b=oCLidVgb; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -2.23 X-Spam-Score: -2.23 X-Migadu-Queue-Id: 36BE144CBC X-TUID: 6oYwHmIruZ6f > Also I didn't really think about the point about having to pay for > the p2p services at some point of time. a quick note here: i forgot to mention that e.g. the Swarm Foundation has p= rograms for supporting opensource projects. so, chances are high that the s= torage needs for Guix would be paid for by the foundation. > In this case we will have to pay for the storage of substitutes both > on the p2p storage backend as well as for storage in the substitute > server am I right? well, the substitute servers are currently already operated (and paid for) = by the Guix team. i don't think that p2p storage solutions have reached a p= oint of maturity that we could rely on them alone. there should definitely = be some time where both infrastructures are running in parallel. somewhere = down the road a choice could be made to stop running the current substitute= servers, but we are far away from that. also, running swarm nodes that serve the network can earn money. so, the co= st of running enough swarm nodes to pay for the storage needs of Guix on th= e swarm network should be in the same ballpark of the costs of running the = current substitute servers. the storage price will be market based (this pa= rt is just being rolled out on the live network), so it's reasonable to exp= ect that people will fire up nodes if the storage price go well above the V= PS costs. and not all p2p storage networks are made equal. e.g. IPFS is only a regist= ry of who is serving what. if you want to keep your data alive on IPFS, the= n you need to run some nodes and make sure that they are serving the conten= t that you care about... and bear the costs of running these nodes. i.e. th= e DoS attack surface of IPFS is much smaller. (IPFS stores only the metadat= a in the DHT (i.e. where is what), while Swarm stores there the data itself= -- they are different architectures with different features) (i need to learn more about GNUnet) > So ideally we will want to eliminate the usage of these substitute > servers and shift totally to p2p services and in this case we will > have to shift the responsibility of re-uploading the blocks to the > clients itself. yep, that's my way of thinking, too. note that 'client' here has two meanings: 1) some part of the codebase 2) a program that is running on the computers of the Guix users i was using it in the first sense. without a functional Web of Trust solution, the Guix team will have to run = nodes that compile packages, sign them with their PGP keys, and make them a= vailable somewhere. currently it's published through a HTTP based service t= hat we call 'substitute servers'. this GSoC project is about adding more st= orage backends. but those backends don't solve the problem that the Guix users need to trus= t someone with a private key who compiles and signs packages, regadless of = the transport mechanism that gets the packages to the clients. i can dream about a future where there's a social network that is based on = digital signatures and encryption, and my Guix client authorizes compiled b= inaries based on some weighted transitive closure of signatures of my trust= ed peers... but we are not there yet. for now it's either trusting the Guix= team's signature, or setting up your own substitute servers and build work= ers (or trusting someone else, but i'm not aware of any third party offerin= g substitutes). > Also if we dont keep the re-uploading blocks option as default for > the users, won't users usually not choose to enable it? Maybe we can > keep it on as default and resource conscious users can choose to > turn it off? Please let me know your thoughts on these points and I > will change the implementation point of my proposal accordingly. first, it's a philosophical question: i value consensual relationships, and= that implies that the other party is well informed. and clogging someone's= network bandwidth is not an expected behavior from installing a linux dist= ribution. there's also a technical issue: these p2p backends will need to be configur= ed. i have my doubts that we could ship a default config with Guix with whi= ch these p2p backends could just work out of the box, but... let's hope tha= t i'm wrong! and then there's the issue of payments: it's not obvious that a random clie= nt can just upload binaries into a p2p storage network. on some p2p network= s someone needs to pay for that, and i don't yet understand well enough how= the data is authorized through payments (they are called postage stamps in= swarm). a good, high level comparison of p2p storage solutions would be useful. --=20 =E2=80=A2 attila lendvai =E2=80=A2 PGP: 963F 5D5F 45C7 DFCD 0A39 -- =E2=80=9CThere is something in the human spirit that will survive and preva= il, there is a tiny and brilliant light burning in the heart of man that wi= ll not go out no matter how dark the world becomes.=E2=80=9D =09=E2=80=94 Leo Tolstoy (1828=E2=80=931910)