From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id iLJqHnJlxmakfgEAqHPOHw:P1 (envelope-from ) for ; Wed, 21 Aug 2024 22:08:50 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id iLJqHnJlxmakfgEAqHPOHw (envelope-from ) for ; Thu, 22 Aug 2024 00:08:50 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=dustycloud.org header.s=fm1 header.b=bLfAyMVs; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="o XfG12l"; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1724278130; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=MsGcyonGr0nV4+NbgsjcIYn9MG4/V3sU3ZmTgiFlDXA=; b=d0BgzO0edI1QkWwQ6WdCyD6LaYqYFzRejkMkdrahiprLmdBmJvoRdWSZYv6bEr95hc9Dy5 ZGjAE4NcjYhcbui4DA4PDPKJsG/eogsFxPplsCWXTNDIjRuQbF8FOeA9kQeaxbiY21y8e2 /ulVsfhd4P1u7ZQJhyyDwUeUNFyraj+k14h7xEZqxmcxFPNXnm8m4WlrEmqN8n+i+wDX7A DX4CZdPGB0VF3Tyce2tBmyR+qpQS6Rn+AlRCR3XYUxN/Ece34VBgTE+FPBs6NwqYxcP3UG pywVOo4AQtZxNk9oxcQ5jo4JpdKVznoebqRVeZk97j3qYXnmuVGCzXzvgHB+Dw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=dustycloud.org header.s=fm1 header.b=bLfAyMVs; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="o XfG12l"; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Seal: i=1; s=key1; d=yhetil.org; t=1724278130; a=rsa-sha256; cv=none; b=VI0XYa3c7EoHkyEb/O5gS5raWhLxliMzsSfTvIJGHlbQ3imvn4lMAkdCZzeaOYwVyOo8NA mfqZeCxhMClgGBx2rHLc03m8/ZH4fHzQf1QXdGGo7v5osAKU2UnSeuh7wjpqBRFempiz8I czBQp2CvqX+E9HRt0xH7p0OZE14qkrbOPCyMYj2nWwUCwo26JD4pN8QQ4yKDL5WEtSnlwI H5zgknBTZ5PEohf8KJ/SrF+DBWQ4AMseZqWa4LtBSiSGIqna7himAb0C5i6AOJ1Wuum3Zr fzaAl7YjTE2vfP/j/1w5UW9uB7IQY8Xb5EU4xQ7ee3bp2+17RKX1cLnjnkAL0w== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4E9FB7C42C for ; Thu, 22 Aug 2024 00:08:50 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sgtVA-0008JQ-BW; Wed, 21 Aug 2024 18:08:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sgtV8-0008D7-GE; Wed, 21 Aug 2024 18:08:22 -0400 Received: from fhigh4-smtp.messagingengine.com ([103.168.172.155]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sgtV4-0001Wb-E9; Wed, 21 Aug 2024 18:08:22 -0400 Received: from phl-compute-02.internal (phl-compute-02.nyi.internal [10.202.2.42]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 736D5114829A; Wed, 21 Aug 2024 18:08:14 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-02.internal (MEProxy); Wed, 21 Aug 2024 18:08:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dustycloud.org; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm1; t=1724278094; x=1724364494; bh=MsGcyonGr0nV4+NbgsjcIYn9MG4/V3sU 3ZmTgiFlDXA=; b=bLfAyMVsPskFpdMFX5DyWwEgpiuWnfQPR5Rwv4EKwii1T103 S3qukv34V2yyOT1ot8LKB3Di7xlNkDoXo9ulAx7srVNrKmzCW/dK2dbDB3PHeEvU wC7dm2UkU0IpqAE2K2wVCjAAOqcz6Lfjzk0b9npRkHipkNaRhrwzgjmW96qQTOUe Matzd2VCMRHWclvMIr0NMNUeWptLWtJhC17wp/NqgUHUcdO83LmkoEQl2vtY0082 LQORXzIqioNhcL/RUCjI94THvf2Qjk1m1eX+CS5aJEj633cyN9H9pJpsihUHtXwz 5SLhDrFkWlykd0VfpwWtm4qB//N/iSxqsmD+Cg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1724278094; x= 1724364494; bh=MsGcyonGr0nV4+NbgsjcIYn9MG4/V3sU3ZmTgiFlDXA=; b=o XfG12lqd0bk5eW6KHR/RAwdVei64RxwGeOAJJgL5esivuoHwiAtplh9zlCn5u8WJ jSMKcL3H8w1onnfzX0BF6tPPkFYBCDXON7CROhsxLfnyqjKkQJJn8z3OlS2zyB8K /v9IJeH5/Qi0wvpzF9VPw5j4FqFvjbQ/yrjn8ogLVYV/4pwJDLAzN2sTsbFddWWx q8aPuW4GXY8NAp9brmQYX7IMs7gj57nvLXnmVTpwQRj3s3t6/U+KtMZzoy7221p0 dL8bX8TYt1FkUpVQ9sV+ZXbd00FF3eaR9A4xE24VT/p3q07/SEWXADBD2BQfA+Y4 fAwwnIGAnz4KvUGKrAZBQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrudduledgtdejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephffvvefujghffgffkfggtgfgsehtqhertddtreej necuhfhrohhmpeevhhhrihhsthhinhgvucfnvghmmhgvrhdqhggvsggsvghruceotgifvg gssggvrhesughushhthigtlhhouhgurdhorhhgqeenucggtffrrghtthgvrhhnpeeukeet gfegudetgfeugedtveehveejkeffhfejgedvteffkeeggfegteffvdelheenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegtfigvsggsvghrsegu uhhsthihtghlohhuugdrohhrghdpnhgspghrtghpthhtohepiedpmhhouggvpehsmhhtph houhhtpdhrtghpthhtohepghhuihigqdhshihsrggumhhinhesghhnuhdrohhrghdprhgt phhtthhopehguhhigidquggvvhgvlhesghhnuhdrohhrghdprhgtphhtthhopehluhguoh esghhnuhdrohhrghdprhgtphhtthhopehmrghrvghksehmrghrvghkphgrshhnihhkohif shhkihdrphhlpdhrtghpthhtohepshgvrhhgihhordhprghsthhorhhpvghrvgiisehouh htlhhoohhkrdgvshdprhgtphhtthhopehjohhnrghthhgrnhesthgvrhhrrggtrhihphht rdhnvght X-ME-Proxy: Feedback-ID: i006446df:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 21 Aug 2024 18:08:05 -0400 (EDT) From: Christine Lemmer-Webber To: "Jonathan Frederickson" Cc: Sergio Pastor =?utf-8?Q?P=C3=A9rez?= , Marek =?utf-8?Q?Pa=C5=9Bnikowski?= , Ludovic =?utf-8?Q?Court=C3=A8s?= , guix-devel@gnu.org, guix-sysadmin Subject: P2P Guix package building and distribution In-Reply-To: (Jonathan Frederickson's message of "Tue, 13 Aug 2024 19:38:41 -0400") References: <87sewr98jd.fsf@gnu.org> <87sevnhp02.fsf@marekpasnikowski.pl> <3ad5baad-2ab6-4fa4-8788-717f827ccf86@app.fastmail.com> User-Agent: mu4e 1.12.4; emacs 29.3 Date: Wed, 21 Aug 2024 18:07:58 -0400 Message-ID: <87msl5o7gh.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=103.168.172.155; envelope-from=cwebber@dustycloud.org; helo=fhigh4-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam-Score: -9.55 X-Migadu-Queue-Id: 4E9FB7C42C X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -9.55 X-TUID: NPZ154MsporS "Jonathan Frederickson" writes: > On Tue, Aug 13, 2024, at 12:23 PM, Sergio Pastor P=C3=A9rez wrote: > >> Wouldn't it be enough to have a few independent seeders that have the >> same derivation output? We could have a field in the p2p service type >> which allows the user to configure a "level of trust", where the user >> specifies the minimum number of seeders with the same output for the >> daemon to accept the substitute. > > This might be enough if you could do it, but the trouble is > identifying "independent" seeders. If you get the same output from > five different seeders, that could be five different people... or I > could have set up five different nodes participating in the swarm > serving my malicious substitutes. (This is known as a Sibyl attack.) > > But maybe taking inspiration from this... perhaps you could do > something more akin to some of the web-of-trust features of > e.g. PGP. In other words, you might have the ability to partially > trust a server's substitutes such that you'll only use a substitute if > N other partially trusted servers (or at least one fully trusted > server) serve up the same content. This would still not let you have a > totally permissionless set of P2P substitutes, but it would allow the > community to build a list of individuals who are at least trusted not > to collude with one another, if not fully trusted. > > Though there's a detail that might need addressing for this to > work... you would want this to be an indication that multiple > individuals were able to reproducibly build the same packages > bit-for-bit. But my impression is that substitutes served by 'guix > publish' are always signed with the substitute server's signing key, > regardless of where they were built. That does mean that if 4 people > were to pull substitutes of a package from one other person, those 5 > people would end up serving substitutes originating from one > person. You may want a way for someone running a substitute server to > additionally attest that they had individually built the derivation in > question. I definitely think that this is a future we'd want with Guix. Goals: - That our software be fully reproducible in the first place - P2P distribution mechanisms for inputs (this one is relatively easy!) - "Community participation" of building derivatives - P2P distribution of built artifacts (actually, if you have p2p distribution of inputs, you can have this one relatively for free/cheap) There are challenges with all of this, but really we know enough what p2p content addressed infrastructure looks like, this isn't the hard part. Figuring out how to build a set of "semi-trusted sources" and the UX around it is the hard part. (In a weird way, compiling and verifying software is a "soft trap door problem", I have been thinking. Certainly not as much so as the functions we require for cryptography to work, but it's still a trap door, which is why build farms are expensive.) I guess a worthwhile question is "where are the costs coming from"? Ludovic said: > The various options and back-of-the-envelope estimates we came up with > are as follows: > > 1. Buying and hosting hardware: > 250k=E2=82=AC for hardware > 3k=E2=82=AC/month (36k=E2=82=AC/year) > > 2. Renting machines (e.g., on Hetzner): > 6k=E2=82=AC/month (72k=E2=82=AC/year) > > 3. Sponsored: > get hardware and/or hosting sponsored (by academic institutions or > companies). So I am guessing bandwith costs are significant but the 250k EUR for hardware indicates this is especially a build farm issue rather than a content distribution / bandwith issue. (Do I have that right?) Regardless, something I have thought about... #2 and #3 are cheaper but "less preferable" than #1 because of security concerns, per my understanding. But... what if we managed to make #2 and #3 *more secure*? Here is an idea that is semi-p2p, and maybe a path towards a more full p2p option, that we could possibly persue. - We have machines hosted that we trust a bit less, at some hosting facility, or possibly sponsored from someplace we trust even less. Let's imagine we went with the imaginary MegaCloud Inc. - We have a set of keys for semi-trusted "Guix Builders", who have machines that we run in our houses/lodgings/etc which sit around compiling Guix packages all day. These could be eg people who aren't even committers but have gained the trust of committers and maybe have even come to something like Guix Days in person. Now imagine for a moment that I wanted to download the latest version of... let's go oldschool in our FOSS references and say some expensive to compile browser named IceWeasel. ;) I want to download the latest version of IceWeasel. I could compile it myself, or I could get a substitute. #1 feels like the most "trustworthy" option at first glance but actually it could be even a single point of failure attack source. Okay, but what if instead I had the option to download something signed off by *all of* the MegaCloud build service and two "Guix Builders", and they all came to the same hash? This seems even better than #1 from a security/integrity perspective, I think. Just speculating... - Christine