From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pjotr Prins Subject: Re: plz is there a roadmap for a more resilient substitutes infrastructure? Date: Sat, 3 Nov 2018 07:10:50 +0100 Message-ID: <20181103061050.crf6564laxir4jze@thebird.nl> References: <87wopv7jzw.fsf@roquette.mug.biscuolo.net> <20181102210451.htzxk6ox7uldr2fr@thebird.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:60657) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gIp99-0001oR-Pd for guix-devel@gnu.org; Sat, 03 Nov 2018 02:11:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gIp94-0007Hc-GK for guix-devel@gnu.org; Sat, 03 Nov 2018 02:10:57 -0400 Received: from mail.thebird.nl ([94.142.245.5]:35564) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gIp93-0007Eh-Lo for guix-devel@gnu.org; Sat, 03 Nov 2018 02:10:54 -0400 Content-Disposition: inline In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Julien Lepiller Cc: guix-devel@gnu.org On Fri, Nov 02, 2018 at 11:51:20PM +0100, Julien Lepiller wrote: > We could easily distribute nar files over distributed networks (IPFS, b= ittorrent, =E2=80=A6) but we still need a "canonical source" that builds = these packages, otherwise how do you know what you are looking for? Don't= we always need some sort of central authority? Yes. A name service which is fed from accredited build servers.=20 It is not hard to keep a few build servers in the 'air' which can be replaced on demand - even run in the cloud or in VMs. What is hard it to create a 100% uptime service that serves many generations of nars. Lot of data, and the data load can be high. This is what we ought to consider fanning out. Guix can support both systems, existing and new. Just add a substitute-url which resolves to an IPFS based naming scheme. Could even be integrated with guix-publish. Anyone who would run a guix-publish server could choose to expose an IPFS node for sharing. But I think it can be lighter weight. If we have a name service we could indeed just make use of any protocol that serves files. As long as the download hash is known. So, guix-named provides pointers to nar entities with their download hash and guix-download is capable of querying guix-named and provides more protocols. IPFS protocol is well defined and there exist implementations in multiple languages.=20 Anyway, this all requires more thought and a proof-of-concept. The point really is to design a distributed system based on existing components. Pj.