From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Thompson, David" Subject: Re: Using a CDN or some other mirror? Date: Tue, 4 Dec 2018 12:58:10 -0500 Message-ID: References: <20181203154335.10366-1-ludo@gnu.org> <87tvju6145.fsf@gnu.org> <34d346ac-fc0a-aac2-f092-8a57e345c4e6@goebel-consult.de> <87woop1j5z.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:35707) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gUExk-00054v-NH for guix-devel@gnu.org; Tue, 04 Dec 2018 12:58:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gUExj-0004sc-9K for guix-devel@gnu.org; Tue, 04 Dec 2018 12:58:24 -0500 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]:39929) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gUExj-0004sC-1l for guix-devel@gnu.org; Tue, 04 Dec 2018 12:58:23 -0500 Received: by mail-wr1-x42d.google.com with SMTP id t27so16936097wra.6 for ; Tue, 04 Dec 2018 09:58:22 -0800 (PST) In-Reply-To: <87woop1j5z.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Cc: guix-devel On Tue, Dec 4, 2018 at 9:06 AM Ludovic Court=C3=A8s wrote: > > Hi Hartmut, > > Hartmut Goebel skribis: > > > Am 03.12.2018 um 17:12 schrieb Ludovic Court=C3=A8s: > >> Thus, I=E2=80=99m thinking about using a similar setup, but hosting th= e mirror > >> on some Big Corp CDN or similar. > > > > Isn't this a contradiction: Building a free infrastructure relaying on > > servers from some Big Corporation? Let allow the privacy concerns > > raising when delivering data via some Big Corporation. > > > > If delivering "packages" works via static data without requiring any > > additional service, we could ask universities to host Guix, too. IMHO > > this is a much preferred solution since this is a decentralized publish > > infrastructure already in place for many GNU/Linux distributions. > > As you know, berlin.guixsd.org is hosted at the Max Delbr=C3=BCck Center = in > Berlin, a public research institute. So in a way, we=E2=80=99re already = doing > that. We shouldn=E2=80=99t take it for granted that public institutes wi= ll > happily host our stuff and donate hardware: without Ricardo=E2=80=99s wor= k and > the generosity of the MDC, we wouldn=E2=80=99t have anything there. > > I understand the reluctance regarding =E2=80=9CBig Corp=E2=80=9D hosting,= and I actually > share it to some extent. However, having put much thought into it (and > also much sweat in build farm sysadmin=E2=80=A6), I think the alternative= is: > commercial hosting, or peer-to-peer. > > Florian has been looking at the latter approach with IPFS, and perhaps > we=E2=80=99ll be able to put it in production in a few months and be happ= y with > it (I have good hopes given what Florian already demonstrated.) > > In the meantime, we need redundant storage, high bandwidth, and high > availability. If you know of non-profit organizations that can provide > such services, please let us know; if not, we=E2=80=99ll resort to a comm= ercial > service. The bottom line is: we cannot reasonably pretend to offer such > a service ourselves. > > (Note that we=E2=80=99re just talking about substitute delivery=E2=80=94I= wouldn=E2=80=99t want > to *build* packages on one of these commercial hosting services.) > > I hope this clarifies my position. Using CloudFront with a custom (non-S3) origin sounds like a reasonable solution to me, though I understand the hesitance to use a commercial service. If AWS CloudFront is the path chosen, it may be worthwhile to follow the "infrastructure as code" practice and use CloudFormation to provision the CloudFront distribution and any other supporting resources. The benefit is that there would be a record of exactly *how* the project is using these commercial services and the setup could be easily reproduced. The timing is interesting here because I just attended the annual AWS conference on behalf of my employer and while I was there I felt inspired to write a Guile API for building CloudFormation "stacks". You can see a small sample of what it does here: https://gist.github.com/davexunit/db4b9d3e67902216fbdbc66cd9c6413e - Dave