From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: Re: CDN performance Date: Fri, 14 Dec 2018 11:35:37 +0100 Message-ID: <87pnu45rba.fsf@gnu.org> References: <20181203154335.10366-1-ludo@gnu.org> <87tvju6145.fsf@gnu.org> <87ftv7l6gy.fsf@gmail.com> <871s6qzo6m.fsf_-_@gnu.org> <87y38tx365.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:33376) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXkos-0007Fi-8T for guix-devel@gnu.org; Fri, 14 Dec 2018 05:35:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXkoo-0006SJ-HE for guix-devel@gnu.org; Fri, 14 Dec 2018 05:35:44 -0500 In-Reply-To: <87y38tx365.fsf@gmail.com> (Chris Marusich's message of "Thu, 13 Dec 2018 00:05:06 -0800") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Chris Marusich Cc: Hartmut Goebel , guix-devel@gnu.org, 33600@debbugs.gnu.org Hello! Chris Marusich skribis: > Ludovic Court=C3=A8s writes: [...] > Judging by that email thread, one of the reasons why Debian considered > using a CDN was because they felt that the cost, in terms of people > power, of maintaining their own "proto-CDN" infrastructure had grown too > great. I believe it! I think it would be ill-advised for the Guix > project to expend effort and capital on building and maintaining its own > CDN. I think it would be wiser to focus on developing a decentralized > substitute solution (GNUnet, IPFS, etc.). > > That said, I still think that today Guix should provide a third-party > CDN option. For many Guix users, a CDN would improve performance and > availability of substitutes. Contracting with a third party to provide > the CDN service would require much less effort and capital than building > and maintaining a CDN from scratch. This would also enable the project > to focus more on building a decentralized substitute solution. And once > that decentralized solution is ready, it will be easy to just "turn off" > the CDN. > > I also understand Hartmut's concerns. The risks he points out are > valid. Because of those risks, even if we make a third-party CDN option > available, some people will choose not to use it. For that reason, we > should not require Guix users to use a third-party CDN, just as we do > not require them to use substitutes from our build farm. > > However, not everyone shares the same threat model. For example, > although some people choose not to trust substitutes from our build > farm, still others do. The choice is based on one's own individual > situation. Similarly, if we make a third-party CDN option available and > explain the risks of using it, Guix users will be able to make an > educated decision for themselves about whether or not to use it. That summarizes the situation very well. In Paris, there was consensus on two things: that a CDN would be helpful for performance and availability (though availability could suddenly be worse if we run=E2=80=A6 out of money), and that it raises privacy concerns. Based on this, some suggested having a way to opt out of the CDN distribution: for example, we might make ci.guix.info point to the CDN and also advertise berlin.guixsd.org as a way to bypass it. Of course in parallel we want to do our best to develop decentralized solutions. > However, I believe you when you say that it's slow the first time you > download the substitute from berlin. What path does the data take from > its origin through berlin? If berlin needs to download the initial file > from another server, perhaps the connection between berlin and that > server is the bottleneck? Maybe we should discuss that in a different > email thread, though. On the first hit, nginx fetches the file from =E2=80=98guix publish=E2=80= =99, which is also on berlin. =E2=80=98guix publish=E2=80=99 gets the file from local st= orage and passes it with sendfile(2), so it=E2=80=99s as fast as can be. Thanks for the info! Ludo=E2=80=99.