Re: CDN Test Results - Should We Continue Using a CDN?

["Ludovic Courtès" <ludo@gnu.org>]

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Pardon me for asking, but how does using a CDN frees up resources?
> Aren't the usual infrastructure preserved (e.g., ci.guix.info)? It
> seems it'd be an extra layer to maintain?

One of the motivations for this is that berlin.guixsd.org
aka. ci.guix.info is a single machine, the head of our main build farm.
If that machine goes down, we have no substitutes.  Having a cache like
a CDN provides some redundancy: if the build farm goes down, we’ll at
least still have cached substitutes, which leaves us time to fix the
build farm.

We can have a cache that’s not a CDN, like we did with
mirror.hydra.gnu.org, which runs an nginx caching proxy for
hydra.gnu.org.  However, that’s another machine to take care of (that’s
not much work in practice, but still, we must be able to quickly respond
to outages), and another single point of failure.

> The heaviest bandwith usage appear to originate from areas already well
> served by the current infrastructure (mirror.hydra.gnu.org -> North
> America, ci.guix.info -> Europe), so I'm not sure spending resources on
> a CDN is worthwhile in this context.

I think the good bandwidth is the second motivation for the CDN, but
it’s true that it still benefits the same groups of people; in
particular we know that Cloudfront is unavailable in China.

Nevertheless the extra performance is welcome IMO.  I think substitute
delivery plays an important role in the user experience so if we can
improve it, the better.

> I'd rather see this (even modest) amount put into the hands of a
> motivated hacker to work on a distributed solution instead of
> encouraging a company which do not share our free software ideals.

As discussed before, I definitely sympathize with this.  Heck, if
someone had told me I’d argue in favor of a CDN after all this time
spent filling in CloudFare CAPTCHAs just because CloudFare decided that
user privacy doesn’t matter and that Tor users should be penalized, I’d
have laughed.  ;-)

So it’s definitely not an easy decision.  Nevertheless, we have to
acknowledge the fact that our current substitute delivery infrastructure
is fragile.  If people volunteer to maintain a set of mirrors with some
load balancing, that’s great, I’m all for it.  But for now, we don’t
have that at all, hence the CDN.

Longer term, I do hope for IPFS to become our main delivery mechanism.
I’ve posted a proof-of-concept that I think should allow us to get
started, play with the idea, and find out how that works in practice.

Thanks,
Ludo’.