CDN Mirrors for GNU Guix

CDN Mirrors for GNU Guix

[Tom Li]

[-- Attachment #1: Type: text/plain, Size: 989 bytes --]

Currently, GNU Guix is still in the early stage of development, and there is a great
lack of mirrors worldwide. For example. in my region, using GNU Guix is incredibly
slow, the speed is around 4 KiB/s and rendering it almost unusable.

Therefore, I created two CDN mirrors of https://mirror.hydra.gnu.org/, by using
CloudFlare and Amazon CloudFront's service. I know some have the concerns about
such type of centralized corporation-controlled service. Personally, I have done my
best to minimized the security risks (HTTPS only, untouched signatures) and set ip
up faithfully. Please use it according to your own judgement.

they are available at:

* https://guix-cloudflare.tomli.me/
* https://guix-amazon.tomli.me/

Since they are identical mirrors of Hydra, you just need to use `--substitute-urls=`
in order to use it.

I have no experience on setting up mirrors. I expected to have some problems
related to cache things for too long. Please report any problems.

Happy Hacking!
Tom.

[-- Attachment #2: Type: text/html, Size: 1357 bytes --]

CDN Mirrors for GNU Guix

[Tom Li]

[-- Attachment #1: Type: text/plain, Size: 989 bytes --]

Currently, GNU Guix is still in the early stage of development, and there is a great
lack of mirrors worldwide. For example. in my region, using GNU Guix is incredibly
slow, the speed is around 4 KiB/s and rendering it almost unusable.

Therefore, I created two CDN mirrors of https://mirror.hydra.gnu.org/, by using
CloudFlare and Amazon CloudFront's service. I know some have the concerns about
such type of centralized corporation-controlled service. Personally, I have done my
best to minimized the security risks (HTTPS only, untouched signatures) and set ip
up faithfully. Please use it according to your own judgement.

they are available at:

* https://guix-cloudflare.tomli.me/
* https://guix-amazon.tomli.me/

Since they are identical mirrors of Hydra, you just need to use `--substitute-urls=`
in order to use it.

I have no experience on setting up mirrors. I expected to have some problems
related to cache things for too long. Please report any problems.

Happy Hacking!
Tom.

[-- Attachment #2: Type: text/html, Size: 1327 bytes --]

Re: CDN Mirrors for GNU Guix

[Ludovic Courtès]

Hello Tom,

Tom Li <tomli@protonmail.ch> skribis:

> Currently, GNU Guix is still in the early stage of development, and there is a great
> lack of mirrors worldwide. For example. in my region, using GNU Guix is incredibly
> slow, the speed is around 4 KiB/s and rendering it almost unusable.

Woow, that sounds really extreme!  Do you always have such a bandwidth,
or did you just happen to be unlucky somehow at that time?

Regardless, I agree that we should have more mirrors and a wider
distribution.

> Therefore, I created two CDN mirrors of https://mirror.hydra.gnu.org/, by using
> CloudFlare and Amazon CloudFront's service. I know some have the concerns about
> such type of centralized corporation-controlled service. Personally, I have done my
> best to minimized the security risks (HTTPS only, untouched signatures) and set ip
> up faithfully. Please use it according to your own judgement.
>
> they are available at:
>
> * https://guix-cloudflare.tomli.me/
> * https://guix-amazon.tomli.me/
>
> Since they are identical mirrors of Hydra, you just need to use `--substitute-urls=`
> in order to use it.

Nice!  (Though I should say that I hate CloudFare for essentially
preventing Tor users from accessing what they host.)

I think it may be time to arrange so that mirror.hydra.gnu.org (or some
other host name?) can somehow redirect users to external mirrors.  I’m
not sure how to achieve this, so if anyone has experience in this area,
help is welcome!

Thanks,
Ludo’.

Re: CDN Mirrors for GNU Guix

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2714 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Hello Tom,
>
> Tom Li <tomli@protonmail.ch> skribis:
>
>> Currently, GNU Guix is still in the early stage of development, and there is a great
>> lack of mirrors worldwide. For example. in my region, using GNU Guix is incredibly
>> slow, the speed is around 4 KiB/s and rendering it almost unusable.
>
> Woow, that sounds really extreme!  Do you always have such a bandwidth,
> or did you just happen to be unlucky somehow at that time?
>
> Regardless, I agree that we should have more mirrors and a wider
> distribution.
>
>> Therefore, I created two CDN mirrors of https://mirror.hydra.gnu.org/, by using
>> CloudFlare and Amazon CloudFront's service. I know some have the concerns about
>> such type of centralized corporation-controlled service. Personally, I have done my
>> best to minimized the security risks (HTTPS only, untouched signatures) and set ip
>> up faithfully. Please use it according to your own judgement.
>>
>> they are available at:
>>
>> * https://guix-cloudflare.tomli.me/
>> * https://guix-amazon.tomli.me/
>>
>> Since they are identical mirrors of Hydra, you just need to use `--substitute-urls=`
>> in order to use it.

I'm curious to know what the cache hit rate of your CloudFront
distribution is.  I have one, too, at
https://hydra-mirror.marusich.info, and the hit rate is surprisingly low
(less than 5%, last I checked).  It's probably because I'm the only one
using it, though :-)

> Nice!  (Though I should say that I hate CloudFare for essentially
> preventing Tor users from accessing what they host.)
>
> I think it may be time to arrange so that mirror.hydra.gnu.org (or some
> other host name?) can somehow redirect users to external mirrors.  I’m
> not sure how to achieve this, so if anyone has experience in this area,
> help is welcome!

One option is to use DNS round robin.  The main problems with
traditional DNS round robin is that you don't always get a "nearby"
(according to some metric such as latency or geographic location)
address, and if one of the addresses is "unhealthy" (according to some
metric such as external network connectivity), some clients might still
get (and cache) the unhealthy address.  Some DNS services (e.g., Amazon
Route 53) provide "smarter" features that will return nearby records or
healthy records, but I do not know how easy it is to use those features
when you are trying to route requests to another domain or address
outside of your own control.  Beyond DNS, I suppose you could implement
some kind of application layer logic, perhaps involving HTTP redirects,
but it would not be as simple as DNS round robin.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: CDN Mirrors for GNU Guix

[Tom Li]

[-- Attachment #1: Type: text/plain, Size: 1974 bytes --]

> Woow, that sounds really extreme! Do you always have such a bandwidth,
> or did you just happen to be unlucky somehow at that time?

Welcome to China, haha.

No. Actually my connectivity is relatively good, and I have plenty
of bandwidth and being able to visit many of the U.S-based servers
directly with a reasonable speed, such as CloudFlare or Amazon CDN
(they do have local servers but still direct most users to the U.S.
due to high ISP costs).

However, apparently Hydra is not one of them. I believe censorship
is less relevant to this specific issue, this looks like a network
routing/capacity issue, as usual. People from other parts of the world
may also have similar problems.

> (Though I should say that I hate CloudFare for essentially
> preventing Tor users from accessing what they host.)

Yes, that's what we all know.

I'm not CloudFlare fan but I have to say, there isn't an
intentional plot against Tor users because any IP address-based
firewall would eventually blacklist all exit nodes, and yes,
they (and any other large providers, e.g. Google Search) need
to work with the community to solve this issue.

For this mirror, I have switched all security features off or to
the minimum level by using a CloudFlare Page Rule.

I'm able to access via Tor without blocking by its firewall, I
don't know if it's just for my lucky exit node through.

> I'm curious to know what the cache hit rate of your CloudFront
> distribution is.
> The hit rate is surprisingly low
> (less than 5%, last I checked). It's probably because I'm the only one
> using it, though :-)

It is very low.

By default, CloudFlare seems to be conservative on caching, I have
seeing lots of people complaining that CloudFlare talks to the origin
too frequently and effectively change a DDoS into a DoS (lol). Some
tuning may help. I think it is beneficial for users even if it acts
only as a reverse proxy. I can tweak the settings a bit after more
people started to use it.

Tom.

[-- Attachment #2: Type: text/html, Size: 2310 bytes --]

Re: CDN Mirrors for GNU Guix

[Ludovic Courtès]

Tom Li <tomli@protonmail.ch> skribis:

> Therefore, I created two CDN mirrors of https://mirror.hydra.gnu.org/, by using
> CloudFlare and Amazon CloudFront's service. I know some have the concerns about
> such type of centralized corporation-controlled service. Personally, I have done my
> best to minimized the security risks (HTTPS only, untouched signatures) and set ip
> up faithfully. Please use it according to your own judgement.
>
> they are available at:
>
> * https://guix-cloudflare.tomli.me/
> * https://guix-amazon.tomli.me/
>
> Since they are identical mirrors of Hydra, you just need to use `--substitute-urls=`
> in order to use it.

Thanks for sharing!

Ludo’.