Re: Yet another Hydra mirror: hydra-mirror.marusich.info

[ludo@gnu.org (Ludovic Courtès)]

Chris Marusich <cmmarusich@gmail.com> skribis:

> No, I'm not using that at the moment. In the future, if I set up any
> nginx servers to accomplish the same task, I will definitely use it. I
> would prefer to run my own servers, but for now this is something I can
> do immediately to help the project, so I decided to do it.
>
> CloudFront is a service, so you use its API (or the AWS Management
> Console, which is a web UI for the API) to create a "distribution" and
> configure it to use hydra.gnu.org as its "origin". A little extra work
> is required to glue everything together. For example, I had to create a
> CNAME pointing from hydra-mirror.marusich.info to
> d2xj50ygrk34qq.cloudfront.net, which is the canonical name of my
> distribution. Once it's configured, all requests sent to
> hydra-mirror.marusich.info are serviced by a nearby point of presence in
> the CloudFront content distribution network, and the results are cached.

OK.

Do you know exactly how much is cached?  Also, when does caching happen?

When using nginx as a proxy as on mirror.guixsd.org, it fetches things
lazily, so on a cache miss it goes connect to hydra.gnu.org.

> I've noticed that Hydra does not include cache-related headers (e.g.,
> Cache-Control). Perhaps for this reason, the nginx config you linked
> seems to pick arbitrary caching settings. When using CloudFront, a
> distribution can be configured to respect the Cache-Control headers sent
> by the origin server. Does nginx provide similar functionality? Would it
> make sense to have hydra.gnu.org return such headers?

Dunno, maybe!  Maybe we could tell nginx to add such headers?  What
would be the right thing?

> For now, I've configured hydra-mirror.marusich.info to cache all
> successful requests for 1 week, and to respect cache-related headers
> from the hydra.gnu.org if it ever decides to send them. This seemed like
> a reasonable configuration for data which is not expected to change.

Yeah, mirror.guixsd.org also caches for one week now.  I wasn’t sure how
much disk space that would represent, but so far we’re around 10G, so
increasing to 1 week seemed reasonable.

Thanks,
Ludo’.