Re: Yet another Hydra mirror: hydra-mirror.marusich.info

[Chris Marusich <cmmarusich@gmail.com>]

ludo@gnu.org (Ludovic Courtès) writes:

> Yeah, mirror.guixsd.org also caches for one week now.  I wasn’t sure how
> much disk space that would represent, but so far we’re around 10G, so
> increasing to 1 week seemed reasonable.

Are there any URL paths a caching proxy should NOT cache for a long
time? My understanding is that it's OK to cache all URL paths for 1
week. Are there any URL paths for which I should return something
different from Hydra (e.g., /nix-cache-info)?

> OK.
>
> Do you know exactly how much is cached?  Also, when does caching happen?

Every OK result is cached for 7 days. My understanding is that negative
results (e.g., 404) are not cached. The storage limit is effectively
unlimited from my perspective, since CloudFront is a service.

> When using nginx as a proxy as on mirror.guixsd.org, it fetches things
> lazily, so on a cache miss it goes connect to hydra.gnu.org.

CloudFront operates the same way.

>> I've noticed that Hydra does not include cache-related headers (e.g.,
>> Cache-Control). Perhaps for this reason, the nginx config you linked
>> seems to pick arbitrary caching settings. When using CloudFront, a
>> distribution can be configured to respect the Cache-Control headers sent
>> by the origin server. Does nginx provide similar functionality? Would it
>> make sense to have hydra.gnu.org return such headers?
>
> Dunno, maybe!  Maybe we could tell nginx to add such headers?  What
> would be the right thing?

I'm not sure. It's obviously not necessary, but if the common proxy
servers (e.g., nginx) obey such cache control headers by default, it
might be nice to return the headers. That way, Hydra could "recommend"
caching behavior, and it would be easier to set up a correctly
functioning caching proxy. However, if servers like nginx don't obey
cache control headers by default, or if Hydra does not require different
objects to be cached differently, it may not worth the effort.

Chris