From: Tobias Geerinckx-Rice <me@tobias.gr>
To: mhw@netris.org
Cc: 26201@debbugs.gnu.org, guix-sysadmin@gnu.org
Subject: bug#26201: hydra.gnu.org uses ‘guix publish’ for nars and narinfos
Date: Sun, 26 Mar 2017 19:35:25 +0200 [thread overview]
Message-ID: <1988d01c-1e67-bf47-2b43-cf3551d0651b@tobias.gr> (raw)
In-Reply-To: <87y3vvozy5.fsf@netris.org>
[-- Attachment #1.1: Type: text/plain, Size: 2091 bytes --]
Mark,
On 24/03/17 09:12, Mark H Weaver wrote:
> IIUC, with "proxy_cache_lock on", we have two choices of how other
> client requests will be treated:
>
> [badly, ed.]
Eh. You're probably (and disappointingly) right.
When configuring my little cache, I had a clear idea of how such a cache
should work (basically, your last scenario below), then looked at the
nginx documentation to find what I had in mind. ‘proxy_cache_lock’ matched.
I should have been more pessimistic and done more testing.
Shame on me, &c. Too much other things on my mind. :-/
> Or at least that's what I'd expect based on my reading of the nginx docs
> linked above. I haven't tried it.
I can try to do some simple tests tomorrow.
> IMO, the best solution is to *never* generate nars on Hydra in response
> to client requests, but rather to have the build slaves pack and
> compress the nars, copy them to Hydra, and then serve them as static
> files using nginx.
A true mirror at last! Do we have the disc space for that?
And could Hydra actually handle compressing *everything*, without an
infinitely growing back-log? I don't have access to any statistics, but
I'm guessing that a fair number of package+versions are never actually
requested, and hence never compressed. This would change that.
> A far inferior solution, but possibly acceptable and closer to the
> current approach, would be to arrange for all concurrent responses for
> the same nar to be sent incrementally from a single nar-packing process.
> More concretely, while packing and sending a nar response to the first
> client, the data would also be written to a file. Subsequent requests
> for the same nar would be serviced using the equivalent of:
>
> tail --bytes=+0 --follow FILENAME
>
> This way, no one would have to wait an hour to receive the first byte.
^ This is so obviously the right solution, that it would be
disappointing if nginx really couldn't be made to do it. It already
buffers proxy responses to a temporary file anyway...
Kind regards,
T G-R
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 476 bytes --]
next prev parent reply other threads:[~2017-03-26 17:36 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-21 1:44 bug#26201: No notification of cache misses when downloading substitutes dian_cecht
2017-03-21 2:46 ` Tobias Geerinckx-Rice
2017-03-21 2:52 ` dian_cecht
2017-03-21 3:57 ` Tobias Geerinckx-Rice
2017-03-21 4:48 ` dian_cecht
2017-03-21 6:21 ` Tobias Geerinckx-Rice
2017-03-21 6:49 ` dian_cecht
2017-03-21 14:55 ` Tobias Geerinckx-Rice
2017-03-21 15:32 ` dian_cecht
2017-03-21 16:07 ` Tobias Geerinckx-Rice
2017-03-24 2:15 ` Maxim Cournoyer
2017-03-21 12:59 ` Florian Pelz
2017-03-21 15:35 ` dian_cecht
2017-03-21 16:43 ` Ludovic Courtès
2017-03-21 17:08 ` Tobias Geerinckx-Rice
2017-03-22 22:06 ` Ludovic Courtès
2017-03-23 19:25 ` bug#26201: hydra.gnu.org uses ‘guix publish’ for nars and narinfos Tobias Geerinckx-Rice
2017-03-22 22:22 ` Ludovic Courtès
2017-03-23 10:29 ` Ricardo Wurmus
2017-03-23 18:36 ` Mark H Weaver
2017-03-23 18:52 ` Tobias Geerinckx-Rice
2017-03-24 8:12 ` Mark H Weaver
2017-03-24 9:25 ` Ludovic Courtès
2017-04-17 21:36 ` Ludovic Courtès
2017-04-18 21:27 ` Ludovic Courtès
2017-04-19 14:24 ` bug#26201: Heads-up: hydra.gnu.org uses ‘guix publish --cache’ Ludovic Courtès
2017-03-26 17:35 ` Tobias Geerinckx-Rice [this message]
2017-03-27 18:47 ` bug#26201: hydra.gnu.org uses ‘guix publish’ for nars and narinfos Tobias Geerinckx-Rice
2017-03-28 14:47 ` Ludovic Courtès
2017-05-03 8:11 ` Mark H Weaver
2017-05-03 9:25 ` Ludovic Courtès
2017-03-27 11:20 ` bug#26201: Bandwidth when retrieving substitutes Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1988d01c-1e67-bf47-2b43-cf3551d0651b@tobias.gr \
--to=me@tobias.gr \
--cc=26201@debbugs.gnu.org \
--cc=guix-sysadmin@gnu.org \
--cc=mhw@netris.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).