* Re: 01/01: nginx: berlin: Disable narinfo caching altogether. [not found] ` <20180621094439.636F520498@vcs0.savannah.gnu.org> @ 2018-06-21 20:06 ` Mark H Weaver 2018-06-21 22:06 ` Ludovic Courtès 0 siblings, 1 reply; 4+ messages in thread From: Mark H Weaver @ 2018-06-21 20:06 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Hi Ludovic, ludo@gnu.org (Ludovic Courtès) writes: > civodul pushed a commit to branch master > in repository maintenance. > > commit 8379ba4119e51151d93589a6ef57cb159d94e9f2 > Author: Ludovic Courtès <ludo@gnu.org> > Date: Thu Jun 21 11:41:06 2018 +0200 > > nginx: berlin: Disable narinfo caching altogether. > > This is a followup to ebbe4c7f402b6d9cf9c6c2ecf120f49697ab2c49. > > * hydra/nginx/berlin-locations.conf (.narinfo): Disable caching. > * hydra/nginx/berlin.conf: Remove 'proxy_cache_path' directive > for narinfos. What's the rationale for this change? Although 'guix publish' does its own caching, I would expect 'nginx' to handle cache hits far more efficiently than guix publish. Therefore, I expect this change to result in higher CPU usage for a given amount of traffic. I also expect that cache hits on narinfos are an important common case for our substitute servers. Does this change bring advantages that outweigh the loss of efficiency? Mark ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: 01/01: nginx: berlin: Disable narinfo caching altogether. 2018-06-21 20:06 ` 01/01: nginx: berlin: Disable narinfo caching altogether Mark H Weaver @ 2018-06-21 22:06 ` Ludovic Courtès 2018-06-22 21:14 ` Mark H Weaver 0 siblings, 1 reply; 4+ messages in thread From: Ludovic Courtès @ 2018-06-21 22:06 UTC (permalink / raw) To: Mark H Weaver; +Cc: guix-devel Hello Mark, Mark H Weaver <mhw@netris.org> skribis: > ludo@gnu.org (Ludovic Courtès) writes: > >> civodul pushed a commit to branch master >> in repository maintenance. >> >> commit 8379ba4119e51151d93589a6ef57cb159d94e9f2 >> Author: Ludovic Courtès <ludo@gnu.org> >> Date: Thu Jun 21 11:41:06 2018 +0200 >> >> nginx: berlin: Disable narinfo caching altogether. >> >> This is a followup to ebbe4c7f402b6d9cf9c6c2ecf120f49697ab2c49. >> >> * hydra/nginx/berlin-locations.conf (.narinfo): Disable caching. >> * hydra/nginx/berlin.conf: Remove 'proxy_cache_path' directive >> for narinfos. > > What's the rationale for this change? From commit ebbe4c7f402b6d9cf9c6c2ecf120f49697ab2c49: Somehow nginx appeared to be caching narinfos for longer than needed, which defeated the atime-based cache eviction strategy of 'guix publish'. In this case, I noticed on berlin that nginx was caching 404s for narinfos longer than expected, for reasons I could not elucidate. Plus there’s this atime story. (Narinfo caching was critical when we were using Hydra, because Hydra would compute them all the fly, which included computing signatures over whole nars.) > Although 'guix publish' does its own caching, I would expect 'nginx' to > handle cache hits far more efficiently than guix publish. Therefore, I > expect this change to result in higher CPU usage for a given amount of > traffic. I also expect that cache hits on narinfos are an important > common case for our substitute servers. > > Does this change bring advantages that outweigh the loss of efficiency? I’m not sure the loss of efficiency is noticeable (‘guix publish’ uses sendfile(2) on 200, and stat(2) is all it takes to distinguish between 200 and 404, so it’s not that bad.) Anyway, nothing’s set in stone. It did seem like the right thing to do now, but if it turned out to be wrong, we can revisit this. In fact, I would very much welcome input from people who are used to tweaking nginx more than I am! Thanks for your feedback, Ludo’. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: 01/01: nginx: berlin: Disable narinfo caching altogether. 2018-06-21 22:06 ` Ludovic Courtès @ 2018-06-22 21:14 ` Mark H Weaver 2018-06-23 22:20 ` Ludovic Courtès 0 siblings, 1 reply; 4+ messages in thread From: Mark H Weaver @ 2018-06-22 21:14 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Hi Ludovic, ludo@gnu.org (Ludovic Courtès) writes: > Mark H Weaver <mhw@netris.org> skribis: > >> ludo@gnu.org (Ludovic Courtès) writes: >> >>> civodul pushed a commit to branch master >>> in repository maintenance. >>> >>> commit 8379ba4119e51151d93589a6ef57cb159d94e9f2 >>> Author: Ludovic Courtès <ludo@gnu.org> >>> Date: Thu Jun 21 11:41:06 2018 +0200 >>> >>> nginx: berlin: Disable narinfo caching altogether. >>> >>> This is a followup to ebbe4c7f402b6d9cf9c6c2ecf120f49697ab2c49. >>> >>> * hydra/nginx/berlin-locations.conf (.narinfo): Disable caching. >>> * hydra/nginx/berlin.conf: Remove 'proxy_cache_path' directive >>> for narinfos. >> >> What's the rationale for this change? > > From commit ebbe4c7f402b6d9cf9c6c2ecf120f49697ab2c49: > > Somehow nginx appeared to be caching narinfos for longer than needed, > which defeated the atime-based cache eviction strategy of 'guix > publish'. > > In this case, I noticed on berlin that nginx was caching 404s for > narinfos longer than expected, for reasons I could not elucidate. Plus > there’s this atime story. Although you didn't mention it here, I now remember one reason why it's a problem for narinfos to be in the cache longer than expected: because if a narinfo is available but the corresponding NAR isn't, it causes problems on the client side. If that's still the case, then it certainly justifies this change. Thanks, Mark ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: 01/01: nginx: berlin: Disable narinfo caching altogether. 2018-06-22 21:14 ` Mark H Weaver @ 2018-06-23 22:20 ` Ludovic Courtès 0 siblings, 0 replies; 4+ messages in thread From: Ludovic Courtès @ 2018-06-23 22:20 UTC (permalink / raw) To: Mark H Weaver; +Cc: guix-devel Hello, Mark H Weaver <mhw@netris.org> skribis: > ludo@gnu.org (Ludovic Courtès) writes: > >> Mark H Weaver <mhw@netris.org> skribis: >> >>> ludo@gnu.org (Ludovic Courtès) writes: >>> >>>> civodul pushed a commit to branch master >>>> in repository maintenance. >>>> >>>> commit 8379ba4119e51151d93589a6ef57cb159d94e9f2 >>>> Author: Ludovic Courtès <ludo@gnu.org> >>>> Date: Thu Jun 21 11:41:06 2018 +0200 >>>> >>>> nginx: berlin: Disable narinfo caching altogether. >>>> >>>> This is a followup to ebbe4c7f402b6d9cf9c6c2ecf120f49697ab2c49. >>>> >>>> * hydra/nginx/berlin-locations.conf (.narinfo): Disable caching. >>>> * hydra/nginx/berlin.conf: Remove 'proxy_cache_path' directive >>>> for narinfos. >>> >>> What's the rationale for this change? >> >> From commit ebbe4c7f402b6d9cf9c6c2ecf120f49697ab2c49: >> >> Somehow nginx appeared to be caching narinfos for longer than needed, >> which defeated the atime-based cache eviction strategy of 'guix >> publish'. >> >> In this case, I noticed on berlin that nginx was caching 404s for >> narinfos longer than expected, for reasons I could not elucidate. Plus >> there’s this atime story. > > Although you didn't mention it here, I now remember one reason why it's > a problem for narinfos to be in the cache longer than expected: because > if a narinfo is available but the corresponding NAR isn't, it causes > problems on the client side. If that's still the case, then it > certainly justifies this change. Yes, that too. We’ll see how it goes and if it performs badly let’s revert or adjust. Ludo’. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2018-06-23 22:21 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20180621094438.32617.70405@vcs0.savannah.gnu.org> [not found] ` <20180621094439.636F520498@vcs0.savannah.gnu.org> 2018-06-21 20:06 ` 01/01: nginx: berlin: Disable narinfo caching altogether Mark H Weaver 2018-06-21 22:06 ` Ludovic Courtès 2018-06-22 21:14 ` Mark H Weaver 2018-06-23 22:20 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).