From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tobias Geerinckx-Rice Subject: bug#26201: No notification of cache misses when downloading substitutes Date: Tue, 21 Mar 2017 15:55:05 +0100 Message-ID: <1bbd8ee3-1745-3642-27ed-f095c732dc11@tobias.gr> References: <20170320184449.5ac06051@khaalida> <144e9ba8-af93-fb18-d2b9-f198ae7c11e9@tobias.gr> <20170320195247.05f72fc9@khaalida> <8e7e07d1-563f-666f-2c32-2a772757c86f@tobias.gr> <20170320214809.466dc5fe@khaalida> <20170320234912.46680062@khaalida> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="sVtwlF7vBIdjQ4PVvfNB2UD97ITDdwMOc" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:43247) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cqLBk-0005XN-3E for bug-guix@gnu.org; Tue, 21 Mar 2017 10:55:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cqLBf-0003Be-4M for bug-guix@gnu.org; Tue, 21 Mar 2017 10:55:08 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:39887) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cqLBe-0003BY-Vs for bug-guix@gnu.org; Tue, 21 Mar 2017 10:55:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1cqLBe-0006AX-G7 for bug-guix@gnu.org; Tue, 21 Mar 2017 10:55:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <20170320234912.46680062@khaalida> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: dian_cecht@zoho.com Cc: 26201@debbugs.gnu.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --sVtwlF7vBIdjQ4PVvfNB2UD97ITDdwMOc Content-Type: multipart/mixed; boundary="iu575CXagERv6leFegvDstT3Ehqs7xP5x"; protected-headers="v1" From: Tobias Geerinckx-Rice To: dian_cecht@zoho.com Cc: 26201@debbugs.gnu.org Message-ID: <1bbd8ee3-1745-3642-27ed-f095c732dc11@tobias.gr> Subject: Re: bug#26201: No notification of cache misses when downloading substitutes References: <20170320184449.5ac06051@khaalida> <144e9ba8-af93-fb18-d2b9-f198ae7c11e9@tobias.gr> <20170320195247.05f72fc9@khaalida> <8e7e07d1-563f-666f-2c32-2a772757c86f@tobias.gr> <20170320214809.466dc5fe@khaalida> <20170320234912.46680062@khaalida> In-Reply-To: <20170320234912.46680062@khaalida> --iu575CXagERv6leFegvDstT3Ehqs7xP5x Content-Type: multipart/mixed; boundary="------------F5B05DD24CBE0FDA7ED8453B" This is a multi-part message in MIME format. --------------F5B05DD24CBE0FDA7ED8453B Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hullo! On 21/03/17 07:49, dian_cecht@zoho.com wrote: > I'm not sure how any of this matters. If you are running a local=20 > Hydra instance or whatever, then I'd assume you'd be aware of what,=20 > if any, problems that could arise. It matters for the reasons mentioned. It's not a =E2=80=98local Hydra=E2=80= =99 & I have no idea what problems you're talking about. My problem is that every invocation of Guix already fills several screens with Guile cache misses. Adding another warning (=E2=80=98warning= ! the system is working exactly as designed!=E2=80=99) will only serve to make = those other warnings look less silly, and I think that would be a shame. To clarify: - Warnings should be scary because warnings should be actionable. There's nothing the user can or needs to do about a cache miss. - It would be randomly shown to everyone, since this happens constantly. - The behaviour warned about is not incorrect or abnormal. - As already noted, it's how caching works. > I don't see how this would have to be "an ugly hack". It's simply a=20 > query and response. The simplest way I can see for this to work would > be for mirror.hydra to either just send the requested file, or a > response that the file isn't cached then start to trickle the file on > to the client. Well, yeah... That's the ugly hack. :-) It's not that your suggestion's hard to implement. In fact, it's just one line for nginx (which it turns out I already had): add_header X-Cache-Status $upstream_cache_status; and 6 lines of lightly-tested Guile (attached)=C2=B9. And presto. This th= ing. Doesn't mean we should. Kind regards, T G-R =C2=B9: Why? Practice. Irony. Light masochism. --------------F5B05DD24CBE0FDA7ED8453B Content-Type: text/x-patch; name="0001-http-client-Warn-on-proxy-cache-misses.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0001-http-client-Warn-on-proxy-cache-misses.patch" =46rom 6d459a442d73628a0628385283c7cf04dff1b797 Mon Sep 17 00:00:00 2001 From: Tobias Geerinckx-Rice Date: Tue, 21 Mar 2017 15:31:56 +0100 Subject: [PATCH] http-client: Warn on proxy cache misses. Still not a good idea. * guix/http-client.scm (http-fetch): Add #:peek-behind-proxy parameter to expose caching proxy implementation details as a scary warning. * guix/scripts/substitute.scm (fetch): Use it. --- guix/http-client.scm | 10 +++++++++- guix/scripts/substitute.scm | 3 ++- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/guix/http-client.scm b/guix/http-client.scm index 6874c51..2366f5e 100644 --- a/guix/http-client.scm +++ b/guix/http-client.scm @@ -2,6 +2,7 @@ ;;; Copyright =C2=A9 2012, 2013, 2014, 2015, 2016, 2017 Ludovic Court=C3= =A8s ;;; Copyright =C2=A9 2015 Mark H Weaver ;;; Copyright =C2=A9 2012, 2015 Free Software Foundation, Inc. +;;; Copyright =C2=A9 2017 Tobias Geerinckx-Rice ;;; ;;; This file is part of GNU Guix. ;;; @@ -222,7 +223,8 @@ or if EOF is reached." =20 (define* (http-fetch uri #:key port (text? #f) (buffered? #t) keep-alive? (verify-certificate? #t) - (headers '((user-agent . "GNU Guile")))) + (headers '((user-agent . "GNU Guile"))) + (peek-behind-cache? #f)) "Return an input port containing the data at URI, and the expected num= ber of bytes available or #f. If TEXT? is true, the data at URI is considered = to be textual. Follow any HTTP redirection. When BUFFERED? is #f, return an @@ -253,8 +255,14 @@ Raise an '&http-get-error' condition if downloading = fails." (http-get uri #:streaming? #t #:port port #:keep-alive? #t #:headers headers)) + ((headers) + (response-headers resp)) ((code) (response-code resp))) + (when (and peek-behind-cache? + (equal? (assoc-ref headers 'x-cache-status) "MISS")) + (warning (_ "the caching proxy is working properly!~%")) + (warning (_ "and there's nothing you can do about it.~%"))= ) (case code ((200) (values data (response-content-length resp))) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index faeb019..4a4f115 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -216,7 +216,8 @@ provide." (unless (or buffered? (not (file-port? port))) (setvbuf port _IONBF))) (http-fetch uri #:text? #f #:port port - #:verify-certificate? #f)))))) + #:verify-certificate? #f + #:peek-behind-cache? #t)))))) (else (leave (_ "unsupported substitute URI scheme: ~a~%") (uri->string uri))))) --=20 2.9.3 --------------F5B05DD24CBE0FDA7ED8453B-- --iu575CXagERv6leFegvDstT3Ehqs7xP5x-- --sVtwlF7vBIdjQ4PVvfNB2UD97ITDdwMOc Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEqBAEBCgAUBQJY0T7KDRxtZUB0b2JpYXMuZ3IACgkQkczbm0hUG5mNSgf+NNsf Eneqqsg7f9QfkOW0iT6JxZ90JEcsVDLoUvx8YvBtOXwVvhnUxt1GQxu0o9yCUUAY 9nLOl+5cmE81Yjz2ibX6z/oUqp4kxzMO/uftnQyYpYKsOkIZppkVcuk2ksoPMrs+ c/kc0tULupvLIJMCWMAxOKAGryR0jsM3aeaq8BgpPLmlcUrR024Yn0RNM8j9vFHz 5b35tVv6M7pCV3VNQGg10fRid4h1OHbnV/KUALMSCuErRwIrg09J+4bllFODCf3M W2QHdYE1q7fn+1gakrkE3BaYeAVlMXu/uGEuR1fUpmK5nnOKdj1Rz+6mOvIAUEK3 qsXIXX8xm0GuiWOIIg== =M77U -----END PGP SIGNATURE----- --sVtwlF7vBIdjQ4PVvfNB2UD97ITDdwMOc--