From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id GEn2AtspuV6NFQAA0tVLHw (envelope-from ) for ; Mon, 11 May 2020 10:32:59 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id GAR5OOgpuV6/HgAAB5/wlQ (envelope-from ) for ; Mon, 11 May 2020 10:33:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 528599400CB for ; Mon, 11 May 2020 10:33:10 +0000 (UTC) Received: from localhost ([::1]:60584 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jY5kE-0004M2-GQ for larch@yhetil.org; Mon, 11 May 2020 06:33:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49792) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jY5k6-0004Ll-N8 for bug-guix@gnu.org; Mon, 11 May 2020 06:33:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:39764) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jY5k6-00061R-D4 for bug-guix@gnu.org; Mon, 11 May 2020 06:33:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jY5k6-00039q-9c for bug-guix@gnu.org; Mon, 11 May 2020 06:33:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#37207: guix.gnu.org returns Last-Modified = Epoch Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 11 May 2020 10:33:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 37207 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 37207-submit@debbugs.gnu.org id=B37207.158919315012087 (code B ref 37207); Mon, 11 May 2020 10:33:02 +0000 Received: (at 37207) by debbugs.gnu.org; 11 May 2020 10:32:30 +0000 Received: from localhost ([127.0.0.1]:51310 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jY5jZ-00038t-NB for submit@debbugs.gnu.org; Mon, 11 May 2020 06:32:30 -0400 Received: from mira.cbaines.net ([212.71.252.8]:54646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jY5jX-00038j-PP for 37207@debbugs.gnu.org; Mon, 11 May 2020 06:32:28 -0400 Received: from localhost (unknown [46.237.174.28]) by mira.cbaines.net (Postfix) with ESMTPSA id D156E27BBE1; Mon, 11 May 2020 11:32:25 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 4450712e; Mon, 11 May 2020 10:32:23 +0000 (UTC) References: <875zmhliqj.fsf@gnu.org> <87o8qwg3te.fsf@cbaines.net> <874ksoaym3.fsf@gnu.org> User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines In-reply-to: <874ksoaym3.fsf@gnu.org> Date: Mon, 11 May 2020 11:32:19 +0100 Message-ID: <87imh2pxsc.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 37207@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 X-Spam-Score: -3.11 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Scan-Result: default: False [-3.11 / 13.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.53975740515929]; TO_DN_SOME(0.00)[]; DWL_DNSWL_FAIL(0.00)[209.51.188.17:server fail]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.07), country: US(-0.00), ip: 209.51.188.17(-0.54)]; R_SPF_ALLOW(-0.20)[+ip4:209.51.188.0/24]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; SIGNED_PGP(-2.00)[]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; RCVD_IN_DNSWL_FAIL(0.00)[209.51.188.17:server fail]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:22989, ipnet:209.51.188.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[larch=yhetil.org]; FROM_NEQ_ENVFROM(0.00)[mail@cbaines.net,bug-guix-bounces@gnu.org]; ARC_NA(0.00)[]; URIBL_BLOCKED(0.00)[cbaines.net:email,gnu.org:email]; FROM_HAS_DN(0.00)[]; SPF_REPUTATION_HAM(0.00)[-0.56555365014368]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DMARC_NA(0.00)[cbaines.net]; HAS_LIST_UNSUB(-0.01)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.51.188.17:from]; RCVD_COUNT_SEVEN(0.00)[9]; FORGED_SENDER_MAILLIST(0.00)[] X-TUID: 3dbRu/g9oUns --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Howdy! > > Christopher Baines skribis: > >> Ludovic Court=C3=A8s writes: >> >>> Since the use of the =E2=80=98static-web-site=E2=80=99 service, which p= uts web site >>> files in the store, nginx returns a =E2=80=98Last-Modified=E2=80=99 hea= der that can >>> trick clients into caching things forever: >>> >>> --8<---------------cut here---------------start------------->8--- >>> $ wget --debug -O /dev/null https://guix.gnu.org/packages.json 2>&1 |= grep Last >>> Last-Modified: Thu, 01 Jan 1970 00:00:01 GMT >>> --8<---------------cut here---------------end--------------->8--- >>> >>> We should tell nginx to do not emit =E2=80=98Last-Modified=E2=80=99, or= to take the >>> state from the /srv/guix.gnu.org symlink, if possible. >> >> I ended up looking at this again in relation to Repology [1]. >> >> 1: https://github.com/repology/repology-updater/issues/218#issuecomment-= 525905704 >> >> Going back to that comment, given that the Last-Modified header (and the >> ETag) is wrong, it's probably sensible to remove them. That might even >> fix the issue with Repology fetching the packages.json file. >> >> Alternatively (or in addition), we could run a really simple Guile web >> server that just serves the packages.json file with the right >> Last-Modified value, and have NGinx proxy requests to that server. This >> would be pretty easy to setup I believe, and would allow providing a >> correct value. >> >> Thoughts? > > I think it wouldn=E2=80=99t really help because the Last-Modified issue is > pervasive. It shows for instance when accessing the web site: one often > has to force the browser to reload pages to get the latest version. > > So I=E2=80=99m all for one of the solutions that were proposed earlier. > > WDYT? So I think removing the Last-Modified header from the responses will fix the issue with the Repology fetcher (as it will stop thinking it's already fetch the file, since it was last modified in 1970), instead it will just always process the file. Removing the Last-Modified header, and maybe the ETag as well from responses should avoid the issue with web browsers using a cached version of the page when they probably shouldn't. I realise what I described with using a Guile web server to serve the packages.json file wouldn't help with other pages (unless they're served as well, which is a possibility), but that was just an optimisation over removing the header entirely, as having the Last-Modified header, with a correct value would help the Repology fetcher cache the file. Does that make sense? It still seems to me that a small change to the NGinx config (I think these lines somewhere in the config would do it [1]) would help with the Repology fetcher issue, and the issue you describe with web browsers. 1: add_header Last-Modified ""; if_modified_since off; etag off; --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl65KbNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdHlA//cd0NZrmdmt1wzM1x48Mi8hbvjM0V1ySm2wJcmgcxuvqxyhJYMZXNghn8 zGQ78Z5ZBSxd4qI9WWFNe69uPewHbmTWQliu+Ju5RJSDwmPKC4EuMOWidGgcDIUh E0PF0549JYkBG2K9YAFDnnGFa+d9suJpAvqfVcNmdcPUbnXPNYj8wql0/EiYBvZU 7DnLFhiQjLxIt6za2dE7+P3kzqpOUW49b/EE8snk8/10reJhxRYuxEObTK1Vx5zL CbceiaxzYiZ5BZLu7J1zh+oH3GbErjoTI2YlDIcp7wQ7GYAhK4eWh3uRzxecCK3H A8A30EkLeKGZItHlo16djuqGKGu8rf29dchKViRaNrJ/aq/OiL+MHLbkE7zmQp9n EZNbAVBQopTuM6VgxeSvkDE+8QfoOeM2tQI6ufS3CEvPBqjIYPd+00IYk2ufjTXR tL4OmcHEbeafqnacrJDelvsIWQZPbfWXcS4hYljf76EVXWS5pnPdwCi7NfZM8yOp MpDRkAmZI/QGdHrOPjLLMm1awzT/J3ee69opMHWM+7NHYQyObysknfSDMw8SIknD bJv/I8QrjIEKEFd4PXClRIgJVBKfJ5V2v8Hz9Ry8AKXxI5NUR8CCSraAwJ2hRBOH t1yEDfQAP3uKKCXSwJ7ZvW3et7RcOzY1ggZG+8O4jH8sURGjTIs= =xvw6 -----END PGP SIGNATURE----- --=-=-=--