From: Leo Famulari <leo@famulari.name>
To: 52338@debbugs.gnu.org
Subject: bug#52338: [maintenance] hydra: berlin: Create robots.txt.
Date: Mon, 6 Dec 2021 17:18:10 -0500 [thread overview]
Message-ID: <2f52f6b48db55f8a79b07dbb242b297ab49d6083.1638828946.git.leo@famulari.name> (raw)
In-Reply-To: <Ya5+t3YAVCs0KlCN@jasmine.lan>
I tested that `guix system build` does succeed with this change, but I
would like a review on whether the resulting Nginx configuration is
correct, and if this is the correct path to disallow. It generates an
Nginx location block like this:
------
location /robots.txt {
add_header Content-Type text/plain;
return 200 "User-agent: *
Disallow: /nar
";
}
------
* hydra/nginx/berlin.scm (berlin-locations): Add a robots.txt Nginx location.
---
hydra/nginx/berlin.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/hydra/nginx/berlin.scm b/hydra/nginx/berlin.scm
index 1f4b0be..3bb2129 100644
--- a/hydra/nginx/berlin.scm
+++ b/hydra/nginx/berlin.scm
@@ -174,7 +174,14 @@ PUBLISH-URL."
(nginx-location-configuration
(uri "/berlin.guixsd.org-export.pub")
(body
- (list "root /var/www/guix;"))))))
+ (list "root /var/www/guix;")))
+
+ (nginx-location-configuration
+ (uri "/robots.txt")
+ (body
+ (list
+ "add_header Content-Type text/plain;"
+ "return 200 \"User-agent: *\nDisallow: /nar/\n\";"))))))
(define guix.gnu.org-redirect-locations
(list
--
2.34.0
next prev parent reply other threads:[~2021-12-06 22:19 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-06 21:20 bug#52338: Crawler bots are downloading substitutes Leo Famulari
2021-12-06 22:18 ` Leo Famulari [this message]
2021-12-09 13:27 ` Mathieu Othacehe
2021-12-09 15:42 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2021-12-10 16:22 ` Leo Famulari
2021-12-10 16:47 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2021-12-11 9:46 ` Mathieu Othacehe
2021-12-19 16:53 ` Mathieu Othacehe
2021-12-10 21:21 ` Mark H Weaver
2021-12-10 22:52 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2f52f6b48db55f8a79b07dbb242b297ab49d6083.1638828946.git.leo@famulari.name \
--to=leo@famulari.name \
--cc=52338@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).