From: "Ludovic Courtès" <ludo@gnu.org>
To: 54241@debbugs.gnu.org
Cc: "Ludovic Courtès" <ludo@gnu.org>,
"Nicolas Goaziou" <mail@nicolasgoaziou.fr>
Subject: [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting
Date: Thu, 3 Mar 2022 22:13:26 +0100 [thread overview]
Message-ID: <20220303211326.19884-1-ludo@gnu.org> (raw)
Hi Guix!
These patches address a famous complaint about “the GitHub problem”
that affects ‘guix refresh’¹, shown here in its naked awfulness:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix refresh
gnu/packages/zig.scm:32:13: zig would be upgraded from 0.9.0 to 0.9.1
[...]
In guix/scripts/refresh.scm:
578:14 5 (_ _)
In srfi/srfi-1.scm:
634:9 4 (for-each #<procedure 7fe85c9a8e00 at guix/scripts/refresh.scm:578:24 (t-916fdc98f4be2f1-1d48)> _)
In guix/scripts/refresh.scm:
378:2 3 (check-for-package-update #<package redshift-wayland@1.12-1.7da875d gnu/packages/xdisorg.scm:1425 7fe85879e790> (#<<upstream…>) …)
In guix/import/github.scm:
232:12 2 (latest-release _)
In ice-9/boot-9.scm:
1685:16 1 (raise-exception _ #:continuable? _)
1685:16 0 (raise-exception _ #:continuable? _)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Error downloading release information through the GitHub
API. This may be fixed by using an access token and setting the environment
variable GUIX_GITHUB_TOKEN, for instance one procured from
https://github.com/settings/tokens
--8<---------------cut here---------------end--------------->8---
With this change, ‘guix refresh’ warns you when the GitHub rate limit
is reached, but it keeps going, falling back to the ‘generic-git’
updater if it’s among the applicable updaters:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix refresh -t github,generic-git
[...]
guix refresh: warning: GitHub rate limit exceeded; disallowing requests for 1477 seconds
hint: You can waive the rate limit by setting the `GUIX_GITHUB_TOKEN' environment variable to a
token obtained from `https://github.com/settings/tokens' with your GitHub account.
Alternatively, you can wait until your rate limit is reset, or use the `generic-git' updater
instead.
gnu/packages/zile.scm:113:14: warning: no tags were found for zile-on-guile
gnu/packages/zig.scm:32:13: zig would be upgraded from 0.9.0 to 0.9.1
gnu/packages/xorg.scm:2830:7: warning: no valid tags found for xf86-video-freedreno
gnu/packages/xml.scm:2132:13: java-kxml2 would be upgraded from 2.4.2 to 2.5.0
--8<---------------cut here---------------end--------------->8---
The GitHub updater becomes functional again once the rate limit has
been reset.
The code to deal with rate limiting is similar to that in (guix swh).
Thoughts?
Thanks,
Ludo’.
¹ https://issues.guix.gnu.org/53818#50
Ludovic Courtès (4):
http-client: Add response headers to '&http-get-error'.
import: github: Gracefully handle rate limit exhaustion.
http-client: Correctly handle redirects when #:keep-alive? #t.
import: github: Reuse HTTP connection for the /tags URL fallback.
.dir-locals.el | 1 +
guix/http-client.scm | 39 +++++++++-----
guix/import/github.scm | 119 +++++++++++++++++++++++++++++------------
3 files changed, 112 insertions(+), 47 deletions(-)
base-commit: be84fb701bf7a36a0eb50147ccbb988aa3f41209
--
2.34.0
next reply other threads:[~2022-03-03 21:29 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-03 21:13 Ludovic Courtès [this message]
2022-03-03 21:14 ` [bug#54241] [PATCH 1/4] http-client: Add response headers to '&http-get-error' Ludovic Courtès
2022-03-03 21:14 ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Ludovic Courtès
2022-03-05 9:35 ` Maxime Devos
2022-03-05 22:00 ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-05 9:37 ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Maxime Devos
2022-03-05 22:01 ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-05 9:48 ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Maxime Devos
2022-03-05 22:09 ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-05 9:48 ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Maxime Devos
2022-03-05 22:03 ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-05 22:16 ` Maxime Devos
2022-03-06 17:18 ` Ludovic Courtès
2022-03-05 22:21 ` Maxime Devos
2022-03-05 9:52 ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Maxime Devos
2022-03-05 22:06 ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-03 21:14 ` [bug#54241] [PATCH 3/4] http-client: Correctly handle redirects when #:keep-alive? #t Ludovic Courtès
2022-03-04 12:39 ` Maxime Devos
2022-03-06 21:55 ` bug#54241: [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-03 21:14 ` [bug#54241] [PATCH 4/4] import: github: Reuse HTTP connection for the /tags URL fallback Ludovic Courtès
2022-03-04 12:07 ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Maxime Devos
2022-03-04 20:45 ` Ludovic Courtès
2022-03-05 9:44 ` Maxime Devos
2022-03-05 21:58 ` Ludovic Courtès
2022-03-05 22:04 ` Maxime Devos
2022-03-05 22:11 ` Maxime Devos
2022-03-06 17:21 ` Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220303211326.19884-1-ludo@gnu.org \
--to=ludo@gnu.org \
--cc=54241@debbugs.gnu.org \
--cc=mail@nicolasgoaziou.fr \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).