unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting
@ 2022-03-03 21:13 Ludovic Courtès
  2022-03-03 21:14 ` [bug#54241] [PATCH 1/4] http-client: Add response headers to '&http-get-error' Ludovic Courtès
  2022-03-04 12:07 ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Maxime Devos
  0 siblings, 2 replies; 27+ messages in thread
From: Ludovic Courtès @ 2022-03-03 21:13 UTC (permalink / raw)
  To: 54241; +Cc: Ludovic Courtès, Nicolas Goaziou

Hi Guix!

These patches address a famous complaint about “the GitHub problem”
that affects ‘guix refresh’¹, shown here in its naked awfulness:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix refresh
gnu/packages/zig.scm:32:13: zig would be upgraded from 0.9.0 to 0.9.1

[...]

In guix/scripts/refresh.scm:
   578:14  5 (_ _)
In srfi/srfi-1.scm:
    634:9  4 (for-each #<procedure 7fe85c9a8e00 at guix/scripts/refresh.scm:578:24 (t-916fdc98f4be2f1-1d48)> _)
In guix/scripts/refresh.scm:
    378:2  3 (check-for-package-update #<package redshift-wayland@1.12-1.7da875d gnu/packages/xdisorg.scm:1425 7fe85879e790> (#<<upstream…>) …)
In guix/import/github.scm:
   232:12  2 (latest-release _)
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Error downloading release information through the GitHub
API. This may be fixed by using an access token and setting the environment
variable GUIX_GITHUB_TOKEN, for instance one procured from
https://github.com/settings/tokens
--8<---------------cut here---------------end--------------->8---

With this change, ‘guix refresh’ warns you when the GitHub rate limit
is reached, but it keeps going, falling back to the ‘generic-git’
updater if it’s among the applicable updaters:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix refresh -t github,generic-git

[...]

guix refresh: warning: GitHub rate limit exceeded; disallowing requests for 1477 seconds
hint: You can waive the rate limit by setting the `GUIX_GITHUB_TOKEN' environment variable to a
token obtained from `https://github.com/settings/tokens' with your GitHub account.

Alternatively, you can wait until your rate limit is reset, or use the `generic-git' updater
instead.

gnu/packages/zile.scm:113:14: warning: no tags were found for zile-on-guile
gnu/packages/zig.scm:32:13: zig would be upgraded from 0.9.0 to 0.9.1
gnu/packages/xorg.scm:2830:7: warning: no valid tags found for xf86-video-freedreno
gnu/packages/xml.scm:2132:13: java-kxml2 would be upgraded from 2.4.2 to 2.5.0
--8<---------------cut here---------------end--------------->8---

The GitHub updater becomes functional again once the rate limit has
been reset.

The code to deal with rate limiting is similar to that in (guix swh).

Thoughts?

Thanks,
Ludo’.

¹ https://issues.guix.gnu.org/53818#50

Ludovic Courtès (4):
  http-client: Add response headers to '&http-get-error'.
  import: github: Gracefully handle rate limit exhaustion.
  http-client: Correctly handle redirects when #:keep-alive? #t.
  import: github: Reuse HTTP connection for the /tags URL fallback.

 .dir-locals.el         |   1 +
 guix/http-client.scm   |  39 +++++++++-----
 guix/import/github.scm | 119 +++++++++++++++++++++++++++++------------
 3 files changed, 112 insertions(+), 47 deletions(-)


base-commit: be84fb701bf7a36a0eb50147ccbb988aa3f41209
-- 
2.34.0





^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2022-03-06 21:57 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-03-03 21:13 [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-03 21:14 ` [bug#54241] [PATCH 1/4] http-client: Add response headers to '&http-get-error' Ludovic Courtès
2022-03-03 21:14   ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Ludovic Courtès
2022-03-05  9:35     ` Maxime Devos
2022-03-05 22:00       ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-05  9:37     ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Maxime Devos
2022-03-05 22:01       ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-05  9:48     ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Maxime Devos
2022-03-05 22:09       ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-05  9:48     ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Maxime Devos
2022-03-05 22:03       ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-05 22:16         ` Maxime Devos
2022-03-06 17:18           ` Ludovic Courtès
2022-03-05 22:21         ` Maxime Devos
2022-03-05  9:52     ` [bug#54241] [PATCH 2/4] import: github: Gracefully handle rate limit exhaustion Maxime Devos
2022-03-05 22:06       ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-03 21:14   ` [bug#54241] [PATCH 3/4] http-client: Correctly handle redirects when #:keep-alive? #t Ludovic Courtès
2022-03-04 12:39     ` Maxime Devos
2022-03-06 21:55       ` bug#54241: [PATCH 0/4] 'github' importer gracefully handles rate limiting Ludovic Courtès
2022-03-03 21:14   ` [bug#54241] [PATCH 4/4] import: github: Reuse HTTP connection for the /tags URL fallback Ludovic Courtès
2022-03-04 12:07 ` [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting Maxime Devos
2022-03-04 20:45   ` Ludovic Courtès
2022-03-05  9:44     ` Maxime Devos
2022-03-05 21:58       ` Ludovic Courtès
2022-03-05 22:04         ` Maxime Devos
2022-03-05 22:11         ` Maxime Devos
2022-03-06 17:21           ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).