From: Attila Lendvai <attila@lendvai.name>
To: 55242@debbugs.gnu.org
Cc: Attila Lendvai <attila@lendvai.name>
Subject: [bug#55242] [PATCH 06/10] guix: import: go: Add a local duplicate of http-fetch.
Date: Tue, 3 May 2022 13:42:57 +0200 [thread overview]
Message-ID: <20220503114301.9524-6-attila@lendvai.name> (raw)
In-Reply-To: <20220503114301.9524-2-attila@lendvai.name>
This is needed when dealing with golang packages, as per:
https://golang.org/ref/mod#vcs-find
A page may return 404, but at the same time also contain the sought after
`go-import` meta tag. An example for such a project/page is:
https://www.gonum.org/v1/gonum?go-get=1
It's not enough to just handle the thrown exception, because we need to be
able to get hold of the fetched content, too.
Discussion why it's duplicated here: https://issues.guix.gnu.org/54836
* guix/import/go.scm (http-fetch): Add a copy and extend it with the
accept-all-response-codes? param.
---
guix/import/go.scm | 107 ++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 105 insertions(+), 2 deletions(-)
diff --git a/guix/import/go.scm b/guix/import/go.scm
index 6b0fbaa8b6..a08005d090 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -33,13 +33,22 @@ (define-module (guix import go)
#:use-module (guix import json)
#:use-module (guix packages)
#:use-module ((guix utils) #:select (string-replace-substring))
- #:use-module (guix http-client)
+ ;; FIXME? We use a local copy of http-fetch.
+ ;; See https://issues.guix.gnu.org/54836
+ #:use-module ((guix http-client) #:hide (http-fetch))
+ #:use-module (guix base64)
+ #:use-module (rnrs bytevectors)
+ #:use-module ((guix build download)
+ #:select (open-socket-for-uri
+ (open-connection-for-uri
+ . guix:open-connection-for-uri)
+ resolve-uri-reference))
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix memoization)
#:autoload (htmlprag) (html->sxml) ;from Guile-Lib
#:autoload (guix serialization) (write-file)
#:autoload (guix base32) (bytevector->nix-base32-string)
- #:autoload (guix build utils) (mkdir-p)
+ #:autoload (guix build utils) (mkdir-p dump-port)
#:autoload (gcrypt hash) (hash-algorithm sha256)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
@@ -69,6 +78,100 @@ (define-module (guix import go)
go-module->guix-package*
go-module-recursive-import))
+;; This is a duplicate from (guix http-client) with the addition of a
+;; #:accept-all-response-codes? param. See https://issues.guix.gnu.org/54836
+(define* (http-fetch uri #:key port (text? #f) (buffered? #t)
+ (open-connection guix:open-connection-for-uri)
+ (keep-alive? #f)
+ (verify-certificate? #t)
+ (headers '((user-agent . "GNU Guile")))
+ (log-port (current-error-port))
+ timeout
+ (accept-all-response-codes? #f))
+ "Return an input port containing the data at URI, and the expected number of
+bytes available or #f. If TEXT? is true, the data at URI is considered to be
+textual. Follow any HTTP redirection. When BUFFERED? is #f, return an
+unbuffered port, suitable for use in `filtered-port'. HEADERS is an alist of
+extra HTTP headers.
+
+When KEEP-ALIVE? is true, the connection is marked as 'keep-alive' and PORT is
+not closed upon completion.
+
+When VERIFY-CERTIFICATE? is true, verify HTTPS server certificates.
+
+TIMEOUT specifies the timeout in seconds for connection establishment; when
+TIMEOUT is #f, connection establishment never times out.
+
+Write information about redirects to LOG-PORT.
+
+When ACCEPT-ALL-RESPONSE-CODES? is false then raise an '&http-get-error'
+condition if downloading fails, otherwise return the response regardless
+of the reponse code."
+ (define parsed-initial-uri
+ (if (string? uri) (string->uri uri) uri))
+
+ (define (open-connection* uri)
+ (open-connection uri
+ #:verify-certificate? verify-certificate?
+ #:timeout timeout))
+
+ (let loop ((current-uri parsed-initial-uri)
+ (current-port (or port (open-connection parsed-initial-uri))))
+ (let ((headers (match (uri-userinfo current-uri)
+ ((? string? str)
+ (cons (cons 'Authorization
+ (string-append "Basic "
+ (base64-encode
+ (string->utf8 str))))
+ headers))
+ (_ headers))))
+ (unless (or buffered? (not (file-port? current-port)))
+ (setvbuf current-port 'none))
+ (let*-values (((resp data)
+ (http-get current-uri #:streaming? #t #:port current-port
+ #:keep-alive? keep-alive?
+ #:headers headers))
+ ((code)
+ (response-code resp)))
+ (case code
+ ((200)
+ (values data (response-content-length resp)))
+ ((301 ; moved permanently
+ 302 ; found (redirection)
+ 303 ; see other
+ 307 ; temporary redirection
+ 308) ; permanent redirection
+ (let ((host (uri-host current-uri))
+ (new-uri (resolve-uri-reference (response-location resp)
+ current-uri)))
+ (if keep-alive?
+ (dump-port data (%make-void-port "w0")
+ (response-content-length resp))
+ (close-port current-port))
+ (format log-port (G_ "following redirection to `~a'...~%")
+ (uri->string new-uri))
+ (loop new-uri
+ (or (and keep-alive?
+ (or (not (uri-host new-uri))
+ (string=? host (uri-host new-uri)))
+ current-port)
+ (open-connection* new-uri)))))
+ (else
+ (if accept-all-response-codes?
+ (values data (response-content-length resp))
+ (raise (condition (&http-get-error
+ (uri current-uri)
+ (code code)
+ (reason (response-reason-phrase resp))
+ (headers (response-headers resp)))
+ (&message
+ (message
+ (format
+ #f
+ (G_ "~a: HTTP download failed: ~a (~s)")
+ (uri->string current-uri) code
+ (response-reason-phrase resp)))))))))))))
+
;;; Commentary:
;;;
;;; (guix import go) attempts to make it easier to create Guix package
--
2.35.1
next prev parent reply other threads:[~2022-05-03 11:44 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-03 11:16 [bug#55242] [PATCH 01/10] guix: import: Print the number of packages at the end Attila Lendvai
2022-05-03 11:42 ` [bug#55242] [PATCH 02/10] guix: import: go: Rename go.pkg.dev-info to pkg.go.dev-info Attila Lendvai
2022-05-03 11:42 ` [bug#55242] [PATCH 03/10] guix: import: go: Add mockup logging facility Attila Lendvai
2022-05-03 16:23 ` Maxime Devos
2022-06-14 19:46 ` [bug#55242] [PATCH 01/10] guix: import: Print the number of packages at the end Maxim Cournoyer
2022-05-03 11:42 ` [bug#55242] [PATCH 04/10] guix: import: go: Fix the interpretation of the replace directive Attila Lendvai
2022-05-03 16:25 ` Maxime Devos
2022-05-03 16:26 ` Maxime Devos
2022-05-03 16:27 ` Maxime Devos
2022-05-03 16:28 ` Maxime Devos
2022-05-03 11:42 ` [bug#55242] [PATCH 05/10] guix: import: go: Harden sxml->texi conversion Attila Lendvai
2022-05-03 16:32 ` Maxime Devos
2022-05-03 11:42 ` Attila Lendvai [this message]
2022-05-03 11:42 ` [bug#55242] [PATCH 07/10] guix: import: go: More resilience wrt network errors; add logging Attila Lendvai
2022-05-03 16:12 ` Maxime Devos
2022-05-03 17:00 ` Attila Lendvai
2022-05-03 17:38 ` Maxime Devos
2022-05-09 12:34 ` Attila Lendvai
2022-05-09 17:45 ` Maxime Devos
2022-05-09 20:02 ` Attila Lendvai
2022-05-09 20:08 ` Maxime Devos
2022-05-03 16:17 ` Maxime Devos
2022-05-03 16:36 ` Maxime Devos
2022-05-03 16:37 ` Maxime Devos
2022-05-03 11:42 ` [bug#55242] [PATCH 08/10] guix: import: go: Modules in a subdir and prefixed tags Attila Lendvai
2022-05-03 16:42 ` Maxime Devos
2022-05-03 16:48 ` Maxime Devos
2022-05-03 16:50 ` Maxime Devos
2022-05-09 12:50 ` Attila Lendvai
2022-05-03 16:51 ` Maxime Devos
2022-05-03 16:53 ` Maxime Devos
2022-05-03 16:55 ` Maxime Devos
2022-05-03 16:56 ` Maxime Devos
2022-05-03 16:59 ` Maxime Devos
2022-05-03 17:00 ` Maxime Devos
2022-05-03 17:01 ` Maxime Devos
2022-05-03 11:43 ` [bug#55242] [PATCH 09/10] guix: import: go: module-name -> module-path to be consistent Attila Lendvai
2022-05-03 11:43 ` [bug#55242] [PATCH 10/10] guix: import: go: Better handling of /v2 in the module path Attila Lendvai
2022-05-09 20:39 ` Maxime Devos
2022-05-03 16:22 ` [bug#55242] [PATCH 02/10] guix: import: go: Rename go.pkg.dev-info to pkg.go.dev-info Maxime Devos
2022-05-03 16:21 ` [bug#55242] [PATCH 01/10] guix: import: Print the number of packages at the end Maxime Devos
2022-05-03 17:29 ` Maxime Devos
2022-05-09 12:37 ` Attila Lendvai
2022-05-14 7:09 ` Maxime Devos
2022-05-14 7:15 ` Maxime Devos
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220503114301.9524-6-attila@lendvai.name \
--to=attila@lendvai.name \
--cc=55242@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).