[PATCH] download: Support 'https_proxy'.

[PATCH] download: Support 'https_proxy'.

[宋文武]

[-- Attachment #1: Type: text/plain, Size: 94 bytes --]

Hello, this patch add 'https_proxy' to 'guix download' (and guix-daemon
if we update guix?):


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-download-Support-https_proxy.patch --]
[-- Type: text/x-patch, Size: 3085 bytes --]

From 424da6e43ba9c928403e3fd9b42e75d0fe90fc23 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>
Date: Fri, 10 May 2019 21:27:40 +0800
Subject: [PATCH] download: Support 'https_proxy'.

* guix/build/download.scm (setup-http-tunnel): New procedure.
(open-connection-for-uri): Honor the 'https_proxy' environment variable.
---
 guix/build/download.scm | 28 ++++++++++++++++++++++------
 1 file changed, 22 insertions(+), 6 deletions(-)

diff --git a/guix/build/download.scm b/guix/build/download.scm
index a64e0f0bd3..92cef76dff 100644
--- a/guix/build/download.scm
+++ b/guix/build/download.scm
@@ -380,6 +380,20 @@ ETIMEDOUT error is raised."
               (apply throw args)
               (loop (cdr addresses))))))))
 
+(define (setup-http-tunnel port uri)
+  "Establish a tunnel to the destination server of URI."
+  (define target
+    (string-append (uri-host uri) ":"
+                   (number->string
+                    (or (uri-port uri)
+                        (match (uri-scheme uri)
+                          ('http 80)
+                          ('https 443))))))
+  (format port "CONNECT ~a HTTP/1.1\r\n" target)
+  (format port "Host: ~a\r\n\r\n" target)
+  (force-output port)
+  (read-response port))
+
 (define* (open-connection-for-uri uri
                                   #:key
                                   timeout
@@ -393,21 +407,20 @@ VERIFY-CERTIFICATE? is true, verify HTTPS server certificates."
   (define https?
     (eq? 'https (uri-scheme uri)))
 
+  (define https-proxy (let ((proxy (getenv "https_proxy")))
+                        (and (not (equal? proxy ""))
+                             proxy)))
+
   (let-syntax ((with-https-proxy
                 (syntax-rules ()
                   ((_ exp)
                    ;; For HTTPS URIs, honor 'https_proxy', not 'http_proxy'.
-                   ;; FIXME: Proxying is not supported for https.
                    (let ((thunk (lambda () exp)))
                      (if (and https?
                               (module-variable
                                (resolve-interface '(web client))
                                'current-http-proxy))
-                         (parameterize ((current-http-proxy #f))
-                           (when (and=> (getenv "https_proxy")
-                                        (negate string-null?))
-                             (format (current-error-port)
-                                     "warning: 'https_proxy' is ignored~%"))
+                         (parameterize ((current-http-proxy https-proxy))
                            (thunk))
                          (thunk)))))))
     (with-https-proxy
@@ -415,6 +428,9 @@ VERIFY-CERTIFICATE? is true, verify HTTPS server certificates."
        ;; Buffer input and output on this port.
        (setvbuf s 'block %http-receive-buffer-size)
 
+       (when https-proxy
+         (setup-http-tunnel s uri))
+
        (if https?
            (tls-wrap s (uri-host uri)
                      #:verify-certificate? verify-certificate?)
-- 
2.19.2


[-- Attachment #3: Type: text/plain, Size: 293 bytes --]



Some problems and questions:

- It assumes ‘https_proxy’ is ‘http://PROXY-SERVER:PORT’, if the scheme
  part is missing, it fail.

- It fails some servers (eg: www.google.com) for me while curl works...

- I think this should go into guile’s ‘(web client)’ module?

Re: [PATCH] download: Support 'https_proxy'.

[Ludovic Courtès]

Hi!

iyzsong@member.fsf.org (宋文武) skribis:

> Hello, this patch add 'https_proxy' to 'guix download' (and guix-daemon
> if we update guix?):

Neat!

> From 424da6e43ba9c928403e3fd9b42e75d0fe90fc23 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>
> Date: Fri, 10 May 2019 21:27:40 +0800
> Subject: [PATCH] download: Support 'https_proxy'.
>
> * guix/build/download.scm (setup-http-tunnel): New procedure.
> (open-connection-for-uri): Honor the 'https_proxy' environment variable.

[...]

> +(define (setup-http-tunnel port uri)
> +  "Establish a tunnel to the destination server of URI."

Maybe “Establish over PORT an HTTP tunnel to the destination server of
URI.”?

Otherwise LGTM!

> Some problems and questions:
> 
> - It assumes ‘https_proxy’ is ‘http://PROXY-SERVER:PORT’, if the scheme
>   part is missing, it fail.

That’s already the case with ‘http_proxy’.

It seems that other tools can happily deal with the lack of a URI
scheme, so perhaps in a subsequent patch we should add code to
automatically add a URI scheme when it’s missing?

> - It fails some servers (eg: www.google.com) for me while curl works...

For www.google.com it fails even without ‘https_proxy’, so that’s OK.
:-)

> - I think this should go into guile’s ‘(web client)’ module?

Yes!  Once we’ve committed it Guix, it’d be great if you could a similar
patch to bug-guile@gnu.org.

Thank you!

Ludo’.

Re: [PATCH] download: Support 'https_proxy'.

[宋文武]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> iyzsong@member.fsf.org (宋文武) skribis:
>
>> Hello, this patch add 'https_proxy' to 'guix download' (and guix-daemon
>> if we update guix?):
>
> Neat!

Pushed, thank you for the review!

>
>> From 424da6e43ba9c928403e3fd9b42e75d0fe90fc23 Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>
>> Date: Fri, 10 May 2019 21:27:40 +0800
>> Subject: [PATCH] download: Support 'https_proxy'.
>>
>> * guix/build/download.scm (setup-http-tunnel): New procedure.
>> (open-connection-for-uri): Honor the 'https_proxy' environment variable.
>
> [...]
>
>> +(define (setup-http-tunnel port uri)
>> +  "Establish a tunnel to the destination server of URI."
>
> Maybe “Establish over PORT an HTTP tunnel to the destination server of
> URI.”?

Sure.

>
> Otherwise LGTM!
>
>> Some problems and questions:
>> 
>> - It assumes ‘https_proxy’ is ‘http://PROXY-SERVER:PORT’, if the scheme
>>   part is missing, it fail.
>
> That’s already the case with ‘http_proxy’.
>
> It seems that other tools can happily deal with the lack of a URI
> scheme, so perhaps in a subsequent patch we should add code to
> automatically add a URI scheme when it’s missing?

Yes, I think the URI scheme can be ‘http’, ‘https’ or ‘socks5’, etc. and
default to ‘http’.  We only have ‘http’ now, other are good exercise :)

>
>> - It fails some servers (eg: www.google.com) for me while curl works...
>
> For www.google.com it fails even without ‘https_proxy’, so that’s OK.
> :-)
>
>> - I think this should go into guile’s ‘(web client)’ module?
>
> Yes!  Once we’ve committed it Guix, it’d be great if you could a similar
> patch to bug-guile@gnu.org.
>
> Thank you!
>
> Ludo’.

Okay, get it!