bug#62598: 29.0.60; url-https-proxy-connect doesn't support multi-stage auth to proxies

bug#62598: 29.0.60; url-https-proxy-connect doesn't support multi-stage auth to proxies

[Spencer Baugh]

url-http knows how to use HTTPS proxies, primarily in
url-https-proxy-connect.  It even knows to authenticate to those
proxies, as fixed in bug#42422.

But some HTTP authentication methods (e.g. NTLM as supported by
url-http-ntlm) require multiple stages of back-and-forth in
authentication.  This works fine with regular HTTP requests and requests
to HTTP (non-S) proxies; it's handled by url-http-handle-authentication
which is called by url-http-parse-headers when it sees a 401 or 407
(auth required and proxy auth required) status.

But this does not work with the HTTPS proxy support, because if it sees
401 or 407 as a response to CONNECT, it just immediately fails.

I'm very interested in adding this but I'm unsure how to approach it.  I
guess that url-https-proxy-after-change-function should be calling
something similar to url-http-handle-authentication.  Or maybe the whole
design of how HTTPS proxy support works today is wrong, and it should be
calling url-http-parse-headers like everything else?

bug#62598: 29.0.60; url-https-proxy-connect doesn't support multi-stage auth to proxies

[Thomas Fitzsimmons]

Hi Spencer,

Spencer Baugh <sbaugh@janestreet.com> writes:

> url-http knows how to use HTTPS proxies, primarily in
> url-https-proxy-connect.  It even knows to authenticate to those
> proxies, as fixed in bug#42422.
>
> But some HTTP authentication methods (e.g. NTLM as supported by
> url-http-ntlm) require multiple stages of back-and-forth in
> authentication.  This works fine with regular HTTP requests and requests
> to HTTP (non-S) proxies; it's handled by url-http-handle-authentication
> which is called by url-http-parse-headers when it sees a 401 or 407
> (auth required and proxy auth required) status.
>
> But this does not work with the HTTPS proxy support, because if it sees
> 401 or 407 as a response to CONNECT, it just immediately fails.

Why can't that code path call url-http-handle-authentication instead of
just failing?  What makes HTTPS different from HTTP in this respect?

> I'm very interested in adding this but I'm unsure how to approach it.  I
> guess that url-https-proxy-after-change-function should be calling
> something similar to url-http-handle-authentication.  Or maybe the whole
> design of how HTTPS proxy support works today is wrong, and it should be
> calling url-http-parse-headers like everything else?

I'd say try to make both approaches work, and see which one results in
the minimum set of changes.

Thomas

bug#62598: 29.0.60; url-https-proxy-connect doesn't support multi-stage auth to proxies

[J.P.]

Hi, just wondering if you might be interested in broadening the scope of
this bug into something more ambitious, namely, making proxy handling
more flexible and predictable for libraries doing business with `url'.
I've been tinkering with

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=53941

off and on for a bit, but I'm not familiar enough with the `url'
landscape to go all in. From your bug description, you seem to have a
good handle on the `url-http' parts, so perhaps you're open to exploring
ideas for improving the overall proxy situation `url'-wide. If so, I'd
be willing to investigate how best to adapt `socks' to whatever you
might propose. Just a thought, though (no pressure).