307 status code handling in url-http-parse-headers

307 status code handling in url-http-parse-headers

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 395 bytes --]

Hi,

I see that `url-http-parse-headers' won't redirect a 307 response for a
POST request.

There's a comment about 301/302 status codes, which says that for other
requests than HEAD and GET, the user agent must not automatically
redirect, which seems to follow the RFC.

However, should we ask the user confirmation for the redirection and do
the redirect depending on the user's choice?

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: 307 status code handling in url-http-parse-headers

[Alexis]

Nicolas Petton <nicolas@petton.fr> writes:

> I see that `url-http-parse-headers' won't redirect a 307 
> response for a POST request.
>
> There's a comment about 301/302 status codes, which says that 
> for other requests than HEAD and GET, the user agent must not 
> automatically redirect, which seems to follow the RFC.
>
> However, should we ask the user confirmation for the redirection 
> and do the redirect depending on the user's choice?

Sorry, do you mean, ask the user for redirect confirmation in the 
context of HEAD and GET requests? The reason i ask is this:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18290

Unfortunately i've not yet been able to find the time to try to 
work on this issue, but asking users for confirmation in the 
context of a PROPFIND request seems like it might not be a good 
idea ....


Alexis.

Re: 307 status code handling in url-http-parse-headers

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 239 bytes --]

Alexis <flexibeast@gmail.com> writes:

> Sorry, do you mean, ask the user for redirect confirmation in the 
> context of HEAD and GET requests? The reason i ask is this:

In my case, I meant in the context of a POST request.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: 307 status code handling in url-http-parse-headers

[Yuri Khan]

On Mon, Feb 1, 2016 at 8:30 PM, Nicolas Petton <nicolas@petton.fr> wrote:

> I see that `url-http-parse-headers' won't redirect a 307 response for a
> POST request.
>
> There's a comment about 301/302 status codes, which says that for other
> requests than HEAD and GET, the user agent must not automatically
> redirect, which seems to follow the RFC.
>
> However, should we ask the user confirmation for the redirection and do
> the redirect depending on the user's choice?

The sad tale of HTTP redirections :(

HTTP/1.0 (RFC 1945) defined 301 and 302 requiring confirmation on
redirecting methods other than HEAD and GET. The intent was that it
was unknown whether the original handler performed the requested
operation before responding with a redirect. The intent was also that
in case of user confirmation the client would follow the redirect with
the original method.

Both browser implementors and web developers misunderstood the spec
and used 301 and 302 redirects to mean “I have performed the operation
you requested; follow the redirect with a GET to see results and/or
further instructions”.

RFC 2068 recognized the above use case and introduced the 303 code to
mean exactly that. It clarified that changing POST to GET on a 301 or
302 is an error. Still requires confirmation before redirecting
non-idempotent methods with 301 or 302, but not for 303.

HTTP/1.1 (RFC 2616) introduced the 307 code with pretty much the same
wording as RFC 2068 for 302, possibly in expectation that this time
developers will get it right. It required a confirmation on the 307,
as well as 301 and 302.

The current version of HTTP/1.1 (RFC 7231) no longer requires
confirmation on 301, 302 or 307. It documents that some clients MAY
(but actually should not) change POST to GET on 301 and 302. It
clarifies that clients MUST NOT change the method on a 307.

Re: 307 status code handling in url-http-parse-headers

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> The current version of HTTP/1.1 (RFC 7231) no longer requires
> confirmation on 301, 302 or 307. It documents that some clients MAY
> (but actually should not) change POST to GET on 301 and 302. It
> clarifies that clients MUST NOT change the method on a 307.

Thank you for the clarifications!

What should we do then?  The current implementation seems to just
ignores the redirection, I would at least ask the user for a
confirmation.

Nico
-- 
Nicolas Petton
http://nicolas-petton.fr

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: 307 status code handling in url-http-parse-headers

[Yuri Khan]

On Mon, Feb 1, 2016 at 11:42 PM, Nicolas Petton <nicolas@petton.fr> wrote:

> What should we do then?  The current implementation seems to just
> ignores the redirection, I would at least ask the user for a
> confirmation.

That’s the million dollar question, of course :)

Well-behaved contemporary sites should probably not respond with 301
and 302 to POST requests, preferring 303 for “I have processed your
request, now please navigate to this new page” and 307 for “I have not
processed your request but the handler on that other URI might”.

Older sites probably expect 301s and 302s to be handled by the client
in the fashion that is correct for 303. Sites that use 301 and 302
redirects as intended expect them to be handled like 307.

If I were designing an HTTP client library now, I would handle 301 and
302 the same way as 307 by default: no confirmation, no method change,
send the same request data. But I would provide an escape hatch — a
single flag that, when explicitly specified, forbids following
redirects and just returns the 30x codes to the application which
should then decide what to do. I would cite RFC 7231 to everybody who
complained about my choice of defaults ;)

cURL does not follow redirects by default. When directed to with the
-L/--location switch, it defaults to changing the method to GET for
301, 302 and 303 but keeping the original method for 307. It has
further switches --post301, --post302 and the bizarre --post303 to
keep POST method for these respective redirects. Its manual says the
default behavior is non-compliant.

Wget, on the other hand, keeps the original method on 301, 302 and
307, and the manual says it is with accordance to RFC 2616.

Re: 307 status code handling in url-http-parse-headers

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 1696 bytes --]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> On Mon, Feb 1, 2016 at 11:42 PM, Nicolas Petton <nicolas@petton.fr> wrote:
>
>> What should we do then?  The current implementation seems to just
>> ignores the redirection, I would at least ask the user for a
>> confirmation.

> If I were designing an HTTP client library now, I would handle 301 and
> 302 the same way as 307 by default: no confirmation, no method change,
> send the same request data. But I would provide an escape hatch — a
> single flag that, when explicitly specified, forbids following
> redirects and just returns the 30x codes to the application which
> should then decide what to do. I would cite RFC 7231 to everybody who
> complained about my choice of defaults ;)
>
> cURL does not follow redirects by default. When directed to with the
> -L/--location switch, it defaults to changing the method to GET for
> 301, 302 and 303 but keeping the original method for 307. It has
> further switches --post301, --post302 and the bizarre --post303 to
> keep POST method for these respective redirects. Its manual says the
> default behavior is non-compliant.
>
> Wget, on the other hand, keeps the original method on 301, 302 and
> 307, and the manual says it is with accordance to RFC 2616.

I'm thinking about adding a variable, which when non-nil would follow
the redirection after a confirmation from the user for 301, 302 and 307
status code responses of all requests, regardless of the HTTP method.

This variable would default to nil, so that the current behavior is
unchanged by default.

Would that be ok with everybody?

Cheers,
Nico
-- 
Nicolas Petton
http://nicolas-petton.fr

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: 307 status code handling in url-http-parse-headers

[Lars Ingebrigtsen]

Nicolas Petton <nicolas@petton.fr> writes:

> I'm thinking about adding a variable, which when non-nil would follow
> the redirection after a confirmation from the user for 301, 302 and 307
> status code responses of all requests, regardless of the HTTP method.
>
> This variable would default to nil, so that the current behavior is
> unchanged by default.
>
> Would that be ok with everybody?

Is there some particular reason you think the user wants to be queried
on a 307 redirect?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: 307 status code handling in url-http-parse-headers

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Nicolas Petton <nicolas@petton.fr> writes:
>
>> I'm thinking about adding a variable, which when non-nil would follow
>> the redirection after a confirmation from the user for 301, 302 and 307
>> status code responses of all requests, regardless of the HTTP method.
>>
>> This variable would default to nil, so that the current behavior is
>> unchanged by default.
>>
>> Would that be ok with everybody?
>
> Is there some particular reason you think the user wants to be queried
> on a 307 redirect?

Do you mean that when this new variable would be non-nil, we could just
follow the redirection without querying the user?  That could also
work.  I'm bringing this in this list because I'm not sure what is the
best behavior.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: 307 status code handling in url-http-parse-headers

[Lars Ingebrigtsen]

Nicolas Petton <nicolas@petton.fr> writes:

> Do you mean that when this new variable would be non-nil, we could just
> follow the redirection without querying the user?  That could also
> work.  I'm bringing this in this list because I'm not sure what is the
> best behavior.

I think that redirection should be done without querying the user, and
that there should be no variable to control this behaviour, I think.
Unless there's a good reason for adding such querying functionality.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: 307 status code handling in url-http-parse-headers

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 435 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I think that redirection should be done without querying the user, and
> that there should be no variable to control this behaviour, I think.
> Unless there's a good reason for adding such querying functionality.

That would work for me, but I tried to come up with a solution that
would not impact users who want to keep the current behavior.  Well,
maybe that's not needed at all.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: 307 status code handling in url-http-parse-headers

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I think that redirection should be done without querying the user, and
> that there should be no variable to control this behaviour, I think.
> Unless there's a good reason for adding such querying functionality.

If that's ok with everybody, then I'll make this change in master.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]