bug#10478: 24.0.50; url-http-parse-headers can silently drop the response when handling BASIC AUTHENTICATION

[David Engster <deng@randomsample.de>]

Jerry Asher writes:
> The next bug occurs in url-http-parse-headers where the 401 and 407 cases which
> deal with authentication do not take into account that the url contents may
> have changed as a result of authentication. And there all I did was to cut and
> paste the previously self-declared hack from the 3XX case in.

[...]

>          (unauthorized                  ; 401
>           ;; The request requires user authentication.  The response
>           ;; MUST include a WWW-Authenticate header field containing a
>           ;; challenge applicable to the requested resource.  The
>           ;; client MAY repeat the request with a suitable
>           ;; Authorization header field.
>
>           ;; bug patch because url-http-handle-authentication
>           ;; might return a new buffer
>
>           (let ((retval (url-http-handle-authentication nil)))
>             (url-http-debug "Url Http Parse Headers: handling
> authentication return buffer TO %s" retval)
>             (when retval
>               ;; Put in the current buffer a forwarding pointer to the new
>               ;; destination buffer.
>               ;; FIXME: This is a hack to fix url-retrieve-synchronously
>               ;; without changing the API.  Instead url-retrieve should
>               ;; either simply not return the "destination" buffer, or it
>               ;; should take an optional `dest-buf' argument.
>               (set (make-local-variable 'url-redirect-buffer)
>                    retval)
>               (url-http-debug "Url Http Parse Headers: handling
> authentication return buffer TO %s -> %s 2:"
>                               retval url-redirect-buffer)
>               (url-mark-buffer-as-dead buffer))))

That did not work for me. The reason is that at the end of this block,
`url-mark-buffer-as-dead' will return non-nil, so the 'success'-flag
will be set. If the callback is called immediately because
Content-Length is '0' (which is often the case for '401'),
`url-retrieve-synchronously' won't look at the url-redirect-buffer
variable.

So I let that block always return 'nil' and that seems to do the trick,
but I'll do some more testing. In the meantime, I'll send you the form
for assigning copyright to the FSF in a separate mail. [Whether a patch
is considered 'trivial' almost entirely depends on quantity. The general
rule is: If the amount of added lines is below ~10, it is considered
trivial, otherwise papers are required.]

-David