bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request

[Dmitry Gutov <dgutov@yandex.ru>]

On 08/02/2016 06:25 PM, Eli Zaretskii wrote:

> How about making the temporary buffer parsed by url-generic-parse-url
> a unibyte buffer?  Does that fix the problem?

It does fix anaconda-mode, yes.

> AFAIU, RFC 3986 doesn't
> allow non-ASCII characters, so we should be okay handling that in a
> unibyte buffer, right?

I don't really know. RFC 3986 or not, I suppose in practice the url 
could be quoted before or after it's parsed. And url-parse-tests.el 
doesn't specify this case.

Lars, what do you think?

> I mean something like this:
>
>     (with-temp-buffer
>       ;; Don't let those temp-buffer modifications accidentally
>       ;; deactivate the mark of the current-buffer.
>       (let ((deactivate-mark nil))
>         (set-syntax-table url-parse-syntax-table)
> 	(erase-buffer)
> 	(set-buffer-multibyte nil)   ;; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
> 	(insert url)
> 	(goto-char (point-min))
> 	...

Heh, that's exactly where I added the line, without looking at your code.

> As for other possible problems like that, are there any that could be
> expected already?  If so, we could try fixing them now.

Nothing else jumps out so far. The function depends on quite a few 
global variables. To be really certain, we'd have to trace how all of 
them are created, and for all that are not directly bound by the user, 
the chains of calls that produce them.

> Alternatively, we could just wait for them to come up;

I'm worried about having a problem crop up in some significant use case 
after we release 25.1. That doesn't feel very probable, but still.

> after all,
> catching those was the main rationale for introducing the length test,
> right?

The most important part was to make sure that the length of the body in 
bytes is equal to the value of the Content-Length header (the difference 
caused actual problems).

But then we decided to make the check wider and test that the whole 
request string is unibyte-ish. Which made sense, but seems to be working 
out less well than we expected.