bug#24117: 25.1; url-http-create-request: Multibyte text in HTTP request

[Lars Ingebrigtsen <larsi@gnus.org>]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Just got around to this. The test I came up with looks like this:
>
> (ert-deftest url-generic-parse-url/multibyte-host-and-path ()
>   (should (equal (url-generic-parse-url "http://банки.рф/фыва/")
>                  (url-parse-make-urlobj "http" nil nil "банки.рф" nil
>                                         "/фыва/" nil nil t))))

That looks like the correct decomposition of this URL, so
url-generic-parse-url does the right thing.

> But! What behavior would this test? If we're making sure here that
> url-generic-parse-url can cope with multibyte characters anywhere in
> the URL, the encode-coding-string/decode-coding-string logic in
> url-encode-url is extraneous. I'm not sure that it is, or is there are
> some edge cases (are they fixable? should we add tests for them?).

(url-encode-url "http://банки.рф/фыва/")
=> "http://банки.рф/%D1%84%D1%8B%D0%B2%D0%B0/"

It is perhaps debatable whether the host name should be encoded (with
punycode) here, but this is otherwise correct.

> So if this test goes in, it should be accompanied with the
> simplification of url-encode-url.
>
> Lars, what do you think?

The utf-8 encoding does seem superfluous, especially since
url-hexify-string also does the encoding...

(url-hexify-string "фыва")
=> "%D1%84%D1%8B%D0%B2%D0%B0"

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no