EWW problems with SNI https sites?

EWW problems with SNI https sites?

[Hideki Saito]

Hello,
I'm having a bit of problem with EWW accessing some sites under Windows,
namely, some sites returning "400 Bad Request" -- the reason being "The
plain HTTP request was sent to HTTPS port"

This seems to affect a lot of CloudFlare hosted sites. But even without
CloudFlare, this seems to be affecting SNI (Server Name Indication -- ones
without dedicated IP address) hosts as well.

Here's the pattern I'm seeing:
1) https://www.hidekisaito.com (CloudFlare) -- doesn't work
2) https://hidekisaito.com (SNI) -- doesn't work
3) https://duckduckgo.com -- works
4) https://twitter.com -- works
5) https://helpa.hidekisaito.com (SNI) -- doesn't work (due to the
maintenance, the site might be down for next few hours.

I have my personal ELPA site on #5 site, and this seems to be causing
issue; M-x list-package fails with the same error, M-x
package-refresh-content somehow successfully fetch the list. (But doesn't
help as fetching and installing the package would fail with 400 Bad Request)

I can live with eww not working, but it's painful that list-package doesn't
work. Do you happen to know if there are anyways to force list-package (and
package download) to use the method used by package-refresh-content?

Please let me know if this is something I should be reporting as a bug.

Thank you.

Hideki Saito (斉藤英樹)

Re: EWW problems with SNI https sites?

[Colin Baxter]

Dear Hideki,
>>>>> Hideki Saito <hidekis@gmail.com> writes:

    > Hello, I'm having a bit of problem with EWW accessing some sites
    > under Windows, namely, some sites returning "400 Bad Request" --
    > the reason being "The plain HTTP request was sent to HTTPS port"

    > This seems to affect a lot of CloudFlare hosted sites. But even
    > without CloudFlare, this seems to be affecting SNI (Server Name
    > Indication -- ones without dedicated IP address) hosts as well.

    > Here's the pattern I'm seeing: 1) https://www.hidekisaito.com
    > (CloudFlare) -- doesn't work 2) https://hidekisaito.com (SNI) --
    > doesn't work 3) https://duckduckgo.com -- works 4)
    > https://twitter.com -- works 5) https://helpa.hidekisaito.com
    > (SNI) -- doesn't work (due to the maintenance, the site might be
    > down for next few hours.

It looks like a MS Windows issue because these sites work fine with eww
under linux. Have you tried emacs-w3m?

Best wishes,

-- 
Colin Baxter

Re: EWW problems with SNI https sites?

[Eli Zaretskii]

> From: Colin Baxter <m43cap@yandex.com>
> Date: Thu, 11 Jul 2019 10:31:11 +0100
> Cc: , help-gnu-emacs@gnu.org
> 
> >>>>> Hideki Saito <hidekis@gmail.com> writes:
> 
>     > Hello, I'm having a bit of problem with EWW accessing some sites
>     > under Windows, namely, some sites returning "400 Bad Request" --
>     > the reason being "The plain HTTP request was sent to HTTPS port"
> 
>     > This seems to affect a lot of CloudFlare hosted sites. But even
>     > without CloudFlare, this seems to be affecting SNI (Server Name
>     > Indication -- ones without dedicated IP address) hosts as well.
> 
>     > Here's the pattern I'm seeing: 1) https://www.hidekisaito.com
>     > (CloudFlare) -- doesn't work 2) https://hidekisaito.com (SNI) --
>     > doesn't work 3) https://duckduckgo.com -- works 4)
>     > https://twitter.com -- works 5) https://helpa.hidekisaito.com
>     > (SNI) -- doesn't work (due to the maintenance, the site might be
>     > down for next few hours.
> 
> It looks like a MS Windows issue because these sites work fine with eww
> under linux. Have you tried emacs-w3m?

They all work for me on Windows, so this is not an issue with Emacs on
Windows, it's probably local to OP's system or maybe the network
setup.

What version of Emacs is the one where it doesn't work?

Re: EWW problems with SNI https sites?

[Lars Magne Ingebrigtsen]

Hideki Saito <hidekis@gmail.com> writes:

> I'm having a bit of problem with EWW accessing some sites under Windows,
> namely, some sites returning "400 Bad Request" -- the reason being "The
> plain HTTP request was sent to HTTPS port"

What Emacs version are you using?  Emacs 26 should have SNI support.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: EWW problems with SNI https sites?

[Hideki Saito]

It's 26.2.
I will see if there are any other variables that might make a difference...

Hideki Saito (斉藤英樹)

Re: EWW problems with SNI https sites?

[Hideki Saito]

I think I've figured out pattern, although I'm not still sure what's going
on.
This probably is not really related with SNI, but related to IPv6 --
perhaps there's some incompatibility with the way Windows IPv6 stack
handles thing; if I disable IPv6, I'm not seeing this is happening. (And
this seems to be Windows specific issue as this does not happen on Linux on
the same network.)

For the same reason, other places like https://wikipedia.org also fails.
(Where it resolves to IPv6.)
Anyone encountering similar issue and any workaround? (Unfortunately I
can't really disable IPv6 as I'm relying certain things for it. Wondering
if there's a way to disable IPv6 just for Emacs...)

Hideki Saito (斉藤英樹)

Re: EWW problems with SNI https sites?

[Robert Pluim]

>>>>> On Sat, 13 Jul 2019 22:24:40 -0700, Hideki Saito <hidekis@gmail.com> said:

    Hideki> I think I've figured out pattern, although I'm not still sure what's going
    Hideki> on.
    Hideki> This probably is not really related with SNI, but related to IPv6 --
    Hideki> perhaps there's some incompatibility with the way Windows IPv6 stack
    Hideki> handles thing; if I disable IPv6, I'm not seeing this is happening. (And
    Hideki> this seems to be Windows specific issue as this does not happen on Linux on
    Hideki> the same network.)

    Hideki> For the same reason, other places like https://wikipedia.org also fails.
    Hideki> (Where it resolves to IPv6.)
    Hideki> Anyone encountering similar issue and any workaround? (Unfortunately I
    Hideki> can't really disable IPv6 as I'm relying certain things for it. Wondering
    Hideki> if there's a way to disable IPv6 just for Emacs...)

Based on your original report of the error message "The plain HTTP
request was sent to HTTPS port", this is an issue Emacs has with
falling back from IP address A + TLS to IP address B + TLS, which
tends to manifest itself when the DNS response for a host returns an
IPv6 address, but that address is not actually reachable over IPv6.

I have a potential fix which I can send you as a patch, assuming you
can rebuild emacs.

Robert