bug#45179: qutebrowser stuck on browser checks forever

bug#45179: qutebrowser stuck on browser checks forever

[bdju via Bug reports for GNU Guix]

guix (GNU Guix) 91e35e32a4938e0e37499c64fa8ed3e7cf51dce3
some example sites with browser checks:
https://gitlab.com/users/sign_in
http://livechart.me/

I already contacted the people in the qutebrowser IRC. The bug persists
when I launch with no config (`qutebrowser --temp-basedir'), and they
could not reproduce the issue on the same version of qutebrowser I'm
using. I suspect it may be an issue with the guix version.

bug#45179: qutebrowser stuck at Cloudflare 'browser checks'

[Michael Rohleder]

[-- Attachment #1: Type: text/plain, Size: 1152 bytes --]

Hello bdju,

"bdju" <bdju@tilde.team> writes:
> guix (GNU Guix) 91e35e32a4938e0e37499c64fa8ed3e7cf51dce3
> some example sites with browser checks:
> https://gitlab.com/users/sign_in
> http://livechart.me/
>
> I already contacted the people in the qutebrowser IRC. The bug persists
> when I launch with no config (`qutebrowser --temp-basedir'), and they
> could not reproduce the issue on the same version of qutebrowser I'm
> using. I suspect it may be an issue with the guix version.
>

A workaround is setting the "User Agent" string to something CF likes, e.g. like this:
qutebrowser --temp-basedir --set content.headers.user_agent 'Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0' https://gitlab.com/users/sign_in

Maybe the reason why the (very helpful) #qutebrowser folks can't
reproduce this, is because they use another qt version (qutebrowser uses
qtwebengine) and perhaps this sets user agent to a string that CF has
whitelisted.

I don't think there is much we could do here (besides updating qt and
mbakke has that in the pipeline, afaik).

-- 
COFFEE.EXE Missing - Insert Cup and Press Any Key

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

bug#45179: qutebrowser stuck at Cloudflare 'browser checks'

[ben--- via Bug reports for GNU Guix]

On Tue, 22 Dec 2020, Michael Rohleder wrote:

> Hello bdju,
>
> "bdju" <bdju@tilde.team> writes:
>> guix (GNU Guix) 91e35e32a4938e0e37499c64fa8ed3e7cf51dce3
>> some example sites with browser checks:
>> https://gitlab.com/users/sign_in
>> http://livechart.me/
>
> A workaround is setting the "User Agent" string to something CF likes, e.g. like this:
> qutebrowser --temp-basedir --set content.headers.user_agent 'Mozilla/5.0
> (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0'
> https://gitlab.com/users/sign_in
>
> Maybe the reason why the (very helpful) #qutebrowser folks can't
> reproduce this, is because they use another qt version (qutebrowser uses
> qtwebengine) and perhaps this sets user agent to a string that CF has
> whitelisted.
>
> I don't think there is much we could do here (besides updating qt and
> mbakke has that in the pipeline, afaik).

I'm also having similar issues accessing gitlab.com with IceCat.
Installing the User Agent Switcher and setting to "Linux / Firefox 82"
fixed this for me.

For context I also tried starting IceCat up in safe mode and switching
tracking protection back to "standard", but no good. The issue appears
to be that Cloudflare are essentially blocking niche user agents.

Regards,
Ben

bug#45179: qutebrowser stuck at Cloudflare 'browser checks'

[Mark H Weaver]

ben--- via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

> I'm also having similar issues accessing gitlab.com with IceCat.
> Installing the User Agent Switcher and setting to "Linux / Firefox 82"
> fixed this for me.
>
> For context I also tried starting IceCat up in safe mode and switching
> tracking protection back to "standard", but no good. The issue appears
> to be that Cloudflare are essentially blocking niche user agents.

The IceCat package in Guix sends the same user-agent string as the
'firefox-esr' package in Debian.

      Mark

bug#45179: qutebrowser stuck at Cloudflare 'browser checks'

[ben--- via Bug reports for GNU Guix]

On Sun, 24 Jan 2021, Mark H. Weaver wrote:

> ben--- via Bug reports for GNU Guix <bug-guix@gnu.org> writes:
>
>> I'm also having similar issues accessing gitlab.com with IceCat.
>> Installing the User Agent Switcher and setting to "Linux / Firefox 82"
>> fixed this for me.
>>
>> For context I also tried starting IceCat up in safe mode and switching
>> tracking protection back to "standard", but no good. The issue appears
>> to be that Cloudflare are essentially blocking niche user agents.
>
> The IceCat package in Guix sends the same user-agent string as the
> 'firefox-esr' package in Debian.

I've done a little more testing for interest sake - I still think
CloudFlare are at fault here.

If use a private tab and attempt to access a private GitLab repository I
get stuck at the CloudFlare "checking your browser message". Here's the
IceCat default user-agent string:

"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0"

If I install User Agent Switcher and do the same thing, it works. The
only difference in the user-agent string is the version number:

"Mozilla/5.0 (X11; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0"

If I try the same on Firefox ESR on Debian Buster, it works no problem,
but as Mark says, the Firefox user-agent string matches exactly to
IceCat:

"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0"

That suggests to me that there's more than the user-agent string at play
here.

Regards,
Ben

bug#45179: qutebrowser stuck at Cloudflare 'browser checks'

[bdju via Bug reports for GNU Guix]

On Tue Dec 22, 2020 at 10:34 AM CST, Michael Rohleder wrote:
> Hello bdju,
>
> "bdju" <bdju@tilde.team> writes:
> > guix (GNU Guix) 91e35e32a4938e0e37499c64fa8ed3e7cf51dce3
> > some example sites with browser checks:
> > https://gitlab.com/users/sign_in
> > http://livechart.me/
> >
> > I already contacted the people in the qutebrowser IRC. The bug persists
> > when I launch with no config (`qutebrowser --temp-basedir'), and they
> > could not reproduce the issue on the same version of qutebrowser I'm
> > using. I suspect it may be an issue with the guix version.
> >
>
> A workaround is setting the "User Agent" string to something CF likes,
> e.g. like this:
> qutebrowser --temp-basedir --set content.headers.user_agent 'Mozilla/5.0
> (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0'
> https://gitlab.com/users/sign_in
>
> Maybe the reason why the (very helpful) #qutebrowser folks can't
> reproduce this, is because they use another qt version (qutebrowser uses
> qtwebengine) and perhaps this sets user agent to a string that CF has
> whitelisted.
>
> I don't think there is much we could do here (besides updating qt and
> mbakke has that in the pipeline, afaik).
>
> --
> COFFEE.EXE Missing - Insert Cup and Press Any Key
While I was able to bypass the cloudflare checks in IceCat with that
User Agent Switcher addon, I found I so far haven't been able to get
past them in qutebrowser by changing my useragent. I tried both your
example and copying one of the useragents generated by UAS in IceCat.
Still caught in a loop. (and I am using --temp-basedir as you said, so
shouldn't be my config)

bug#45179: qutebrowser stuck at Cloudflare 'browser checks'

[bdju via Bug reports for GNU Guix]

On Tue Apr 6, 2021 at 4:41 PM CDT, bdju wrote:
> While I was able to bypass the cloudflare checks in IceCat with that
> User Agent Switcher addon, I found I so far haven't been able to get
> past them in qutebrowser by changing my useragent. I tried both your
> example and copying one of the useragents generated by UAS in IceCat.
> Still caught in a loop. (and I am using --temp-basedir as you said, so
> shouldn't be my config)
I was too hasty! I both had to change my useragent and wait a long time!
I didn't measure the time, but I'd guess over 10 minutes for Gitlab to
work. Livechart took even longer. Thanks so much for this workaround.
Oh, and this works without --temp-basedir, by the way. I should be able
to actually use these sites in my normal session now I think!

bug#45179: qutebrowser stuck at Cloudflare 'browser checks'

[Michael Rohleder]

[-- Attachment #1: Type: text/plain, Size: 1018 bytes --]

"bdju" <bdju@tilde.team> writes:
> On Tue Apr 6, 2021 at 4:41 PM CDT, bdju wrote:
>> While I was able to bypass the cloudflare checks in IceCat with that
>> User Agent Switcher addon, I found I so far haven't been able to get
>> past them in qutebrowser by changing my useragent. I tried both your
>> example and copying one of the useragents generated by UAS in IceCat.
>> Still caught in a loop. (and I am using --temp-basedir as you said, so
>> shouldn't be my config)
> I was too hasty! I both had to change my useragent and wait a long time!
> I didn't measure the time, but I'd guess over 10 minutes for Gitlab to
> work. Livechart took even longer. Thanks so much for this workaround.
> Oh, and this works without --temp-basedir, by the way. I should be able
> to actually use these sites in my normal session now I think!

Yay!  That's great to hear, so let's close this ;)

-- 
Practical people would be more practical if they would take a little
more time for dreaming.   -   J. P. McEvoy

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]