Changing user agent on eww

Changing user agent on eww

[Vibhav Pant]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Is it possible to change the user agent for fetching pages on eww (something like a variable or a command)?
- --
Vibhav Pant
vibhavp@gmail.com

Sent from my Android device with K-9 Mail. Please excuse my brevity.
-----BEGIN PGP SIGNATURE-----
Version: APG v1.0.9

iQI9BAEBCAAnBQJS2n0qIBxWaWJoYXYgUGFudCA8dmliaGF2cEBnbWFpbC5jb20+
AAoJEO86onTybWtcxt0P/iHQo6QlKrIH3pQ8jm0wg36eEAHadIcTewc0JFQcc9q/
f94nhXtJVeoP8Zr2JfIZ2wftIbeLtJd5OEA+9aFrgFg7aGr4TMaBYRo4PcLi2HKS
l9NS6y+U06KSBMmVraWL1OCxAYxHyLyMkb8YXMTfx4+cKlJKG6YYkWgfx4uWiJEG
43/Z/K/2Wqwnr0A5HFYhtbH5n2WuduOakaJGulNQFmJCkspFkW1JwoOeRcKkyJOJ
LoVR2aNvo52CDExpavgt8H5zbBrgTpZCQyclbn0yzSz3ThGqB3SXFbtqUdkwNA0A
LNogV7snNi2HC/KpHxNop3CD9lGAWfjeLKZ930i9JI1CXej62lNJLfDe2rAyXEaC
AxvhqlQAP/9sQs6u8gYulryzIZ1ShLBfu/DP7kqcIITEUuueWwKChJ3waCGRNfNY
xUrOQPilm/VtUoT0CtV3w8jSgP2xIdx5SGjiMwooR5k2s9Y7NNvn5n0GcB5QRvU/
oUE6fmKFDGWLCif0SumcKov9cznb+UiIvNriObbM3REYYHtYNbpczaKOTOCr3soC
8INgFai/fP50bgEwX2yvsKAcZxvOYujU+nFYpglIuNdKnjFcpeBPCr6VFUgEj1DB
sB1R6QY1FgZh8baWauHt4poaMandcwHYRQGUL3atgQpxN1I0blKYP4BTWlAErrvf
=Pxfv
-----END PGP SIGNATURE-----

Re: Changing user agent on eww

[Rüdiger Sonderfeld]

On Saturday 18 January 2014 18:40:02 Vibhav Pant wrote:
> Is it possible to change the user agent for fetching pages on eww (something
> like a variable or a command)? --

Kinda, eww uses url.el to fetch webpages.  Url.el has no direct variable to 
set the User-agent.  But if you set `url-privacy-level' to `paranoid' no User-
agent is sent at all.  The function `url-http-user-agent-string' is used to 
create the User-agent string.  You should be able to override it, to send 
whatever User-agent string you prefer.  Maybe we should add a variable for the 
user agent to make it easier to customize.

Regards,
Rüdiger

Re: Changing user agent on eww

[Vibhav Pant]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512


Rüdiger Sonderfeld writes:

>  The function `url-http-user-agent-string' is used to 
> create the User-agent string.  You should be able to override it, to send 
> whatever User-agent string you prefer.  Maybe we should add a variable for the 
> user agent to make it easier to customize.

I had a look at the function url-http-create-request. The
user-agent string is set by simply calling
url-http-user-agent-string, which in turn concats the "User Agent: URL/"
and the URL package version.
This could be customised by adding a user-defined variable (string)
url-(custom-)user-agent-string, which the user-agent-string could
return. This could be nil by default, in which case the vanila string
would be returned.
If this sounds good, I can create a patch this.

- -- 
Vibhav Pant
vibhavp@gmail.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBCgAGBQJS2rOSAAoJEO86onTybWtcSfQP/RF+6bpn/nG5+l7e0H1V8qQR
1iWwj90yj8mljnnnEGixnezCPvxudR/UIrXgDl0qzP7qUGF94LSzdI4MJGDmHhOV
BYycEIbe2kw1LwzQWX38cSk6LRGaa8Ad5tJu82in8RIn8dU6WysfLEAYXfcDuen8
qa5I3CV1hX+IybrKIQ1wkPM7pQ4FhJW4EsFXTiN1SyIvYRQDit0DUSBuc8/1OVFN
taGfQEa66JG5subZvtyESBPh/SXHEbbq89FG3S1ESMnOhg1d8VwS0TI7pT7iBXwT
J5PwmyTdM45paa6EcXy1ZUI3ru62UuQXIDdVASRWCQuaqH/Xcgh8vlgR8rd3W8nl
KkpGyV2O6BTJ1QEDtP2J4jmPzl4Szh/X1XLCljZKAuqENcusAaK3N5IDT0b2QQCy
DE1kp7h57kp0NthxGDY0hnLmtMYRIt59boVZewg8nhIi/TUnfTQacBtsKb78zZFi
JDSLmAoHHV0r8QMm5hbDskAwWTYYKt76dmnMODKecV4QEVYFL+C1DAvVj1Aur8sK
+v7bjYc4/c0IPCPm+jBDfAvsq4b+NBc7BLDyyHReppOXjmnaEfyHFEyjm40zSbuS
kLFqfzG6dO6By8IDZVDllycejqFwrcmsW7yrJ72vnP8lQZwCT+k84E2yexTd6DzR
yP5ffQXBBZwXnvNlMmeu
=Ewxz
-----END PGP SIGNATURE-----

Re: Changing user agent on eww [OT]

[Trent W. Buck]

Rüdiger Sonderfeld <ruediger@c-plusplus.de> writes:

> On Saturday 18 January 2014 18:40:02 Vibhav Pant wrote:
>> Is it possible to change the user agent for fetching pages on eww (something
>> like a variable or a command)? --
>
> Kinda, eww uses url.el to fetch webpages.  Url.el has no direct
> variable to set the User-agent.  But if you set `url-privacy-level' to
> `paranoid' no User- agent is sent at all.

I used to do that, until Wikipedia (and her sister projects) started
returning an empty page if UA wasn't set, on the theory that you must be
a really inept leecher -- so inept you didn't start with wget or curl.

Hmm... I wonder if https://panopticlick.eff.org/ rates "no UA" as being
*more* identifiable than spoofing it to something commonplace? :-)

Re: Changing user agent on eww [OT]

[Stefan Monnier]

> Hmm... I wonder if https://panopticlick.eff.org/ rates "no UA" as being
> *more* identifiable than spoofing it to something commonplace? :-)

BTW, I was recently thinking about this "http header fingerprinting"
problem and was wondering if anybody has tried to randomize
their header.
More specifically, change part of the header for each request.
An obvious option is to add a counter to the "user-agent", and to add
a "blur" factor to the language options.

E.g. my "en-us,fr-ch;q=0.8,es-ar;q=0.6,en;q=0.4,de;q=0.2" appears to be very
rare, but if it keeps changing from
   en-us,fr-ch;q=0.80000234,es-ar;q=0.60000765,en;q=0.40000345,de;q=0.20000123
to en-us,fr-ch;q=0.80000983,es-ar;q=0.60000923,en;q=0.40000186,de;q=0.20000236
to ...
then a naive fingerprinting will be fooled into thinking it's coming
from a different user.


        Stefan

Re: Changing user agent on eww [OT]

[chad]

[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]

On Thu, Jan 23, 2014 at 5:50 PM, Stefan Monnier <monnier@iro.umontreal.ca>wrote:

> > Hmm... I wonder if https://panopticlick.eff.org/ rates "no UA" as being
> > *more* identifiable than spoofing it to something commonplace? :-)
>
> BTW, I was recently thinking about this "http header fingerprinting"
> problem and was wondering if anybody has tried to randomize
> their header.
> More specifically, change part of the header for each request.
> An obvious option is to add a counter to the "user-agent", and to add
> a "blur" factor to the language options.
>
> E.g. my "en-us,fr-ch;q=0.8,es-ar;q=0.6,en;q=0.4,de;q=0.2" appears to be
> very
> rare, but if it keeps changing from
>
>  en-us,fr-ch;q=0.80000234,es-ar;q=0.60000765,en;q=0.40000345,de;q=0.20000123
> to
> en-us,fr-ch;q=0.80000983,es-ar;q=0.60000923,en;q=0.40000186,de;q=0.20000236
> to ...
> then a naive fingerprinting will be fooled into thinking it's coming
> from a different user.
>

It's not a bad idea, but I wouldn't try it specifically with q-values, as
they're likely discarded by the server before they get to the browser-id
step.

~Chad

[-- Attachment #2: Type: text/html, Size: 1664 bytes --]

Re: Changing user agent on eww [OT]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    More specifically, change part of the header for each request.
    An obvious option is to add a counter to the "user-agent", and to add
    a "blur" factor to the language options.

If you have the same IP address each time, they will know anyway that
it is you.

However, if you change the user agent every time the IP address changes,
that will do more good.  But you need to vary among user agent strings
which are themselves fairly common.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Changing user agent on eww [OT]

[Stefan Monnier]

> If you have the same IP address each time, they will know anyway that
> it is you.

As soon as we switch to IPv6, that's true.  But for now, I often share
(at the same time) a single IPv4 address with various people.

> However, if you change the user agent every time the IP address changes,
> that will do more good.  But you need to vary among user agent strings
> which are themselves fairly common.

The idea is to completely give up on the idea of "blending into the
mass", since header-finger-printing lets them distinguish individuals
among the mass anyway.  So instead your user-agent string will be unique
(instead of common), but you'll never reuse the same one.


        Stefan

Re: Changing user agent on eww [OT]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    As soon as we switch to IPv6, that's true.

We will need to change our IPv6 addresses, not stick with the same one.

						But for now, I often share
    (at the same time) a single IPv4 address with various people.

In the same computer?  In different computers?

    The idea is to completely give up on the idea of "blending into the
    mass", since header-finger-printing lets them distinguish individuals
    among the mass anyway.  So instead your user-agent string will be unique
    (instead of common), but you'll never reuse the same one.

I am not convinced.  If they can tell it's always you, from your IP,
the fact that your header data changes in a specific would only
identify which browser you use.  To use a header string that blends in
is the only way.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Changing user agent on eww [OT]

[Stefan Monnier]

>     But for now, I often share
>     (at the same time) a single IPv4 address with various people.
> In the same computer?  In different computers?

Different computers, all behind a NAT router.


        Stefan

Re: Changing user agent on eww [OT]

[Simon Leinen]

> We will need to change our IPv6 addresses, not stick with the same one.

FYI, changing the source IPv6 address for outgoing connections is
foreseen as a host-level privacy mechanism, see RFC 4941.  A typical
OS implementation is to change the address roughly every 24 hours.  I
think this has been the default on Windows for several years; it can
also be enabled on GNU/Linux and Mac OS X, but I'm not sure it's the
default.

So at least a basic level of address-changing can be achieved
relatively easily using today's OSes.
-- 
Simon.

Re: Changing user agent on eww [OT]

[Yuri Khan]

On Sun, Jan 26, 2014 at 8:47 PM, Simon Leinen <simon.leinen@gmail.com> wrote:
>> We will need to change our IPv6 addresses, not stick with the same one.
>
> FYI, changing the source IPv6 address for outgoing connections is
> foreseen as a host-level privacy mechanism, see RFC 4941.

On the other hand, the end user can only change their IPv6 address
within the range allocated to them. An individual user is likely to
have a /48 network. Sites will most probably drop the lower 80 bits
when tracking users.