Making package.el talk over Tor

Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I would like to arrange for package.el
to always connect to ELPA across the Tor network.
But it is 4600 lines of code and I would rather not have to read it all.

Can someone tell me where to find the code that actually
communicates with the ELPA repos?  Where is the best place
to make that change?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Akib Azmain Turja]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> I would like to arrange for package.el
> to always connect to ELPA across the Tor network.
> But it is 4600 lines of code and I would rather not have to read it all.
>
> Can someone tell me where to find the code that actually
> communicates with the ELPA repos?  Where is the best place
> to make that change?

Isearching for 'url-' reveals that the following functions use the URL
package to access the HTTP server: 'package--with-work-buffer',
'package--archive-file-exists-p' and'package--with-response-buffer-1'.

But I think a better/safer solution will be to use torsocks.

-- 
Akib Azmain Turja, GPG key: 70018CE5819F17A3BBA666AFE74F0EFA922AE7F5
Fediverse: akib@hostux.social
Codeberg: akib
emailselfdefense.fsf.org | "Nothing can be secure without encryption."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Making package.el talk over Tor

[Emanuel Berg]

Akib Azmain Turja wrote:

>> I would like to arrange for package.el to always connect to
>> ELPA across the Tor network. But it is 4600 lines of code
>> and I would rather not have to read it all.
>>
>> Can someone tell me where to find the code that actually
>> communicates with the ELPA repos? Where is the best place
>> to make that change?
>
> Isearching for 'url-' reveals that the following functions
> use the URL package to access the HTTP server:
> 'package--with-work-buffer',
> 'package--archive-file-exists-p'
> and'package--with-response-buffer-1'.
>
> But I think a better/safer solution will be to use torsocks.

Holy socks! But won't there be a connection speed disadvantage
from doing this?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Isearching for 'url-' reveals that the following functions use the URL
  > package to access the HTTP server: 'package--with-work-buffer',
  > 'package--archive-file-exists-p' and'package--with-response-buffer-1'.

  > But I think a better/safer solution will be to use torsocks.

Better/safer solution than which other solution?

Using torsocks was what I had in mind -- the question is
where it would be easy to do that.  Would it be easy to do
in those three functions?  If not, then where?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Thanks for replying to my question a month ago.
I got so backlogged I just saw the reply.

  > > Can someone tell me where to find the code that actually
  > > communicates with the ELPA repos?  Where is the best place
  > > to make that change?

  > Isearching for 'url-' reveals that the following functions use the URL
  > package to access the HTTP server: 'package--with-work-buffer',
  > 'package--archive-file-exists-p' and'package--with-response-buffer-1'.

  > But I think a better/safer solution will be to use torsocks.

I agree, and I plan to use torsocks.  But in order to do thst, I need
to know where to do that.  That's really what my question was about.

Can you suggest where I should do that?

Does the url package have a variable to specify the command to use?
I could specify torsocks there.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> Thanks for replying to my question a month ago.
> I got so backlogged I just saw the reply.
>
>   > > Can someone tell me where to find the code that actually
>   > > communicates with the ELPA repos?  Where is the best place
>   > > to make that change?
>
>   > Isearching for 'url-' reveals that the following functions use the URL
>   > package to access the HTTP server: 'package--with-work-buffer',
>   > 'package--archive-file-exists-p' and'package--with-response-buffer-1'.
>
>   > But I think a better/safer solution will be to use torsocks.
>
> I agree, and I plan to use torsocks.  But in order to do thst, I need
> to know where to do that.  That's really what my question was about.
>
> Can you suggest where I should do that?
>
> Does the url package have a variable to specify the command to use?
> I could specify torsocks there.

No, url.el eventually calls `make-network-process', that directly
invokes the respective networking system calls, not making it possible
to interject torsocks.  What I believe Akib meant was to start Emacs
with torsocks, but to my understanding this is not a recommended
practice either, because one will continue to leak fingerprintable
metadata (specially inside of Emacs) that would undermine the point of
using Tor to begin with.

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > No, url.el eventually calls `make-network-process', that directly
  > invokes the respective networking system calls, not making it possible
  > to interject torsocks.

Is this the code you mean?

		      (open-network-stream
		       name buffer host service
		       :type gw-method
		       ;; Use non-blocking socket if we can.
		       :nowait (and (featurep 'make-network-process)
                                    (url-asynchronous url-current-object)
                                    '(:nowait t)))

How can I make that use TOR?

I could set `url-gateway-method' to specify some other method, but
what other value would make it possible to put in torsocks?

  > What I believe Akib meant was to start Emacs
  > with torsocks,

I think that is no good -- all subprocesses of Emacs would be
compelled to use torsocks too.

  >  because one will continue to leak fingerprintable
  > metadata (specially inside of Emacs)

Could you give me an example of what you mean?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: akib@disroot.org, emacs-devel@gnu.org
> Date: Sat, 18 Nov 2023 22:39:36 -0500
> 
>   > No, url.el eventually calls `make-network-process', that directly
>   > invokes the respective networking system calls, not making it possible
>   > to interject torsocks.
> 
> Is this the code you mean?
> 
> 		      (open-network-stream
> 		       name buffer host service
> 		       :type gw-method
> 		       ;; Use non-blocking socket if we can.
> 		       :nowait (and (featurep 'make-network-process)
>                                     (url-asynchronous url-current-object)
>                                     '(:nowait t)))

Yes.

> How can I make that use TOR?

Perhaps the ':type shell' connection is the solution?

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Is this the code you mean?
  > > 
  > > 		      (open-network-stream
  > > 		       name buffer host service
  > > 		       :type gw-method
  > > 		       ;; Use non-blocking socket if we can.
  > > 		       :nowait (and (featurep 'make-network-process)
  > >                                     (url-asynchronous url-current-object)
  > >                                     '(:nowait t)))

  > > How can I make that use TOR?

  > Perhaps the ':type shell' connection is the solution?

I have no idea.  How could I use that to acheieve the goal?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: philipk@posteo.net, akib@disroot.org, emacs-devel@gnu.org
> Date: Fri, 08 Dec 2023 23:06:38 -0500
> 
>   > > Is this the code you mean?
>   > > 
>   > > 		      (open-network-stream
>   > > 		       name buffer host service
>   > > 		       :type gw-method
>   > > 		       ;; Use non-blocking socket if we can.
>   > > 		       :nowait (and (featurep 'make-network-process)
>   > >                                     (url-asynchronous url-current-object)
>   > >                                     '(:nowait t)))
> 
>   > > How can I make that use TOR?
> 
>   > Perhaps the ':type shell' connection is the solution?
> 
> I have no idea.  How could I use that to acheieve the goal?

I never tried this, and I'm not familiar with TOR, so all I have is
what the ELisp manual says where it documents open-network-stream
(which is the workhorse of all Emacs commands that use network
connections):

 -- Function: open-network-stream name buffer host service &rest
          parameters
     This function opens a TCP connection, with optional encryption, and
     returns a process object that represents the connection.
 [...]
     The remaining arguments PARAMETERS are keyword/argument pairs that
     are mainly relevant to encrypted connections:
 [...]
     ‘:type TYPE’
          The type of connection.  Options are:
          ‘plain’
               An ordinary, unencrypted connection.
          ‘tls’
          ‘ssl’
               A TLS (Transport Layer Security) connection.
          ‘nil’
          ‘network’
               Start with a plain connection, and if parameters
               ‘:success’ and ‘:capability-command’ are supplied, try to
               upgrade to an encrypted connection via STARTTLS.  If that
               fails, retain the unencrypted connection.
          ‘starttls’
               As for ‘nil’, but if STARTTLS fails drop the connection.
          ‘shell’
               A shell connection.
 [...]
     ‘:shell-command STRING-OR-NIL’
          If the connection ‘type’ is ‘shell’, this parameter will be
          interpreted as a format-spec string that will be executed to
          make the connection.  The specs available are ‘%s’ for the
          host name and ‘%p’ for the port number.  For instance, if you
          want to first ssh to ‘gateway’ before making a plain
          connection, then this parameter could be something like ‘ssh
          gateway nc %s %p’.

This sounds like a way to make the connection by running a shell
command.  Maybe you could arrange for using this facility to do what
you want?  Apologies if this makes no sense to someone who does
understand how TOR works; I don't.

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This sounds like a way to make the connection by running a shell
  > command.  Maybe you could arrange for using this facility to do what
  > you want?

I have no idea.  I don't know how to make a program talk to anything
over the net, either with Tor or without.

I think all Emacs features to do something over the internet should
give users a way to say "go through Tor".  (For most applicaions I
think a global variable is the preferable way to specify this.)

Can someone who knows how to do that please implement it for Emacs package.el?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > This sounds like a way to make the connection by running a shell
>   > command.  Maybe you could arrange for using this facility to do what
>   > you want?
>
> I have no idea.  I don't know how to make a program talk to anything
> over the net, either with Tor or without.

I tried out Eli's suggestion, and it seems to work:

(let ((conn
       (open-network-stream
	"test" (current-buffer) "gnu.org" "http"
	:type 'shell
	:shell-command "torsocks nc %s %p")))
  (process-send-string conn "GET / HTTP/1.0\r\n\r\n"))

>
> I think all Emacs features to do something over the internet should
> give users a way to say "go through Tor".  (For most applicaions I
> think a global variable is the preferable way to specify this.)

If you have a Tor daemon running on your system, it should provide a
SOCKS proxy by default, meaning that you can configure every package
that uses url.el to go through this proxy (unless I have misunderstood
something):

--8<---------------cut here---------------start------------->8---
(setq url-gateway-method 'socks
      socks-server '("Tor" "localhost" 9050 5))
--8<---------------cut here---------------end--------------->8---

That being said, while testing I noticed that when connecting to a
server I have access to, I always receive two requests, one from a Tor
exit node and one from my current IP address.  Unless I missed something
obvious, that might be a bug.

> Can someone who knows how to do that please implement it for Emacs package.el?

FWIW As package.el doesn't directly use any low level networking, so the
:shell trick wouldn't be something that could be easily implemented.

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If you have a Tor daemon running on your system, it should provide a
  > SOCKS proxy by default,

I believe it does that.

                            meaning that you can configure every package
  > that uses url.el to go through this proxy (unless I have misunderstood
  > something):

Maybe that is possible.  But I know very little about sockets or how
to use them.  I would appreciate precise advice about what I should do
to tell package.el to connect using Tor.

You sent this code

    (let ((conn
           (open-network-stream
            "test" (current-buffer) "gnu.org" "http"
            :type 'shell
            :shell-command "torsocks nc %s %p")))
      (process-send-string conn "GET / HTTP/1.0\r\n\r\n"))

but I don't think that is a solution ready to use.  It looks like a
proposed approach for designing that solution.

Also, the end of your message MAY mean that this approach is not
applicable to package.el.

  > If you have a Tor daemon running on your system, it should provide a
  > SOCKS proxy by default, meaning that you can configure every package
  > that uses url.el to go through this proxy (unless I have misunderstood
  > something):

  > --8<---------------cut here---------------start------------->8---
  > (setq url-gateway-method 'socks
  >       socks-server '("Tor" "localhost" 9050 5))
  > --8<---------------cut here---------------end--------------->8---

  > That being said, while testing I noticed that when connecting to a
  > server I have access to, I always receive two requests, one from a Tor
  > exit node and one from my current IP address.  Unless I missed something
  > obvious, that might be a bug.

I await word about further progress.

When I know enough to be able to do it, I will add a feature to
package.el for a user option to tell it to go through Tor always.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Would it be possible to add this code to eww.el in the right place

      > (let ((conn
      >        (open-network-stream
      >         "test" (current-buffer) "gnu.org" "http"
      >         :type 'shell
      >         :shell-command "torsocks nc %s %p")))
      >   (process-send-string conn "GET / HTTP/1.0\r\n\r\n"))

and cause all use of eww to go through Tor?
If so, can someone show me how and where this is needed?



-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Tomas Hlavaty]

> and cause all use of eww to go through Tor?

I have this in .emacs:

(custom-set-variables
 '(url-cookie-confirmation nil)
 '(url-cookie-untrusted-urls '(".*"))
 '(url-gateway-method 'native)
 '(url-privacy-level 'paranoid)
 '(url-proxy-services
   '(("no_proxy" . "localhost")
     ("http" . "127.0.0.1:8118")
     ("https" . "127.0.0.1:8118")))
 '(url-queue-timeout 4)

and eww goes through tor running on port 8118

why is this not a sufficient solution?

Re: Making package.el talk over Tor

[Tomas Hlavaty]

On Mon 18 Dec 2023 at 09:05, Tomas Hlavaty <tom@logand.com> wrote:
>> and cause all use of eww to go through Tor?
>
> I have this in .emacs:
>
> (custom-set-variables
>  '(url-cookie-confirmation nil)
>  '(url-cookie-untrusted-urls '(".*"))
>  '(url-gateway-method 'native)
>  '(url-privacy-level 'paranoid)
>  '(url-proxy-services
>    '(("no_proxy" . "localhost")
>      ("http" . "127.0.0.1:8118")
>      ("https" . "127.0.0.1:8118")))
>  '(url-queue-timeout 4)
>
> and eww goes through tor running on port 8118

actually iirc I have it like this: eww -> privoxy -> tor (because of
additional rules in privoxy, e.g. some sites should not go via tor).
but privoxy could probably be dropped if desired

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > (custom-set-variables
  >  '(url-cookie-confirmation nil)
  >  '(url-cookie-untrusted-urls '(".*"))
  >  '(url-gateway-method 'native)
  >  '(url-privacy-level 'paranoid)
  >  '(url-proxy-services
  >    '(("no_proxy" . "localhost")
  >      ("http" . "127.0.0.1:8118")
  >      ("https" . "127.0.0.1:8118")))
  >  '(url-queue-timeout 4)

I will give that a try.  Thanks.

  > why is this not a sufficient solution?

I am the wrong person to ask.  I don't have the background knowledge
to tell whether it solves the problem.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > (custom-set-variables
>   >  '(url-cookie-confirmation nil)
>   >  '(url-cookie-untrusted-urls '(".*"))
>   >  '(url-gateway-method 'native)
>   >  '(url-privacy-level 'paranoid)
>   >  '(url-proxy-services
>   >    '(("no_proxy" . "localhost")
>   >      ("http" . "127.0.0.1:8118")
>   >      ("https" . "127.0.0.1:8118")))

For this to work, one has to add

   HTTPTunnelPort localhost:8118

and it only appears to work for https:// URLs, not unencrypted http://.

>   >  '(url-queue-timeout 4)
>
> I will give that a try.  Thanks.
>
>   > why is this not a sufficient solution?
>
> I am the wrong person to ask.  I don't have the background knowledge
> to tell whether it solves the problem.

You can use https://check.torproject.org/ to check, if the request goes
through the Tor proxy, the page will say "Congratulations. This browser
is configured to use Tor."

Re: Making package.el talk over Tor

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>> [[[ whether defending the US Constitution against all enemies,     ]]]
>> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>>
>>   > (custom-set-variables
>>   >  '(url-cookie-confirmation nil)
>>   >  '(url-cookie-untrusted-urls '(".*"))
>>   >  '(url-gateway-method 'native)
>>   >  '(url-privacy-level 'paranoid)
>>   >  '(url-proxy-services
>>   >    '(("no_proxy" . "localhost")
>>   >      ("http" . "127.0.0.1:8118")
>>   >      ("https" . "127.0.0.1:8118")))
>
> For this to work, one has to add
>
>    HTTPTunnelPort localhost:8118

* to /etc/tor/torrc

> and it only appears to work for https:// URLs, not unencrypted http://.
>
>>   >  '(url-queue-timeout 4)
>>
>> I will give that a try.  Thanks.
>>
>>   > why is this not a sufficient solution?
>>
>> I am the wrong person to ask.  I don't have the background knowledge
>> to tell whether it solves the problem.
>
> You can use https://check.torproject.org/ to check, if the request goes
> through the Tor proxy, the page will say "Congratulations. This browser
> is configured to use Tor."

Re: Making package.el talk over Tor

[Tomas Hlavaty]

On Thu 21 Dec 2023 at 09:55, Philip Kaludercic <philipk@posteo.net> wrote:
>>>   >  '(url-proxy-services
>>>   >    '(("no_proxy" . "localhost")
>>>   >      ("http" . "127.0.0.1:8118")
>>>   >      ("https" . "127.0.0.1:8118")))
>>
>> For this to work, one has to add
>>
>>    HTTPTunnelPort localhost:8118
>
> * to /etc/tor/torrc
>
>> and it only appears to work for https:// URLs, not unencrypted http://.

interesting

I use it with privoxy:

  emacs -> privoxy on port 8118 -> tor on port 9050

iirc the protocols are:

  emacs -> privoxy on port 8118 -> tor on port 9050
       http(s)                socks4a

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >>   >  '(url-proxy-services
  > >>   >    '(("no_proxy" . "localhost")
  > >>   >      ("http" . "127.0.0.1:8118")
  > >>   >      ("https" . "127.0.0.1:8118")))

Is there a reason for using the number 9118?  Tor's default seems to
be 9050; if I specify that number here, would it work
without changing /etc/tor/torrc?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Tomas Hlavaty]

On Sat 23 Dec 2023 at 22:57, Richard Stallman <rms@gnu.org> wrote:
>   > >>   >  '(url-proxy-services
>   > >>   >    '(("no_proxy" . "localhost")
>   > >>   >      ("http" . "127.0.0.1:8118")
>   > >>   >      ("https" . "127.0.0.1:8118")))
>
> Is there a reason for using the number 9118?

in my case:
8118 has privoxy running
so it looks like this:
emacs -> 8118 privoxy -> 9050 tor

8118 is a HTTP proxy
9050 is a socks proxy

> Tor's default seems to
> be 9050; if I specify that number here, would it work
> without changing /etc/tor/torrc?

I don't think so.

iirc emacs using socks proxy did not work for me
emacs using http proxy works fine
and I can have other rules in privoxy
which I do not thing is possible to specify directly in emacs
without changing emacs

Re: Making package.el talk over Tor

[Philip Kaludercic]

Tomas Hlavaty <tom@logand.com> writes:

> On Sat 23 Dec 2023 at 22:57, Richard Stallman <rms@gnu.org> wrote:
>>   > >>   >  '(url-proxy-services
>>   > >>   >    '(("no_proxy" . "localhost")
>>   > >>   >      ("http" . "127.0.0.1:8118")
>>   > >>   >      ("https" . "127.0.0.1:8118")))
>>
>> Is there a reason for using the number 9118?
>
> in my case:
> 8118 has privoxy running
> so it looks like this:
> emacs -> 8118 privoxy -> 9050 tor
>
> 8118 is a HTTP proxy
> 9050 is a socks proxy
>
>> Tor's default seems to
>> be 9050; if I specify that number here, would it work
>> without changing /etc/tor/torrc?
>
> I don't think so.
>
> iirc emacs using socks proxy did not work for me

Do you recall what your configuration was.  I gave an example earlier on
in this thread, but noticed that url wouldn't redirect all messages over
the SOCKS proxy.

> emacs using http proxy works fine
> and I can have other rules in privoxy
> which I do not thing is possible to specify directly in emacs
> without changing emacs

Re: Making package.el talk over Tor

[Tomas Hlavaty]

On Sun 24 Dec 2023 at 15:19, Philip Kaludercic <philipk@posteo.net> wrote:
> Tomas Hlavaty <tom@logand.com> writes:
>> iirc emacs using socks proxy did not work for me
>
> Do you recall what your configuration was.  I gave an example earlier on
> in this thread, but noticed that url wouldn't redirect all messages over
> the SOCKS proxy.

I don't remember, it was a long time ago.
The configuration I sent is something I figured out at that time
and it still works reliably without any changes.

It would be nice if it was possible for me to drop privoxy
and specify per host/ip proxying rules in emacs directly,
but privoxy works well so this has low priority i guess.

Re: Making package.el talk over Tor

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

>   >  because one will continue to leak fingerprintable
>   > metadata (specially inside of Emacs)
>
> Could you give me an example of what you mean?

As mention in my other message, I was testing what my web server was
logging when accessing the server via Tor, and this was the log entry:

185.220.101.26 - - [14/Dec/2023:13:04:00 +0100] "GET /test HTTP/1.1" 301 169 "https://amodernist.com/" "URL/Emacs Emacs/30.0.50 (PureGTK; x86_64-pc-linux-gnu)"

As you can see the User-Agent indicates that I am using Emacs, what
version and even my architecture.  Compare that to the user agent that
you'd regularly encounter from an average browser:

31.10.139.153 - - [14/Dec/2023:00:18:33 +0100] "GET / HTTP/1.1" 200 10585 "-" "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Mobile Safari/537.36"

This can be remedied by setting the `url-privacy-level' user option to
'paranoid, but in that case you are still identifiable because there is
no user agent, which carries some information.

Other than the user-agent, there are certainly other bits of behaviour
that a malicious actor can use to track a user, such as the order in
which HTTP headers are transmitted, the size of chunks by which the
client sends and receives data and of course what requests aren't being
sent (e.g. due to a lack of Javascript in EWW).

The EFF has more information on the topic here:
https://coveryourtracks.eff.org/learn.

That being said: All of this doesn't matter that much for package.el,
since most people are accessing it via Emacs.

Re: Making package.el talk over Tor

[Emanuel Berg]

Philip Kaludercic wrote:

> 185.220.101.26 - - [14/Dec/2023:13:04:00 +0100] "GET /test
> HTTP/1.1" 301 169 "https://amodernist.com/" "URL/Emacs
> Emacs/30.0.50 (PureGTK; x86_64-pc-linux-gnu)"

I tried to visit this page with Emacs-w3m

  https://www.site24x7.com/tools/browser-fingerprint-test.html

and it outputted the below data (last).

Summary: There is no information except the the IP details.
The "IP Address" is correct, that's my public IP. The Location
is partly correct (the country is, the city and region isn't).
The ISP is correct.

This is without torsocks.

OS

   patch     -    
patch_minor  -    
   major     -    
   minor     -    
   family    Other

Device

family  Other

Browser

 patch  -    
 major  -    
 minor  -    
family  Other

IP Details

IP Address 85.230.218.84                      
 Location  BORAS ,VASTRA GOTALANDS LAN ,SWEDEN
   ISP     TELENOR SVERIGE AB                 

Misc Details

CPU cores (logical)  
        GPU          
  Touch enabled ?    
     Resolution      
     View Port       
    Orientation      
    Color Depth      
      Battery        
      Language       
     Time Zone       
  Connection type    
        RAM          
 Cookies enabled ?   
Do not track header  
  Fonts installed

-- 
underground experts united
https://dataswamp.org/~incal

Re: Making package.el talk over Tor

[Emanuel Berg]

>> 185.220.101.26 - - [14/Dec/2023:13:04:00 +0100] "GET /test
>> HTTP/1.1" 301 169 "https://amodernist.com/" "URL/Emacs
>> Emacs/30.0.50 (PureGTK; x86_64-pc-linux-gnu)"
>
> I tried to visit this page with Emacs-w3m
>
>   https://www.site24x7.com/tools/browser-fingerprint-test.html
>
> and it outputted the below data (last).
>
> Summary: There is no information except the the IP details.
> The "IP Address" is correct, that's my public IP. The Location
> is partly correct (the country is, the city and region isn't).
> The ISP is correct.
>
> This is without torsocks.

With torsocks, i.e. to run Emacs like this

  $ torsocks emacs

everything else is still blank, and also those three things
are incorrect.

But with torsocks everything is much slower and half the stuff
don't work, e.g. Google doesn't work (429/302 Moved).

-- 
underground experts united
https://dataswamp.org/~incal

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > 185.220.101.26 - - [14/Dec/2023:13:04:00 +0100] "GET /test HTTP/1.1" 301 169 "https://amodernist.com/" "URL/Emacs Emacs/30.0.50 (PureGTK; x86_64-pc-linux-gnu)"

  > As you can see the User-Agent indicates that I am using Emacs, what
  > version and even my architecture.  Compare that to the user agent that
  > you'd regularly encounter from an average browser:

We should (1) let users specify what User-Agent to send, and (2) maybe
choose a different default.

Icecat, by default, identifies itself as some widely used proprietary
browser running on Windows.

  > Other than the user-agent, there are certainly other bits of behaviour
  > that a malicious actor can use to track a user, such as the order in
  > which HTTP headers are transmitted, the size of chunks by which the
  > client sends and receives data and of course what requests aren't being
  > sent (e.g. due to a lack of Javascript in EWW).

We could work on making Emacs-based browsing more similar to the most
common browsers, in such aspects of visible behavior.

  >  and of course what requests aren't being
  > sent (e.g. due to a lack of Javascript in EWW).

Compareed with the harm done by _running_ the page's Javascript,
giving evidence of not running Javascript is arguably a far lesser
evil.

That said, one important method for preventing sites from effectively
profiling you is to connect to them through Tor.  In fact, connecting
directly enables OTHERS that observe your network traffic to figure
out what you are talking to!

That is why I want to connect to the Emacs package repo via Tor.
I am not worried about being profiled by the Emacs package repo!

More generally, if all that distinguishes you in the actual
interaction with a site is that you don't run the Javascript, and you
connect through Tor, whatever site you are talking to will have
trouble distinguishing you from other users that don't run the
Javascript.

  > That being said: All of this doesn't matter that much for package.el,
  > since most people are accessing it via Emacs.

I agree.  However, these issues may have some real importance for the case
of using EWW to look at pages _other than_ the Emacs package repo.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

>   > 185.220.101.26 - - [14/Dec/2023:13:04:00 +0100] "GET /test HTTP/1.1" 301 169 "https://amodernist.com/" "URL/Emacs Emacs/30.0.50 (PureGTK; x86_64-pc-linux-gnu)"
>
>   > As you can see the User-Agent indicates that I am using Emacs, what
>   > version and even my architecture.  Compare that to the user agent that
>   > you'd regularly encounter from an average browser:
>
> We should (1) let users specify what User-Agent to send, and (2) maybe
> choose a different default.
>
> Icecat, by default, identifies itself as some widely used proprietary
> browser running on Windows.

Should we bump the default to 'paranoid'?  Do what icecat does?

Does the remote ever need to know if we're using X11 or PureGTK?
I think they don't, and we should never add that information, in any
configuration.

>   > Other than the user-agent, there are certainly other bits of behaviour
>   > that a malicious actor can use to track a user, such as the order in
>   > which HTTP headers are transmitted, the size of chunks by which the
>   > client sends and receives data and of course what requests aren't being
>   > sent (e.g. due to a lack of Javascript in EWW).
>
> We could work on making Emacs-based browsing more similar to the most
> common browsers, in such aspects of visible behavior.

If you are very concerned about your privacy, it's probably better to
browse the web using the Tor web browser and eschew Emacs altogether.

How about telling users about this in the EWW manual?

Re: Making package.el talk over Tor

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 17 Dec 2023 00:23:27 -0800
> Cc: akib@disroot.org, emacs-devel@gnu.org
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> >   > 185.220.101.26 - - [14/Dec/2023:13:04:00 +0100] "GET /test HTTP/1.1" 301 169 "https://amodernist.com/" "URL/Emacs Emacs/30.0.50 (PureGTK; x86_64-pc-linux-gnu)"
> >
> >   > As you can see the User-Agent indicates that I am using Emacs, what
> >   > version and even my architecture.  Compare that to the user agent that
> >   > you'd regularly encounter from an average browser:
> >
> > We should (1) let users specify what User-Agent to send, and (2) maybe
> > choose a different default.
> >
> > Icecat, by default, identifies itself as some widely used proprietary
> > browser running on Windows.
> 
> Should we bump the default to 'paranoid'?  Do what icecat does?
> 
> Does the remote ever need to know if we're using X11 or PureGTK?
> I think they don't, and we should never add that information, in any
> configuration.
> 
> >   > Other than the user-agent, there are certainly other bits of behaviour
> >   > that a malicious actor can use to track a user, such as the order in
> >   > which HTTP headers are transmitted, the size of chunks by which the
> >   > client sends and receives data and of course what requests aren't being
> >   > sent (e.g. due to a lack of Javascript in EWW).
> >
> > We could work on making Emacs-based browsing more similar to the most
> > common browsers, in such aspects of visible behavior.
> 
> If you are very concerned about your privacy, it's probably better to
> browse the web using the Tor web browser and eschew Emacs altogether.
> 
> How about telling users about this in the EWW manual?

It looks like a changeset was installed on master which changes how
URL behaves in this matter, see commit 346e571230.  I'm worried that
this is a backward-incompatible change which doesn't seem to have any
way for users to get back old behavior.  I think we should provide
such a way, and I think this change should be called out in the
"Incompatible changes" section of NEWS.

Thanks.

Never send user email address in HTTP requests

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> It looks like a changeset was installed on master which changes how
> URL behaves in this matter, see commit 346e571230.  I'm worried that
> this is a backward-incompatible change which doesn't seem to have any
> way for users to get back old behavior.  I think we should provide
> such a way, and I think this change should be called out in the
> "Incompatible changes" section of NEWS.

Thanks, I moved it to "Incompatible changes".

The TL;DR here is that I think the issue fixed in 346e571230 is a
serious issue, and that we should not provide a way to get back to the
old behavior.

The other issues we discussed in this thread had to do with
fingerprinting, which is also a real concern.  However, more steps are
required for someone to figure out your real identity.

The basic problem is that a mere misconfiguration of `url-privacy-level'
will lead a user's privacy to be fully compromised.

For example, a typo like:

    (setq url-privacy-level '(eemail))

will make Emacs announce your email (that you customized separately, for
Gnus or Notmuch) to the remote server in every HTTP request.

In fact, it's enough to customize that setting to anything that is not
`high', `paranoid', or a list containing the symbol `email'.

You best not assume you can set it to `medium', or anything like that,
because trying that will be _silently_accepted_ and then: your email
will be revealed.  That's a pretty huge gotcha, and certainly not the
way to design a security feature.

But it gets even worse: url.el used to do these acrobatics to make sure
that there is indeed something privacy breaking in there:

    (or url-personal-mail-address
        user-mail-address
        (format "%s@%s" (user-real-login-name)
                        (system-name)))

AFAIK, no other browser out there provide this misfeature.  It seems
like something from the happy 1990's that has completely outlived any
usefulness, assuming that it was at all useful even to begin with.

Providing a way to get back to the old behaviour is just re-introducing
a bad, bad footgun.  Keeping it around puts users at risk.  So I think
we shouldn't do that.

Re: Never send user email address in HTTP requests

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 17 Dec 2023 04:02:09 -0800
> Cc: rms@gnu.org, philipk@posteo.net, akib@disroot.org, emacs-devel@gnu.org, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It looks like a changeset was installed on master which changes how
> > URL behaves in this matter, see commit 346e571230.  I'm worried that
> > this is a backward-incompatible change which doesn't seem to have any
> > way for users to get back old behavior.  I think we should provide
> > such a way, and I think this change should be called out in the
> > "Incompatible changes" section of NEWS.
> 
> Thanks, I moved it to "Incompatible changes".
> 
> The TL;DR here is that I think the issue fixed in 346e571230 is a
> serious issue, and that we should not provide a way to get back to the
> old behavior.

Sorry, but I disagree.  Emacs should not second-guess the users, and
should certainly NOT force them into what we consider to be the secure
environment.  It is okay to behave securely by default, but if someone
wants to be insecure, for whatever reasons, we should let them have
the old, insecure behavior.  Certainly when we first change the
default, since there's a possibility that something will break for
someone due to this change, and we need to let users have a fire
escape in those cases, until we get our act together in the next
release.

> The basic problem is that a mere misconfiguration of `url-privacy-level'
> will lead a user's privacy to be fully compromised.
> 
> For example, a typo like:
> 
>     (setq url-privacy-level '(eemail))
> 
> will make Emacs announce your email (that you customized separately, for
> Gnus or Notmuch) to the remote server in every HTTP request.

If typos are the problem, it should be okay to detect that and holler
or signal an error.  But the changes disallow even values that have no
typos, under the premise that "we know better".  I very much object to
software that "knows better", and hope Emacs will never behave like
that.

> In fact, it's enough to customize that setting to anything that is not
> `high', `paranoid', or a list containing the symbol `email'.
> 
> You best not assume you can set it to `medium', or anything like that,
> because trying that will be _silently_accepted_ and then: your email
> will be revealed.  That's a pretty huge gotcha, and certainly not the
> way to design a security feature.

Again: it is okay to refrain from silently accepting invalid values.
That's not my concern.  In the Emacs where I'm typing this the value
of url-privacy-level is (email), and that has no typos.  I see no
reasons to disallow such values, as long as they are not the default.

> But it gets even worse: url.el used to do these acrobatics to make sure
> that there is indeed something privacy breaking in there:
> 
>     (or url-personal-mail-address
>         user-mail-address
>         (format "%s@%s" (user-real-login-name)
>                         (system-name)))
> 
> AFAIK, no other browser out there provide this misfeature.  It seems
> like something from the happy 1990's that has completely outlived any
> usefulness, assuming that it was at all useful even to begin with.
> 
> Providing a way to get back to the old behaviour is just re-introducing
> a bad, bad footgun.  Keeping it around puts users at risk.  So I think
> we shouldn't do that.

See above: I don't agree to be smarter than everybody.  The defaults
should be secure, but completely locking out users is not.

Re: Never send user email address in HTTP requests

[Yuri Khan]

On Sun, 17 Dec 2023 at 19:36, Eli Zaretskii <eliz@gnu.org> wrote:

> Sorry, but I disagree.  Emacs should not second-guess the users, and
> should certainly NOT force them into what we consider to be the secure
> environment.  It is okay to behave securely by default, but if someone
> wants to be insecure, for whatever reasons, we should let them have
> the old, insecure behavior.  Certainly when we first change the
> default, since there's a possibility that something will break for
> someone due to this change, and we need to let users have a fire
> escape in those cases, until we get our act together in the next
> release.

The header in question, From, is governed by RFC 9110 § 10.1.2[0], which says:

    The From header field is rarely sent by non-robotic user agents.
    A user agent SHOULD NOT send a From header field
    without explicit configuration by the user,
    since that might conflict with the user's privacy interests
    or their site's security policy.

    A robotic user agent SHOULD send a valid From header field
    so that the person responsible for running the robot can be contacted
    if problems occur on servers,
    such as if the robot is sending excessive, unwanted, or invalid requests.

[0]: https://www.rfc-editor.org/rfc/rfc9110.html#section-10.1.2

That is, it’s not intended for web browsers or interactive
applications acting on the user’s behalf. It’s for spiders and other
automation.

Surely a user who is writing a spider in Elisp can take the extra
conscious step of filling out the general fire escape,
‘url-request-extra-headers’, and deciding which email address to
expose.

It is good that the default value of ‘url-privacy-level’ is (email),
preventing the leak by default, but there is no reason to make it
possible to configure url.el to leak it with every request made from
Emacs. If you’re running a spider and also just browsing the Web with
EWW, you probably only want requests from your spider to be attributed
to you as the spider maintainer.

Re: Never send user email address in HTTP requests

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 17 Dec 2023 21:05:00 +0700
> Cc: Stefan Kangas <stefankangas@gmail.com>, rms@gnu.org, philipk@posteo.net, 
>  akib@disroot.org, emacs-devel@gnu.org, monnier@iro.umontreal.ca
> 
> On Sun, 17 Dec 2023 at 19:36, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Sorry, but I disagree.  Emacs should not second-guess the users, and
> > should certainly NOT force them into what we consider to be the secure
> > environment.  It is okay to behave securely by default, but if someone
> > wants to be insecure, for whatever reasons, we should let them have
> > the old, insecure behavior.  Certainly when we first change the
> > default, since there's a possibility that something will break for
> > someone due to this change, and we need to let users have a fire
> > escape in those cases, until we get our act together in the next
> > release.
> 
> The header in question, From, is governed by RFC 9110 § 10.1.2[0], which says:

Thanks, but this isn't relevant to the issue at hand.

> It is good that the default value of ‘url-privacy-level’ is (email),
> preventing the leak by default, but there is no reason to make it
> possible to configure url.el to leak it with every request made from
> Emacs. If you’re running a spider and also just browsing the Web with
> EWW, you probably only want requests from your spider to be attributed
> to you as the spider maintainer.

I remain convinced that we should allow users who actively want that
to make their Emacs behave against any RFCs, when Emacs has been
behaving like that for many years until now.

Re: Never send user email address in HTTP requests

[T.V Raman]

Yuri Khan <yuri.v.khan@gmail.com> writes:


Just let-bind url-privacy-level to paranoid
> On Sun, 17 Dec 2023 at 19:36, Eli Zaretskii <eliz@gnu.org> wrote:
>
>> Sorry, but I disagree.  Emacs should not second-guess the users, and
>> should certainly NOT force them into what we consider to be the secure
>> environment.  It is okay to behave securely by default, but if someone
>> wants to be insecure, for whatever reasons, we should let them have
>> the old, insecure behavior.  Certainly when we first change the
>> default, since there's a possibility that something will break for
>> someone due to this change, and we need to let users have a fire
>> escape in those cases, until we get our act together in the next
>> release.
>
> The header in question, From, is governed by RFC 9110 § 10.1.2[0], which says:
>
>     The From header field is rarely sent by non-robotic user agents.
>     A user agent SHOULD NOT send a From header field
>     without explicit configuration by the user,
>     since that might conflict with the user's privacy interests
>     or their site's security policy.
>
>     A robotic user agent SHOULD send a valid From header field
>     so that the person responsible for running the robot can be contacted
>     if problems occur on servers,
>     such as if the robot is sending excessive, unwanted, or invalid requests.
>
> [0]: https://www.rfc-editor.org/rfc/rfc9110.html#section-10.1.2
>
> That is, it’s not intended for web browsers or interactive
> applications acting on the user’s behalf. It’s for spiders and other
> automation.
>
> Surely a user who is writing a spider in Elisp can take the extra
> conscious step of filling out the general fire escape,
> ‘url-request-extra-headers’, and deciding which email address to
> expose.
>
> It is good that the default value of ‘url-privacy-level’ is (email),
> preventing the leak by default, but there is no reason to make it
> possible to configure url.el to leak it with every request made from
> Emacs. If you’re running a spider and also just browsing the Web with
> EWW, you probably only want requests from your spider to be attributed
> to you as the spider maintainer.
>

-- 

Re: Never send user email address in HTTP requests

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

      > A user agent SHOULD NOT send a From header field
      > without explicit configuration by the user,

If the user sets a user option to request something,
that is explicit configuration by the user.

In addition, the words "SHOULD NOT" are vehement, but they do not show
a reason that could convince us do or not to do something.  If we saw
a reason -- some trouble that would result, for someone or other --
that could be a reason, but it would depend on what the trouble would
be, etc.
-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If you are very concerned about your privacy, it's probably better to
  > browse the web using the Tor web browser and eschew Emacs altogether.

The issue here is how to make package.el talk using Tor with the Emacs
package site.  Can package.el do this using an external browser?
If so, that might do the job.  How can one do this?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > 185.220.101.26 - - [14/Dec/2023:13:04:00 +0100] "GET /test HTTP/1.1" 301 169 "https://amodernist.com/" "URL/Emacs Emacs/30.0.50 (PureGTK; x86_64-pc-linux-gnu)"
>
>   > As you can see the User-Agent indicates that I am using Emacs, what
>   > version and even my architecture.  Compare that to the user agent that
>   > you'd regularly encounter from an average browser:
>
> We should (1) let users specify what User-Agent to send, 

That is already possible using the `url-user-agent' user option.

>                                                          and (2) maybe
> choose a different default.

1+

> Icecat, by default, identifies itself as some widely used proprietary
> browser running on Windows.

While helpful, it could also be dangerous if the server decides to send
the user content that only a widely used proprietary browser running on
Windows could process.

>   > Other than the user-agent, there are certainly other bits of behaviour
>   > that a malicious actor can use to track a user, such as the order in
>   > which HTTP headers are transmitted, the size of chunks by which the
>   > client sends and receives data and of course what requests aren't being
>   > sent (e.g. due to a lack of Javascript in EWW).
>
> We could work on making Emacs-based browsing more similar to the most
> common browsers, in such aspects of visible behavior.
>
>   >  and of course what requests aren't being
>   > sent (e.g. due to a lack of Javascript in EWW).
>
> Compareed with the harm done by _running_ the page's Javascript,
> giving evidence of not running Javascript is arguably a far lesser
> evil.

The way I see this, this is a security/privacy vs freedom issue.  If you
really want to blend in, you have to behave the way most people do.

> That said, one important method for preventing sites from effectively
> profiling you is to connect to them through Tor.  In fact, connecting
> directly enables OTHERS that observe your network traffic to figure
> out what you are talking to!

Tor hides your IP address, in which sense it acts like a trusted VPN
service, but I am just trying to emphasise that (in general) just using
Tor as a transport layer can give users a false sense of security.

> That is why I want to connect to the Emacs package repo via Tor.
> I am not worried about being profiled by the Emacs package repo!
>
> More generally, if all that distinguishes you in the actual
> interaction with a site is that you don't run the Javascript, and you
> connect through Tor, whatever site you are talking to will have
> trouble distinguishing you from other users that don't run the
> Javascript.

True, but considering how infrequent this is on a global scale, "no
Javascript" still holds a lot of information.

>   > That being said: All of this doesn't matter that much for package.el,
>   > since most people are accessing it via Emacs.
>
> I agree.  However, these issues may have some real importance for the case
> of using EWW to look at pages _other than_ the Emacs package repo.

Right, that is where my concerns from above apply.

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > If you have a Tor daemon running on your system, it should provide a
>   > SOCKS proxy by default,
>
> I believe it does that.
>
>                             meaning that you can configure every package
>   > that uses url.el to go through this proxy (unless I have misunderstood
>   > something):
>
> Maybe that is possible.  But I know very little about sockets or how
> to use them.  I would appreciate precise advice about what I should do
> to tell package.el to connect using Tor.

That was the code you quoted below with `url-gateway-method'.

> You sent this code
>
>     (let ((conn
>            (open-network-stream
>             "test" (current-buffer) "gnu.org" "http"
>             :type 'shell
>             :shell-command "torsocks nc %s %p")))
>       (process-send-string conn "GET / HTTP/1.0\r\n\r\n"))
>
> but I don't think that is a solution ready to use.  It looks like a
> proposed approach for designing that solution.
>
> Also, the end of your message MAY mean that this approach is not
> applicable to package.el.

Yes, this was just a proof of concept integrating torsocks into a
network connection open from within Emacs.

>   > If you have a Tor daemon running on your system, it should provide a
>   > SOCKS proxy by default, meaning that you can configure every package
>   > that uses url.el to go through this proxy (unless I have misunderstood
>   > something):
>
>   > --8<---------------cut here---------------start------------->8---
>   > (setq url-gateway-method 'socks
>   >       socks-server '("Tor" "localhost" 9050 5))
>   > --8<---------------cut here---------------end--------------->8---
>
>   > That being said, while testing I noticed that when connecting to a
>   > server I have access to, I always receive two requests, one from a Tor
>   > exit node and one from my current IP address.  Unless I missed something
>   > obvious, that might be a bug.
>
> I await word about further progress.

I will try and look into if I am mis-configuring something, and
otherwise report a bug.

> When I know enough to be able to do it, I will add a feature to
> package.el for a user option to tell it to go through Tor always.

Stefan Kangas <stefankangas@gmail.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>>   > 185.220.101.26 - - [14/Dec/2023:13:04:00 +0100] "GET /test
>>   > HTTP/1.1" 301 169 "https://amodernist.com/" "URL/Emacs
>>   > Emacs/30.0.50 (PureGTK; x86_64-pc-linux-gnu)"
>>
>>   > As you can see the User-Agent indicates that I am using Emacs, what
>>   > version and even my architecture.  Compare that to the user agent that
>>   > you'd regularly encounter from an average browser:
>>
>> We should (1) let users specify what User-Agent to send, and (2) maybe
>> choose a different default.
>>
>> Icecat, by default, identifies itself as some widely used proprietary
>> browser running on Windows.
>
> Should we bump the default to 'paranoid'?  Do what icecat does?

That wouldn't send any Use-Agent header at all, which is still
fingerprintable.  Richard mentioned that Icecat uses a false user agent like

  Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.3

but that would have to be updated on a regular basis, to keep up with
whatever is being used.  That being said, I don't believe that this is
enough, if we take the threat model into account that a Tor user is
operating under.  In their case, it probably is the best to just use the
Tor Browser.

> Does the remote ever need to know if we're using X11 or PureGTK?
> I think they don't, and we should never add that information, in any
> configuration.

Yes, it might make sense to just add `os' to `url-privacy-level'.

>>   > Other than the user-agent, there are certainly other bits of behaviour
>>   > that a malicious actor can use to track a user, such as the order in
>>   > which HTTP headers are transmitted, the size of chunks by which the
>>   > client sends and receives data and of course what requests aren't being
>>   > sent (e.g. due to a lack of Javascript in EWW).
>>
>> We could work on making Emacs-based browsing more similar to the most
>> common browsers, in such aspects of visible behavior.
>
> If you are very concerned about your privacy, it's probably better to
> browse the web using the Tor web browser and eschew Emacs altogether.
>
> How about telling users about this in the EWW manual?

1+

-- 
Philip Kaludercic

Re: Making package.el talk over Tor

[Yuri Khan]

On Sun, 17 Dec 2023 at 20:04, Philip Kaludercic <philipk@posteo.net> wrote:

> > Icecat, by default, identifies itself as some widely used proprietary
> > browser running on Windows.
>
> While helpful, it could also be dangerous if the server decides to send
> the user content that only a widely used proprietary browser running on
> Windows could process.

It also inflates That Other Browser’s usage statistics. Which is
harmful because web site authors say “yay, Firefox and its forks have
fallen below our support threshold, we can start using features only
supported in That Other Browser”.

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > While helpful, it could also be dangerous if the server decides to send
  > > the user content that only a widely used proprietary browser running on
  > > Windows could process.

It could be a little annoying, but minor annoyances are widespread on
the web.  So this wouldn't be a reason to change a decision.

  > It also inflates That Other Browser’s usage statistics. 

That effect is tiny -- not enough to worry about.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

> Compareed with the harm done by _running_ the page's Javascript,
> giving evidence of not running Javascript is arguably a far lesser
> evil.

The person who set this default for IceCat has sttudied the situation
csrefully; to dismiss his conclusions hastily is not wise.

  > The way I see this, this is a security/privacy vs freedom issue.  If you
  > really want to blend in, you have to behave the way most people do.

That simple heuristic is valid in many situations, but not in this
one.  With Javascript, the option you're calling "blending in" means
letting each site take your fingerprints in its usual way.  Trying to
"blend in" that way will backfire completely: each site can precisely
recognize you each time, and distinguish you from all the other
visitors.

  > Tor hides your IP address, in which sense it acts like a trusted VPN
  > service, but I am just trying to emphasise that (in general) just using
  > Tor as a transport layer can give users a false sense of security.

Indeed, just using Tor is not sufficient to stop a web site from
recognizing me when I come back.  I take other precautions too.

Meanwhile, can anyone help me make it easy to use Tor when
communicating with the Emacs package archive site.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > If you are very concerned about your privacy, it's probably better to
  > > browse the web using the Tor web browser and eschew Emacs altogether.

I'm looking for a way I could make package.el to communicate with the
Emacs package repository via the Tor network.  I think you've drifted
onto a totally different subject.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I have been trying to understand how package.el calls url.el
but I can't begin to understand URL.  What are its entry points?
How is it supposed to be used?

The file url.el gives no information whatsoever about how that package
works or how to use it.  It creates a process only inside
url-open-rlogin, url-open-telnet and url-gateway-nslookup-host, but
these are buried deep inside its calling structure.

package.el calls a few url-... functions.  They are called from three
places that are subroutines or auxiliary macros, two of which have
next to no documentation of what they do.  The one that has some
information is package--with-work-buffer, whose doc string is very
abstract.  It could be used for almost anything.

      "Run BODY in a buffer containing the contents of FILE at LOCATION.
    LOCATION is the base location of a package archive, and should be
    one of the URLs (or file names) specified in `package-archives'.
    FILE is the name of a file relative to that base location.

    This macro retrieves FILE from LOCATION into a temporary buffer,
    and evaluates BODY while that buffer is current.  This work
    buffer is killed afterwards.  Return the last value in BODY."

I decided to change the calls to start-process.  I can do that without
undertsnading url.el.  Maybe it will work.

Would someone like to clean up the URL package and document
its structure, its entry points, and how to use them?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Fri, 17 Nov 2023 22:03:46 -0500
> 
> I have been trying to understand how package.el calls url.el
> but I can't begin to understand URL.  What are its entry points?
> How is it supposed to be used?
> 
> The file url.el gives no information whatsoever about how that package
> works or how to use it.

The url package has its own Info manual, see doc/misc/url.texi.  If
that manual doesn't answer your questions, we need to improve it, but
in that case, please tell what is missing from that manual that would
have helped you to get the information you needed.

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The url package has its own Info manual, see doc/misc/url.texi.  If
  > that manual doesn't answer your questions, we need to improve it, but
  > in that case, please tell what is missing from that manual that would
  > have helped you to get the information you needed.

Thanks for telling me about that.  It does say how other packages
should _use_ URL.  It does not explain the internal structure of URL
itself, which is not straightforward.  I think the URL source needs
comments to explain that.

The manual helped me get started towards solving my problem, but not
enough to get me to a solution.

The section Proxies and Gatewaying suggests that perhaps it is
possible to use a Socks gateway to make URL connect through Tor.  But
this short explanation of Socks gateways

    @item socks
    @cindex @sc{socks}
    Use if the firewall has a @sc{socks} gateway running on it.  The
    @sc{socks} v5 protocol is defined in RFC 1928.

doesn't explain enough to show concretely how to do that.

Can anyone tell me how to do that?

The RFC is not likely to give much help, and it might not even mention
the crucial systen-specific details needed to actually do this.

I think each of these gateway methods should have examples showing
concretely how to use it. and one example for `socks' could be using
the Tor demon as a gateway.

In the rest of the manual, I noted these items

    @item dav
    @cindex DAV
    A list of numbers specifying what DAV protocol/schema versions are
    supported.

    @item dasl
    @cindex DASL
    A list of supported DASL search types supported (string form).

which need some sort of explanation of what DAV and DASL mean,
and where to find more info on them.  I don't know those terms.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

> I would like to arrange for package.el
> to always connect to ELPA across the Tor network.
> But it is 4600 lines of code and I would rather not have to read it all.
>
> Can someone tell me where to find the code that actually
> communicates with the ELPA repos?  Where is the best place
> to make that change?

I found these macros by searching for the string "(url":

    package--with-response-buffer-1
    package--with-work-buffer

I don't know if you want this to affect package-vc, but I guess a new
option would be even more useful if it could cover that too.

Re: Making package.el talk over Tor

[Philip Kaludercic]

Stefan Kangas <stefankangas@gmail.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> I would like to arrange for package.el
>> to always connect to ELPA across the Tor network.
>> But it is 4600 lines of code and I would rather not have to read it all.
>>
>> Can someone tell me where to find the code that actually
>> communicates with the ELPA repos?  Where is the best place
>> to make that change?
>
> I found these macros by searching for the string "(url":
>
>     package--with-response-buffer-1
>     package--with-work-buffer
>
> I don't know if you want this to affect package-vc, but I guess a new
> option would be even more useful if it could cover that too.

IIRC all HTTP requests by package-vc go through these functions as well,
so if they were to be made Tor-compatible and vc-tor was set, then this
/could/ be safe, setting aside issues such as detailed fingerprinting.

-- 
Philip Kaludercic