Making package.el talk over Tor

Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I would like to arrange for package.el
to always connect to ELPA across the Tor network.
But it is 4600 lines of code and I would rather not have to read it all.

Can someone tell me where to find the code that actually
communicates with the ELPA repos?  Where is the best place
to make that change?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Akib Azmain Turja]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> I would like to arrange for package.el
> to always connect to ELPA across the Tor network.
> But it is 4600 lines of code and I would rather not have to read it all.
>
> Can someone tell me where to find the code that actually
> communicates with the ELPA repos?  Where is the best place
> to make that change?

Isearching for 'url-' reveals that the following functions use the URL
package to access the HTTP server: 'package--with-work-buffer',
'package--archive-file-exists-p' and'package--with-response-buffer-1'.

But I think a better/safer solution will be to use torsocks.

-- 
Akib Azmain Turja, GPG key: 70018CE5819F17A3BBA666AFE74F0EFA922AE7F5
Fediverse: akib@hostux.social
Codeberg: akib
emailselfdefense.fsf.org | "Nothing can be secure without encryption."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Making package.el talk over Tor

[Emanuel Berg]

Akib Azmain Turja wrote:

>> I would like to arrange for package.el to always connect to
>> ELPA across the Tor network. But it is 4600 lines of code
>> and I would rather not have to read it all.
>>
>> Can someone tell me where to find the code that actually
>> communicates with the ELPA repos? Where is the best place
>> to make that change?
>
> Isearching for 'url-' reveals that the following functions
> use the URL package to access the HTTP server:
> 'package--with-work-buffer',
> 'package--archive-file-exists-p'
> and'package--with-response-buffer-1'.
>
> But I think a better/safer solution will be to use torsocks.

Holy socks! But won't there be a connection speed disadvantage
from doing this?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Making package.el talk over Tor

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

> I would like to arrange for package.el
> to always connect to ELPA across the Tor network.
> But it is 4600 lines of code and I would rather not have to read it all.
>
> Can someone tell me where to find the code that actually
> communicates with the ELPA repos?  Where is the best place
> to make that change?

I found these macros by searching for the string "(url":

    package--with-response-buffer-1
    package--with-work-buffer

I don't know if you want this to affect package-vc, but I guess a new
option would be even more useful if it could cover that too.

Re: Making package.el talk over Tor

[Philip Kaludercic]

Stefan Kangas <stefankangas@gmail.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> I would like to arrange for package.el
>> to always connect to ELPA across the Tor network.
>> But it is 4600 lines of code and I would rather not have to read it all.
>>
>> Can someone tell me where to find the code that actually
>> communicates with the ELPA repos?  Where is the best place
>> to make that change?
>
> I found these macros by searching for the string "(url":
>
>     package--with-response-buffer-1
>     package--with-work-buffer
>
> I don't know if you want this to affect package-vc, but I guess a new
> option would be even more useful if it could cover that too.

IIRC all HTTP requests by package-vc go through these functions as well,
so if they were to be made Tor-compatible and vc-tor was set, then this
/could/ be safe, setting aside issues such as detailed fingerprinting.

-- 
Philip Kaludercic

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Isearching for 'url-' reveals that the following functions use the URL
  > package to access the HTTP server: 'package--with-work-buffer',
  > 'package--archive-file-exists-p' and'package--with-response-buffer-1'.

  > But I think a better/safer solution will be to use torsocks.

Better/safer solution than which other solution?

Using torsocks was what I had in mind -- the question is
where it would be easy to do that.  Would it be easy to do
in those three functions?  If not, then where?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Thanks for replying to my question a month ago.
I got so backlogged I just saw the reply.

  > > Can someone tell me where to find the code that actually
  > > communicates with the ELPA repos?  Where is the best place
  > > to make that change?

  > Isearching for 'url-' reveals that the following functions use the URL
  > package to access the HTTP server: 'package--with-work-buffer',
  > 'package--archive-file-exists-p' and'package--with-response-buffer-1'.

  > But I think a better/safer solution will be to use torsocks.

I agree, and I plan to use torsocks.  But in order to do thst, I need
to know where to do that.  That's really what my question was about.

Can you suggest where I should do that?

Does the url package have a variable to specify the command to use?
I could specify torsocks there.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> Thanks for replying to my question a month ago.
> I got so backlogged I just saw the reply.
>
>   > > Can someone tell me where to find the code that actually
>   > > communicates with the ELPA repos?  Where is the best place
>   > > to make that change?
>
>   > Isearching for 'url-' reveals that the following functions use the URL
>   > package to access the HTTP server: 'package--with-work-buffer',
>   > 'package--archive-file-exists-p' and'package--with-response-buffer-1'.
>
>   > But I think a better/safer solution will be to use torsocks.
>
> I agree, and I plan to use torsocks.  But in order to do thst, I need
> to know where to do that.  That's really what my question was about.
>
> Can you suggest where I should do that?
>
> Does the url package have a variable to specify the command to use?
> I could specify torsocks there.

No, url.el eventually calls `make-network-process', that directly
invokes the respective networking system calls, not making it possible
to interject torsocks.  What I believe Akib meant was to start Emacs
with torsocks, but to my understanding this is not a recommended
practice either, because one will continue to leak fingerprintable
metadata (specially inside of Emacs) that would undermine the point of
using Tor to begin with.

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I have been trying to understand how package.el calls url.el
but I can't begin to understand URL.  What are its entry points?
How is it supposed to be used?

The file url.el gives no information whatsoever about how that package
works or how to use it.  It creates a process only inside
url-open-rlogin, url-open-telnet and url-gateway-nslookup-host, but
these are buried deep inside its calling structure.

package.el calls a few url-... functions.  They are called from three
places that are subroutines or auxiliary macros, two of which have
next to no documentation of what they do.  The one that has some
information is package--with-work-buffer, whose doc string is very
abstract.  It could be used for almost anything.

      "Run BODY in a buffer containing the contents of FILE at LOCATION.
    LOCATION is the base location of a package archive, and should be
    one of the URLs (or file names) specified in `package-archives'.
    FILE is the name of a file relative to that base location.

    This macro retrieves FILE from LOCATION into a temporary buffer,
    and evaluates BODY while that buffer is current.  This work
    buffer is killed afterwards.  Return the last value in BODY."

I decided to change the calls to start-process.  I can do that without
undertsnading url.el.  Maybe it will work.

Would someone like to clean up the URL package and document
its structure, its entry points, and how to use them?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Fri, 17 Nov 2023 22:03:46 -0500
> 
> I have been trying to understand how package.el calls url.el
> but I can't begin to understand URL.  What are its entry points?
> How is it supposed to be used?
> 
> The file url.el gives no information whatsoever about how that package
> works or how to use it.

The url package has its own Info manual, see doc/misc/url.texi.  If
that manual doesn't answer your questions, we need to improve it, but
in that case, please tell what is missing from that manual that would
have helped you to get the information you needed.

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > No, url.el eventually calls `make-network-process', that directly
  > invokes the respective networking system calls, not making it possible
  > to interject torsocks.

Is this the code you mean?

		      (open-network-stream
		       name buffer host service
		       :type gw-method
		       ;; Use non-blocking socket if we can.
		       :nowait (and (featurep 'make-network-process)
                                    (url-asynchronous url-current-object)
                                    '(:nowait t)))

How can I make that use TOR?

I could set `url-gateway-method' to specify some other method, but
what other value would make it possible to put in torsocks?

  > What I believe Akib meant was to start Emacs
  > with torsocks,

I think that is no good -- all subprocesses of Emacs would be
compelled to use torsocks too.

  >  because one will continue to leak fingerprintable
  > metadata (specially inside of Emacs)

Could you give me an example of what you mean?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Making package.el talk over Tor

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: akib@disroot.org, emacs-devel@gnu.org
> Date: Sat, 18 Nov 2023 22:39:36 -0500
> 
>   > No, url.el eventually calls `make-network-process', that directly
>   > invokes the respective networking system calls, not making it possible
>   > to interject torsocks.
> 
> Is this the code you mean?
> 
> 		      (open-network-stream
> 		       name buffer host service
> 		       :type gw-method
> 		       ;; Use non-blocking socket if we can.
> 		       :nowait (and (featurep 'make-network-process)
>                                     (url-asynchronous url-current-object)
>                                     '(:nowait t)))

Yes.

> How can I make that use TOR?

Perhaps the ':type shell' connection is the solution?

Re: Making package.el talk over Tor

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The url package has its own Info manual, see doc/misc/url.texi.  If
  > that manual doesn't answer your questions, we need to improve it, but
  > in that case, please tell what is missing from that manual that would
  > have helped you to get the information you needed.

Thanks for telling me about that.  It does say how other packages
should _use_ URL.  It does not explain the internal structure of URL
itself, which is not straightforward.  I think the URL source needs
comments to explain that.

The manual helped me get started towards solving my problem, but not
enough to get me to a solution.

The section Proxies and Gatewaying suggests that perhaps it is
possible to use a Socks gateway to make URL connect through Tor.  But
this short explanation of Socks gateways

    @item socks
    @cindex @sc{socks}
    Use if the firewall has a @sc{socks} gateway running on it.  The
    @sc{socks} v5 protocol is defined in RFC 1928.

doesn't explain enough to show concretely how to do that.

Can anyone tell me how to do that?

The RFC is not likely to give much help, and it might not even mention
the crucial systen-specific details needed to actually do this.

I think each of these gateway methods should have examples showing
concretely how to use it. and one example for `socks' could be using
the Tor demon as a gateway.

In the rest of the manual, I noted these items

    @item dav
    @cindex DAV
    A list of numbers specifying what DAV protocol/schema versions are
    supported.

    @item dasl
    @cindex DASL
    A list of supported DASL search types supported (string form).

which need some sort of explanation of what DAV and DASL mean,
and where to find more info on them.  I don't know those terms.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)