From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Thanos Apollo <public@thanosapollo.org> Newsgroups: gmane.emacs.devel Subject: Re: Using tor & url Date: Sun, 11 Feb 2024 09:43:50 +0200 Message-ID: <87frxzbdbd.fsf@thanosapollo.org> References: <87ttr67rkt.fsf@thanosapollo.com> <CADwFkm=OV5rmDvVuyhrn8HusxsbJRe_THPNUrMNPjLzMssPjMg@mail.gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28139"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.10.8; emacs 29.2 Cc: Thanos Apollo <public@thanosapollo.com>, emacs-devel@gnu.org, rms@gnu.org To: Stefan Kangas <stefankangas@gmail.com> Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Feb 11 10:01:30 2024 Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org> Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>) id 1rZ5iJ-00077z-9W for ged-emacs-devel@m.gmane-mx.org; Sun, 11 Feb 2024 10:01:29 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <emacs-devel-bounces@gnu.org>) id 1rZ5hZ-0002tq-EN; Sun, 11 Feb 2024 04:00:41 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <public@thanosapollo.org>) id 1rZ5RY-0000xi-V8 for emacs-devel@gnu.org; Sun, 11 Feb 2024 03:44:08 -0500 Original-Received: from out3-smtp.messagingengine.com ([66.111.4.27]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <public@thanosapollo.org>) id 1rZ5RW-0007bz-CZ; Sun, 11 Feb 2024 03:44:08 -0500 Original-Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 0ADCC5C0093; Sun, 11 Feb 2024 03:44:04 -0500 (EST) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Sun, 11 Feb 2024 03:44:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= thanosapollo.org; h=cc:cc:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1707641044; x=1707727444; bh=gowleOfiPT8M3hrIrTVKFyS0zaNDZmocMF2UF9c2txk=; b= SDlBCd7qg9CyvZN51ulna68TFyrmbezTDfi0o6+FvKfAzTbNm8yVYWR4NQUpmxbP UDwhFJqT5YluwMA3YIIHd93dGwejB06f9XEpx3d0Y4/nMYadTUYeOVUUBmn8sKff uNVPspY4nEJphG4kqU46JX8jVulW509LpC3FIGqgpDyUcmqgjIu6OYaCUOkq+h+n zfO4vK3R81spQNEiAO5YN19NlZQfMXCKjtwTpKvfgHPHVRljo0WKbZzw95weaYH7 xA4GMffxYsbiYBfaapFwB+Wj3CPrwfZ3ZlZ4MDeHDhFoyunN6AU7p3xuQKNUR+FW KnJ7OnyiiedmRkb5QbomuQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1707641044; x=1707727444; bh=gowleOfiPT8M3hrIrTVKFyS0zaND ZmocMF2UF9c2txk=; b=CqfMMCn9Cg21nJ8Qa9IYqPRxEh4qlR7uQ98tPFap5MLZ Yui2kZMgnbKZOiFKUSwGfAswodVaAqPb0tVVG9VRBs1BBtbbIltnATsKXG1noR9b 9URLOaUICQIJkbKza8kLaZy1FkYCiD6PQy9+WUgq4Fajkj9zktJOFH34z3tTbdQ3 nWUAMUkFoRfnamH1LFU11VT8gB7Jh6PskQKcs5N5nIk+qMdfmkZEtRmztrXUPQ88 AZ/7/lsD7XSrriupckTI6PItdCI01KPH9o7qUZ2Nek0JfNsvRU71Dsf/yAlcu3Im Ywvz3ONPZbHsnxMPu/885MaLr9SPzmt+sIBhmWVqDA== X-ME-Sender: <xms:04jIZSFE7CArI2AlLAaMHjDnD8us7NrbEWpncQs3AI_vc5mQWra3TQ> <xme:04jIZTXEMP7jPk_gDlWjKeotgSegCSmuUv_PNx5hstSgTLGGdCovpZmjueK-zXdhP 5AGqxYpjmO24Byw4dc> X-ME-Received: <xmr:04jIZcKAR9n31fzk2NYwRW_1wdWOzTovIILn1LfdkGcp8T4ZdpzVtw> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddtgdduvdefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfhgfhffvvefuffgjkfggtgesghdtreertdertdenucfhrhhomhepvfhhrghn ohhsucetphholhhlohcuoehpuhgslhhitgesthhhrghnohhsrghpohhllhhordhorhhgqe enucggtffrrghtthgvrhhnpeelueefgfeuheeghfduveekgedukeegudekveeileefteff ieejlefgleejhfdvheenucffohhmrghinhepihhnvhhiughiohhushdrihhopdhgihhthh husgdrtghomhdpthhhrghnohhsrghpohhllhhordhorhhgnecuvehluhhsthgvrhfuihii vgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhusghlihgtsehthhgrnhhoshgrph holhhlohdrohhrgh X-ME-Proxy: <xmx:04jIZcGK2_SyA8SeXiryvgm96KUvFOdctFUDwG9ENkPIu2GvsRn5SQ> <xmx:04jIZYU-UwSRzVLMBKXKjOhfl_-xT838MH9KMXKw8dfmBKpVecfSRQ> <xmx:04jIZfO4HxuySHwCKHtuES52aq201HQzK1mAckgTPR7GfgU8Y1vQ1A> <xmx:1IjIZfQpvcDYi2OcRPUJIj2D0Z8M0pq9yaAB4heCfppLXed6u1syDA> Feedback-ID: if38949d4:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 11 Feb 2024 03:44:02 -0500 (EST) In-reply-to: <CADwFkm=OV5rmDvVuyhrn8HusxsbJRe_THPNUrMNPjLzMssPjMg@mail.gmail.com> Received-SPF: pass client-ip=66.111.4.27; envelope-from=public@thanosapollo.org; helo=out3-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Sun, 11 Feb 2024 04:00:39 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." <emacs-devel.gnu.org> List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-devel>, <mailto:emacs-devel-request@gnu.org?subject=unsubscribe> List-Archive: <https://lists.gnu.org/archive/html/emacs-devel> List-Post: <mailto:emacs-devel@gnu.org> List-Help: <mailto:emacs-devel-request@gnu.org?subject=help> List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-devel>, <mailto:emacs-devel-request@gnu.org?subject=subscribe> Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:316109 Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/316109> --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Quite a late reply to a dead thread, but for anyone reading this in the future and is interested in implementing tor support, it might be useful. Stefan Kangas <stefankangas@gmail.com> writes: [...] > > From a very quick glance at the code: > > - Is there anything more needed to add support for Individuous > instances? See: https://invidious.io/ > (FYI this has been added) > - Any chance you could add an option like `vc-tor'? Is it recommended to use 'socks for this? e.g using a macro like this one: =3D=3D=3D=3D (defmacro yeetube-with-tor-socks (&rest body) `(let ((url-gateway-method 'socks) (socks-noproxy '("localhost")) (socks-server '("Default server" "127.0.0.1" 9050 5))) ,@body)) =3D=3D=3D=3D This was the easiest way I could find to route traffic using #'url-retrieve-synchronously to tor. A built in option like this for url to route all traffic through tor might be possible? Additionally, I found the performance of url-retrieve-synchronously to improve when I set 'url-request-extra-headers' to something like this: =3D=3D=3D=3D '(("Accept-Language" . "Accept-Language: en-US,en;q=3D0.9") ("Accept" . "text/html,application/xhtml+xml,application/xml;q=3D0.9,im= age/webp,image/apng,*/*;q=3D0.8") ("User-Agent" . "Mozilla/5.0 (Windows NT 10.0; rv:122.0) Gecko/20100101= Firefox/122.0")) =3D=3D=3D=3D I'm using the user-agent from arkenfox user.js <https://github.com/arkenfox/user.js/> I had url-privacy-level set to 'high, but despite that my user-agent was "URL/Emacs Emacs/29.2 (X11; x86_64-pc-linux-gnu)". Changing this improved the performance of #'url-retrieve-synchronously (maybe because it's not a common user-agent and it required further verification?) & makes websites that utilize user-agent blocking accessible. Also, when it comes to 'url-mime-language-string', shouldn't that either be set to "en" by default (instead of nil) or get a locale value using something like getenv? I was located in Greece with locale set to en-US but I was getting Spanish by default on most websites (still trying to figure out why). =2D-=20 Information wants to be free. Thanos Apollo https://thanosapollo.org --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJMBAEBCAA2FiEEYrdY0PZxmTi8Cc7KM59zbDpyCSgFAmXIiNYYHHB1YmxpY0B0 aGFub3NhcG9sbG8ub3JnAAoJEDOfc2w6cgkoz5YP/A77DrVlbz8YmrFg/3wIIQcn bvz3aPZuFnjeGq2INY7Fhymo9bQDtrUgms3IweCZWyozfjoBuoxocA/n3iY25JD2 3+XFj4P3l3/NC0IiWwG74RfjpVfq16zSHJdIrbALMWoQ70pHWWLf7nGB3ocM/TLz MdnD4a2hJbsNDYGqBgpOoBA+pOVQdZ3ot+ZZd+LWxQAiIYeXP6kqanD+KvQ/BpJB usXdqaGuoWVnGJ2N8kl99TmHpAilS9QuRqmgijboPIuge4FTVgF/PPlSOMy+0orb zB4wtADbl/NF/AZ8TSbIPkzK4v2M6/+6gkxHZ2LNGz6Afs0qVt4YjbokQVwh3pUX y/p1v0Ey5JAlCCvx1zByV29g6u2oY/1Bne4jkHmZMkd0YVznpflpiKV32k/NCyB+ +99IRiqhvutGL3Ml33yRSNPvkm/hscI5nHEasuF/Jey5U5IGf0tWS5gnGH1b8XSx HMns/fdoGJUapLa+1b/y56tGDE+63bVTLe8Znd2rFy93r9HW3ID2PAd5xaTkDxd2 oZ3EhB42TSaFwYMl+AsR0PZaBKdd8xAmuuAaRqTDor38ENthL4gk0awq0fuwCG3G /fnHKpuy7DnQL5Bm4aQXfDQHar1WqLQYGZXKKp/AEOvQ4lDc6fDoyD1D3b3xxVO5 f/5Qc9GRs7ATrYHrlhbk =6H1p -----END PGP SIGNATURE----- --=-=-=--