WIP: searching all of lore

[Eric Wong <e@80x24.org>]

Requires Tor, for now:

http://rskvuqcfnfizkjg6h5jvovwb3wkikzcwskf54lfpymus6mxrzw67b5ad.onion/all/
http://lore.czquwvybam4bgbro.onion/all/

It seems v3 onions with longer URLs are more secure, these days;
but requires newer Tor.

On Debian or RH-based systems, it's as easy as:
	<apt|yum> install tor torsocks

	# assuming w3m is installed:
	torsocks w3m http://rskvuqcfnfizkjg6h5jvovwb3wkikzcwskf54lfpymus6mxrzw67b5ad.onion/all/

Other browsers can configure SOCKS5 proxies to 127.0.0.1:9050
or be wrapped via torsocks (which uses LD_PRELOAD)

Disclaimers:

  I don't know much about Tor security (or security in general).
  I see Tor as an alternative to paying corrupt organizations
  (ICANN) and lets me self-host without a static IP address.

  I've also had numerous ISP and power outages this year,
  probably because more neighbors are home due to the pandemic,
  so don't expect 99.999% uptime, either.  And I'm a klutz who
  always trips over cables on (relatively) good days, and I've
  only had bad days since March :<

How to replicate
----------------

	# I'm using the following to update mirror and lore.kernel.org
	# (see grokmirror docs) for more.  Old command:
	grok-pull -v -c repos.conf
	public-inbox-index --all # update per-inbox indices

	# add "-L basic" or "-L medium" to reduce space requirements
	# to either -extindex or -index commands

	# The new command, not finalized yet:
	public-inbox-extindex --all -v /path/to/ALL


The following changes in a otherwise boring ~/.public-inbox/config
(or whatever $PI_CONFIG is set to)

; not yet stable or finalized, yet:
; this section allows, "all" is a special case, currently
[extindex "all"]
	topdir = /path/to/ALL

; these are already documented in public-inbox-config(5)
[publicinbox]
	; 'all' ignores domain name matching,
	; useful for inboxes served via multiple domains
	wwwlisting = all
	grokManifest = all

	; users with larger machines may want to bump this,
	; the default is for machines w/ 256MB RAM
	; (which I still use, sometimes)
	indexBatchSize = 100m

# I'm using the following to update from lore.kernel.org
# (see grokmirror docs) for more
grok-pull -v -c repos.conf
public-inbox-index --all # update per-inbox indices
public-inbox-extindex --all -v /path/to/ALL # index [extindex "all"]

This is running commit 95cb3e48fc5c4e847cdc111c2c8c9f0b70bdea56
git clone https://public-inbox.org/public-inbox.git

More changes coming (JMAP, speedups), and there's probably still
lots of stuff broken and need fixing (including my brain :<)