Re: Setup woes

[Eric Wong <e@80x24.org>]

Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote:
> Hello,
> 
> I'm currently (again) trying to setup a public-inbox instance and have
> two problems:
> 
>  1) Import of older mails
>     I used Konstantin Ryabitsev script[1] to generate a directory of
>     mbob files. It's unclear to me how to easily import that into the
>     archive. Not knowing a better way, I'd put them all in a Maildir and
>     use public-inbox-watch to import them. Is there a nicer way?

scripts/import_vger_from_mbox is probably the best maintained
of the stuff in scripts for mass imports (it supports v2, unlike
the rest of scripts/).

I will work on making -watch faster (and modeled after lei
internals) once lei is farther along in public-inbox.git...

import_maildir is v1, only, atm; but some of the stuff developed
for lei can make it easier to maintain going foward.

>  2) Apache configuration
>     I was able to setup Apache to serve individual archives but I fail
>     to get a list listing at the root. That is I can access the barebox
>     archive at http://lore.pengutronix.de/barebox/, but not the overview
>     of the archived lists at http://lore.pengutronix.de/.
> 
> 	$ curl -D - http://lore.pengutronix.de/
> 	HTTP/1.1 404 Not Found
> 	Date: Tue, 23 Feb 2021 15:31:54 GMT
> 	Server: Apache/2.4.38 (Debian)
> 	Expires: Fri, 01 Jan 1980 00:00:00 GMT
> 	Pragma: no-cache
> 	Cache-Control: no-cache, max-age=0, must-revalidate
> 	Content-Length: 9
> 	Content-Type: text/plain
> 
> 	Not Found

You might need to configure publicinbox.wwwlisting to
"all" or match=domain".  See publicinbox.wwwlisting in
public-inbox-config(5).

The default behavior is to 404 since the listing is a
fairly recent addition.

>     The Apache config looks as follows:
> 
> 	<VirtualHost *:80>
> 		ServerName lore.pengutronix.de
> 
> 		ServerAdmin webmaster@localhost
> 		DocumentRoot /srv/www/lore.pengutronix.de/root
> 
> 		<Directory "/srv/www/lore.pengutronix.de/root">
> 			AllowOverride None
> 			Require all granted
> 		</Directory>
> 
> 		LogLevel debug
> 
> 		ErrorLog ${APACHE_LOG_DIR}/error.log
> 		CustomLog ${APACHE_LOG_DIR}/access.log combined
> 
> 		Include conf-available/serve-cgi-bin.conf
> 		Include mods-available/rewrite.load
> 		SetEnv PI_DIR /home/lore/.public-inbox
> 		SetEnv NO_SCRIPT_NAME 1
> 
> 		Options +ExecCGI
> 		ScriptAlias /public-inbox.cgi "/usr/lib/cgi-bin/public-inbox.cgi"
> 
> 		DirectoryIndex public-inbox.cgi
> 		RewriteEngine On
> 		RewriteCond %{REQUEST_FILENAME} !-f
> 		RewriteCond %{REQUEST_FILENAME} !-d

Perhaps it's the -d check?  I haven't tested WwwListing with
Apache.

> 		RewriteRule ^.* /public-inbox.cgi/$0 [L,PT]
> 
> 	</VirtualHost>
> 
>      My guess is that the "SetEnv NO_SCRIPT_NAME 1" part doesn't work as
>      expected, but after some searching in the net I didn't find how to
>      do this properly.

Not sure, I haven't used Apache in ages and some of the configs
in examples/ are likely out-of-date and never tested with
WwwListing.

If you're not using public-inbox-httpd; then I strongly suggest
using at least mod_perl (and not plain CGI) since Perl startup
time is atrocious.

If you're dealing with large inboxes and supporting smart HTTP
clone/fetch, then use public-inbox-httpd (which works with
varnish, mod_proxy, or any other front-end proxy).

public-inbox-httpd can throttle CPU/memory intensive tasks
(e.g. git-http-backend, cgit) independently of other work
(the "limiter" stuff in public-inbox-config(5)).

As of public-inbox 1.6, -httpd also does git blob retrievals
around 20% faster on /T/, /t/ and t.mbox.gz requests and
decouples high-latency HDD from network ops.

> Any hints are welcome
> 
> Best regards and thanks in advance
> Uwe
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/mricon/korg-helpers.git/plain/list-archive-maker.py