From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id EDE171F9FD; Tue, 23 Feb 2021 20:01:24 +0000 (UTC) Date: Tue, 23 Feb 2021 20:01:24 +0000 From: Eric Wong To: Uwe =?utf-8?Q?Kleine-K=C3=B6nig?= Cc: meta@public-inbox.org Subject: Re: Setup woes Message-ID: <20210223200124.GA26348@dcvr> References: <20210223154246.uso4tc2qf5bz6qym@pengutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20210223154246.uso4tc2qf5bz6qym@pengutronix.de> List-Id: Uwe Kleine-König wrote: > Hello, > > I'm currently (again) trying to setup a public-inbox instance and have > two problems: > > 1) Import of older mails > I used Konstantin Ryabitsev script[1] to generate a directory of > mbob files. It's unclear to me how to easily import that into the > archive. Not knowing a better way, I'd put them all in a Maildir and > use public-inbox-watch to import them. Is there a nicer way? scripts/import_vger_from_mbox is probably the best maintained of the stuff in scripts for mass imports (it supports v2, unlike the rest of scripts/). I will work on making -watch faster (and modeled after lei internals) once lei is farther along in public-inbox.git... import_maildir is v1, only, atm; but some of the stuff developed for lei can make it easier to maintain going foward. > 2) Apache configuration > I was able to setup Apache to serve individual archives but I fail > to get a list listing at the root. That is I can access the barebox > archive at http://lore.pengutronix.de/barebox/, but not the overview > of the archived lists at http://lore.pengutronix.de/. > > $ curl -D - http://lore.pengutronix.de/ > HTTP/1.1 404 Not Found > Date: Tue, 23 Feb 2021 15:31:54 GMT > Server: Apache/2.4.38 (Debian) > Expires: Fri, 01 Jan 1980 00:00:00 GMT > Pragma: no-cache > Cache-Control: no-cache, max-age=0, must-revalidate > Content-Length: 9 > Content-Type: text/plain > > Not Found You might need to configure publicinbox.wwwlisting to "all" or match=domain". See publicinbox.wwwlisting in public-inbox-config(5). The default behavior is to 404 since the listing is a fairly recent addition. > The Apache config looks as follows: > > > ServerName lore.pengutronix.de > > ServerAdmin webmaster@localhost > DocumentRoot /srv/www/lore.pengutronix.de/root > > > AllowOverride None > Require all granted > > > LogLevel debug > > ErrorLog ${APACHE_LOG_DIR}/error.log > CustomLog ${APACHE_LOG_DIR}/access.log combined > > Include conf-available/serve-cgi-bin.conf > Include mods-available/rewrite.load > SetEnv PI_DIR /home/lore/.public-inbox > SetEnv NO_SCRIPT_NAME 1 > > Options +ExecCGI > ScriptAlias /public-inbox.cgi "/usr/lib/cgi-bin/public-inbox.cgi" > > DirectoryIndex public-inbox.cgi > RewriteEngine On > RewriteCond %{REQUEST_FILENAME} !-f > RewriteCond %{REQUEST_FILENAME} !-d Perhaps it's the -d check? I haven't tested WwwListing with Apache. > RewriteRule ^.* /public-inbox.cgi/$0 [L,PT] > > > > My guess is that the "SetEnv NO_SCRIPT_NAME 1" part doesn't work as > expected, but after some searching in the net I didn't find how to > do this properly. Not sure, I haven't used Apache in ages and some of the configs in examples/ are likely out-of-date and never tested with WwwListing. If you're not using public-inbox-httpd; then I strongly suggest using at least mod_perl (and not plain CGI) since Perl startup time is atrocious. If you're dealing with large inboxes and supporting smart HTTP clone/fetch, then use public-inbox-httpd (which works with varnish, mod_proxy, or any other front-end proxy). public-inbox-httpd can throttle CPU/memory intensive tasks (e.g. git-http-backend, cgit) independently of other work (the "limiter" stuff in public-inbox-config(5)). As of public-inbox 1.6, -httpd also does git blob retrievals around 20% faster on /T/, /t/ and t.mbox.gz requests and decouples high-latency HDD from network ops. > Any hints are welcome > > Best regards and thanks in advance > Uwe > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/mricon/korg-helpers.git/plain/list-archive-maker.py