unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
From: Eric Wong <e@80x24.org>
To: "Uwe Kleine-König" <u.kleine-koenig@pengutronix.de>
Cc: meta@public-inbox.org
Subject: Re: Setup woes
Date: Tue, 23 Feb 2021 20:01:24 +0000	[thread overview]
Message-ID: <20210223200124.GA26348@dcvr> (raw)
In-Reply-To: <20210223154246.uso4tc2qf5bz6qym@pengutronix.de>

Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote:
> Hello,
> 
> I'm currently (again) trying to setup a public-inbox instance and have
> two problems:
> 
>  1) Import of older mails
>     I used Konstantin Ryabitsev script[1] to generate a directory of
>     mbob files. It's unclear to me how to easily import that into the
>     archive. Not knowing a better way, I'd put them all in a Maildir and
>     use public-inbox-watch to import them. Is there a nicer way?

scripts/import_vger_from_mbox is probably the best maintained
of the stuff in scripts for mass imports (it supports v2, unlike
the rest of scripts/).

I will work on making -watch faster (and modeled after lei
internals) once lei is farther along in public-inbox.git...

import_maildir is v1, only, atm; but some of the stuff developed
for lei can make it easier to maintain going foward.

>  2) Apache configuration
>     I was able to setup Apache to serve individual archives but I fail
>     to get a list listing at the root. That is I can access the barebox
>     archive at http://lore.pengutronix.de/barebox/, but not the overview
>     of the archived lists at http://lore.pengutronix.de/.
> 
> 	$ curl -D - http://lore.pengutronix.de/
> 	HTTP/1.1 404 Not Found
> 	Date: Tue, 23 Feb 2021 15:31:54 GMT
> 	Server: Apache/2.4.38 (Debian)
> 	Expires: Fri, 01 Jan 1980 00:00:00 GMT
> 	Pragma: no-cache
> 	Cache-Control: no-cache, max-age=0, must-revalidate
> 	Content-Length: 9
> 	Content-Type: text/plain
> 
> 	Not Found

You might need to configure publicinbox.wwwlisting to
"all" or match=domain".  See publicinbox.wwwlisting in
public-inbox-config(5).

The default behavior is to 404 since the listing is a
fairly recent addition.

>     The Apache config looks as follows:
> 
> 	<VirtualHost *:80>
> 		ServerName lore.pengutronix.de
> 
> 		ServerAdmin webmaster@localhost
> 		DocumentRoot /srv/www/lore.pengutronix.de/root
> 
> 		<Directory "/srv/www/lore.pengutronix.de/root">
> 			AllowOverride None
> 			Require all granted
> 		</Directory>
> 
> 		LogLevel debug
> 
> 		ErrorLog ${APACHE_LOG_DIR}/error.log
> 		CustomLog ${APACHE_LOG_DIR}/access.log combined
> 
> 		Include conf-available/serve-cgi-bin.conf
> 		Include mods-available/rewrite.load
> 		SetEnv PI_DIR /home/lore/.public-inbox
> 		SetEnv NO_SCRIPT_NAME 1
> 
> 		Options +ExecCGI
> 		ScriptAlias /public-inbox.cgi "/usr/lib/cgi-bin/public-inbox.cgi"
> 
> 		DirectoryIndex public-inbox.cgi
> 		RewriteEngine On
> 		RewriteCond %{REQUEST_FILENAME} !-f
> 		RewriteCond %{REQUEST_FILENAME} !-d

Perhaps it's the -d check?  I haven't tested WwwListing with
Apache.

> 		RewriteRule ^.* /public-inbox.cgi/$0 [L,PT]
> 
> 	</VirtualHost>
> 
>      My guess is that the "SetEnv NO_SCRIPT_NAME 1" part doesn't work as
>      expected, but after some searching in the net I didn't find how to
>      do this properly.

Not sure, I haven't used Apache in ages and some of the configs
in examples/ are likely out-of-date and never tested with
WwwListing.

If you're not using public-inbox-httpd; then I strongly suggest
using at least mod_perl (and not plain CGI) since Perl startup
time is atrocious.

If you're dealing with large inboxes and supporting smart HTTP
clone/fetch, then use public-inbox-httpd (which works with
varnish, mod_proxy, or any other front-end proxy).

public-inbox-httpd can throttle CPU/memory intensive tasks
(e.g. git-http-backend, cgit) independently of other work
(the "limiter" stuff in public-inbox-config(5)).

As of public-inbox 1.6, -httpd also does git blob retrievals
around 20% faster on /T/, /t/ and t.mbox.gz requests and
decouples high-latency HDD from network ops.

> Any hints are welcome
> 
> Best regards and thanks in advance
> Uwe
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/mricon/korg-helpers.git/plain/list-archive-maker.py

  reply	other threads:[~2021-02-23 20:01 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-23 15:42 Setup woes Uwe Kleine-König
2021-02-23 20:01 ` Eric Wong [this message]
2021-02-23 21:31   ` Uwe Kleine-König
2021-02-23 22:11     ` Eric Wong
2021-02-24 10:25       ` Uwe Kleine-König
2021-02-24 11:09         ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210223200124.GA26348@dcvr \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    --cc=u.kleine-koenig@pengutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).