Re: more debugging for gentoo usage & supporting feature requests

[Eric Wong <e@80x24.org>]

"Robin H. Johnson" <robbat2@gentoo.org> wrote:
> Hi,
> 
> I moved the Gentoo instance to a much beefier machine & newer kernel,
> ingest is a lot faster; but there's still some hiccups.
> 
> 1. Request for more debugging details about mails: Seems that many of
> our oldest mails don't get ingested - and there's no output about why.
> I don't know if -watch actually scanned that folder or not.

I started working on it, but got sidetracked with some bugs in the
FakeInotify implementation on low-time-resolution FS :x

> 1.1. Possibly related:
> Intended config is that the mail should be ingested regardless of the
> email address on the headers. Way back in time, the Gentoo lists were
> renamed a few times, and the files are sorted into the correct folders.
> I think this impacted any attempted ingest via -mda because there's no
> other way to override what list a given mail on stdin should be
> associated with.
> 
> The headers may be inconsistent, changed style, name, or even be absent
> in a few cases.

Fwiw, there can be multiple publicinbox.*.address directives for
a given inbox.  You can also use publicinbox.*.watchheader to
match arbitrary headers (e.g. List-Id, X-BeenThere, etc...)
I think "public-inbox-ctl import" will be needed to handle
odd messages without any matching headers

> 2.
> What's the intended way for public-inbox-mda to function with no
> SpamAssassin installed at all? "spamcheck = " doesn't seem to do it.

spamcheck=none

You can also use --no-precheck to disabl some builtin rules.

> 3.
> As a formal feature request:
> Change the arguments of: public-inbox-watch 
> - Add --all to mean all lists in the config
> - no arguments => implicit --all
> - $LISTNAME/$INBOXPATH => one *OR* more inboxes manually specified.
> 
> I did a hacky split of the configuration for Gentoo, and things are a
> LOT more stable with 120 instances; but it's a little wasteful: I'd like
> to give the high-traffic lists their own instance, and group the
> low-traffic instances together.

Fwiw, the IMAP code for watch is already 1:1
process:IMAP-mailbox because of the Mail::IMAPClient API.

How about making that an option for Maildirs, too and at least get
some benefit from copy-on-write memory savings...

> 4.
> Make public-inbox-init NOT attempt to write to any configuration files.
> 
> Trying to implement segregation of roles:
> - config files owned by root only; readable by public-inbox users.

OK, it'd probably have to write a $INBOX_DIR/config.snippet.sample
file with comments, then..

> - source maildirs read-only to user running public-inbox-watch
> - public-inbox dirs writable to user running public-inbox-watch
> - public-inbox dirs readable to user running public-inbox-httpd

The last 3 has been what I've been doing since the beginning.