Re: public-inbox skipping new inboxes or many mails

["Robin H. Johnson" <robbat2@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 2421 bytes --]

On Mon, Jul 15, 2024 at 11:58:08PM +0000, Eric Wong wrote:
> > On Mon, Jul 15, 2024 at 09:03:40PM +0000, Eric Wong wrote:
> > > Currently, public-inbox-watch is the only public-inbox-* tool which
> > > works directly with Maildirs.
> > HMm, I had excluded public-inbox-watch initially because it didn't seem
> > to be doing anything after the very long startup. I'm thinking that the
> > inotify is not working as expected, maybe relating to the huge number of
> > folders we watch.
> 
> -watch is (or should be) doing a full scan every startup, but it
> switches between inboxes every few messages and tries to
> prioritize new messages from inotify.   Curious to see the
> strace immediately after startup to see if it's indeed doing the
> full scan.  I should probably add a stderr diagnostic for full
> scan completion...
It's definitely very busy after scan, but I can't tell if it's the full
set.

At an admin level, is there a way to dump out all of the paths it's
indexes, to compare against the paths on disk?

> How many Maildirs are you watching?  I wonder if it's hitting
> RLIMIT_NOFILE... (errors should be logged to stderr).
6774 Maildirs right now.
I should probably improve the OpenRC script for it, I think we're
throwing away stderr right now for -watch.

RLIMIT_NOFILE 1M
sysctl fs.file-max 16M

> > Terminal 1:
> > # strace -p $(pidof /usr/bin/public-inbox-watch)  -ff 
> > strace: Process 93260 attached
> > pselect6(8, [3 4], NULL, NULL, NULL, NULL
> > (nothing more)
> Curious, which architecture is that and is it using
> Linux::Inotify2 or inotify via the `syscall' perlop?
> (I expect 3 is the inotify FD).
x86-64, dev-perl/Linux-Inotify2 is installed on the host, but I can't
tell at a glance if -watch used perlop or package.

fd 3 is inotify & fd 4 is signalfd.

> 
> You may also be running into fs.inotify.max_user_watches or
> max_queued_events sysctl limits; each Maildir costs 2 inotify
> watches; and it looks like I forgot to handle IN_Q_OVERFLOW
> properly :x
Already set quite high:
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 1024
fs.inotify.max_user_watches = 65536

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]