Re: public-inbox skipping new inboxes or many mails

[Eric Wong <e@80x24.org>]

"Robin H. Johnson" <robbat2@gentoo.org> wrote:
> On Mon, Jul 15, 2024 at 11:58:08PM +0000, Eric Wong wrote:
> > > On Mon, Jul 15, 2024 at 09:03:40PM +0000, Eric Wong wrote:
> > > > Currently, public-inbox-watch is the only public-inbox-* tool which
> > > > works directly with Maildirs.
> > > HMm, I had excluded public-inbox-watch initially because it didn't seem
> > > to be doing anything after the very long startup. I'm thinking that the
> > > inotify is not working as expected, maybe relating to the huge number of
> > > folders we watch.
> > 
> > -watch is (or should be) doing a full scan every startup, but it
> > switches between inboxes every few messages and tries to
> > prioritize new messages from inotify.   Curious to see the
> > strace immediately after startup to see if it's indeed doing the
> > full scan.  I should probably add a stderr diagnostic for full
> > scan completion...
> It's definitely very busy after scan, but I can't tell if it's the full
> set.

OK, I think adding stderr diagnostic messages for full scans
shouldn't be too noisy.

> At an admin level, is there a way to dump out all of the paths it's
> indexes, to compare against the paths on disk?

No, path information isn't stored for public-facing inboxes
since it's too unstable.  It should be possible to reverse map
things at real-time and add better diagnostic tools, but the
philosophy has always been to store||index as little as possible
and be able to infer/regenerate needed data on-the-fly to avoid
data consistency problems.

(lei stores path info, but it's been a problematic
implementation, too :<)

> > How many Maildirs are you watching?  I wonder if it's hitting
> > RLIMIT_NOFILE... (errors should be logged to stderr).
> 6774 Maildirs right now.
> I should probably improve the OpenRC script for it, I think we're
> throwing away stderr right now for -watch.

Yeah, watch stderr is important for diagnosing problems.

Fwiw, I run it inside a screen(1) session on one system,
and rely on systemd to redirect stderr to syslog on another

<snip>
OK, various limits seem fine.

> > Curious, which architecture is that and is it using
> > Linux::Inotify2 or inotify via the `syscall' perlop?
> > (I expect 3 is the inotify FD).
> x86-64, dev-perl/Linux-Inotify2 is installed on the host, but I can't
> tell at a glance if -watch used perlop or package.

Probably, yes; but it can/should favor the pure Perl version
soon.  Since it's Gentoo I trust it's up-to-date with broadcast
and overflow support?