unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
* High polling frequency of public-inbox-watch
@ 2019-08-11 16:43 Jan Kiszka
  2019-08-11 16:51 ` Eric Wong
  0 siblings, 1 reply; 5+ messages in thread
From: Jan Kiszka @ 2019-08-11 16:43 UTC (permalink / raw)
  To: meta

Hi,

I'm running public-inbox for some months very successfully on my server. Great work!

On thing though: I realized that the public-inbox-watch service is scanning the
configured maildirs with a sleep delay of just 2 seconds, and that causes a CPU
load of 20-30% on my box. Can this be configured to a more moderate rate? I
would have patched this already, but I'm not seeing in the code where these 2
seconds may come from.

Thanks,
Jan

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: High polling frequency of public-inbox-watch
  2019-08-11 16:43 High polling frequency of public-inbox-watch Jan Kiszka
@ 2019-08-11 16:51 ` Eric Wong
  2019-08-11 17:01   ` Jan Kiszka
  0 siblings, 1 reply; 5+ messages in thread
From: Eric Wong @ 2019-08-11 16:51 UTC (permalink / raw)
  To: Jan Kiszka; +Cc: meta

Jan Kiszka <jan.kiszka@web.de> wrote:
> Hi,
> 
> I'm running public-inbox for some months very successfully on my server. Great work!

Good to know :>

> On thing though: I realized that the public-inbox-watch service is scanning the
> configured maildirs with a sleep delay of just 2 seconds, and that causes a CPU
> load of 20-30% on my box. Can this be configured to a more moderate rate? I
> would have patched this already, but I'm not seeing in the code where these 2
> seconds may come from.

I guess it's from Filesys::Notify::Simple.  Which OS are you using?

It should be able to use inotify / kqueue depending on whether
you're on Linux or FreeBSD (I haven't tested other BSDs).  It
uses Linux::Inotify2 on Linux which AFAIK all deb-based distros
will pull in, and Filesys::Notify::KQueue on KQueue-supported
systems.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: High polling frequency of public-inbox-watch
  2019-08-11 16:51 ` Eric Wong
@ 2019-08-11 17:01   ` Jan Kiszka
  2019-08-11 17:33     ` Eric Wong
  0 siblings, 1 reply; 5+ messages in thread
From: Jan Kiszka @ 2019-08-11 17:01 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

On 11.08.19 18:51, Eric Wong wrote:
> Jan Kiszka <jan.kiszka@web.de> wrote:
>> Hi,
>>
>> I'm running public-inbox for some months very successfully on my server. Great work!
>
> Good to know :>
>
>> On thing though: I realized that the public-inbox-watch service is scanning the
>> configured maildirs with a sleep delay of just 2 seconds, and that causes a CPU
>> load of 20-30% on my box. Can this be configured to a more moderate rate? I
>> would have patched this already, but I'm not seeing in the code where these 2
>> seconds may come from.
>
> I guess it's from Filesys::Notify::Simple.  Which OS are you using?
>

Debian Linux, Stretch.

> It should be able to use inotify / kqueue depending on whether
> you're on Linux or FreeBSD (I haven't tested other BSDs).  It
> uses Linux::Inotify2 on Linux which AFAIK all deb-based distros
> will pull in, and Filesys::Notify::KQueue on KQueue-supported
> systems.
>

I was expecting inotify usage as well, but I'm not seeing that in strace, rather
a long *stat series, and then nanosleep(2)... Hah!

     apt install liblinux-inotify2-perl

solved it. Did I miss a note in the readme?

Jan

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: High polling frequency of public-inbox-watch
  2019-08-11 17:01   ` Jan Kiszka
@ 2019-08-11 17:33     ` Eric Wong
  2019-10-15  3:59       ` [PATCH] INSTALL: recommend inotify|kqueue modules for -watch Eric Wong
  0 siblings, 1 reply; 5+ messages in thread
From: Eric Wong @ 2019-08-11 17:33 UTC (permalink / raw)
  To: Jan Kiszka; +Cc: meta

Jan Kiszka <jan.kiszka@web.de> wrote:
> On 11.08.19 18:51, Eric Wong wrote:
> > Linux::Inotify2 on Linux which AFAIK all deb-based distros
> > will pull in,

Oops, I was wrong on that :x

> I was expecting inotify usage as well, but I'm not seeing that in strace, rather
> a long *stat series, and then nanosleep(2)... Hah!
> 
>     apt install liblinux-inotify2-perl
> 
> solved it. Did I miss a note in the readme?

Nope, but it should probably be noted as an optional dependency
in INSTALL, along with KQueue dependencies for *BSDs.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH] INSTALL: recommend inotify|kqueue modules for -watch
  2019-08-11 17:33     ` Eric Wong
@ 2019-10-15  3:59       ` Eric Wong
  0 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2019-10-15  3:59 UTC (permalink / raw)
  To: meta; +Cc: Jan Kiszka

Jan Kiszka reported high polling frequency when using -watch.

It turns out OS-specific packages for Filesys::Notify::Simple do
not pull in interfaces to use kqueue or inotify, which are
required to perform power-efficient event-based wakeups on
Maildir writes.

Fix the name of the Filesys::Notify::Simple for FreeBSD while
we're at it.

Link: https://public-inbox.org/meta/c85803c6-6d77-a300-491a-9f310dd284c1@web.de/
---
 INSTALL | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/INSTALL b/INSTALL
index f7e5999e..aad52c7b 100644
--- a/INSTALL
+++ b/INSTALL
@@ -130,10 +130,17 @@ above, so there is no need to explicitly install them:
                                     typically installed alongside Perl5)
 
 - Filesys::Notify::Simple          deb: libfilesys-notify-simple-perl
-                                   pkg: pkg-Filesys-Notify-Simple
+                                   pkg: p5-Filesys-Notify-Simple
                                    rpm: perl-Filesys-Notify-Simple
                                    (for public-inbox-watch, pulled in by Plack)
 
+- Linux::Inotify                   deb: liblinux-inotify2-perl
+                                   rpm: perl-Linux-Inotify2
+                                   (for public-inbox-watch on Linux)
+
+- Filesys::Notify::KQueue          pkg: p5-Filesys-Notify-KQueue
+                                   (for public-inbox-watch on FreeBSD)
+
 - IO::Compress::Gzip               deb: perl-modules (or libio-compress-perl)
                                    pkg: perl5
                                    rpm: perl-IO-Compress

^ permalink raw reply related	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2019-10-15  3:59 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-08-11 16:43 High polling frequency of public-inbox-watch Jan Kiszka
2019-08-11 16:51 ` Eric Wong
2019-08-11 17:01   ` Jan Kiszka
2019-08-11 17:33     ` Eric Wong
2019-10-15  3:59       ` [PATCH] INSTALL: recommend inotify|kqueue modules for -watch Eric Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).