From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00, URIBL_BLOCKED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id AC01E1F66E for ; Mon, 31 Aug 2020 04:41:40 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 00/11] watch: fix contention w/ Maildir & NNTP Date: Mon, 31 Aug 2020 04:41:29 +0000 Message-Id: <20200831044140.17027-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Here's a bunch of fixes to improve watch performance when both Maildirs and NNTP are being watched (possibly on the same inbox, or if `watchspam' is configured for spam removals). Wakeups are reduced, and inbox.lock contention is minimized by using read-only ->over to check for `watchspam' removals. These affect IMAP, too; but I've been mainly using NNTP. Eric Wong (11): watch: limit batch size of NNTP and IMAP workers, too watchmaildir: use v5.10.1, drop warnings rename WatchMaildir => Watch watch: log signal activities to STDERR watch: avoid unnecessary spawning on spam removals watch: block signals before fork on non-signalfd/kevent systems watch: comments and tiny cleanups ds: avoid excessive queueing when reaping PIDs watch: use EOFpipe to reduce dwaitpid wakeups ds: avoid unnecessary timer for waitpid replace ParentPipe with EOFpipe MANIFEST | 4 +- lib/PublicInbox/DS.pm | 38 +++--- lib/PublicInbox/Daemon.pm | 6 +- lib/PublicInbox/EOFpipe.pm | 24 ++++ lib/PublicInbox/Import.pm | 3 + lib/PublicInbox/ParentPipe.pm | 23 ---- lib/PublicInbox/V2Writable.pm | 3 + lib/PublicInbox/{WatchMaildir.pm => Watch.pm} | 111 +++++++++++++----- script/public-inbox-watch | 34 ++++-- t/imapd.t | 2 +- t/nntpd.t | 2 +- t/watch_filter_rubylang.t | 4 +- t/watch_imap.t | 4 +- t/watch_maildir.t | 18 +-- t/watch_maildir_v2.t | 22 ++-- t/watch_multiple_headers.t | 4 +- t/watch_nntp.t | 4 +- 17 files changed, 190 insertions(+), 116 deletions(-) create mode 100644 lib/PublicInbox/EOFpipe.pm delete mode 100644 lib/PublicInbox/ParentPipe.pm rename lib/PublicInbox/{WatchMaildir.pm => Watch.pm} (92%)