From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id A3AB31F452; Tue, 28 Mar 2023 11:12:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1680001956; bh=dp6sUbwwezz+I5nWwnAOzhb1WJDw69/7KwTRwxfz6BU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=albShUcNgmNLUBBLs2lF0m0olLR/W31/wr0E2mSuFG+Vs6MzBHyZBhRkCbDH5UHYZ LxHIqvO2GDW9+FD5dv7ubaCsjCuwqe7ZOqkdT4e3qZoXU137GivoPOuvbj6LZlY6FR GBiqCHtxPaYqiSjKoSM476G+iKni3HwbCWi5wGSI= Date: Tue, 28 Mar 2023 11:12:36 +0000 From: Eric Wong To: Louis DeLosSantos Cc: meta@public-inbox.org Subject: [PATCH] inotify: wrap with informative error message Message-ID: <20230328111236.M627198@dcvr> References: <20230328013226.M125137@dcvr> <20230328025205.M639038@dcvr> <20230328033816.M690495@dcvr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: List-Id: Louis DeLosSantos wrote: > > Thats a lot of tail processes.... > Ugh, sorry to waste your time. Not at all. Thank you for sharing your experience with us as it will benefit future users, especially since lei is still in its early days. I think the patch below can help future users. > All the tails were from a run away UI program I'm working on. > Once I killed them, `let` runs just fine as non-root. If you're adminning your own system (which it sounds like), increasing the fs.inotify.max_user_instances should let you run as many `tail' processes as you desire :> Perhaps distros and/or kernel developers may raise limits some day for single-user systems. My distro uses /etc/sysctl.d/*.conf and /etc/sysctl.conf to make those knobs persistent across reboots; but distros vary/change a lot. > Thanks for the free tech support, hope I didn't steal your attention > from something valuable :-D. No problem, and thank you for being one of the brave, early lei users :> User feedback is valuable :> ------8<------ Subject: [PATCH] inotify: wrap with informative error message As encountered by Louis DeLosSantos, Linux inotify is capped by a lesser-known limit than the standard RLIMIT_NOFILE (`ulimit -n`) value. Give the user a hint about the fs.inotify.max_user_instances sysctl knob on EMFILE, since EMFILE alone may mislead users into thinking they've hit the (typically higher) RLIMIT_NOFILE limit. I can test this on my system using: perl -I lib -MPublicInbox::Inotify -E \ 'my @x = map { PublicInbox::Inotify->new } (1..128)' But I hesitate to include it in the test suite since triggering the limit can cause unrelated processes to fail. Link: https://public-inbox.org/meta/CAE6jdTo8iQfNM9Yuk0Dwi-ARMxmQxX-onL8buXcQ9Ze3r0hKrg@mail.gmail.com/ Reported-by: Louis DeLosSantos --- MANIFEST | 1 + lib/PublicInbox/DirIdle.pm | 4 ++-- lib/PublicInbox/InboxIdle.pm | 4 ++-- lib/PublicInbox/Inotify.pm | 30 ++++++++++++++++++++++++++++++ 4 files changed, 35 insertions(+), 4 deletions(-) create mode 100644 lib/PublicInbox/Inotify.pm diff --git a/MANIFEST b/MANIFEST index 40535233..3c421645 100644 --- a/MANIFEST +++ b/MANIFEST @@ -217,6 +217,7 @@ lib/PublicInbox/In2Tie.pm lib/PublicInbox/Inbox.pm lib/PublicInbox/InboxIdle.pm lib/PublicInbox/InboxWritable.pm +lib/PublicInbox/Inotify.pm lib/PublicInbox/InputPipe.pm lib/PublicInbox/Isearch.pm lib/PublicInbox/KQNotify.pm diff --git a/lib/PublicInbox/DirIdle.pm b/lib/PublicInbox/DirIdle.pm index 55c3982f..af99811c 100644 --- a/lib/PublicInbox/DirIdle.pm +++ b/lib/PublicInbox/DirIdle.pm @@ -9,14 +9,14 @@ use PublicInbox::Syscall qw(EPOLLIN); use PublicInbox::In2Tie; my ($MAIL_IN, $MAIL_GONE, $ino_cls); -if ($^O eq 'linux' && eval { require Linux::Inotify2; 1 }) { +if ($^O eq 'linux' && eval { require PublicInbox::Inotify; 1 }) { $MAIL_IN = Linux::Inotify2::IN_MOVED_TO() | Linux::Inotify2::IN_CREATE(); $MAIL_GONE = Linux::Inotify2::IN_DELETE() | Linux::Inotify2::IN_DELETE_SELF() | Linux::Inotify2::IN_MOVE_SELF() | Linux::Inotify2::IN_MOVED_FROM(); - $ino_cls = 'Linux::Inotify2'; + $ino_cls = 'PublicInbox::Inotify'; # Perl 5.22+ is needed for fileno(DIRHANDLE) support: } elsif ($^V ge v5.22 && eval { require PublicInbox::KQNotify }) { $MAIL_IN = PublicInbox::KQNotify::MOVED_TO_OR_CREATE(); diff --git a/lib/PublicInbox/InboxIdle.pm b/lib/PublicInbox/InboxIdle.pm index f0d8a972..4231c0a0 100644 --- a/lib/PublicInbox/InboxIdle.pm +++ b/lib/PublicInbox/InboxIdle.pm @@ -10,9 +10,9 @@ use parent qw(PublicInbox::DS); use PublicInbox::Syscall qw(EPOLLIN); my $IN_MODIFY = 0x02; # match Linux inotify my $ino_cls; -if ($^O eq 'linux' && eval { require Linux::Inotify2; 1 }) { +if ($^O eq 'linux' && eval { require PublicInbox::Inotify }) { $IN_MODIFY = Linux::Inotify2::IN_MODIFY(); - $ino_cls = 'Linux::Inotify2'; + $ino_cls = 'PublicInbox::Inotify'; } elsif (eval { require PublicInbox::KQNotify }) { $IN_MODIFY = PublicInbox::KQNotify::NOTE_WRITE(); $ino_cls = 'PublicInbox::KQNotify'; diff --git a/lib/PublicInbox/Inotify.pm b/lib/PublicInbox/Inotify.pm new file mode 100644 index 00000000..3ef271c8 --- /dev/null +++ b/lib/PublicInbox/Inotify.pm @@ -0,0 +1,30 @@ +# Copyright (C) all contributors +# License: AGPL-3.0+ + +# wrap Linux::Inotify2 XS module, support pure Perl via `syscall' someday +package PublicInbox::Inotify; +use v5.12; +our @ISA; +BEGIN { + eval { require Linux::Inotify2 }; + if ($@) { # TODO: get rid of XS dependency + die "W: Linux::Inotify2 missing: $@\n"; + } else { + push @ISA, 'Linux::Inotify2'; + } +}; + +sub new { + $_[0]->SUPER::new // do { + my $msg = $!{EMFILE} ? <new: $!\n"; +inotify_init/inotify_init1: $! +You may need to raise the `fs.inotify.max_user_instances' sysctl limit. +Consult your OS documentation and/or sysctl(8) + sysctl.conf(5) manpages. +EOM + $msg =~ s/^/E: /smg; + require Carp; + Carp::croak($msg); + } +} + +1;