* Issues with `lei` as non-root @ 2023-03-28 1:00 Louis DeLosSantos 2023-03-28 1:32 ` Eric Wong 0 siblings, 1 reply; 9+ messages in thread From: Louis DeLosSantos @ 2023-03-28 1:00 UTC (permalink / raw) To: meta Hello, I'm experimenting with `lei` as a nice search tool for `lore.kernel.org` Everything works fine with the caveat that it seems to break if I'm not root. When using `lei` as non-root we get this error: ``` E: Linux::Inotify2->new: Too many open files at /usr/share/perl5/vendor_perl/PublicInbox/DirIdle.pm line 40. connect(/run/user/1000/lei/5.seq.sock): Connection refused (after attempted daemon start) ``` Any ideas why this may occur? Is `lei` designed to only be ran as root or is Fedora installing perl in an odd fashion which results in root needing to be used? ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root 2023-03-28 1:00 Issues with `lei` as non-root Louis DeLosSantos @ 2023-03-28 1:32 ` Eric Wong 2023-03-28 1:36 ` public-inbox.org downtime expected in the next 6-8 hours Eric Wong 2023-03-28 2:30 ` Issues with `lei` as non-root Louis DeLosSantos 0 siblings, 2 replies; 9+ messages in thread From: Eric Wong @ 2023-03-28 1:32 UTC (permalink / raw) To: Louis DeLosSantos; +Cc: meta Louis DeLosSantos <louis.delos@gmail.com> wrote: > Hello, > > I'm experimenting with `lei` as a nice search tool for `lore.kernel.org` > > Everything works fine with the caveat that it seems to break if I'm not root. > > When using `lei` as non-root we get this error: I've never used lei as root nor has any part of public-inbox ever been intended to run as root. > ``` > E: Linux::Inotify2->new: Too many open files at > /usr/share/perl5/vendor_perl/PublicInbox/DirIdle.pm line 40. > connect(/run/user/1000/lei/5.seq.sock): Connection refused (after > attempted daemon start) > ``` > > Any ideas why this may occur? Is `lei` designed to only be ran as root > or is Fedora installing perl in an odd fashion which results in root > needing to be used? What's the output of `ulimit -n` and `lsof -p $(lei daemon-pid)`? (you may need to use `ps -ef |grep lei-daemon` to get the PID if lei is broken and using too many FDs, though) `ulimit -n' is the open file limit, typically 1024 or higher. If `lsof -p $PID` may reveal a bug in lei which leaves too many files open. lei (especially with inotify on Linux) should use far less than 1024. (FreeBSD may end up using far more open files, but that's a different story) ^ permalink raw reply [flat|nested] 9+ messages in thread
* public-inbox.org downtime expected in the next 6-8 hours 2023-03-28 1:32 ` Eric Wong @ 2023-03-28 1:36 ` Eric Wong 2023-03-28 2:30 ` Issues with `lei` as non-root Louis DeLosSantos 1 sibling, 0 replies; 9+ messages in thread From: Eric Wong @ 2023-03-28 1:36 UTC (permalink / raw) To: meta; +Cc: Louis DeLosSantos Btw, I need to finish off a HW downgrade due to increasing hosting costs. So retry sending any messages if you get bounces or if your message doesn't show up on https://public-inbox.org/meta/ after a few minutes. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root 2023-03-28 1:32 ` Eric Wong 2023-03-28 1:36 ` public-inbox.org downtime expected in the next 6-8 hours Eric Wong @ 2023-03-28 2:30 ` Louis DeLosSantos 2023-03-28 2:52 ` Eric Wong 1 sibling, 1 reply; 9+ messages in thread From: Louis DeLosSantos @ 2023-03-28 2:30 UTC (permalink / raw) To: Eric Wong; +Cc: meta > What's the output of `ulimit -n` and `lsof -p $(lei daemon-pid)`? 🖳 ulimit -n 1024 🖳 ps -ef |grep lei-daemon root 861005 1 0 19:24 ? 00:00:00 lei-daemon /tmp/lei-0/5.seq.sock louis 1025477 1015489 0 22:22 pts/6 00:00:00 grep --color=auto lei-daemon ~ 🖳 sudo lsof -p 861005 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME lei-daemo 861005 root cwd DIR 0,34 246 2902791 /home/louis/Mail/linux-bpf lei-daemo 861005 root rtd DIR 0,34 158 256 / lei-daemo 861005 root txt REG 0,34 15984 746278 /usr/bin/perl lei-daemo 861005 root mem REG 0,32 746278 /usr/bin/perl (path dev=0,34) lei-daemo 861005 root mem REG 0,32 669987 /usr/lib64/libstdc++.so.6.0.30 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 654072 /usr/lib64/libgcc_s-12-20221121.so.1 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 707780 /usr/lib64/libxapian.so.30.12.1 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 744976 /usr/lib64/perl5/vendor_perl/auto/Data/Dumper/Dumper.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2847838 /usr/lib64/perl5/vendor_perl/auto/Search/Xapian/Xapian.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 670001 /usr/lib64/libz.so.1.2.12 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 670059 /usr/lib64/libsqlite3.so.0.8.6 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 1142922 /usr/lib/locale/locale-archive (path dev=0,34) lei-daemo 861005 root mem REG 0,32 746285 /usr/lib64/perl5/auto/Sys/Hostname/Hostname.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 670032 /usr/lib64/libuuid.so.1.3.0 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2846948 /usr/lib64/perl5/vendor_perl/auto/Compress/Raw/Zlib/Zlib.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2848056 /usr/lib64/perl5/vendor_perl/auto/DBD/SQLite/SQLite.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 748851 /usr/lib64/perl5/vendor_perl/auto/DBI/DBI.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745669 /usr/lib64/perl5/auto/attributes/attributes.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2847712 /usr/lib64/perl5/vendor_perl/auto/Cpanel/JSON/XS/XS.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2848586 /usr/lib64/perl5/vendor_perl/auto/Linux/Inotify2/Inotify2.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2849070 /usr/lib64/perl5/vendor_perl/auto/Socket/MsgHdr/MsgHdr.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2847193 /usr/lib64/perl5/vendor_perl/auto/Sys/Syslog/Syslog.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745481 /usr/lib64/perl5/vendor_perl/auto/MIME/Base64/Base64.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745670 /usr/lib64/perl5/auto/re/re.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745506 /usr/lib64/perl5/vendor_perl/auto/Storable/Storable.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745611 /usr/lib64/perl5/vendor_perl/auto/Encode/Encode.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2849930 /usr/lib64/perl5/vendor_perl/auto/Email/Address/XS/XS.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 1143694 /usr/lib64/libm.so.6 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 1143691 /usr/lib64/libc.so.6 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745650 /usr/lib64/libperl.so.5.36.0 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2846914 /usr/lib64/perl5/vendor_perl/auto/Digest/SHA/SHA.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745663 /usr/lib64/perl5/auto/File/Glob/Glob.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745449 /usr/lib64/perl5/auto/IO/IO.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745355 /usr/lib64/perl5/vendor_perl/auto/Socket/Socket.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745492 /usr/lib64/perl5/vendor_perl/auto/List/Util/Util.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745259 /usr/lib64/perl5/auto/POSIX/POSIX.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 1019649 /usr/lib64/libcrypt.so.2.0.0 (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745556 /usr/lib64/perl5/vendor_perl/auto/Cwd/Cwd.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 2846928 /usr/lib64/perl5/vendor_perl/auto/Time/HiRes/HiRes.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 745427 /usr/lib64/perl5/auto/Fcntl/Fcntl.so (path dev=0,34) lei-daemo 861005 root mem REG 0,32 1143688 /usr/lib64/ld-linux-x86-64.so.2 (path dev=0,34) lei-daemo 861005 root 0u REG 0,38 0 3448 /tmp/lei-0/errors.log lei-daemo 861005 root 1u REG 0,38 0 3448 /tmp/lei-0/errors.log lei-daemo 861005 root 2u REG 0,38 0 3448 /tmp/lei-0/errors.log lei-daemo 861005 root 3u a_inode 0,14 0 15364 [eventpoll:4,5,7,8,14] lei-daemo 861005 root 4u unix 0x0000000033b859c6 0t0 4095008 /tmp/lei-0/5.seq.sock type=SEQPACKET (LISTEN) lei-daemo 861005 root 5u unix 0x000000007a56c532 0t0 4085549 type=SEQPACKET (CONNECTED) lei-daemo 861005 root 6u unix 0x00000000c6d0e27b 0t0 4085550 type=SEQPACKET (CONNECTED) lei-daemo 861005 root 7r a_inode 0,14 0 15364 inotify lei-daemo 861005 root 8u a_inode 0,14 0 15364 [signalfd] lei-daemo 861005 root 14r FIFO 0,13 0t0 4098533 pipe lei-daemo 861005 root 16u unix 0x00000000557447d0 0t0 4098534 type=SEQPACKET (CONNECTED) lei-daemo 861005 root 17u unix 0x00000000f60edd15 0t0 4098535 type=SEQPACKET (CONNECTED) Above is 54 open sockets. Which seems fine. Should daemon be running as root, if I intend to only use lei as user? On Mon, Mar 27, 2023 at 9:32 PM Eric Wong <e@80x24.org> wrote: > > Louis DeLosSantos <louis.delos@gmail.com> wrote: > > Hello, > > > > I'm experimenting with `lei` as a nice search tool for `lore.kernel.org` > > > > Everything works fine with the caveat that it seems to break if I'm not root. > > > > When using `lei` as non-root we get this error: > > I've never used lei as root nor has any part of public-inbox > ever been intended to run as root. > > > ``` > > E: Linux::Inotify2->new: Too many open files at > > /usr/share/perl5/vendor_perl/PublicInbox/DirIdle.pm line 40. > > connect(/run/user/1000/lei/5.seq.sock): Connection refused (after > > attempted daemon start) > > ``` > > > > Any ideas why this may occur? Is `lei` designed to only be ran as root > > or is Fedora installing perl in an odd fashion which results in root > > needing to be used? > > What's the output of `ulimit -n` and `lsof -p $(lei daemon-pid)`? > > (you may need to use `ps -ef |grep lei-daemon` to get the PID > if lei is broken and using too many FDs, though) > > `ulimit -n' is the open file limit, typically 1024 or higher. > > If `lsof -p $PID` may reveal a bug in lei which leaves too many > files open. lei (especially with inotify on Linux) should use > far less than 1024. > > (FreeBSD may end up using far more open files, but that's a > different story) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root 2023-03-28 2:30 ` Issues with `lei` as non-root Louis DeLosSantos @ 2023-03-28 2:52 ` Eric Wong 2023-03-28 3:05 ` Louis DeLosSantos 0 siblings, 1 reply; 9+ messages in thread From: Eric Wong @ 2023-03-28 2:52 UTC (permalink / raw) To: Louis DeLosSantos; +Cc: meta Louis DeLosSantos <louis.delos@gmail.com> wrote: <snip> > Above is 54 open sockets. Which seems fine. Actually, only 18 (0..17). The `mem' stuff is mmap-ed and doesn't count against `ulimit -n` (RLIMIT_NOFILE). > Should daemon be running as root, if I intend to only use lei as user? Definitely not; the lei-daemon is per-user. I also forgot, inotify has its own per-user limits; perhaps you're hitting those? # show system-wide limits $ head /proc/sys/fs/inotify/max_* # show per-user inotify FDs (-nP speeds up lsof by avoiding lookups) $ lsof -nP -u $USER |grep inotify ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root 2023-03-28 2:52 ` Eric Wong @ 2023-03-28 3:05 ` Louis DeLosSantos 2023-03-28 3:38 ` Eric Wong 0 siblings, 1 reply; 9+ messages in thread From: Louis DeLosSantos @ 2023-03-28 3:05 UTC (permalink / raw) To: Eric Wong; +Cc: meta > Actually, only 18 (0..17). The `mem' stuff is mmap-ed and doesn't count against `ulimit -n` (RLIMIT_NOFILE). Yup, you are right, did a quick "wc -l" without thinking about it. > Definitely not; the lei-daemon is per-user. Okay, maybe this is the issue to begin with? I installed lei from dnf. I'm not sure what launches the daemon, is it launched on first run? If that is the case, it was probably launched when I restored to `sudo lei q ....` command. But, if its running as systemd service, I could move it to user service. # show system-wide limits ~ 🖳 head /proc/sys/fs/inotify/max_* ==> /proc/sys/fs/inotify/max_queued_events <== 16384 ==> /proc/sys/fs/inotify/max_user_instances <== 128 ==> /proc/sys/fs/inotify/max_user_watches <== 524288 # show per-user inotify FDs (-nP speeds up lsof by avoiding lookups) systemd 1686 louis 6r a_inode 0,14 0 15364 inotify systemd 1686 louis 11r a_inode 0,14 0 15364 inotify systemd 1686 louis 13r a_inode 0,14 0 15364 inotify dbus-brok 1727 louis 7r a_inode 0,14 0 15364 inotify swaync 1868 louis 15r a_inode 0,14 0 15364 inotify dbus-brok 1898 louis 7r a_inode 0,14 0 15364 inotify xdg-deskt 1914 louis 10r a_inode 0,14 0 15364 inotify xdg-deskt 1938 louis 12r a_inode 0,14 0 15364 inotify xdg-deskt 1968 louis 14r a_inode 0,14 0 15364 inotify wireplumb 1979 louis 19r a_inode 0,14 0 15364 inotify wireplumb 1979 louis 28r a_inode 0,14 0 15364 inotify wireplumb 1979 louis 30r a_inode 0,14 0 15364 inotify code 2362 louis 63r a_inode 0,14 0 15364 inotify code 2362 louis 86r a_inode 0,14 0 15364 inotify code 2362 louis 109r a_inode 0,14 0 15364 inotify code 2456 louis 21r a_inode 0,14 0 15364 inotify code 2493 louis 51r a_inode 0,14 0 15364 inotify code 2547 louis 49r a_inode 0,14 0 15364 inotify code 2547 louis 54r a_inode 0,14 0 15364 inotify firefox 4168 louis 80r a_inode 0,14 0 15364 inotify cgroupify 4347 louis 6r a_inode 0,14 0 15364 inotify flatpak-s 53051 louis 7r a_inode 0,14 0 15364 inotify obsidian 53073 louis 69r a_inode 0,14 0 15364 inotify flatpak-p 53080 louis 7r a_inode 0,14 0 15364 inotify obsidian 53135 louis 21r a_inode 0,14 0 15364 inotify obsidian 53141 louis 56r a_inode 0,14 0 15364 inotify tail 235359 louis 4r a_inode 0,14 0 15364 inotify tail 251716 louis 4r a_inode 0,14 0 15364 inotify tail 251821 louis 4r a_inode 0,14 0 15364 inotify tail 252069 louis 4r a_inode 0,14 0 15364 inotify tail 252266 louis 4r a_inode 0,14 0 15364 inotify tail 252399 louis 4r a_inode 0,14 0 15364 inotify tail 252434 louis 4r a_inode 0,14 0 15364 inotify tail 252749 louis 4r a_inode 0,14 0 15364 inotify tail 252953 louis 4r a_inode 0,14 0 15364 inotify tail 253611 louis 4r a_inode 0,14 0 15364 inotify tail 253772 louis 4r a_inode 0,14 0 15364 inotify tail 253858 louis 4r a_inode 0,14 0 15364 inotify tail 253932 louis 4r a_inode 0,14 0 15364 inotify tail 254159 louis 4r a_inode 0,14 0 15364 inotify tail 254323 louis 4r a_inode 0,14 0 15364 inotify tail 254425 louis 4r a_inode 0,14 0 15364 inotify tail 255070 louis 4r a_inode 0,14 0 15364 inotify tail 255592 louis 4r a_inode 0,14 0 15364 inotify tail 256816 louis 4r a_inode 0,14 0 15364 inotify tail 256939 louis 4r a_inode 0,14 0 15364 inotify tail 257302 louis 4r a_inode 0,14 0 15364 inotify tail 257435 louis 4r a_inode 0,14 0 15364 inotify tail 257746 louis 4r a_inode 0,14 0 15364 inotify tail 258071 louis 4r a_inode 0,14 0 15364 inotify tail 258169 louis 4r a_inode 0,14 0 15364 inotify tail 258279 louis 4r a_inode 0,14 0 15364 inotify tail 258413 louis 4r a_inode 0,14 0 15364 inotify tail 258715 louis 4r a_inode 0,14 0 15364 inotify tail 259167 louis 4r a_inode 0,14 0 15364 inotify tail 259289 louis 4r a_inode 0,14 0 15364 inotify tail 259484 louis 4r a_inode 0,14 0 15364 inotify tail 259623 louis 4r a_inode 0,14 0 15364 inotify tail 259876 louis 4r a_inode 0,14 0 15364 inotify tail 260275 louis 4r a_inode 0,14 0 15364 inotify tail 260412 louis 4r a_inode 0,14 0 15364 inotify tail 260518 louis 4r a_inode 0,14 0 15364 inotify tail 260650 louis 4r a_inode 0,14 0 15364 inotify tail 260818 louis 4r a_inode 0,14 0 15364 inotify tail 261665 louis 4r a_inode 0,14 0 15364 inotify tail 262636 louis 4r a_inode 0,14 0 15364 inotify tail 262963 louis 4r a_inode 0,14 0 15364 inotify tail 263781 louis 4r a_inode 0,14 0 15364 inotify tail 264183 louis 4r a_inode 0,14 0 15364 inotify tail 264242 louis 4r a_inode 0,14 0 15364 inotify tail 264484 louis 4r a_inode 0,14 0 15364 inotify tail 264769 louis 4r a_inode 0,14 0 15364 inotify tail 264970 louis 4r a_inode 0,14 0 15364 inotify tail 265488 louis 4r a_inode 0,14 0 15364 inotify tail 265939 louis 4r a_inode 0,14 0 15364 inotify tail 266145 louis 4r a_inode 0,14 0 15364 inotify tail 267183 louis 4r a_inode 0,14 0 15364 inotify tail 267352 louis 4r a_inode 0,14 0 15364 inotify tail 267951 louis 4r a_inode 0,14 0 15364 inotify tail 268135 louis 4r a_inode 0,14 0 15364 inotify tail 268320 louis 4r a_inode 0,14 0 15364 inotify tail 268892 louis 4r a_inode 0,14 0 15364 inotify tail 269418 louis 4r a_inode 0,14 0 15364 inotify tail 269631 louis 4r a_inode 0,14 0 15364 inotify tail 339541 louis 4r a_inode 0,14 0 15364 inotify tail 339854 louis 4r a_inode 0,14 0 15364 inotify tail 340349 louis 4r a_inode 0,14 0 15364 inotify tail 340566 louis 4r a_inode 0,14 0 15364 inotify tail 340712 louis 4r a_inode 0,14 0 15364 inotify tail 340858 louis 4r a_inode 0,14 0 15364 inotify tail 341122 louis 4r a_inode 0,14 0 15364 inotify tail 341356 louis 4r a_inode 0,14 0 15364 inotify tail 341499 louis 4r a_inode 0,14 0 15364 inotify tail 341643 louis 4r a_inode 0,14 0 15364 inotify tail 341858 louis 4r a_inode 0,14 0 15364 inotify tail 341948 louis 4r a_inode 0,14 0 15364 inotify tail 342271 louis 4r a_inode 0,14 0 15364 inotify tail 342410 louis 4r a_inode 0,14 0 15364 inotify tail 342557 louis 4r a_inode 0,14 0 15364 inotify tail 342802 louis 4r a_inode 0,14 0 15364 inotify tail 342944 louis 4r a_inode 0,14 0 15364 inotify tail 343409 louis 4r a_inode 0,14 0 15364 inotify tail 343690 louis 4r a_inode 0,14 0 15364 inotify tail 343864 louis 4r a_inode 0,14 0 15364 inotify tail 344049 louis 4r a_inode 0,14 0 15364 inotify tail 344285 louis 4r a_inode 0,14 0 15364 inotify tail 344429 louis 4r a_inode 0,14 0 15364 inotify tail 345251 louis 4r a_inode 0,14 0 15364 inotify tail 345541 louis 4r a_inode 0,14 0 15364 inotify tail 345972 louis 4r a_inode 0,14 0 15364 inotify tail 346163 louis 4r a_inode 0,14 0 15364 inotify tail 346755 louis 4r a_inode 0,14 0 15364 inotify tail 347063 louis 4r a_inode 0,14 0 15364 inotify tail 350273 louis 4r a_inode 0,14 0 15364 inotify tail 350316 louis 4r a_inode 0,14 0 15364 inotify tail 350781 louis 4r a_inode 0,14 0 15364 inotify tail 350942 louis 4r a_inode 0,14 0 15364 inotify tail 351111 louis 4r a_inode 0,14 0 15364 inotify tail 351746 louis 4r a_inode 0,14 0 15364 inotify tail 353489 louis 4r a_inode 0,14 0 15364 inotify tail 353801 louis 4r a_inode 0,14 0 15364 inotify tail 354352 louis 4r a_inode 0,14 0 15364 inotify tail 354638 louis 4r a_inode 0,14 0 15364 inotify tail 354923 louis 4r a_inode 0,14 0 15364 inotify tail 362029 louis 4r a_inode 0,14 0 15364 inotify tail 367093 louis 4r a_inode 0,14 0 15364 inotify waybar 811099 louis 37r a_inode 0,14 0 15364 inotify waybar 811099 louis 38r a_inode 0,14 0 15364 inotify On Mon, Mar 27, 2023 at 10:52 PM Eric Wong <e@80x24.org> wrote: > > Louis DeLosSantos <louis.delos@gmail.com> wrote: > > <snip> > > > Above is 54 open sockets. Which seems fine. > > Actually, only 18 (0..17). The `mem' stuff is mmap-ed and > doesn't count against `ulimit -n` (RLIMIT_NOFILE). > > > Should daemon be running as root, if I intend to only use lei as user? > > Definitely not; the lei-daemon is per-user. > > I also forgot, inotify has its own per-user limits; perhaps > you're hitting those? > > # show system-wide limits > $ head /proc/sys/fs/inotify/max_* > > # show per-user inotify FDs (-nP speeds up lsof by avoiding lookups) > $ lsof -nP -u $USER |grep inotify ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root 2023-03-28 3:05 ` Louis DeLosSantos @ 2023-03-28 3:38 ` Eric Wong 2023-03-28 4:08 ` Louis DeLosSantos 0 siblings, 1 reply; 9+ messages in thread From: Eric Wong @ 2023-03-28 3:38 UTC (permalink / raw) To: Louis DeLosSantos; +Cc: meta Louis DeLosSantos <louis.delos@gmail.com> wrote: > > Definitely not; the lei-daemon is per-user. > > Okay, maybe this is the issue to begin with? I installed lei from dnf. > I'm not sure what launches the daemon, is it launched on first run? > > If that is the case, it was probably launched when I restored to `sudo > lei q ....` command. > But, if its running as systemd service, I could move it to user service. You shouldn't need to manage it as a service; it's auto-started and killing it is harmless in most cases. I'm considering it have it auto-exit if it stays idle for a long time and there's no active inotify watches. lei-daemon doesn't start until any other lei command is invoked; so it shouldn't be started on installation. > # show system-wide limits > ==> /proc/sys/fs/inotify/max_user_instances <== > 128 <snip> > tail 367093 louis 4r a_inode 0,14 0 Thats a lot of tail processes.... I wonder if they were spawned by `lei q -v' for emitting curl stderr? They should be auto-killed. (or if you have some other reason for running tail on your system). ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root 2023-03-28 3:38 ` Eric Wong @ 2023-03-28 4:08 ` Louis DeLosSantos 2023-03-28 11:12 ` [PATCH] inotify: wrap with informative error message Eric Wong 0 siblings, 1 reply; 9+ messages in thread From: Louis DeLosSantos @ 2023-03-28 4:08 UTC (permalink / raw) To: Eric Wong; +Cc: meta > Thats a lot of tail processes.... Ugh, sorry to waste your time. All the tails were from a run away UI program I'm working on. Once I killed them, `let` runs just fine as non-root. Thanks for the free tech support, hope I didn't steal your attention from something valuable :-D. On Mon, Mar 27, 2023 at 11:38 PM Eric Wong <e@80x24.org> wrote: > > Louis DeLosSantos <louis.delos@gmail.com> wrote: > > > Definitely not; the lei-daemon is per-user. > > > > Okay, maybe this is the issue to begin with? I installed lei from dnf. > > I'm not sure what launches the daemon, is it launched on first run? > > > > If that is the case, it was probably launched when I restored to `sudo > > lei q ....` command. > > But, if its running as systemd service, I could move it to user service. > > You shouldn't need to manage it as a service; it's auto-started > and killing it is harmless in most cases. I'm considering it > have it auto-exit if it stays idle for a long time and there's > no active inotify watches. > > lei-daemon doesn't start until any other lei command is invoked; > so it shouldn't be started on installation. > > > # show system-wide limits > > > ==> /proc/sys/fs/inotify/max_user_instances <== > > 128 > > <snip> > > > tail 367093 louis 4r a_inode 0,14 0 > > Thats a lot of tail processes.... > I wonder if they were spawned by `lei q -v' for emitting curl stderr? > They should be auto-killed. > (or if you have some other reason for running tail on your system). ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH] inotify: wrap with informative error message 2023-03-28 4:08 ` Louis DeLosSantos @ 2023-03-28 11:12 ` Eric Wong 0 siblings, 0 replies; 9+ messages in thread From: Eric Wong @ 2023-03-28 11:12 UTC (permalink / raw) To: Louis DeLosSantos; +Cc: meta Louis DeLosSantos <louis.delos@gmail.com> wrote: > > Thats a lot of tail processes.... > Ugh, sorry to waste your time. Not at all. Thank you for sharing your experience with us as it will benefit future users, especially since lei is still in its early days. I think the patch below can help future users. > All the tails were from a run away UI program I'm working on. > Once I killed them, `let` runs just fine as non-root. If you're adminning your own system (which it sounds like), increasing the fs.inotify.max_user_instances should let you run as many `tail' processes as you desire :> Perhaps distros and/or kernel developers may raise limits some day for single-user systems. My distro uses /etc/sysctl.d/*.conf and /etc/sysctl.conf to make those knobs persistent across reboots; but distros vary/change a lot. > Thanks for the free tech support, hope I didn't steal your attention > from something valuable :-D. No problem, and thank you for being one of the brave, early lei users :> User feedback is valuable :> ------8<------ Subject: [PATCH] inotify: wrap with informative error message As encountered by Louis DeLosSantos, Linux inotify is capped by a lesser-known limit than the standard RLIMIT_NOFILE (`ulimit -n`) value. Give the user a hint about the fs.inotify.max_user_instances sysctl knob on EMFILE, since EMFILE alone may mislead users into thinking they've hit the (typically higher) RLIMIT_NOFILE limit. I can test this on my system using: perl -I lib -MPublicInbox::Inotify -E \ 'my @x = map { PublicInbox::Inotify->new } (1..128)' But I hesitate to include it in the test suite since triggering the limit can cause unrelated processes to fail. Link: https://public-inbox.org/meta/CAE6jdTo8iQfNM9Yuk0Dwi-ARMxmQxX-onL8buXcQ9Ze3r0hKrg@mail.gmail.com/ Reported-by: Louis DeLosSantos <louis.delos@gmail.com> --- MANIFEST | 1 + lib/PublicInbox/DirIdle.pm | 4 ++-- lib/PublicInbox/InboxIdle.pm | 4 ++-- lib/PublicInbox/Inotify.pm | 30 ++++++++++++++++++++++++++++++ 4 files changed, 35 insertions(+), 4 deletions(-) create mode 100644 lib/PublicInbox/Inotify.pm diff --git a/MANIFEST b/MANIFEST index 40535233..3c421645 100644 --- a/MANIFEST +++ b/MANIFEST @@ -217,6 +217,7 @@ lib/PublicInbox/In2Tie.pm lib/PublicInbox/Inbox.pm lib/PublicInbox/InboxIdle.pm lib/PublicInbox/InboxWritable.pm +lib/PublicInbox/Inotify.pm lib/PublicInbox/InputPipe.pm lib/PublicInbox/Isearch.pm lib/PublicInbox/KQNotify.pm diff --git a/lib/PublicInbox/DirIdle.pm b/lib/PublicInbox/DirIdle.pm index 55c3982f..af99811c 100644 --- a/lib/PublicInbox/DirIdle.pm +++ b/lib/PublicInbox/DirIdle.pm @@ -9,14 +9,14 @@ use PublicInbox::Syscall qw(EPOLLIN); use PublicInbox::In2Tie; my ($MAIL_IN, $MAIL_GONE, $ino_cls); -if ($^O eq 'linux' && eval { require Linux::Inotify2; 1 }) { +if ($^O eq 'linux' && eval { require PublicInbox::Inotify; 1 }) { $MAIL_IN = Linux::Inotify2::IN_MOVED_TO() | Linux::Inotify2::IN_CREATE(); $MAIL_GONE = Linux::Inotify2::IN_DELETE() | Linux::Inotify2::IN_DELETE_SELF() | Linux::Inotify2::IN_MOVE_SELF() | Linux::Inotify2::IN_MOVED_FROM(); - $ino_cls = 'Linux::Inotify2'; + $ino_cls = 'PublicInbox::Inotify'; # Perl 5.22+ is needed for fileno(DIRHANDLE) support: } elsif ($^V ge v5.22 && eval { require PublicInbox::KQNotify }) { $MAIL_IN = PublicInbox::KQNotify::MOVED_TO_OR_CREATE(); diff --git a/lib/PublicInbox/InboxIdle.pm b/lib/PublicInbox/InboxIdle.pm index f0d8a972..4231c0a0 100644 --- a/lib/PublicInbox/InboxIdle.pm +++ b/lib/PublicInbox/InboxIdle.pm @@ -10,9 +10,9 @@ use parent qw(PublicInbox::DS); use PublicInbox::Syscall qw(EPOLLIN); my $IN_MODIFY = 0x02; # match Linux inotify my $ino_cls; -if ($^O eq 'linux' && eval { require Linux::Inotify2; 1 }) { +if ($^O eq 'linux' && eval { require PublicInbox::Inotify }) { $IN_MODIFY = Linux::Inotify2::IN_MODIFY(); - $ino_cls = 'Linux::Inotify2'; + $ino_cls = 'PublicInbox::Inotify'; } elsif (eval { require PublicInbox::KQNotify }) { $IN_MODIFY = PublicInbox::KQNotify::NOTE_WRITE(); $ino_cls = 'PublicInbox::KQNotify'; diff --git a/lib/PublicInbox/Inotify.pm b/lib/PublicInbox/Inotify.pm new file mode 100644 index 00000000..3ef271c8 --- /dev/null +++ b/lib/PublicInbox/Inotify.pm @@ -0,0 +1,30 @@ +# Copyright (C) all contributors <meta@public-inbox.org> +# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> + +# wrap Linux::Inotify2 XS module, support pure Perl via `syscall' someday +package PublicInbox::Inotify; +use v5.12; +our @ISA; +BEGIN { + eval { require Linux::Inotify2 }; + if ($@) { # TODO: get rid of XS dependency + die "W: Linux::Inotify2 missing: $@\n"; + } else { + push @ISA, 'Linux::Inotify2'; + } +}; + +sub new { + $_[0]->SUPER::new // do { + my $msg = $!{EMFILE} ? <<EOM : "$_[0]->new: $!\n"; +inotify_init/inotify_init1: $! +You may need to raise the `fs.inotify.max_user_instances' sysctl limit. +Consult your OS documentation and/or sysctl(8) + sysctl.conf(5) manpages. +EOM + $msg =~ s/^/E: /smg; + require Carp; + Carp::croak($msg); + } +} + +1; ^ permalink raw reply related [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-03-28 11:12 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-03-28 1:00 Issues with `lei` as non-root Louis DeLosSantos 2023-03-28 1:32 ` Eric Wong 2023-03-28 1:36 ` public-inbox.org downtime expected in the next 6-8 hours Eric Wong 2023-03-28 2:30 ` Issues with `lei` as non-root Louis DeLosSantos 2023-03-28 2:52 ` Eric Wong 2023-03-28 3:05 ` Louis DeLosSantos 2023-03-28 3:38 ` Eric Wong 2023-03-28 4:08 ` Louis DeLosSantos 2023-03-28 11:12 ` [PATCH] inotify: wrap with informative error message Eric Wong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).