* Issues with `lei` as non-root
@ 2023-03-28 1:00 Louis DeLosSantos
2023-03-28 1:32 ` Eric Wong
0 siblings, 1 reply; 9+ messages in thread
From: Louis DeLosSantos @ 2023-03-28 1:00 UTC (permalink / raw)
To: meta
Hello,
I'm experimenting with `lei` as a nice search tool for `lore.kernel.org`
Everything works fine with the caveat that it seems to break if I'm not root.
When using `lei` as non-root we get this error:
```
E: Linux::Inotify2->new: Too many open files at
/usr/share/perl5/vendor_perl/PublicInbox/DirIdle.pm line 40.
connect(/run/user/1000/lei/5.seq.sock): Connection refused (after
attempted daemon start)
```
Any ideas why this may occur? Is `lei` designed to only be ran as root
or is Fedora installing perl in an odd fashion which results in root
needing to be used?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root
2023-03-28 1:00 Issues with `lei` as non-root Louis DeLosSantos
@ 2023-03-28 1:32 ` Eric Wong
2023-03-28 1:36 ` public-inbox.org downtime expected in the next 6-8 hours Eric Wong
2023-03-28 2:30 ` Issues with `lei` as non-root Louis DeLosSantos
0 siblings, 2 replies; 9+ messages in thread
From: Eric Wong @ 2023-03-28 1:32 UTC (permalink / raw)
To: Louis DeLosSantos; +Cc: meta
Louis DeLosSantos <louis.delos@gmail.com> wrote:
> Hello,
>
> I'm experimenting with `lei` as a nice search tool for `lore.kernel.org`
>
> Everything works fine with the caveat that it seems to break if I'm not root.
>
> When using `lei` as non-root we get this error:
I've never used lei as root nor has any part of public-inbox
ever been intended to run as root.
> ```
> E: Linux::Inotify2->new: Too many open files at
> /usr/share/perl5/vendor_perl/PublicInbox/DirIdle.pm line 40.
> connect(/run/user/1000/lei/5.seq.sock): Connection refused (after
> attempted daemon start)
> ```
>
> Any ideas why this may occur? Is `lei` designed to only be ran as root
> or is Fedora installing perl in an odd fashion which results in root
> needing to be used?
What's the output of `ulimit -n` and `lsof -p $(lei daemon-pid)`?
(you may need to use `ps -ef |grep lei-daemon` to get the PID
if lei is broken and using too many FDs, though)
`ulimit -n' is the open file limit, typically 1024 or higher.
If `lsof -p $PID` may reveal a bug in lei which leaves too many
files open. lei (especially with inotify on Linux) should use
far less than 1024.
(FreeBSD may end up using far more open files, but that's a
different story)
^ permalink raw reply [flat|nested] 9+ messages in thread
* public-inbox.org downtime expected in the next 6-8 hours
2023-03-28 1:32 ` Eric Wong
@ 2023-03-28 1:36 ` Eric Wong
2023-03-28 2:30 ` Issues with `lei` as non-root Louis DeLosSantos
1 sibling, 0 replies; 9+ messages in thread
From: Eric Wong @ 2023-03-28 1:36 UTC (permalink / raw)
To: meta; +Cc: Louis DeLosSantos
Btw, I need to finish off a HW downgrade due to increasing
hosting costs. So retry sending any messages if you get bounces
or if your message doesn't show up on
https://public-inbox.org/meta/ after a few minutes.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root
2023-03-28 1:32 ` Eric Wong
2023-03-28 1:36 ` public-inbox.org downtime expected in the next 6-8 hours Eric Wong
@ 2023-03-28 2:30 ` Louis DeLosSantos
2023-03-28 2:52 ` Eric Wong
1 sibling, 1 reply; 9+ messages in thread
From: Louis DeLosSantos @ 2023-03-28 2:30 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
> What's the output of `ulimit -n` and `lsof -p $(lei daemon-pid)`?
🖳 ulimit -n
1024
🖳 ps -ef |grep lei-daemon
root 861005 1 0 19:24 ? 00:00:00 lei-daemon
/tmp/lei-0/5.seq.sock
louis 1025477 1015489 0 22:22 pts/6 00:00:00 grep --color=auto lei-daemon
~
🖳 sudo lsof -p 861005
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
lei-daemo 861005 root cwd DIR 0,34 246
2902791 /home/louis/Mail/linux-bpf
lei-daemo 861005 root rtd DIR 0,34 158 256 /
lei-daemo 861005 root txt REG 0,34 15984
746278 /usr/bin/perl
lei-daemo 861005 root mem REG 0,32
746278 /usr/bin/perl (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
669987 /usr/lib64/libstdc++.so.6.0.30 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
654072 /usr/lib64/libgcc_s-12-20221121.so.1 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
707780 /usr/lib64/libxapian.so.30.12.1 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
744976 /usr/lib64/perl5/vendor_perl/auto/Data/Dumper/Dumper.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
2847838 /usr/lib64/perl5/vendor_perl/auto/Search/Xapian/Xapian.so
(path dev=0,34)
lei-daemo 861005 root mem REG 0,32
670001 /usr/lib64/libz.so.1.2.12 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
670059 /usr/lib64/libsqlite3.so.0.8.6 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
1142922 /usr/lib/locale/locale-archive (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
746285 /usr/lib64/perl5/auto/Sys/Hostname/Hostname.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
670032 /usr/lib64/libuuid.so.1.3.0 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
2846948 /usr/lib64/perl5/vendor_perl/auto/Compress/Raw/Zlib/Zlib.so
(path dev=0,34)
lei-daemo 861005 root mem REG 0,32
2848056 /usr/lib64/perl5/vendor_perl/auto/DBD/SQLite/SQLite.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
748851 /usr/lib64/perl5/vendor_perl/auto/DBI/DBI.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
745669 /usr/lib64/perl5/auto/attributes/attributes.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
2847712 /usr/lib64/perl5/vendor_perl/auto/Cpanel/JSON/XS/XS.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
2848586 /usr/lib64/perl5/vendor_perl/auto/Linux/Inotify2/Inotify2.so
(path dev=0,34)
lei-daemo 861005 root mem REG 0,32
2849070 /usr/lib64/perl5/vendor_perl/auto/Socket/MsgHdr/MsgHdr.so
(path dev=0,34)
lei-daemo 861005 root mem REG 0,32
2847193 /usr/lib64/perl5/vendor_perl/auto/Sys/Syslog/Syslog.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
745481 /usr/lib64/perl5/vendor_perl/auto/MIME/Base64/Base64.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
745670 /usr/lib64/perl5/auto/re/re.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
745506 /usr/lib64/perl5/vendor_perl/auto/Storable/Storable.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
745611 /usr/lib64/perl5/vendor_perl/auto/Encode/Encode.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
2849930 /usr/lib64/perl5/vendor_perl/auto/Email/Address/XS/XS.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
1143694 /usr/lib64/libm.so.6 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
1143691 /usr/lib64/libc.so.6 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
745650 /usr/lib64/libperl.so.5.36.0 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
2846914 /usr/lib64/perl5/vendor_perl/auto/Digest/SHA/SHA.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
745663 /usr/lib64/perl5/auto/File/Glob/Glob.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
745449 /usr/lib64/perl5/auto/IO/IO.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
745355 /usr/lib64/perl5/vendor_perl/auto/Socket/Socket.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
745492 /usr/lib64/perl5/vendor_perl/auto/List/Util/Util.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
745259 /usr/lib64/perl5/auto/POSIX/POSIX.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
1019649 /usr/lib64/libcrypt.so.2.0.0 (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
745556 /usr/lib64/perl5/vendor_perl/auto/Cwd/Cwd.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
2846928 /usr/lib64/perl5/vendor_perl/auto/Time/HiRes/HiRes.so (path
dev=0,34)
lei-daemo 861005 root mem REG 0,32
745427 /usr/lib64/perl5/auto/Fcntl/Fcntl.so (path dev=0,34)
lei-daemo 861005 root mem REG 0,32
1143688 /usr/lib64/ld-linux-x86-64.so.2 (path dev=0,34)
lei-daemo 861005 root 0u REG 0,38 0
3448 /tmp/lei-0/errors.log
lei-daemo 861005 root 1u REG 0,38 0
3448 /tmp/lei-0/errors.log
lei-daemo 861005 root 2u REG 0,38 0
3448 /tmp/lei-0/errors.log
lei-daemo 861005 root 3u a_inode 0,14 0
15364 [eventpoll:4,5,7,8,14]
lei-daemo 861005 root 4u unix 0x0000000033b859c6 0t0
4095008 /tmp/lei-0/5.seq.sock type=SEQPACKET (LISTEN)
lei-daemo 861005 root 5u unix 0x000000007a56c532 0t0
4085549 type=SEQPACKET (CONNECTED)
lei-daemo 861005 root 6u unix 0x00000000c6d0e27b 0t0
4085550 type=SEQPACKET (CONNECTED)
lei-daemo 861005 root 7r a_inode 0,14 0 15364 inotify
lei-daemo 861005 root 8u a_inode 0,14 0
15364 [signalfd]
lei-daemo 861005 root 14r FIFO 0,13 0t0 4098533 pipe
lei-daemo 861005 root 16u unix 0x00000000557447d0 0t0
4098534 type=SEQPACKET (CONNECTED)
lei-daemo 861005 root 17u unix 0x00000000f60edd15 0t0
4098535 type=SEQPACKET (CONNECTED)
Above is 54 open sockets. Which seems fine.
Should daemon be running as root, if I intend to only use lei as user?
On Mon, Mar 27, 2023 at 9:32 PM Eric Wong <e@80x24.org> wrote:
>
> Louis DeLosSantos <louis.delos@gmail.com> wrote:
> > Hello,
> >
> > I'm experimenting with `lei` as a nice search tool for `lore.kernel.org`
> >
> > Everything works fine with the caveat that it seems to break if I'm not root.
> >
> > When using `lei` as non-root we get this error:
>
> I've never used lei as root nor has any part of public-inbox
> ever been intended to run as root.
>
> > ```
> > E: Linux::Inotify2->new: Too many open files at
> > /usr/share/perl5/vendor_perl/PublicInbox/DirIdle.pm line 40.
> > connect(/run/user/1000/lei/5.seq.sock): Connection refused (after
> > attempted daemon start)
> > ```
> >
> > Any ideas why this may occur? Is `lei` designed to only be ran as root
> > or is Fedora installing perl in an odd fashion which results in root
> > needing to be used?
>
> What's the output of `ulimit -n` and `lsof -p $(lei daemon-pid)`?
>
> (you may need to use `ps -ef |grep lei-daemon` to get the PID
> if lei is broken and using too many FDs, though)
>
> `ulimit -n' is the open file limit, typically 1024 or higher.
>
> If `lsof -p $PID` may reveal a bug in lei which leaves too many
> files open. lei (especially with inotify on Linux) should use
> far less than 1024.
>
> (FreeBSD may end up using far more open files, but that's a
> different story)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root
2023-03-28 2:30 ` Issues with `lei` as non-root Louis DeLosSantos
@ 2023-03-28 2:52 ` Eric Wong
2023-03-28 3:05 ` Louis DeLosSantos
0 siblings, 1 reply; 9+ messages in thread
From: Eric Wong @ 2023-03-28 2:52 UTC (permalink / raw)
To: Louis DeLosSantos; +Cc: meta
Louis DeLosSantos <louis.delos@gmail.com> wrote:
<snip>
> Above is 54 open sockets. Which seems fine.
Actually, only 18 (0..17). The `mem' stuff is mmap-ed and
doesn't count against `ulimit -n` (RLIMIT_NOFILE).
> Should daemon be running as root, if I intend to only use lei as user?
Definitely not; the lei-daemon is per-user.
I also forgot, inotify has its own per-user limits; perhaps
you're hitting those?
# show system-wide limits
$ head /proc/sys/fs/inotify/max_*
# show per-user inotify FDs (-nP speeds up lsof by avoiding lookups)
$ lsof -nP -u $USER |grep inotify
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root
2023-03-28 2:52 ` Eric Wong
@ 2023-03-28 3:05 ` Louis DeLosSantos
2023-03-28 3:38 ` Eric Wong
0 siblings, 1 reply; 9+ messages in thread
From: Louis DeLosSantos @ 2023-03-28 3:05 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
> Actually, only 18 (0..17). The `mem' stuff is mmap-ed and
doesn't count against `ulimit -n` (RLIMIT_NOFILE).
Yup, you are right, did a quick "wc -l" without thinking about it.
> Definitely not; the lei-daemon is per-user.
Okay, maybe this is the issue to begin with? I installed lei from dnf.
I'm not sure what launches the daemon, is it launched on first run?
If that is the case, it was probably launched when I restored to `sudo
lei q ....` command.
But, if its running as systemd service, I could move it to user service.
# show system-wide limits
~
🖳 head /proc/sys/fs/inotify/max_*
==> /proc/sys/fs/inotify/max_queued_events <==
16384
==> /proc/sys/fs/inotify/max_user_instances <==
128
==> /proc/sys/fs/inotify/max_user_watches <==
524288
# show per-user inotify FDs (-nP speeds up lsof by avoiding lookups)
systemd 1686 louis 6r a_inode 0,14 0
15364 inotify
systemd 1686 louis 11r a_inode 0,14 0
15364 inotify
systemd 1686 louis 13r a_inode 0,14 0
15364 inotify
dbus-brok 1727 louis 7r a_inode 0,14 0
15364 inotify
swaync 1868 louis 15r a_inode 0,14 0
15364 inotify
dbus-brok 1898 louis 7r a_inode 0,14 0
15364 inotify
xdg-deskt 1914 louis 10r a_inode 0,14 0
15364 inotify
xdg-deskt 1938 louis 12r a_inode 0,14 0
15364 inotify
xdg-deskt 1968 louis 14r a_inode 0,14 0
15364 inotify
wireplumb 1979 louis 19r a_inode 0,14 0
15364 inotify
wireplumb 1979 louis 28r a_inode 0,14 0
15364 inotify
wireplumb 1979 louis 30r a_inode 0,14 0
15364 inotify
code 2362 louis 63r a_inode 0,14 0
15364 inotify
code 2362 louis 86r a_inode 0,14 0
15364 inotify
code 2362 louis 109r a_inode 0,14 0
15364 inotify
code 2456 louis 21r a_inode 0,14 0
15364 inotify
code 2493 louis 51r a_inode 0,14 0
15364 inotify
code 2547 louis 49r a_inode 0,14 0
15364 inotify
code 2547 louis 54r a_inode 0,14 0
15364 inotify
firefox 4168 louis 80r a_inode 0,14 0
15364 inotify
cgroupify 4347 louis 6r a_inode 0,14 0
15364 inotify
flatpak-s 53051 louis 7r a_inode 0,14 0
15364 inotify
obsidian 53073 louis 69r a_inode 0,14 0
15364 inotify
flatpak-p 53080 louis 7r a_inode 0,14 0
15364 inotify
obsidian 53135 louis 21r a_inode 0,14 0
15364 inotify
obsidian 53141 louis 56r a_inode 0,14 0
15364 inotify
tail 235359 louis 4r a_inode 0,14 0
15364 inotify
tail 251716 louis 4r a_inode 0,14 0
15364 inotify
tail 251821 louis 4r a_inode 0,14 0
15364 inotify
tail 252069 louis 4r a_inode 0,14 0
15364 inotify
tail 252266 louis 4r a_inode 0,14 0
15364 inotify
tail 252399 louis 4r a_inode 0,14 0
15364 inotify
tail 252434 louis 4r a_inode 0,14 0
15364 inotify
tail 252749 louis 4r a_inode 0,14 0
15364 inotify
tail 252953 louis 4r a_inode 0,14 0
15364 inotify
tail 253611 louis 4r a_inode 0,14 0
15364 inotify
tail 253772 louis 4r a_inode 0,14 0
15364 inotify
tail 253858 louis 4r a_inode 0,14 0
15364 inotify
tail 253932 louis 4r a_inode 0,14 0
15364 inotify
tail 254159 louis 4r a_inode 0,14 0
15364 inotify
tail 254323 louis 4r a_inode 0,14 0
15364 inotify
tail 254425 louis 4r a_inode 0,14 0
15364 inotify
tail 255070 louis 4r a_inode 0,14 0
15364 inotify
tail 255592 louis 4r a_inode 0,14 0
15364 inotify
tail 256816 louis 4r a_inode 0,14 0
15364 inotify
tail 256939 louis 4r a_inode 0,14 0
15364 inotify
tail 257302 louis 4r a_inode 0,14 0
15364 inotify
tail 257435 louis 4r a_inode 0,14 0
15364 inotify
tail 257746 louis 4r a_inode 0,14 0
15364 inotify
tail 258071 louis 4r a_inode 0,14 0
15364 inotify
tail 258169 louis 4r a_inode 0,14 0
15364 inotify
tail 258279 louis 4r a_inode 0,14 0
15364 inotify
tail 258413 louis 4r a_inode 0,14 0
15364 inotify
tail 258715 louis 4r a_inode 0,14 0
15364 inotify
tail 259167 louis 4r a_inode 0,14 0
15364 inotify
tail 259289 louis 4r a_inode 0,14 0
15364 inotify
tail 259484 louis 4r a_inode 0,14 0
15364 inotify
tail 259623 louis 4r a_inode 0,14 0
15364 inotify
tail 259876 louis 4r a_inode 0,14 0
15364 inotify
tail 260275 louis 4r a_inode 0,14 0
15364 inotify
tail 260412 louis 4r a_inode 0,14 0
15364 inotify
tail 260518 louis 4r a_inode 0,14 0
15364 inotify
tail 260650 louis 4r a_inode 0,14 0
15364 inotify
tail 260818 louis 4r a_inode 0,14 0
15364 inotify
tail 261665 louis 4r a_inode 0,14 0
15364 inotify
tail 262636 louis 4r a_inode 0,14 0
15364 inotify
tail 262963 louis 4r a_inode 0,14 0
15364 inotify
tail 263781 louis 4r a_inode 0,14 0
15364 inotify
tail 264183 louis 4r a_inode 0,14 0
15364 inotify
tail 264242 louis 4r a_inode 0,14 0
15364 inotify
tail 264484 louis 4r a_inode 0,14 0
15364 inotify
tail 264769 louis 4r a_inode 0,14 0
15364 inotify
tail 264970 louis 4r a_inode 0,14 0
15364 inotify
tail 265488 louis 4r a_inode 0,14 0
15364 inotify
tail 265939 louis 4r a_inode 0,14 0
15364 inotify
tail 266145 louis 4r a_inode 0,14 0
15364 inotify
tail 267183 louis 4r a_inode 0,14 0
15364 inotify
tail 267352 louis 4r a_inode 0,14 0
15364 inotify
tail 267951 louis 4r a_inode 0,14 0
15364 inotify
tail 268135 louis 4r a_inode 0,14 0
15364 inotify
tail 268320 louis 4r a_inode 0,14 0
15364 inotify
tail 268892 louis 4r a_inode 0,14 0
15364 inotify
tail 269418 louis 4r a_inode 0,14 0
15364 inotify
tail 269631 louis 4r a_inode 0,14 0
15364 inotify
tail 339541 louis 4r a_inode 0,14 0
15364 inotify
tail 339854 louis 4r a_inode 0,14 0
15364 inotify
tail 340349 louis 4r a_inode 0,14 0
15364 inotify
tail 340566 louis 4r a_inode 0,14 0
15364 inotify
tail 340712 louis 4r a_inode 0,14 0
15364 inotify
tail 340858 louis 4r a_inode 0,14 0
15364 inotify
tail 341122 louis 4r a_inode 0,14 0
15364 inotify
tail 341356 louis 4r a_inode 0,14 0
15364 inotify
tail 341499 louis 4r a_inode 0,14 0
15364 inotify
tail 341643 louis 4r a_inode 0,14 0
15364 inotify
tail 341858 louis 4r a_inode 0,14 0
15364 inotify
tail 341948 louis 4r a_inode 0,14 0
15364 inotify
tail 342271 louis 4r a_inode 0,14 0
15364 inotify
tail 342410 louis 4r a_inode 0,14 0
15364 inotify
tail 342557 louis 4r a_inode 0,14 0
15364 inotify
tail 342802 louis 4r a_inode 0,14 0
15364 inotify
tail 342944 louis 4r a_inode 0,14 0
15364 inotify
tail 343409 louis 4r a_inode 0,14 0
15364 inotify
tail 343690 louis 4r a_inode 0,14 0
15364 inotify
tail 343864 louis 4r a_inode 0,14 0
15364 inotify
tail 344049 louis 4r a_inode 0,14 0
15364 inotify
tail 344285 louis 4r a_inode 0,14 0
15364 inotify
tail 344429 louis 4r a_inode 0,14 0
15364 inotify
tail 345251 louis 4r a_inode 0,14 0
15364 inotify
tail 345541 louis 4r a_inode 0,14 0
15364 inotify
tail 345972 louis 4r a_inode 0,14 0
15364 inotify
tail 346163 louis 4r a_inode 0,14 0
15364 inotify
tail 346755 louis 4r a_inode 0,14 0
15364 inotify
tail 347063 louis 4r a_inode 0,14 0
15364 inotify
tail 350273 louis 4r a_inode 0,14 0
15364 inotify
tail 350316 louis 4r a_inode 0,14 0
15364 inotify
tail 350781 louis 4r a_inode 0,14 0
15364 inotify
tail 350942 louis 4r a_inode 0,14 0
15364 inotify
tail 351111 louis 4r a_inode 0,14 0
15364 inotify
tail 351746 louis 4r a_inode 0,14 0
15364 inotify
tail 353489 louis 4r a_inode 0,14 0
15364 inotify
tail 353801 louis 4r a_inode 0,14 0
15364 inotify
tail 354352 louis 4r a_inode 0,14 0
15364 inotify
tail 354638 louis 4r a_inode 0,14 0
15364 inotify
tail 354923 louis 4r a_inode 0,14 0
15364 inotify
tail 362029 louis 4r a_inode 0,14 0
15364 inotify
tail 367093 louis 4r a_inode 0,14 0
15364 inotify
waybar 811099 louis 37r a_inode 0,14 0
15364 inotify
waybar 811099 louis 38r a_inode 0,14 0
15364 inotify
On Mon, Mar 27, 2023 at 10:52 PM Eric Wong <e@80x24.org> wrote:
>
> Louis DeLosSantos <louis.delos@gmail.com> wrote:
>
> <snip>
>
> > Above is 54 open sockets. Which seems fine.
>
> Actually, only 18 (0..17). The `mem' stuff is mmap-ed and
> doesn't count against `ulimit -n` (RLIMIT_NOFILE).
>
> > Should daemon be running as root, if I intend to only use lei as user?
>
> Definitely not; the lei-daemon is per-user.
>
> I also forgot, inotify has its own per-user limits; perhaps
> you're hitting those?
>
> # show system-wide limits
> $ head /proc/sys/fs/inotify/max_*
>
> # show per-user inotify FDs (-nP speeds up lsof by avoiding lookups)
> $ lsof -nP -u $USER |grep inotify
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root
2023-03-28 3:05 ` Louis DeLosSantos
@ 2023-03-28 3:38 ` Eric Wong
2023-03-28 4:08 ` Louis DeLosSantos
0 siblings, 1 reply; 9+ messages in thread
From: Eric Wong @ 2023-03-28 3:38 UTC (permalink / raw)
To: Louis DeLosSantos; +Cc: meta
Louis DeLosSantos <louis.delos@gmail.com> wrote:
> > Definitely not; the lei-daemon is per-user.
>
> Okay, maybe this is the issue to begin with? I installed lei from dnf.
> I'm not sure what launches the daemon, is it launched on first run?
>
> If that is the case, it was probably launched when I restored to `sudo
> lei q ....` command.
> But, if its running as systemd service, I could move it to user service.
You shouldn't need to manage it as a service; it's auto-started
and killing it is harmless in most cases. I'm considering it
have it auto-exit if it stays idle for a long time and there's
no active inotify watches.
lei-daemon doesn't start until any other lei command is invoked;
so it shouldn't be started on installation.
> # show system-wide limits
> ==> /proc/sys/fs/inotify/max_user_instances <==
> 128
<snip>
> tail 367093 louis 4r a_inode 0,14 0
Thats a lot of tail processes....
I wonder if they were spawned by `lei q -v' for emitting curl stderr?
They should be auto-killed.
(or if you have some other reason for running tail on your system).
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Issues with `lei` as non-root
2023-03-28 3:38 ` Eric Wong
@ 2023-03-28 4:08 ` Louis DeLosSantos
2023-03-28 11:12 ` [PATCH] inotify: wrap with informative error message Eric Wong
0 siblings, 1 reply; 9+ messages in thread
From: Louis DeLosSantos @ 2023-03-28 4:08 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
> Thats a lot of tail processes....
Ugh, sorry to waste your time.
All the tails were from a run away UI program I'm working on.
Once I killed them, `let` runs just fine as non-root.
Thanks for the free tech support, hope I didn't steal your attention
from something valuable :-D.
On Mon, Mar 27, 2023 at 11:38 PM Eric Wong <e@80x24.org> wrote:
>
> Louis DeLosSantos <louis.delos@gmail.com> wrote:
> > > Definitely not; the lei-daemon is per-user.
> >
> > Okay, maybe this is the issue to begin with? I installed lei from dnf.
> > I'm not sure what launches the daemon, is it launched on first run?
> >
> > If that is the case, it was probably launched when I restored to `sudo
> > lei q ....` command.
> > But, if its running as systemd service, I could move it to user service.
>
> You shouldn't need to manage it as a service; it's auto-started
> and killing it is harmless in most cases. I'm considering it
> have it auto-exit if it stays idle for a long time and there's
> no active inotify watches.
>
> lei-daemon doesn't start until any other lei command is invoked;
> so it shouldn't be started on installation.
>
> > # show system-wide limits
>
> > ==> /proc/sys/fs/inotify/max_user_instances <==
> > 128
>
> <snip>
>
> > tail 367093 louis 4r a_inode 0,14 0
>
> Thats a lot of tail processes....
> I wonder if they were spawned by `lei q -v' for emitting curl stderr?
> They should be auto-killed.
> (or if you have some other reason for running tail on your system).
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH] inotify: wrap with informative error message
2023-03-28 4:08 ` Louis DeLosSantos
@ 2023-03-28 11:12 ` Eric Wong
0 siblings, 0 replies; 9+ messages in thread
From: Eric Wong @ 2023-03-28 11:12 UTC (permalink / raw)
To: Louis DeLosSantos; +Cc: meta
Louis DeLosSantos <louis.delos@gmail.com> wrote:
> > Thats a lot of tail processes....
> Ugh, sorry to waste your time.
Not at all. Thank you for sharing your experience with us as it
will benefit future users, especially since lei is still in its
early days. I think the patch below can help future users.
> All the tails were from a run away UI program I'm working on.
> Once I killed them, `let` runs just fine as non-root.
If you're adminning your own system (which it sounds like),
increasing the fs.inotify.max_user_instances should let you
run as many `tail' processes as you desire :>
Perhaps distros and/or kernel developers may raise limits
some day for single-user systems.
My distro uses /etc/sysctl.d/*.conf and /etc/sysctl.conf
to make those knobs persistent across reboots; but distros
vary/change a lot.
> Thanks for the free tech support, hope I didn't steal your attention
> from something valuable :-D.
No problem, and thank you for being one of the brave, early lei users :>
User feedback is valuable :>
------8<------
Subject: [PATCH] inotify: wrap with informative error message
As encountered by Louis DeLosSantos, Linux inotify is capped by
a lesser-known limit than the standard RLIMIT_NOFILE (`ulimit -n`)
value. Give the user a hint about the fs.inotify.max_user_instances
sysctl knob on EMFILE, since EMFILE alone may mislead users into
thinking they've hit the (typically higher) RLIMIT_NOFILE limit.
I can test this on my system using:
perl -I lib -MPublicInbox::Inotify -E \
'my @x = map { PublicInbox::Inotify->new } (1..128)'
But I hesitate to include it in the test suite since triggering
the limit can cause unrelated processes to fail.
Link: https://public-inbox.org/meta/CAE6jdTo8iQfNM9Yuk0Dwi-ARMxmQxX-onL8buXcQ9Ze3r0hKrg@mail.gmail.com/
Reported-by: Louis DeLosSantos <louis.delos@gmail.com>
---
MANIFEST | 1 +
lib/PublicInbox/DirIdle.pm | 4 ++--
lib/PublicInbox/InboxIdle.pm | 4 ++--
lib/PublicInbox/Inotify.pm | 30 ++++++++++++++++++++++++++++++
4 files changed, 35 insertions(+), 4 deletions(-)
create mode 100644 lib/PublicInbox/Inotify.pm
diff --git a/MANIFEST b/MANIFEST
index 40535233..3c421645 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -217,6 +217,7 @@ lib/PublicInbox/In2Tie.pm
lib/PublicInbox/Inbox.pm
lib/PublicInbox/InboxIdle.pm
lib/PublicInbox/InboxWritable.pm
+lib/PublicInbox/Inotify.pm
lib/PublicInbox/InputPipe.pm
lib/PublicInbox/Isearch.pm
lib/PublicInbox/KQNotify.pm
diff --git a/lib/PublicInbox/DirIdle.pm b/lib/PublicInbox/DirIdle.pm
index 55c3982f..af99811c 100644
--- a/lib/PublicInbox/DirIdle.pm
+++ b/lib/PublicInbox/DirIdle.pm
@@ -9,14 +9,14 @@ use PublicInbox::Syscall qw(EPOLLIN);
use PublicInbox::In2Tie;
my ($MAIL_IN, $MAIL_GONE, $ino_cls);
-if ($^O eq 'linux' && eval { require Linux::Inotify2; 1 }) {
+if ($^O eq 'linux' && eval { require PublicInbox::Inotify; 1 }) {
$MAIL_IN = Linux::Inotify2::IN_MOVED_TO() |
Linux::Inotify2::IN_CREATE();
$MAIL_GONE = Linux::Inotify2::IN_DELETE() |
Linux::Inotify2::IN_DELETE_SELF() |
Linux::Inotify2::IN_MOVE_SELF() |
Linux::Inotify2::IN_MOVED_FROM();
- $ino_cls = 'Linux::Inotify2';
+ $ino_cls = 'PublicInbox::Inotify';
# Perl 5.22+ is needed for fileno(DIRHANDLE) support:
} elsif ($^V ge v5.22 && eval { require PublicInbox::KQNotify }) {
$MAIL_IN = PublicInbox::KQNotify::MOVED_TO_OR_CREATE();
diff --git a/lib/PublicInbox/InboxIdle.pm b/lib/PublicInbox/InboxIdle.pm
index f0d8a972..4231c0a0 100644
--- a/lib/PublicInbox/InboxIdle.pm
+++ b/lib/PublicInbox/InboxIdle.pm
@@ -10,9 +10,9 @@ use parent qw(PublicInbox::DS);
use PublicInbox::Syscall qw(EPOLLIN);
my $IN_MODIFY = 0x02; # match Linux inotify
my $ino_cls;
-if ($^O eq 'linux' && eval { require Linux::Inotify2; 1 }) {
+if ($^O eq 'linux' && eval { require PublicInbox::Inotify }) {
$IN_MODIFY = Linux::Inotify2::IN_MODIFY();
- $ino_cls = 'Linux::Inotify2';
+ $ino_cls = 'PublicInbox::Inotify';
} elsif (eval { require PublicInbox::KQNotify }) {
$IN_MODIFY = PublicInbox::KQNotify::NOTE_WRITE();
$ino_cls = 'PublicInbox::KQNotify';
diff --git a/lib/PublicInbox/Inotify.pm b/lib/PublicInbox/Inotify.pm
new file mode 100644
index 00000000..3ef271c8
--- /dev/null
+++ b/lib/PublicInbox/Inotify.pm
@@ -0,0 +1,30 @@
+# Copyright (C) all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+
+# wrap Linux::Inotify2 XS module, support pure Perl via `syscall' someday
+package PublicInbox::Inotify;
+use v5.12;
+our @ISA;
+BEGIN {
+ eval { require Linux::Inotify2 };
+ if ($@) { # TODO: get rid of XS dependency
+ die "W: Linux::Inotify2 missing: $@\n";
+ } else {
+ push @ISA, 'Linux::Inotify2';
+ }
+};
+
+sub new {
+ $_[0]->SUPER::new // do {
+ my $msg = $!{EMFILE} ? <<EOM : "$_[0]->new: $!\n";
+inotify_init/inotify_init1: $!
+You may need to raise the `fs.inotify.max_user_instances' sysctl limit.
+Consult your OS documentation and/or sysctl(8) + sysctl.conf(5) manpages.
+EOM
+ $msg =~ s/^/E: /smg;
+ require Carp;
+ Carp::croak($msg);
+ }
+}
+
+1;
^ permalink raw reply related [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-03-28 11:12 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-03-28 1:00 Issues with `lei` as non-root Louis DeLosSantos
2023-03-28 1:32 ` Eric Wong
2023-03-28 1:36 ` public-inbox.org downtime expected in the next 6-8 hours Eric Wong
2023-03-28 2:30 ` Issues with `lei` as non-root Louis DeLosSantos
2023-03-28 2:52 ` Eric Wong
2023-03-28 3:05 ` Louis DeLosSantos
2023-03-28 3:38 ` Eric Wong
2023-03-28 4:08 ` Louis DeLosSantos
2023-03-28 11:12 ` [PATCH] inotify: wrap with informative error message Eric Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).