* Test failures with 1.7.0 @ 2021-12-08 1:07 Julien Moutinho 2021-12-08 4:08 ` Eric Wong 0 siblings, 1 reply; 31+ messages in thread From: Julien Moutinho @ 2021-12-08 1:07 UTC (permalink / raw) To: meta Hi! I'm trying to package the new 1.7.0 public-inbox into NixOS at https://github.com/NixOS/nixpkgs/pull/104457 but two reviewers are facing different test failures that I cannot reproduce: One reviewer and a bot fail with: > t/lei_to_mail.t .............. 1/? Use of uninitialized value in open at t/lei_to_mail.t line 263. > Bailout called. Further testing stopped: No such file or directory > FAILED--Further testing stopped: No such file or directory And another reviewer fails 3 times with: > t/extsearch.t (Wstat: 2048 Tests: 145 Failed: 8) > Failed tests: 68-69, 75-76, 86, 98, 102, 139 > Non-zero exit status: 8 > t/imapd.t (Wstat: 256 Tests: 186 Failed: 1) > Failed test: 183 > Non-zero exit status: 1 > t/nntpd.t (Wstat: 256 Tests: 110 Failed: 1) > Failed test: 104 > Non-zero exit status: 1 > Files=149, Tests=4730, 107 wallclock secs ( 1.14 usr 0.21 sys + 50.13 cusr 54.98 csys = 106.46 CPU) > Result: FAIL > Failed 3/149 test programs. 10/4730 subtests failed. Nix being Nix we can assume the exact same code is used for all, but our systems are different (eg. CPU(s), filesystem(s), etc.) Could you provide some help? Thanks ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Test failures with 1.7.0 2021-12-08 1:07 Test failures with 1.7.0 Julien Moutinho @ 2021-12-08 4:08 ` Eric Wong 2021-12-08 10:56 ` Dominique Martinet 2022-03-01 2:30 ` Test failures with 1.7.0 Julien Moutinho 0 siblings, 2 replies; 31+ messages in thread From: Eric Wong @ 2021-12-08 4:08 UTC (permalink / raw) To: Julien Moutinho; +Cc: meta Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > Hi! > > I'm trying to package the new 1.7.0 public-inbox into NixOS > at https://github.com/NixOS/nixpkgs/pull/104457 > but two reviewers are facing different test failures > that I cannot reproduce: > > One reviewer and a bot fail with: > > t/lei_to_mail.t .............. 1/? Use of uninitialized value in open at t/lei_to_mail.t line 263. > > Bailout called. Further testing stopped: No such file or directory > > FAILED--Further testing stopped: No such file or directory > > And another reviewer fails 3 times with: > > t/extsearch.t (Wstat: 2048 Tests: 145 Failed: 8) > > Failed tests: 68-69, 75-76, 86, 98, 102, 139 > > Non-zero exit status: 8 > > t/imapd.t (Wstat: 256 Tests: 186 Failed: 1) > > Failed test: 183 > > Non-zero exit status: 1 > > t/nntpd.t (Wstat: 256 Tests: 110 Failed: 1) > > Failed test: 104 > > Non-zero exit status: 1 Sorry for the problems.... I wonder if it's a missing dependency that the tests forget to account for... Can they run the tests individually using "prove" and show more output? e.g.: make && prove -bvw t/nntpd.t Also, "./lei.sh sucks" will dump the relevant deps+versions (either Inline::C or Socket::MsgHdr is required for lei to work) > Nix being Nix we can assume the exact same code is used for all, > but our systems are different (eg. CPU(s), filesystem(s), etc.) Perhaps you guys can compare installed package lists easily and help narrow it down? There's a lot of optional stuff in public-inbox since we try to support some old systems and users who don't want extra dependencies; but yes, it gets difficult to support so many possible combinations. > Could you provide some help? Hope the above was helpful, in any case I should be around-ish the next few days. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Test failures with 1.7.0 2021-12-08 4:08 ` Eric Wong @ 2021-12-08 10:56 ` Dominique Martinet 2021-12-08 18:22 ` [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] Eric Wong 2021-12-09 1:37 ` Test failures with 1.7.0 Julien Moutinho 2022-03-01 2:30 ` Test failures with 1.7.0 Julien Moutinho 1 sibling, 2 replies; 31+ messages in thread From: Dominique Martinet @ 2021-12-08 10:56 UTC (permalink / raw) To: Eric Wong; +Cc: Julien Moutinho, meta Hi, Eric Wong wrote on Wed, Dec 08, 2021 at 04:08:36AM +0000: > > And another reviewer fails 3 times with: > > > t/extsearch.t (Wstat: 2048 Tests: 145 Failed: 8) > > > Failed tests: 68-69, 75-76, 86, 98, 102, 139 > > > Non-zero exit status: 8 > > > t/imapd.t (Wstat: 256 Tests: 186 Failed: 1) > > > Failed test: 183 > > > Non-zero exit status: 1 > > > t/nntpd.t (Wstat: 256 Tests: 110 Failed: 1) > > > Failed test: 104 > > > Non-zero exit status: 1 (I'm that one) So instead of giving the output of prove -bvw t/nntpd.t (I couldn't reproduce manually..), I'll go straight to the solution: it looks like there's a problem with the chattr fallback when btrfs is present on the machine (to disable cow, lib/PublicInbox/NDC_PP.pm) I had initial messages about chattr not being found; adding chattr to the build inputs gave another error message that chattr +C doesn't work on tmpfs. The problem here seems to be that the fallback here just checks /proc/self/mounts and calls chattr "just in case", saying it's ok if it fails, but some tests probably rely on stderr output being empty? If so it looks more like a test problem than anything else to me, but perhaps the fallback path could be a bit more prudent in its calling of chattr (or simply try silencing its output?) For testing, being a bit forceful and removing that chattr call made all the tests pass for me. Now the $100 question is I don't know why the Inline::C version wasn't used; this one properly calls statfs() and does the ioctl directly only if required so would have worked. As far as I can see we're installing Inline::C in the build chroot, I see it in the PERL5LIB paths; and make/gcc are also available so it probably should work. Is there a way to check that? > Sorry for the problems.... > > I wonder if it's a missing dependency that the tests forget to > account for... Can they run the tests individually using "prove" > and show more output? > > e.g.: make && prove -bvw t/nntpd.t > > Also, "./lei.sh sucks" will dump the relevant deps+versions > (either Inline::C or Socket::MsgHdr is required for lei to work) Here's what this gives me: --- lei 1.7.0 perl 5.34.0 / Linux 5.10.81 / x86_64 ptrsize=8 nproc=4 git 2.33.1 / JSON::PP 4.06 SQLite 3.32.3, DBI 1.643, DBD::SQLite 1.66 Xapian 1.4.18, bindings: Search::Xapian 1.2.25.4 public-inbox blob OIDs of loaded features: 2c9c4395c02edbb86683573612f5b454115ce719 PublicInbox/Address.pm 0f002e5e62c260fbb5078bbbb25c67b1468fbbec PublicInbox/Config.pm bacc9cdda12498abbb0ada5d2a2e2faec10190f2 PublicInbox/ContentHash.pm bf8c4466218a37a43688aaaca2b063c65e98c86e PublicInbox/DS.pm 9206da9cb710186462ffefd10cb4af0764f7e615 PublicInbox/DirIdle.pm 485f637a3e7b41a9117cb37a9acfd8e7649dadfa PublicInbox/Eml.pm 80fc7364fc9a1943759585c30487ab0c5d36ca0c PublicInbox/EmlContentFoo.pm 309f80dbfcf001e51f130cddee1a14ee38303dd3 PublicInbox/Git.pm 3e299448b334629fb6427e7df02976381ec19242 PublicInbox/IPC.pm 60ce7b66419b3ee0e9ad2dca4b93644497539378 PublicInbox/Import.pm ffe26a44ab3d9e38d46447fd69543e0a9d4e5c8d PublicInbox/In2Tie.pm 1579d500136a8f814f532275f486bc117c3e0441 PublicInbox/Inbox.pm 887025de5dedb608576b2263b4f20c0596fce8e5 PublicInbox/LEI.pm 30bb1a4579c74d8245d1eec7b66b9687bd526e71 PublicInbox/LeiExternal.pm fa0e78667b3b051cdf3dda96d0b8fba24e1f19f2 PublicInbox/LeiHelp.pm 352ee60131aaf6bc9baabfbd4ae735c4a0f06568 PublicInbox/LeiQuery.pm 8e866fc96655a9dd5a6342de43217da0a57e14f5 PublicInbox/LeiSucks.pm 7cedc3493f872df866755e1c2c9d5d545df7c319 PublicInbox/Listener.pm 0ee2a8bd60bd964a13c0c24f958f43e0ca9a6b5b PublicInbox/Lock.pm f82194a347b867aa22d416b75576e3ac624527ee PublicInbox/MDA.pm 35b517e09f5d12ff264bcd0bd8afe77f1b96e69d PublicInbox/MID.pm dd28417b70c043b092f231147d82580ba94e31cf PublicInbox/MsgIter.pm 5ee087fd50fea18612798d094046b4ada12d16e1 PublicInbox/MsgTime.pm 615bc450998beea1e82606db35c0d456450d966b PublicInbox/OnDestroy.pm 30ad949dd027153f30835f928c962d00e5d9f82a PublicInbox/Over.pm 4c434566d31f90104d921b76adf1844a9d6aefde PublicInbox/PktOp.pm 97e9c268f8d1e91aa0f8c1225112dc42cd623768 PublicInbox/ProcessPipe.pm 523003b3c269ae42e9ee210e91415b2846735daa PublicInbox/Search.pm 81e5a1b1dd88e99d25bf37aadbb1f520e8a6503f PublicInbox/Sigfd.pm 260ce6bb065e930adfd9030d50f5a06359e809ce PublicInbox/Smsg.pm 6ca1ca2a0be3538b3cb9dc663422698b6b94952b PublicInbox/Spawn.pm c00385b94db84b63facf7a8d57296ac76b3b1421 PublicInbox/Syscall.pm 3040dd77d457d02f4517088edcad4f4cf858228f PublicInbox/Tmpfile.pm 950bd17052a569b7e6792f875791d801525d821d PublicInbox/WQWorker.pm Let us know how it sucks! Please include the above and any other relevant information when sending plain-text mail to us at: meta@public-inbox.org -- archives: https://public-inbox.org/meta/ --- > > Nix being Nix we can assume the exact same code is used for all, > > but our systems are different (eg. CPU(s), filesystem(s), etc.) > > Perhaps you guys can compare installed package lists easily and > help narrow it down? There's a lot of optional stuff in > public-inbox since we try to support some old systems and users > who don't want extra dependencies; but yes, it gets difficult to > support so many possible combinations. That's the thing with nixos, in theory the package list really should be identical, so we're down to environmental differences (like me and btrfs present in /proc/self/mounts). The build happens in a chroot with just the "input" packages present (which also incidentally makes it somewhat harder to debug, I couldn't reproduce my error manually...). > > One reviewer and a bot fail with: > > > t/lei_to_mail.t .............. 1/? Use of uninitialized value in open at t/lei_to_mail.t line 263. > > > Bailout called. Further testing stopped: No such file or directory > > > FAILED--Further testing stopped: No such file or directory (waiting to see if the ones who produce this can give more details about their environment) Thanks, -- Dominique Martinet | Asmadeus ^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] 2021-12-08 10:56 ` Dominique Martinet @ 2021-12-08 18:22 ` Eric Wong 2021-12-08 21:14 ` Dominique Martinet 2021-12-09 1:37 ` Test failures with 1.7.0 Julien Moutinho 1 sibling, 1 reply; 31+ messages in thread From: Eric Wong @ 2021-12-08 18:22 UTC (permalink / raw) To: Dominique Martinet; +Cc: Julien Moutinho, meta Dominique Martinet <asmadeus@codewreck.org> wrote: > Eric Wong wrote on Wed, Dec 08, 2021 at 04:08:36AM +0000: > > > And another reviewer fails 3 times with: > > > > t/extsearch.t (Wstat: 2048 Tests: 145 Failed: 8) > > > > Failed tests: 68-69, 75-76, 86, 98, 102, 139 > > > > Non-zero exit status: 8 > > > > t/imapd.t (Wstat: 256 Tests: 186 Failed: 1) > > > > Failed test: 183 > > > > Non-zero exit status: 1 > > > > t/nntpd.t (Wstat: 256 Tests: 110 Failed: 1) > > > > Failed test: 104 > > > > Non-zero exit status: 1 > > (I'm that one) > > So instead of giving the output of prove -bvw t/nntpd.t (I couldn't > reproduce manually..), I'll go straight to the solution: it looks like > there's a problem with the chattr fallback when btrfs is present on the > machine (to disable cow, lib/PublicInbox/NDC_PP.pm) Hi, thanks for the extra info. Note: You might want to rm -rf ~/.cache/public-inbox/inline-c or try a different HOME directory to get a clean repro (more on this later). I also forget to note, setting TAIL='tail -F' can dump daemon output for tests (works better for GNU tail w/ inotify support). > I had initial messages about chattr not being found; adding chattr to > the build inputs gave another error message that chattr +C doesn't work > on tmpfs. > The problem here seems to be that the fallback here just checks > /proc/self/mounts and calls chattr "just in case", saying it's ok if it > fails, but some tests probably rely on stderr output being empty? Yes on tests requiring stderr to be empty. Below is a patch which should fix it; however it should only be calling chattr on btrfs mounts. So I wonder if our /proc/self/mounts parsing is broken... Can you share your contents of /proc/self/mounts? (maybe spaces/odd chars aren't handled properly...) You can also try: BTRFS_TESTDIR=/path/to/your/btrfs-mount prove -bvw t/nodatacow.t ? > If so it looks more like a test problem than anything else to me, but > perhaps the fallback path could be a bit more prudent in its calling of > chattr (or simply try silencing its output?) Agreed, the patch below silences stderr. > For testing, being a bit forceful and removing that chattr call made all > the tests pass for me. > Now the $100 question is I don't know why the Inline::C version wasn't > used; this one properly calls statfs() and does the ioctl directly only > if required so would have worked. > As far as I can see we're installing Inline::C in the build chroot, I > see it in the PERL5LIB paths; and make/gcc are also available so it > probably should work. Is there a way to check that? For public-inbox-* stuff, Inline::C isn't enabled by default even if installed. Either ~/.cache/public-inbox/inline-c needs to exist or PERL_INLINE_DIRECTORY must be set in env. However, lei will auto-create ~/.cache/public-inbox/inline-c if Socket::MsgHdr is missing (since lei is new w/ no existing users). Thus running lei will enable Inline::C for public-inbox-* commands. <snip> Thanks again for all the extra info. Hopefully this below is one step towards making things work more smoothly, but I also suspect /proc/self/mount parsing is off... ------8<----- Subject: [PATCH] nodatacow: quiet chattr errors Maybe this helps when chattr is missing or our /proc/self/mount parsing is broken, somehow. Link: https://public-inbox.org/meta/YbCPWGaJEkV6eWfo@codewreck.org/ --- lib/PublicInbox/NDC_PP.pm | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/lib/PublicInbox/NDC_PP.pm b/lib/PublicInbox/NDC_PP.pm index 57abccbe0625..a84682d371c8 100644 --- a/lib/PublicInbox/NDC_PP.pm +++ b/lib/PublicInbox/NDC_PP.pm @@ -16,11 +16,13 @@ sub nodatacow_dir ($) { # weird chars are escaped as octal $mnt_path =~ s/\\(0[0-9]{2})/chr(oct($1))/egs; $mnt_path .= '/' unless $mnt_path =~ m!/\z!; - if (index($path, $mnt_path) == 0) { - # error goes to stderr, but non-fatal for us - system('chattr', '+C', $path); - last; - } + next if index($path, $mnt_path) < 0; + + open my $olderr, '>&', \*STDERR or die "dup(STDERR): $!"; + open STDERR, '>', '/dev/null' or die "open(/dev/null): $!"; + system(qw(chattr +C), $path); # ignore error + open STDERR, '>&', $olderr or die "dup2(..,STDERR): $!"; + last; } } ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] 2021-12-08 18:22 ` [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] Eric Wong @ 2021-12-08 21:14 ` Dominique Martinet 2021-12-08 22:01 ` Dominique Martinet 0 siblings, 1 reply; 31+ messages in thread From: Dominique Martinet @ 2021-12-08 21:14 UTC (permalink / raw) To: Eric Wong; +Cc: Julien Moutinho, meta [-- Attachment #1: Type: text/plain, Size: 2180 bytes --] Eric Wong wrote on Wed, Dec 08, 2021 at 06:22:47PM +0000: > Note: You might want to rm -rf ~/.cache/public-inbox/inline-c or > try a different HOME directory to get a clean repro (more on > this later). Ah, the builder does not have a home ($HOME points to a non-existing directory), so that'd explain why inline-c isn't used for tests. I'll try giving it one, in my opinion it's more representative to test with inline-c working. > I also forget to note, setting TAIL='tail -F' can dump daemon > output for tests (works better for GNU tail w/ inotify support). Thanks! > > I had initial messages about chattr not being found; adding chattr to > > the build inputs gave another error message that chattr +C doesn't work > > on tmpfs. > > The problem here seems to be that the fallback here just checks > > /proc/self/mounts and calls chattr "just in case", saying it's ok if it > > fails, but some tests probably rely on stderr output being empty? > > Yes on tests requiring stderr to be empty. Below is a patch > which should fix it; however it should only be calling chattr on > btrfs mounts. I'll give this a try as well. > So I wonder if our /proc/self/mounts parsing is broken... > Can you share your contents of /proc/self/mounts? > (maybe spaces/odd chars aren't handled properly...) It's a tad big, I've put it down as attachment. I had missed the `index($path, $mnt_path) == 0` check of the script, so I agree it should only be called on a btrfs mount... But the problem in my case is that / is btrfs, so it will always match, even if the build itself happens in /build (which is a tmpfs) So the check doesn't just need to check if it matches, but it must be the longest match's type that is considered e.g. loop through all the entries, remember the longest match's type, and then check if that is btrfs. > You can also try: > > BTRFS_TESTDIR=/path/to/your/btrfs-mount prove -bvw t/nodatacow.t I'll try something similar as well. It turns out the other people with failing test are both using zfs, so the last problem might be somewhat related; I'll give that a try too. I'll send another reply when I'm done, hopefully before work :) -- Dominique [-- Attachment #2: proc mounts file --] [-- Type: text/plain, Size: 41817 bytes --] /dev/dm-2 / btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/05c1kr7hvjy8igz0m1msjg6qfplj2wz7-perl5.34.0-HTTP-Cookies-6.09 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/091dni27c2w1d8y248d8jr8y1ll3jdk2-perl5.34.0-Email-MIME-1.949 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0c7k0znjz1j67vyh7rkas655daxh0dd5-git-2.33.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0cnvyl1npancz3smm2v1qigp1f1lif58-perl5.34.0-Plack-Middleware-ReverseProxy-0.16 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0cz35r5126dziwxz6cs6rqh3lcmb81ig-perl5.34.0-Module-Implementation-0.09 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0dbkbza24kinyv0sb21hdmd0n4cs7vly-perl5.34.0-Params-ValidationCompiler-0.30 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0f7w6agnx47hq22drpy6bd0ha455hfp8-perl5.34.0-IO-Handle-Util-0.02 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0fi0432kdh46x9kbngnmz2y7z0q68cdz-xz-5.2.5-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0grbkshm5cqa6bpx80xk3nrnkji9fsnw-perl5.34.0-Cookie-Baker-0.11 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0hw94rkaqg2l23ajh20qyy4l5qdvnsz7-zstd-1.5.0-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0j6sw51j043rvrv6csg9wkxzjslj8bcj-hook btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0q85a7ygm5s6ym7phb2vnvmp7fgnfilr-perl5.34.0-Email-MIME-ContentType-1.024 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/0rizskpri8d8qawx6qjqcnvlxcvzr1bm-keyutils-1.6.3-lib btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/1hdx7bxjwqrfnalalkgbwi32l45h8z7b-perl5.34.0-Mail-IMAPClient-3.42 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/1l4r0r4ab3v3a3ppir4jwiah3icalk9d-zlib-1.2.11 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/1m35q3vym6v2wpfm6drlay2hkaqyc3bb-nghttp2-1.43.0-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/1yywwq3awjfvrpv3yfa4rvl27zmp9nij-perl5.34.0-HTML-Parser-3.75 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/1zcjxz5z0wvm43z2bq8vziwf4d9l01cr-perl5.34.0-Stream-Buffered-0.03 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/1zsj6v2ry3kirgfdr1qqjsg90ws4f5fm-tzdata-2021c btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/29m4iqpk6s03ld2lz2ydnr254k3jgv5d-gettext-0.21 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/2hgrzcmdn8rsd3v1axxd9c1rklbll6va-git-2.33.1-doc btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/2kl7mfz0ii381n58i719qrvk7rfqb49i-perl5.34.0-MIME-Types-2.18 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/2qb32vc3g91rvv83myywzzrgmgaimf03-openssh-8.8p1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/2s6k3rvk7x936218b32n04kl385m1w8x-libssh2-1.10.0-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/2zs5kkw0lwivr5rs2qzix0iwzgmxxf9v-perl5.34.0-Sub-Install-0.928 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/3g6fhn6pplnfjz8m4mzg500sj2y86fzp-linux-pam-1.5.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/3gq0lgn8620kw2j9axbgxcn1nb4jyizn-xapian-1.4.18 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/3ibnw61rlgj2lj5hycy2dn3ybpq7wapm-libev-4.33 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/3igs7wc8wys0n5w53binlz3af7qhidhj-perl5.34.0-Pegex-0.75 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/3q65fybq3dlzzy0jjabncg0f9vw8dc9k-perl5.34.0-DBI-1.643 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/4kx13qnwgwyb1zq57x0gnvzvdaq91m5r-curl-7.79.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/4nkxq7xgqpyq2v2zd1p25bq7f6kck8l1-perl5.34.0-Module-Runtime-0.016 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/4nw7qp9mhx4yfgga66664717qnkb96yv-util-linux-2.37.2-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/545xan6vhb42f39rjn9ni7rhmpqv8n37-perl5.34.0-HTTP-Request-AsCGI-1.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/55icsrszy4idkdwjqh7nvr9b3qnlgwaz-perl5.34.0-Encode-Locale-1.05 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/5ana57yikvgq34zbfdn6khldr09pm0aq-perl5.34.0-CGI-4.51 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/5bh6rpya1ar6l49vrhx1rg58dsa42906-python3-3.9.6 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/5jpkzx56n4p9aspnz00dsil44bxv13hb-perl5.34.0-IO-Tty-1.15 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/5vzawr6hfi91c8ndf7cj0lfisjn3mckr-perl5.34.0-Sub-Uplevel-0.2800 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/5xwvhw1maymf68qp44mhyvcykprxc5cx-expand-response-params btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/5ylz3dd6fdch62jznchcg85d4p5jmv2y-perl5.34.0-LWP-MediaTypes-6.04 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/61wgnz3xxfmabwxay9gpy200y07xqipl-perl5.34.0-Eval-Closure-0.14 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/649xkl75pcg9iwcfrq7bcpm459s5d0nx-curl-7.79.1-man btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/65jw2h2al04lnwx38r06n2jjbh96ah32-busybox-static-x86_64-unknown-linux-musl-1.34.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/67x6kxbanrqafx3hg7pb3bc83i3d1v3f-gzip-1.11 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/6h32ddjbnjng7gbdla7ag3hmxygm0wq5-perl5.34.0-common-sense-3.75 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/6irxz4fbf1d1ac7wvdjf8cqb3sgmnvg8-zlib-1.2.11-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/6j67z3vp0b89xvklfxgghp6va1xzn5lm-perl5.34.0-Data-OptList-0.110 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/6kgaicmynv53fa6cjabixp4a11girx5d-perl5.34.0-Test-Needs-0.002006 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/6kgfmzx90c1a6afqnbkz6qprkzss476k-mime-types-9 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/6s81ck7vrma1i6pbxxpdy2324xhz1py7-perl5.34.0-IPC-Run-20200505.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/6znzq5yfa9g004acgiz1gmd0kwd399qs-audit-2.8.5 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/71pachqc22wlvf3xjhwjh2rqbl6l3ngg-diffutils-3.8 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/724iipkqbcrh4gaiv8xi4crwckj3lzs8-perl5.34.0-Linux-Inotify2-2.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/72vdginhi637lxyii5fnpmvf4jzc4gbk-ncurses-6.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/7fpwn95ial009l1y8zy1jvswjzcn0nff-perl5.34.0-Sub-Exporter-Progressive-0.001013 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/7kw4fpy721jv2jr5ili6kyv3nqrxylyx-perl5.34.0-LWP-Protocol-http10-6.03 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/7lacd75rxv7mqbrm44xhclx3da7h199b-perl5.34.0-IO-HTML-1.004 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/7n91vqn1kb6vrk11sr30wca9qayis5s1-lua-5.2.4 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/7rfaw11na5ajdgwr55ffzwfibbrdpk8z-glibc-2.33-56-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/7vq4ail5lyw7n9qrmwf3y5879fdsjb4j-perl5.34.0-Plack-1.0048 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/7z5ai3qarfa3xvw0rfpy9dnqqilwjppf-binutils-wrapper-2.35.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/8334m6k80h0fd5acqkmp8hfrcgagm684-perl5.34.0-URI-5.05 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/84k3qb7hpv231p52zmrsxd4x186vc0zm-libkrb5-1.18 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/8jqps5s2wbxqlwiq7wnv2vhh4148q3ak-perl5.34.0-Test-Requires-0.11 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/8vdbp4jzbj82d8c4wlqd4pxrxirf3mnd-perl5.34.0-Any-URI-Escape-0.01 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/90y23lrznwmkdnczk1dzdsq4m35zj8ww-bash-interactive-5.1-p8 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/93igc1gfl7zw6yayxvai014h125cw0x0-perl5.34.0-Package-Stash-0.39 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/9hhv4qc4fb07jr0cfnif3fcym397xp4h-perl5.34.0-TermReadKey-2.38 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/9m6wkb91sjifc50m1fdgky9abnhk75zm-systemd-minimal-249.5 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/9mp06ni69a44dmrjhn28mn15brdry52w-gnused-4.8 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/9ppi191zsi7zvynkm8vy2bi22lci9iwg-bzip2-1.0.6.0.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/9xlk6zzmcf9yh89pva7ffza3xk6zwvag-libfido2-1.8.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ai4y385z26ka668xhg21g3bpprmk32cs-perl5.34.0-FCGI-ProcManager-0.28 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/aq5k7hrbq50mry8xlxx71b08m80vcpjr-perl5.34.0-DBD-SQLite-1.66 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/aqkrqs09i9wzrr5s2pfr96yhnyw1laqg-perl5.34.0-HTTP-Negotiate-6.01 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/b0436zyxfg3n29vya7rbps8cchxh6ncj-libidn-1.38 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/baacy0v9iz6m045hnffa0fi0zr6wnnzz-openssl-1.1.1l-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/bl8czy1y3qb9j3phbrqklg8j2r6p8i1i-perl5.34.0-Test-MockTime-HiRes-0.08 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/bmv164bdc1dmw67pa2bbh6rai4ra9w73-perl5.34.0-Dist-CheckConflicts-0.11 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/bvn90wp9d97ay7z0i3rfkp2kjdxcv7jl-perl5.34.0-XXX-0.35 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/byixsnjgvzxy9rx0jadwjdyzs0rj75g5-zstd-1.5.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/byzidx23xdyp1why6h4hkq3812bgf243-perl5.34.0-FCGI-0.79 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/bzg7k9pp9jifybp7128bvm1szyhg5hdb-perl5.34.0-CPAN-Meta-Check-0.014 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/c1v3ghcmqplp79dpialpx8v9bnmx3j34-perl5.34.0-Try-Tiny-0.30 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/c5bd4lrnsck51cll6s14wkp93ni4zj1j-glibc-2.33-56-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/c88mwrfh761csp2n74ddgrxl6706amid-readline-8.1p0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/c9f15p1kwm0mw5p13wsnvd1ixrhbhb12-gcc-10.3.0-lib btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ca8bls3hlz043czn2myhp6lh8mah89vq-hook btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/cdkpv9b9ln34050yi4j8f3y8fsl9ra7r-nghttp2-1.43.0-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/chddm1kigcgj62qkps2dhvc0g2yqrf8j-perl5.34.0-HTML-Tagset-3.20 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/d0aj1lpk19jns5s3qnnk79w23mmqvdmy-libpipeline-1.5.3 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/d27pv0vwsawvdgwafsras3zfv06barpq-perl5.34.0-MRO-Compat-0.13 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/d65wn83yzlxdl36k6vs1mnhycyrmnggh-kbd-2.4.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/d78xlj699g6nnnn3qwa7d9x1cfivyakj-libidn-1.38-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/d9nqdg99263q2hdj3chgq58ryzsxrx0d-perl5.34.0-Exception-Class-1.44 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/da9l6ifizmh883j94gzx5n51h8lrrp00-gcc-wrapper-10.3.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/dcb3y8l0zc2xv2crxdsfxfll9xadsp77-pcre2-10.37 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/f2m83ypabavw5ria5fqzf56gcjg42qi8-perl5.34.0-Class-Inspector-1.36 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/f3vr8qxbqzv81mdyd47nxcmp4bjbnmwf-perl5.34.0-HTTP-Server-Simple-PSGI-0.16 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/fa793x4d84wslc2pj9spq1gc5j3b1sl1-perl5.34.0-namespace-clean-0.27 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/fic8j6k9hsgx3f89kavlkc4jqqn51s8l-db-5.3.28 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/fijrw5ir2ajkx55sxy3mwxr3p3f7p5vm-perl5.34.0-Apache-LogFormat-Compiler-0.36 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/fjiwamxwjifspds1cy0q9fk10cnw55qh-perl5.34.0-libwww-perl-6.49 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/fl56aypsak2a8ka4khsafacqvm742jn6-perl5.34.0-Log-Dispatch-2.70 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/fnvnllcgsxzl5w99sil57nfjkjypyhrd-perl5.34.0-HTML-TagCloud-0.38 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/fvprxgcxf4px865gdjd81fbwnxcjrg41-coreutils-9.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/g0bvs0w27ijhw6s1lqgpscg54k4spvcr-perl5.34.0-Sub-Exporter-0.987 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/g46cby42hjz6ksbwikmwqw1ha2nb6ngp-perl5.34.0-HTTP-Entity-Parser-0.25 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/gcy03zd1c0xjzmzv30hxapf0nlbq5s1c-libkrb5-1.18-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/gdrwgfz3lr0dvrkwprqv2gd7d02fbww7-libffi-3.4.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/gjmf46lar6hi5qdmsf1vckwfs01p90rj-libseccomp-2.5.2-lib btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/gjyk1v4ri3k1sdl2xnpjk942nsl402v9-libcbor-0.8.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/gmnh4jfjhx83aggwgwzcnrwmpmqr8fwf-gnutar-1.34 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/gmzhclix3kzhir5jmmwakwhpg6j5zwf1-acl-2.3.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/h1yj1jkx57dsj4czl0mpiy2zlc3ybysw-perl5.34.0-Test-RequiresInternet-0.05 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/h6d0c7lnxx65i925xmnxgwdfdvc7hc2p-gcc-10.3.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/h97sr1q1rpv1ry83031q51jbkba7q0m4-bzip2-1.0.6.0.2-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/hggi0lvp1d5vyzkxssv7ybp2kk9if9fk-perl5.34.0-Test-Deep-1.130 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/hjp3p5szxam9wzx7mqb6dj8mzy27npzw-brotli-1.0.9-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/hl09r5178s52savxh4pmkg86jjlakd4p-openssl-1.1.1l btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/hwgcj5gjp0a4z8mfn3b8y04x28smcxmw-perl5.34.0-IO-String-1.08 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/hz9zz1pba77x1vfv1hx9nmyxi9wj127d-perl5.34.0-WWW-RobotRules-6.02 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/i783jhamhw6wb12qqjasa53hfgffwqfn-perl5.34.0-HTTP-Server-Simple-0.52 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/iin3z0l3wmi21h2z20bnl0zkhh8jhvzb-kexec-tools-2.0.23 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ikvp5db9hygc14da45lvxi1c9b4ylna9-pcre-8.44 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ilcwbbjd20yi6904fni1plz3c0cnvk7m-perl5.34.0-JSON-MaybeXS-1.004003 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/in0c2wbqmbxi5kyajh76wkxpcg1pii30-perl5.34.0-Crypt-PasswdMD5-1.40 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/iniphhwn91s358y4akbccahgxjgmz1k4-expat-2.4.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/iqj4idp0pf84hhwmslhkm5jza8vackmm-perl5.34.0-Filesys-Notify-Simple-0.14 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ir1nx41pprndndnaqqrzpdx60p178d32-db-4.8.30 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/j5v7dhq6nnvhvslzfk6ica51ahyk1dyk-gdbm-1.20 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/j7nqx56g28jr2ka1sk590q74d0rhv6dm-perl5.34.0-File-ShareDir-Install-0.13 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/jmwnx0256q8jxxq0bdw98hhgxhx6g7rl-perl5.34.0-asa-1.04 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/jp0a4523pcaksa4almi01371z9vw8i09-libidn-1.38-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/js66s0xwjnzg0ggi2lq9bcvlk6x2za13-binutils-2.35.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/k0qa3rjifblr2vrgx4g54a59zxlfhg90-xz-5.2.5 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/k80ap8bdw4dqs0771a4hwg67z77wbll8-perl5.34.0-Email-Date-Format-1.005 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/kd14wd2wfmb56zpv5y71yq2lqs11l06k-attr-2.5.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/khv33kn327fpdx2311bcjj98kigd2rac-shadow-4.8.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ki2qi1gnjn40ac7fqdz8bw0ncdvdr8kv-man-db-2.9.4 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/kraak5cql3zjqsb3jjv8d7i0qq30ikzs-curl-7.79.1-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ksfz9v7lr3nwhg16sv36rizzll3n31ha-perl5.34.0-Params-Util-1.102 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ksqy6mszsld4z3w8ybxa2vkjf5cqxw3f-c-ares-1.17.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/l0wlqpbsvh1pgvhcdhw7qkka3d31si7k-bash-5.1-p8 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/l3a4yxvd952r0bdpnwii3mn74dafkzcb-perl5.34.0-Test-SharedFork-0.35 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/l4ahhym84c1c57g549qa1pjb3h72dcf4-perl5.34.0-Log-Dispatch-Array-1.003 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/lf6zhg0mxghj1cjngbgj5mfdmzj51a3l-perl5.34.0-Net-SSLeay-1.88 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/lhambyc1v2c7qzzr5sq7p449xs1j6pg8-gnugrep-3.7 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/lkcsgql5nx2fq0h86fyf859cgkz1xcsq-perl5.34.0-Text-Unidecode-1.30 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/lsrahrxyjnsnxzwiacf3dpf0cs7wyn28-perl5.34.0-Inline-0.86 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/m4cpim69iv3mbxxdgvf5cph0hv1abyg5-sqlite-3.36.0-bin btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/m7prz5hx0v9sxj1kdavlh137zhny0vfv-perl5.34.0-Specio-0.46 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/m9lg325s9icr09d1yyc7n26j65alyfaw-perl5.34.0-HTTP-Daemon-6.01 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/mdsjr15hahrkv33k609kvicicsxjh40z-nghttp2-1.43.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/mfnfwmffw8vflgkp9i07c0cxrppw90gg-curl-7.79.1-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/mq9pycra4p4i0gl79gxwzw603dx49n72-stdenv-linux btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/n72iwxf0z2znppa2yr7ndkk502vvznqq-libssh2-1.10.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/n8ifh1z6wkz96j745hsd9p13agdp7md1-perl5.34.0-File-Listing-6.14 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/nalr2sh5p2vjd06kplvna8yqbdpykpkw-perl5.34.0-Test-TCP-2.22 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ni7avvif6pafhqlknzy9yr1cnv9kn702-brotli-1.0.9-lib btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/nii22vkgxhl5x3kkvgddq9yfglpn133z-perl5.34.0-POSIX-strftime-Compiler-0.44 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/nirbgzlcw8zzv3ra6qz2vwi0lw6rzbbg-perl5.34.0-TimeDate-2.33 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/nlvg888wp8jlpmz7bycgc3xfna9xjymz-libcap-2.49-lib btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/nnqmxaydza237jn0dnjvcqqq0f4vvaar-perl5.34.0-Hash-MultiValue-0.16 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ns92nil6y1iff1hvvxh8k6rgfy01qzym-perl5.34.0-HTTP-MultiPartParser-0.02 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/nwbwp5pr2z3wyf6sbwrgc9lncabd26nr-perl5.34.0-namespace-autoclean-0.29 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ny5qpm0y2h44j8kqh740zy6zbf0gzipj-perl5.34.0-Authen-Simple-0.5 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/p2m569gcvx0hr9x0my2002qa01sk351b-groff-1.22.4 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/p7j7hzy5arwx62rd7l67j1b3rii1v2kl-perl5.34.0-Email-MIME-Encodings-1.315 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/pmmlsbl7zf4mda7k6z09qxbmjd11n82r-perl5.34.0-Net-HTTP-6.19 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/psqacrv7k5fxz6mdiawc28sxcdchb4c9-ed-1.17 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/pybllh3323clcqrmaq3vsp4afhwxc5fv-perl5.34.0-YAML-PP-0.026 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/pyk9bdmc36d29367w3rhmn4a9xxnpl7j-perl5.34.0-File-ShareDir-1.118 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/q0zgpg2v1lkaa0j5yw10v9ydlqb4bx0b-perl5.34.0-Authen-Simple-Passwd-0.6 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/qbdsd82q5fyr0v31cvfxda0n0h7jh03g-libunistring-0.9.10 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/qf26s1qwwrifgjvjdqlkdwdfrlq7slh7-perl5.34.0-Sub-Identify-0.14 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/qg8jgfxhn3zmrvh8w52xybijf5nw8h9d-perl5.34.0-Sub-Quote-2.006006 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/qzr7r4w5gm5m20afn2wz4vlv7ah4sr89-gnumake-4.3 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/r5niwjr8r8qags2bzv9z583r9vajxag3-patchelf-0.13 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/r7va5lgm2nlfffvgarl6r738d625s3iy-brotli-1.0.9 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/rhjqb9k87d3v2s4vpjal7sn8p28gdma2-perl5.34.0-IO-Socket-SSL-2.068 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/rnx655nq2qs53yb5arv2gapa91r1wsbn-findutils-4.8.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/rpkbgh8y5k1yvsq04zls6r5003s0a9d5-sqlite-3.36.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/s2pzral55ps1n389m6270f1n4mpic8fl-sqlite-3.36.0-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/s41hl4f321ifhchzc963kkipr7rgkq2p-perl5.34.0-Test-Warn-0.36 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/s92s53kaiapsvw9kyk24sgcgi4ys6rlj-perl5.34.0-Email-MessageID-1.406 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/s97m83hrzzsj3d4pjvzk7r3jvddhm52w-perl5.34.0-Devel-StackTrace-2.04 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/scz4zbxirykss3hh5iahgl39wk9wpaps-libidn2-2.3.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/skc5bc5ih1s19hbsjmbvn9jwzr81h4ga-kmod-29 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/sn0h44frwy9rfmzf0f5xs26h9gj9qxc7-nghttp2-1.43.0-lib btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/sqn31ly001033hsz0dpxwcsay5qdbk2w-gawk-5.1.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/sx3ggwg4aszamg4kcz2q629n3gw6l9xw-perl5.34.0-Devel-StackTrace-AsHTML-0.15 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/syx9zczfg4p6x013lyg4j6nr1fqj4l9m-openssl-1.1.1l-dev btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/v7c1gih8m690hhm504cxisp6b1pwzxjs-perl5.34.0-Params-Validate-1.30 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/vafyy0lx82py8403bvkaki34i5bpin2n-linux-headers-5.14 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/vh8vr0avs8585clpw9k183sc2kb64v0l-perl5.34.0-Test-Fatal-0.016 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/vid3qqjlrjgld5qyiazi74qy6sd7kfad-perl5.34.0-Class-Accessor-0.51 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/vq8fq8p13wvgfaylncnbqjjdxqxxn5p2-perl5.34.0-CGI-Emulate-PSGI-0.23 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/vslsa0l17xjcrdgm2knwj0z5hlvf73m7-perl-5.34.0 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/w0hmsxmgsbv7vzf2y4lyx3lnhmr5d4vx-perl5.34.0-Email-Simple-2.216 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/wh9cwmidi60x1c6q42hp1ifjbz889chv-perl5.34.0-CGI-Fast-2.15 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/wi9g0kykpwajbbi01gxr8i6dxmg3zija-perl5.34.0-Parse-RecDescent-1.967015 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/wp10khjwd6f3ypw9pxgfj1rhscb1m896-perl5.34.0-Inline-C-0.81 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/wrjn1knw7ras686pr3nbaqgxqvr395ad-readline-6.3p08 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ww7nzw9jdzghpwxzn56nwl0k6gxska6l-perl5.34.0-HTTP-Date-6.05 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/x3nivnifc568rwficrczwz4gsn98872z-perl5.34.0-HTTP-Headers-Fast-0.22 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/x5d4w3xnlvsg7ihywgbfwg0ylmbybam0-perl5.34.0-Class-Data-Inheritable-0.08 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/x8m12f6ppfyk28zh1i03nwimyig7d4vx-perl5.34.0-Search-Xapian-1.2.25.4 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/xf9gvzb883bf4391wvyz82bzlz0c8rzy-libcap-ng-0.8.2 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/xqchk05qaxvz1x43ki64qsizsz7yf2k0-perl5.34.0-Mozilla-CA-20200520 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/xra89qnn4m2lfjhcid1b5yd2z7jxd5sl-perl5.34.0-WWW-Form-UrlEncoded-0.26 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/xvyzi7cr0icnyavi5pm9rywjc4d8l7sx-libedit-20210714-3.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/xzrmb5fj7hw44a6fwp99b9ivw5dlzqd0-perl5.34.0-HTTP-Message-6.26 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/y3nw0pf6r4xrhyfxm670mlasjs26zl5b-perl5.34.0-Email-Address-XS-1.04 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/y53ma353xbpv6nnjzxhwcid087h6dp36-perl5.34.0-Devel-GlobalDestruction-0.14 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/y64kgxy85z6j1fdbh9lsfjzwzbwyz81w-highlight-4.1 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/ywmf6vnn7rxh95mmzlfx7yb431mgx6kl-perl5.34.0-Role-Tiny-2.001004 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/yxflij8cg4fgnzqmda91jx4d94jvkjf5-util-linux-2.37.2-lib btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/z56jcx3j1gfyk4sv7g8iaan0ssbdkhz1-glibc-2.33-56 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/zjm4xv4nr872mdhvv3j22bzb08rgf1hk-patch-2.7.6 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /nix/store/zp0pqbkcxi79ar8rca9sykqq90104n89-perl5.34.0-B-Hooks-EndOfScope-0.24 btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 /dev/dm-2 /bin/sh btrfs rw,relatime,degraded,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/root 0 0 tmpfs /build tmpfs rw,relatime,size=14680064k 0 0 devtmpfs /dev/full devtmpfs rw,nosuid,size=814296k,nr_inodes=2032488,mode=755 0 0 devtmpfs /dev/kvm devtmpfs rw,nosuid,size=814296k,nr_inodes=2032488,mode=755 0 0 devtmpfs /dev/null devtmpfs rw,nosuid,size=814296k,nr_inodes=2032488,mode=755 0 0 devtmpfs /dev/random devtmpfs rw,nosuid,size=814296k,nr_inodes=2032488,mode=755 0 0 devtmpfs /dev/tty devtmpfs rw,nosuid,size=814296k,nr_inodes=2032488,mode=755 0 0 devtmpfs /dev/urandom devtmpfs rw,nosuid,size=814296k,nr_inodes=2032488,mode=755 0 0 devtmpfs /dev/zero devtmpfs rw,nosuid,size=814296k,nr_inodes=2032488,mode=755 0 0 none /proc proc rw,relatime 0 0 none /dev/shm tmpfs rw,relatime 0 0 none /dev/pts devpts rw,relatime,mode=620,ptmxmode=000 0 0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] 2021-12-08 21:14 ` Dominique Martinet @ 2021-12-08 22:01 ` Dominique Martinet 2022-01-30 21:49 ` Eric Wong 0 siblings, 1 reply; 31+ messages in thread From: Dominique Martinet @ 2021-12-08 22:01 UTC (permalink / raw) To: Eric Wong; +Cc: Julien Moutinho, meta Dominique Martinet wrote on Thu, Dec 09, 2021 at 06:14:36AM +0900: > I'll try giving it one, in my opinion it's more representative to test > with inline-c working. So giving tests a home appear to make another test hang (t/lei-refresh-mail-sync.t) I've run out of time, will provide more traces tonight > > Yes on tests requiring stderr to be empty. Below is a patch > > which should fix it; however it should only be calling chattr on > > btrfs mounts. > > I'll give this a try as well. This patch makes the tests pass as expected. > > You can also try: > > > > BTRFS_TESTDIR=/path/to/your/btrfs-mount prove -bvw t/nodatacow.t > > I'll try something similar as well. I can confirm this one works as well after installing chattr and running on btrfs, so there's no problem if mounts parsing is fixed. I'd say this hints at a problem so we're probably better off not silencing chattr warnings, but would also need to check if the chattr binary is present... Probably not worth the hassle, I don't know. > It turns out the other people with failing test are both using zfs, so > the last problem might be somewhat related; I'll give that a try too. Julien could also reproduce this one by running on ZFS, we'll provide more traces when we have them! -- Dominique ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] 2021-12-08 22:01 ` Dominique Martinet @ 2022-01-30 21:49 ` Eric Wong 2022-01-30 23:18 ` Dominique Martinet 0 siblings, 1 reply; 31+ messages in thread From: Eric Wong @ 2022-01-30 21:49 UTC (permalink / raw) To: Dominique Martinet; +Cc: Julien Moutinho, meta Dominique Martinet <asmadeus@codewreck.org> wrote: > Dominique Martinet wrote on Thu, Dec 09, 2021 at 06:14:36AM +0900: > > I'll try giving it one, in my opinion it's more representative to test > > with inline-c working. > > So giving tests a home appear to make another test hang > (t/lei-refresh-mail-sync.t) > > I've run out of time, will provide more traces tonight > > > > Yes on tests requiring stderr to be empty. Below is a patch > > > which should fix it; however it should only be calling chattr on > > > btrfs mounts. > > > > I'll give this a try as well. > > This patch makes the tests pass as expected. > > > > You can also try: > > > > > > BTRFS_TESTDIR=/path/to/your/btrfs-mount prove -bvw t/nodatacow.t > > > > I'll try something similar as well. > > I can confirm this one works as well after installing chattr and running > on btrfs, so there's no problem if mounts parsing is fixed. > > I'd say this hints at a problem so we're probably better off not > silencing chattr warnings, but would also need to check if the chattr > binary is present... Probably not worth the hassle, I don't know. Thanks for testing the previous patch. Actually, I prefer we drop previous implementations and instead rely on Linux ABI stability while shifting maintenance burden to maintainers. Can you test the patch below? It supercedes the other one. Apologies for the delay, I've pretty much lost all motivation for everything in life :< Will try to find some energy to look into the other issues in a bit... Thanks. ---------8<---------- Subject: [PATCH] rewrite Linux nodatacow use in pure Perl w/o system btrfs is Linux-only at the moment (and likely to remain that way for practical purposes). So rely on Linux ABI stability and use the `syscall' and `ioctl' perlops rather than relying on Inline::C. Inline::C (and gcc||clang) are monstrous dependencies which we can't expect users to have. This makes supporting new architectures more difficult, but new architectures come along rarely and this reduces the burden for the majority of Linux users on popular architectures (while still avoiding the distribution of pre-built binaries). Link: https://public-inbox.org/meta/YbCPWGaJEkV6eWfo@codewreck.org/ --- MANIFEST | 1 - devel/syscall-list | 10 ++++- lib/PublicInbox/IMAPTracker.pm | 6 +-- lib/PublicInbox/LeiMailSync.pm | 6 +-- lib/PublicInbox/MiscIdx.pm | 6 +-- lib/PublicInbox/Msgmap.pm | 6 +-- lib/PublicInbox/NDC_PP.pm | 34 --------------- lib/PublicInbox/Over.pm | 7 ++-- lib/PublicInbox/SearchIdx.pm | 7 ++-- lib/PublicInbox/SharedKV.pm | 6 +-- lib/PublicInbox/Spawn.pm | 76 +++------------------------------- lib/PublicInbox/Syscall.pm | 46 +++++++++++++++++++- lib/PublicInbox/Xapcmd.pm | 11 ++--- t/nodatacow.t | 35 +++++++--------- 14 files changed, 101 insertions(+), 156 deletions(-) delete mode 100644 lib/PublicInbox/NDC_PP.pm diff --git a/MANIFEST b/MANIFEST index 1287182d..ca840210 100644 --- a/MANIFEST +++ b/MANIFEST @@ -289,7 +289,6 @@ lib/PublicInbox/MsgIter.pm lib/PublicInbox/MsgTime.pm lib/PublicInbox/Msgmap.pm lib/PublicInbox/MultiGit.pm -lib/PublicInbox/NDC_PP.pm lib/PublicInbox/NNTP.pm lib/PublicInbox/NNTPD.pm lib/PublicInbox/NNTPdeflate.pm diff --git a/devel/syscall-list b/devel/syscall-list index 3d55df1f..a6b1bfa7 100755 --- a/devel/syscall-list +++ b/devel/syscall-list @@ -26,8 +26,10 @@ system($cc, '-o', $x, $f, @cflags) == 0 or die "cc failed \$?=$?"; exec($x); __DATA__ #define _GNU_SOURCE -#include <unistd.h> #include <sys/syscall.h> +#include <sys/ioctl.h> +#include <linux/fs.h> +#include <unistd.h> #include <stdio.h> #define D(x) printf("$" #x " = %ld;\n", (long)x) @@ -46,6 +48,12 @@ int main(void) D(SYS_inotify_add_watch); D(SYS_inotify_rm_watch); D(SYS_prctl); + D(SYS_fstatfs); +#ifdef FS_IOC_GETFLAGS + printf("FS_IOC_GETFLAGS=%#lx\nFS_IOC_SETFLAGS=%#lx\n", + (unsigned long)FS_IOC_GETFLAGS, (unsigned long)FS_IOC_SETFLAGS); +#endif + #ifdef SYS_renameat2 D(SYS_renameat2); #endif diff --git a/lib/PublicInbox/IMAPTracker.pm b/lib/PublicInbox/IMAPTracker.pm index 2fd66440..4efa8a7e 100644 --- a/lib/PublicInbox/IMAPTracker.pm +++ b/lib/PublicInbox/IMAPTracker.pm @@ -1,4 +1,4 @@ -# Copyright (C) 2018-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> package PublicInbox::IMAPTracker; use strict; @@ -75,11 +75,11 @@ sub new { } if (!-f $dbname) { require File::Path; - require PublicInbox::Spawn; + require PublicInbox::Syscall; my ($dir) = ($dbname =~ m!(.*?/)[^/]+\z!); File::Path::mkpath($dir); + PublicInbox::Syscall::nodatacow_dir($dir); open my $fh, '+>>', $dbname or die "failed to open $dbname: $!"; - PublicInbox::Spawn::nodatacow_fd(fileno($fh)); } my $self = bless { lock_path => "$dbname.lock", url => $url }, $class; $self->lock_acquire; diff --git a/lib/PublicInbox/LeiMailSync.pm b/lib/PublicInbox/LeiMailSync.pm index 124eb969..182b0c22 100644 --- a/lib/PublicInbox/LeiMailSync.pm +++ b/lib/PublicInbox/LeiMailSync.pm @@ -1,4 +1,4 @@ -# Copyright (C) 2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> # for maintaining synchronization between lei/store <=> Maildir|MH|IMAP|JMAP @@ -15,9 +15,9 @@ sub dbh_new { my $f = $self->{filename}; my $creat = $rw && !-s $f; if ($creat) { - require PublicInbox::Spawn; + require PublicInbox::Syscall; open my $fh, '+>>', $f or Carp::croak "open($f): $!"; - PublicInbox::Spawn::nodatacow_fd(fileno($fh)); + PublicInbox::Syscall::nodatacow_fh($fh); } my $dbh = DBI->connect("dbi:SQLite:dbname=$f",'','', { AutoCommit => 1, diff --git a/lib/PublicInbox/MiscIdx.pm b/lib/PublicInbox/MiscIdx.pm index f5a374b2..dc15442d 100644 --- a/lib/PublicInbox/MiscIdx.pm +++ b/lib/PublicInbox/MiscIdx.pm @@ -1,4 +1,4 @@ -# Copyright (C) 2020-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> # like PublicInbox::SearchIdx, but for searching for non-mail messages. @@ -16,11 +16,11 @@ use v5.10.1; use PublicInbox::InboxWritable; use PublicInbox::Search; # for SWIG Xapian and Search::Xapian compat use PublicInbox::SearchIdx qw(index_text term_generator add_val); -use PublicInbox::Spawn qw(nodatacow_dir); use Carp qw(croak); use File::Path (); use PublicInbox::MiscSearch; use PublicInbox::Config; +use PublicInbox::Syscall; my $json; sub new { @@ -28,7 +28,7 @@ sub new { PublicInbox::SearchIdx::load_xapian_writable(); my $mi_dir = "$eidx->{xpfx}/misc"; File::Path::mkpath($mi_dir); - nodatacow_dir($mi_dir); + PublicInbox::Syscall::nodatacow_dir($mi_dir); my $flags = $PublicInbox::SearchIdx::DB_CREATE_OR_OPEN; $flags |= $PublicInbox::SearchIdx::DB_NO_SYNC if $eidx->{-no_fsync}; $json //= PublicInbox::Config::json(); diff --git a/lib/PublicInbox/Msgmap.pm b/lib/PublicInbox/Msgmap.pm index 699a8bf0..1041cd17 100644 --- a/lib/PublicInbox/Msgmap.pm +++ b/lib/PublicInbox/Msgmap.pm @@ -1,4 +1,4 @@ -# Copyright (C) 2015-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> # bidirectional Message-ID <-> Article Number mapping for the NNTP @@ -13,7 +13,6 @@ use v5.10.1; use DBI; use DBD::SQLite; use PublicInbox::Over; -use PublicInbox::Spawn; use Scalar::Util qw(blessed); sub new_file { @@ -53,7 +52,8 @@ sub tmp_clone { require File::Temp; my $tmp = "mm_tmp-$$-XXXX"; my ($fh, $fn) = File::Temp::tempfile($tmp, EXLOCK => 0, DIR => $dir); - PublicInbox::Spawn::nodatacow_fd(fileno($fh)); + require PublicInbox::Syscall; + PublicInbox::Syscall::nodatacow_fh($fh); $self->{dbh}->sqlite_backup_to_file($fn); $tmp = ref($self)->new_file($fn, 2); $tmp->{dbh}->do('PRAGMA journal_mode = MEMORY'); diff --git a/lib/PublicInbox/NDC_PP.pm b/lib/PublicInbox/NDC_PP.pm deleted file mode 100644 index 57abccbe..00000000 --- a/lib/PublicInbox/NDC_PP.pm +++ /dev/null @@ -1,34 +0,0 @@ -# Copyright (C) 2020-2021 all contributors <meta@public-inbox.org> -# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> - -# Pure-perl class for Linux non-Inline::C users to disable COW for btrfs -package PublicInbox::NDC_PP; -use strict; -use v5.10.1; - -sub nodatacow_dir ($) { - my ($path) = @_; - open my $mh, '<', '/proc/self/mounts' or return; - for (grep(/ btrfs /, <$mh>)) { - my (undef, $mnt_path, $type) = split(/ /); - next if $type ne 'btrfs'; # in case of false-positive from grep - - # weird chars are escaped as octal - $mnt_path =~ s/\\(0[0-9]{2})/chr(oct($1))/egs; - $mnt_path .= '/' unless $mnt_path =~ m!/\z!; - if (index($path, $mnt_path) == 0) { - # error goes to stderr, but non-fatal for us - system('chattr', '+C', $path); - last; - } - } -} - -sub nodatacow_fd ($) { - my ($fd) = @_; - return if $^O ne 'linux'; - defined(my $path = readlink("/proc/self/fd/$fd")) or return; - nodatacow_dir($path); -} - -1; diff --git a/lib/PublicInbox/Over.pm b/lib/PublicInbox/Over.pm index 30ad949d..786f9d92 100644 --- a/lib/PublicInbox/Over.pm +++ b/lib/PublicInbox/Over.pm @@ -1,4 +1,4 @@ -# Copyright (C) 2018-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> # for XOVER, OVER in NNTP, and feeds/homepage/threads in PSGI @@ -18,11 +18,10 @@ sub dbh_new { my $f = delete $self->{filename}; if (!-s $f) { # SQLite defaults mode to 0644, we want 0666 if ($rw) { - require PublicInbox::Spawn; + require PublicInbox::Syscall; my ($dir) = ($f =~ m!(.+)/[^/]+\z!); - PublicInbox::Spawn::nodatacow_dir($dir); + PublicInbox::Syscall::nodatacow_dir($dir); open my $fh, '+>>', $f or die "failed to open $f: $!"; - PublicInbox::Spawn::nodatacow_fd(fileno($fh)); } else { $self->{filename} = $f; # die on stat() below: } diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm index 4e5d7d44..95b14c3a 100644 --- a/lib/PublicInbox/SearchIdx.pm +++ b/lib/PublicInbox/SearchIdx.pm @@ -1,4 +1,4 @@ -# Copyright (C) 2015-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> # based on notmuch, but with no concept of folders, files # @@ -20,7 +20,7 @@ use Carp qw(croak carp); use POSIX qw(strftime); use Time::Local qw(timegm); use PublicInbox::OverIdx; -use PublicInbox::Spawn qw(spawn nodatacow_dir); +use PublicInbox::Spawn qw(spawn); use PublicInbox::Git qw(git_unquote); use PublicInbox::MsgTime qw(msg_timestamp msg_datestamp); use PublicInbox::Address; @@ -139,7 +139,8 @@ sub idx_acquire { if (!-d $dir && (!$is_shard || ($is_shard && need_xapian($self)))) { File::Path::mkpath($dir); - nodatacow_dir($dir); + require PublicInbox::Syscall; + PublicInbox::Syscall::nodatacow_dir($dir); $self->{-set_has_threadid_once} = 1; } } diff --git a/lib/PublicInbox/SharedKV.pm b/lib/PublicInbox/SharedKV.pm index 4297efed..95a3cb14 100644 --- a/lib/PublicInbox/SharedKV.pm +++ b/lib/PublicInbox/SharedKV.pm @@ -1,4 +1,4 @@ -# Copyright (C) 2020-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> # fork()-friendly key-value store. Will be used for making @@ -49,9 +49,9 @@ sub new { my $f = $self->{filename} = "$dir/$base.sqlite3"; $self->{lock_path} = $opt->{lock_path} // "$dir/$base.flock"; unless (-s $f) { - PublicInbox::Spawn::nodatacow_dir($dir); # for journal/shm/wal + require PublicInbox::Syscall; + PublicInbox::Syscall::nodatacow_dir($dir); # for journal/shm/wal open my $fh, '+>>', $f or die "failed to open $f: $!"; - PublicInbox::Spawn::nodatacow_fd(fileno($fh)); } $self; } diff --git a/lib/PublicInbox/Spawn.pm b/lib/PublicInbox/Spawn.pm index e0a51c21..137b8087 100644 --- a/lib/PublicInbox/Spawn.pm +++ b/lib/PublicInbox/Spawn.pm @@ -1,4 +1,4 @@ -# Copyright (C) 2016-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> # # This allows vfork to be used for spawning subprocesses if @@ -21,7 +21,7 @@ use Symbol qw(gensym); use Fcntl qw(LOCK_EX SEEK_SET); use IO::Handle (); use PublicInbox::ProcessPipe; -our @EXPORT_OK = qw(which spawn popen_rd run_die nodatacow_dir); +our @EXPORT_OK = qw(which spawn popen_rd run_die); our @RLIMITS = qw(RLIMIT_CPU RLIMIT_CORE RLIMIT_DATA); BEGIN { @@ -268,62 +268,12 @@ void recv_cmd4(PerlIO *s, SV *buf, STRLEN n) #endif /* defined(CMSG_SPACE) && defined(CMSG_LEN) */ ALL_LIBC -# btrfs on Linux is copy-on-write (COW) by default. As of Linux 5.7, -# this still leads to fragmentation for SQLite and Xapian files where -# random I/O happens, so we disable COW just for SQLite files and Xapian -# directories. Disabling COW disables checksumming, so we only do this -# for regeneratable files, and not canonical git storage (git doesn't -# checksum refs, only data under $GIT_DIR/objects). - my $set_nodatacow = $^O eq 'linux' ? <<'SET_NODATACOW' : ''; -#include <sys/ioctl.h> -#include <sys/vfs.h> -#include <linux/magic.h> -#include <linux/fs.h> -#include <dirent.h> - -void nodatacow_fd(int fd) -{ - struct statfs buf; - int val = 0; - - if (fstatfs(fd, &buf) < 0) { - fprintf(stderr, "fstatfs: %s\\n", strerror(errno)); - return; - } - - /* only btrfs is known to have this problem, so skip for non-btrfs */ - if (buf.f_type != BTRFS_SUPER_MAGIC) - return; - - if (ioctl(fd, FS_IOC_GETFLAGS, &val) < 0) { - fprintf(stderr, "FS_IOC_GET_FLAGS: %s\\n", strerror(errno)); - return; - } - val |= FS_NOCOW_FL; - if (ioctl(fd, FS_IOC_SETFLAGS, &val) < 0) - fprintf(stderr, "FS_IOC_SET_FLAGS: %s\\n", strerror(errno)); -} - -void nodatacow_dir(const char *dir) -{ - DIR *dh = opendir(dir); - int fd; - - if (!dh) croak("opendir(%s): %s", dir, strerror(errno)); - fd = dirfd(dh); - if (fd >= 0) - nodatacow_fd(fd); - /* ENOTSUP probably won't happen under Linux... */ - closedir(dh); -} -SET_NODATACOW - my $inline_dir = $ENV{PERL_INLINE_DIRECTORY} //= ( $ENV{XDG_CACHE_HOME} // ( ($ENV{HOME} // '/nonexistent').'/.cache' ) ).'/public-inbox/inline-c'; warn "$inline_dir exists, not writable\n" if -e $inline_dir && !-w _; - $set_nodatacow = $all_libc = undef unless -d _ && -w _; + $all_libc = undef unless -d _ && -w _; if (defined $all_libc) { my $f = "$inline_dir/.public-inbox.lock"; open my $oldout, '>&', \*STDOUT or die "dup(1): $!"; @@ -337,17 +287,10 @@ SET_NODATACOW # CentOS 7.x ships Inline 0.53, 0.64+ has built-in locking flock($fh, LOCK_EX) or die "LOCK_EX($f): $!"; eval <<'EOM'; -use Inline C => $all_libc.$set_nodatacow, BUILD_NOISY => 1; +use Inline C => $all_libc, BUILD_NOISY => 1; EOM my $err = $@; my $ndc_err = ''; - if ($err && $set_nodatacow) { # missing Linux kernel headers - $ndc_err = "with set_nodatacow: <\n$err\n>\n"; - undef $set_nodatacow; - eval <<'EOM'; -use Inline C => $all_libc, BUILD_NOISY => 1; -EOM - }; $err = $@; open(STDERR, '>&', $olderr) or warn "restore stderr: $!"; open(STDOUT, '>&', $oldout) or warn "restore stdout: $!"; @@ -356,22 +299,13 @@ EOM my @msg = <$fh>; warn "Inline::C build failed:\n", $ndc_err, $err, "\n", @msg; - $set_nodatacow = $all_libc = undef; - } elsif ($ndc_err) { - warn "Inline::C build succeeded w/o set_nodatacow\n", - "error $ndc_err"; + $all_libc = undef; } } unless ($all_libc) { require PublicInbox::SpawnPP; *pi_fork_exec = \&PublicInbox::SpawnPP::pi_fork_exec } - unless ($set_nodatacow) { - require PublicInbox::NDC_PP; - no warnings 'once'; - *nodatacow_fd = \&PublicInbox::NDC_PP::nodatacow_fd; - *nodatacow_dir = \&PublicInbox::NDC_PP::nodatacow_dir; - } } # /BEGIN sub which ($) { diff --git a/lib/PublicInbox/Syscall.pm b/lib/PublicInbox/Syscall.pm index c00385b9..bcfae2cb 100644 --- a/lib/PublicInbox/Syscall.pm +++ b/lib/PublicInbox/Syscall.pm @@ -5,7 +5,7 @@ # This license differs from the rest of public-inbox # # This module is Copyright (c) 2005 Six Apart, Ltd. -# Copyright (C) 2019-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # # All rights reserved. # @@ -68,6 +68,7 @@ our ( $SYS_renameat2, ); +my $SYS_fstatfs; # don't need fstatfs64, just statfs.f_type my $SFD_CLOEXEC = 02000000; # Perl does not expose O_CLOEXEC our $no_deprecated = 0; @@ -96,18 +97,21 @@ if ($^O eq "linux") { $SYS_epoll_wait = 256; $SYS_signalfd4 = 327; $SYS_renameat2 //= 353; + $SYS_fstatfs = 100; } elsif ($machine eq "x86_64") { $SYS_epoll_create = 213; $SYS_epoll_ctl = 233; $SYS_epoll_wait = 232; $SYS_signalfd4 = 289; $SYS_renameat2 //= 316; + $SYS_fstatfs = 138; } elsif ($machine eq 'x32') { $SYS_epoll_create = 1073742037; $SYS_epoll_ctl = 1073742057; $SYS_epoll_wait = 1073742056; $SYS_signalfd4 = 1073742113; $SYS_renameat2 //= 0x40000000 + 316; + $SYS_fstatfs = 138; } elsif ($machine eq 'sparc64') { $SYS_epoll_create = 193; $SYS_epoll_ctl = 194; @@ -116,6 +120,7 @@ if ($^O eq "linux") { $SYS_signalfd4 = 317; $SYS_renameat2 //= 345; $SFD_CLOEXEC = 020000000; + $SYS_fstatfs = 158; } elsif ($machine =~ m/^parisc/) { $SYS_epoll_create = 224; $SYS_epoll_ctl = 225; @@ -129,6 +134,7 @@ if ($^O eq "linux") { $u64_mod_8 = 1; $SYS_signalfd4 = 313; $SYS_renameat2 //= 357; + $SYS_fstatfs = 100; } elsif ($machine eq "ppc") { $SYS_epoll_create = 236; $SYS_epoll_ctl = 237; @@ -136,6 +142,7 @@ if ($^O eq "linux") { $u64_mod_8 = 1; $SYS_signalfd4 = 313; $SYS_renameat2 //= 357; + $SYS_fstatfs = 100; } elsif ($machine =~ m/^s390/) { $SYS_epoll_create = 249; $SYS_epoll_ctl = 250; @@ -143,6 +150,7 @@ if ($^O eq "linux") { $u64_mod_8 = 1; $SYS_signalfd4 = 322; $SYS_renameat2 //= 347; + $SYS_fstatfs = 100; } elsif ($machine eq "ia64") { $SYS_epoll_create = 1243; $SYS_epoll_ctl = 1244; @@ -165,6 +173,7 @@ if ($^O eq "linux") { $no_deprecated = 1; $SYS_signalfd4 = 74; $SYS_renameat2 //= 276; + $SYS_fstatfs = 44; } elsif ($machine =~ m/arm(v\d+)?.*l/) { # ARM OABI $SYS_epoll_create = 250; @@ -173,6 +182,7 @@ if ($^O eq "linux") { $u64_mod_8 = 1; $SYS_signalfd4 = 355; $SYS_renameat2 //= 382; + $SYS_fstatfs = 100; } elsif ($machine =~ m/^mips64/) { $SYS_epoll_create = 5207; $SYS_epoll_ctl = 5208; @@ -180,6 +190,7 @@ if ($^O eq "linux") { $u64_mod_8 = 1; $SYS_signalfd4 = 5283; $SYS_renameat2 //= 5311; + $SYS_fstatfs = 5135; } elsif ($machine =~ m/^mips/) { $SYS_epoll_create = 4248; $SYS_epoll_ctl = 4249; @@ -187,6 +198,7 @@ if ($^O eq "linux") { $u64_mod_8 = 1; $SYS_signalfd4 = 4324; $SYS_renameat2 //= 4351; + $SYS_fstatfs = 4100; } else { # as a last resort, try using the *.ph files which may not # exist or may be wrong @@ -323,6 +335,38 @@ sub rename_noreplace ($$) { } } +sub nodatacow_fh { + return if !defined($SYS_fstatfs); + my $buf = ''; + vec($buf, 120 * 8 - 1, 1) = 0; + my ($fh) = @_; + syscall($SYS_fstatfs, fileno($fh), $buf) == 0 or + return warn("fstatfs: $!\n"); + my $f_type = unpack('l!', $buf); # statfs.f_type is a signed word + return if $f_type != 0x9123683E; # BTRFS_SUPER_MAGIC + + state ($FS_IOC_GETFLAGS, $FS_IOC_SETFLAGS); + unless (defined $FS_IOC_GETFLAGS) { + if (substr($Config{byteorder}, 0, 4) eq '1234') { + $FS_IOC_GETFLAGS = 0x80086601; + $FS_IOC_SETFLAGS = 0x40086602; + } else { # Big endian + $FS_IOC_GETFLAGS = 0x40086601; + $FS_IOC_SETFLAGS = 0x80086602; + } + } + ioctl($fh, $FS_IOC_GETFLAGS, $buf) // + return warn("FS_IOC_GET_FLAGS: $!\n"); + my $attr = unpack('l!', $buf); + return if ($attr & 0x00800000); # FS_NOCOW_FL; + ioctl($fh, $FS_IOC_SETFLAGS, pack('l', $attr | 0x00800000)) // + warn("FS_IOC_SET_FLAGS: $!\n"); +} + +sub nodatacow_dir { + if (open my $fh, '<', $_[0]) { nodatacow_fh($fh) } +} + 1; =head1 WARRANTY diff --git a/lib/PublicInbox/Xapcmd.pm b/lib/PublicInbox/Xapcmd.pm index 44e0f8e5..10685636 100644 --- a/lib/PublicInbox/Xapcmd.pm +++ b/lib/PublicInbox/Xapcmd.pm @@ -1,8 +1,9 @@ -# Copyright (C) 2018-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> package PublicInbox::Xapcmd; use strict; -use PublicInbox::Spawn qw(which popen_rd nodatacow_dir); +use PublicInbox::Spawn qw(which popen_rd); +use PublicInbox::Syscall; use PublicInbox::Admin qw(setup_signals); use PublicInbox::Over; use PublicInbox::SearchIdx; @@ -211,7 +212,7 @@ sub prepare_run { my $v = PublicInbox::Search::SCHEMA_VERSION(); my $wip = File::Temp->newdir("xapian$v-XXXX", DIR => $dir); $tmp->{$old} = $wip; - nodatacow_dir($wip->dirname); + PublicInbox::Syscall::nodatacow_dir($wip->dirname); push @queue, [ $old, $wip ]; } elsif (defined $old) { opendir my $dh, $old or die "Failed to opendir $old: $!\n"; @@ -242,7 +243,7 @@ sub prepare_run { same_fs_or_die($old, $wip->dirname); my $cur = "$old/$dn"; push @queue, [ $src // $cur , $wip ]; - nodatacow_dir($wip->dirname); + PublicInbox::Syscall::nodatacow_dir($wip->dirname); $tmp->{$cur} = $wip; } # mark old shards to be unlinked @@ -443,7 +444,7 @@ sub cpdb ($$) { # cb_spawn callback $ft = File::Temp->newdir("$new.compact-XXXX", DIR => $dir); setup_signals(); $tmp = $ft->dirname; - nodatacow_dir($tmp); + PublicInbox::Syscall::nodatacow_dir($tmp); } else { $tmp = $new; } diff --git a/t/nodatacow.t b/t/nodatacow.t index 19247c10..83aa227f 100644 --- a/t/nodatacow.t +++ b/t/nodatacow.t @@ -1,48 +1,41 @@ #!perl -w -# Copyright (C) 2020-2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> use strict; use v5.10.1; use PublicInbox::TestCommon; use File::Temp 0.19; -use_ok 'PublicInbox::NDC_PP'; +use_ok 'PublicInbox::Syscall'; + +# btrfs on Linux is copy-on-write (COW) by default. As of Linux 5.7, +# this still leads to fragmentation for SQLite and Xapian files where +# random I/O happens, so we disable COW just for SQLite files and Xapian +# directories. Disabling COW disables checksumming, so we only do this +# for regeneratable files, and not canonical git storage (git doesn't +# checksum refs, only data under $GIT_DIR/objects). SKIP: { my $nr = 2; skip 'test is Linux-only', $nr if $^O ne 'linux'; my $dir = $ENV{BTRFS_TESTDIR}; skip 'BTRFS_TESTDIR not defined', $nr unless defined $dir; - require_cmd('chattr', 1) or skip 'chattr(1) not installed', $nr; + my $lsattr = require_cmd('lsattr', 1) or skip 'lsattr(1) not installed', $nr; + my $tmp = File::Temp->newdir('nodatacow-XXXX', DIR => $dir); my $dn = $tmp->dirname; my $name = "$dn/pp.f"; open my $fh, '>', $name or BAIL_OUT "open($name): $!"; - my $pp_sub = \&PublicInbox::NDC_PP::nodatacow_fd; - $pp_sub->(fileno($fh)); + PublicInbox::Syscall::nodatacow_fh($fh); my $res = xqx([$lsattr, $name]); like($res, qr/C.*\Q$name\E/, "`C' attribute set on fd with pure Perl"); + $name = "$dn/pp.d"; mkdir($name) or BAIL_OUT "mkdir($name) $!"; - PublicInbox::NDC_PP::nodatacow_dir($name); + PublicInbox::Syscall::nodatacow_dir($name); $res = xqx([$lsattr, '-d', $name]); like($res, qr/C.*\Q$name\E/, "`C' attribute set on dir with pure Perl"); - - $name = "$dn/ic.f"; - my $ic_sub = \&PublicInbox::Spawn::nodatacow_fd; - $pp_sub == $ic_sub and - skip 'Inline::C or Linux kernel headers missing', 2; - open $fh, '>', $name or BAIL_OUT "open($name): $!"; - $ic_sub->(fileno($fh)); - $res = xqx([$lsattr, $name]); - like($res, qr/C.*\Q$name\E/, "`C' attribute set on fd with Inline::C"); - - $name = "$dn/ic.d"; - mkdir($name) or BAIL_OUT "mkdir($name) $!"; - PublicInbox::Spawn::nodatacow_dir($name); - $res = xqx([$lsattr, '-d', $name]); - like($res, qr/C.*\Q$name\E/, "`C' attribute set on dir with Inline::C"); }; done_testing; ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] 2022-01-30 21:49 ` Eric Wong @ 2022-01-30 23:18 ` Dominique Martinet 2022-01-31 2:03 ` Eric Wong 0 siblings, 1 reply; 31+ messages in thread From: Dominique Martinet @ 2022-01-30 23:18 UTC (permalink / raw) To: Eric Wong; +Cc: Julien Moutinho, meta Eric Wong wrote on Sun, Jan 30, 2022 at 09:49:08PM +0000: > Thanks for testing the previous patch. Actually, I prefer we > drop previous implementations and instead rely on Linux ABI > stability while shifting maintenance burden to maintainers. This makes sense. > Can you test the patch below? It supercedes the other one. I've tested the patch with a btrfs test directory and this test: BTRFS_TESTDIR=... prove -bvw t/nodatacow.t after adding the local git repo to PERL5LIBs With a BTRFS dir as BTRFS it all works, the ioctl are passed and files are indeed nocow (checked with lsattr afterwards as files aren't deleted) With a tmpfs instead the tests fail as it tried to call the ioctl anyway -- I didn't take the time to check if thest mtab parsing was removed, is the ioctl tried anyway or does setting the environment variable skip this part? If so then it's probably fine. (the failure is as below: ------ $ BTRFS_TESTDIR=/tmp/test strace -o /tmp/strace -f prove -bvw t/nodatacow.t t/nodatacow.t .. ok 1 - use PublicInbox::Syscall; /run/current-system/sw/bin/lsattr: Operation not supported While reading flags on /tmp/test/nodatacow-rfqh/pp.f not ok 2 - `C' attribute set on fd with pure Perl # Failed test '`C' attribute set on fd with pure Perl' # at t/nodatacow.t line 31. # '' # doesn't match '(?^:C.*\/tmp\/test\/nodatacow\-rfqh\/pp\.f)' /run/current-system/sw/bin/lsattr: Operation not supported While reading flags on /tmp/test/nodatacow-rfqh/pp.d not ok 3 - `C' attribute set on dir with pure Perl # Failed test '`C' attribute set on dir with pure Perl' # at t/nodatacow.t line 38. # '' # doesn't match '(?^:C.*\/tmp\/test\/nodatacow\-rfqh\/pp\.d)' 1..3 # Looks like you failed 2 tests of 3. Dubious, test returned 2 (wstat 512, 0x200) Failed 2/3 subtests Test Summary Report ------------------- t/nodatacow.t (Wstat: 512 Tests: 3 Failed: 2) Failed tests: 2-3 Non-zero exit status: 2 Files=1, Tests=3, 1 wallclock secs ( 0.04 usr 0.08 sys + 0.15 cusr 0.21 csys = 0.48 CPU) Result: FAIL -------- > Apologies for the delay, I've pretty much lost all motivation for > everything in life :< Will try to find some energy to look into the > other issues in a bit... No worry, one step at a time is good :) Wish you the best, -- Dominique ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] 2022-01-30 23:18 ` Dominique Martinet @ 2022-01-31 2:03 ` Eric Wong 2022-01-31 3:34 ` Dominique Martinet 0 siblings, 1 reply; 31+ messages in thread From: Eric Wong @ 2022-01-31 2:03 UTC (permalink / raw) To: Dominique Martinet; +Cc: Julien Moutinho, meta Dominique Martinet <asmadeus@codewreck.org> wrote: > Eric Wong wrote on Sun, Jan 30, 2022 at 09:49:08PM +0000: > > Thanks for testing the previous patch. Actually, I prefer we > > drop previous implementations and instead rely on Linux ABI > > stability while shifting maintenance burden to maintainers. > > This makes sense. > > > Can you test the patch below? It supercedes the other one. > > I've tested the patch with a btrfs test directory and this test: > BTRFS_TESTDIR=... prove -bvw t/nodatacow.t > > after adding the local git repo to PERL5LIBs > > With a BTRFS dir as BTRFS it all works, the ioctl are passed and files > are indeed nocow (checked with lsattr afterwards as files aren't > deleted) > > With a tmpfs instead the tests fail as it tried to call the ioctl > anyway -- I didn't take the time to check if thest mtab parsing was > removed, is the ioctl tried anyway or does setting the environment > variable skip this part? > If so then it's probably fine. > > (the failure is as below: > ------ > $ BTRFS_TESTDIR=/tmp/test strace -o /tmp/strace -f prove -bvw t/nodatacow.t > t/nodatacow.t .. > ok 1 - use PublicInbox::Syscall; > /run/current-system/sw/bin/lsattr: Operation not supported While reading flags on /tmp/test/nodatacow-rfqh/pp.f > not ok 2 - `C' attribute set on fd with pure Perl > > # Failed test '`C' attribute set on fd with pure Perl' > # at t/nodatacow.t line 31. > # '' > # doesn't match '(?^:C.*\/tmp\/test\/nodatacow\-rfqh\/pp\.f)' > /run/current-system/sw/bin/lsattr: Operation not supported While reading flags on /tmp/test/nodatacow-rfqh/pp.d Ah, intentionally setting BTRFS_TESTDIR to something that isn't btrfs will break, yes. I suppose an explicit BAIL_OUT is in order, here: diff --git a/t/nodatacow.t b/t/nodatacow.t index 83aa227f..0940d908 100644 --- a/t/nodatacow.t +++ b/t/nodatacow.t @@ -28,8 +28,9 @@ SKIP: { open my $fh, '>', $name or BAIL_OUT "open($name): $!"; PublicInbox::Syscall::nodatacow_fh($fh); my $res = xqx([$lsattr, $name]); - like($res, qr/C.*\Q$name\E/, "`C' attribute set on fd with pure Perl"); + BAIL_OUT "lsattr(1) fails in $dir" if $?; + like($res, qr/C.*\Q$name\E/, "`C' attribute set on fd with pure Perl"); $name = "$dn/pp.d"; mkdir($name) or BAIL_OUT "mkdir($name) $!"; ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] 2022-01-31 2:03 ` Eric Wong @ 2022-01-31 3:34 ` Dominique Martinet 2022-02-01 1:27 ` Eric Wong 0 siblings, 1 reply; 31+ messages in thread From: Dominique Martinet @ 2022-01-31 3:34 UTC (permalink / raw) To: Eric Wong; +Cc: Julien Moutinho, meta Eric Wong wrote on Mon, Jan 31, 2022 at 02:03:11AM +0000: > Ah, intentionally setting BTRFS_TESTDIR to something that isn't > btrfs will break, yes. Ok, so this is specific to the test. Checking now nodatacow_fh skips files with non-btrfs magic, so it looks good to me! I've taken a look at the code now, just one more question: I don't understand why you've made the ioctl value depend on endianness ? get and set are respectively _IOR('f', 1, long) and _IOW('f', 2, long) which translate to (2 << 30) | (8 << 16) | ('f' << 8) | 1 (1 << 30) | (8 << 16) | ('f' << 8) | 2 so should be 0x80086601 0x40086602 regardless of endianness ? as far as I understand the memory representation might change but values themselves are handled the same, and I don't see anything in the defines that'd swap just the R/W bits around, but then again I don't have any big endian machine around and never really played with one. That aside the code looks good to me, if you do Reviewed-by tags feel free to add mine (Dominique Martinet <asmadeus@codewreck.org>) once that question is answered. If you don't care, I don't care either :) > I suppose an explicit BAIL_OUT is in order, here: Well, tmpfs does not have attrs so lsattr fails and this triggers this bailout (this would also "work" for nixos tests which likely don't have lsattr available), but for example ext4 and xfs have working lsattr so we get another failure: ========= $ BTRFS_TESTDIR=/boot0/test strace -o /tmp/strace -f prove -bvw t/nodatacow.t t/nodatacow.t .. ok 1 - use PublicInbox::Syscall; not ok 2 - `C' attribute set on fd with pure Perl # Failed test '`C' attribute set on fd with pure Perl' # at t/nodatacow.t line 33. # '--------------e------- /boot0/test/nodatacow-80SC/pp.f # ' # doesn't match '(?^:C.*\/boot0\/test\/nodatacow\-80SC\/pp\.f)' not ok 3 - `C' attribute set on dir with pure Perl ========= But anyway, the test itself is skipped unless BTRFS_TESTDIR is set so this isn't really a problem, I've only tried because I'm a monkey :) Thanks again for the support, don't hesitate to ask if you need further info or tests for the zfs problems. -- Dominique ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] 2022-01-31 3:34 ` Dominique Martinet @ 2022-02-01 1:27 ` Eric Wong 0 siblings, 0 replies; 31+ messages in thread From: Eric Wong @ 2022-02-01 1:27 UTC (permalink / raw) To: Dominique Martinet; +Cc: Julien Moutinho, meta Dominique Martinet <asmadeus@codewreck.org> wrote: > Eric Wong wrote on Mon, Jan 31, 2022 at 02:03:11AM +0000: > > Ah, intentionally setting BTRFS_TESTDIR to something that isn't > > btrfs will break, yes. > > Ok, so this is specific to the test. > Checking now nodatacow_fh skips files with non-btrfs magic, so it looks > good to me! > > I've taken a look at the code now, just one more question: I don't > understand why you've made the ioctl value depend on endianness ? Actually, it's not endianess, it's architecture specific :x The nasty patch below fixes it. I really wish there are better options for scripting languages to use C headers :< > That aside the code looks good to me, if you do Reviewed-by tags feel > free to add mine (Dominique Martinet <asmadeus@codewreck.org>) once that > question is answered. > If you don't care, I don't care either :) Thanks, just added the Noticed-by: as an attribution to the below patch. > this isn't really a problem, I've only tried because I'm a monkey :) Yeah. Setting BTRFS_TESTDIR to a non-btrfs dir isn't going to be supported :> > Thanks again for the support, don't hesitate to ask if you need further > info or tests for the zfs problems. You're welcome, and will do. -----8<----- Subject: [PATCH] syscall: FS_IOC_*FLAGS: define on per-architecture basis It turns out these Linux ioctls are unfortunately architecture-dependent, and not endian-dependent. Fixup some warning messages while we're at it, too. Fixes: 14fa0abdcc7b6513 ("rewrite Linux nodatacow use in pure Perl w/o system") Link: https://public-inbox.org/meta/YfdYqLhDVQRQ9NGT@codewreck.org/ Noticed-by: Dominique Martinet <asmadeus@codewreck.org> --- lib/PublicInbox/Syscall.pm | 35 +++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 12 deletions(-) diff --git a/lib/PublicInbox/Syscall.pm b/lib/PublicInbox/Syscall.pm index bcfae2cb..31c67a14 100644 --- a/lib/PublicInbox/Syscall.pm +++ b/lib/PublicInbox/Syscall.pm @@ -69,6 +69,7 @@ our ( ); my $SYS_fstatfs; # don't need fstatfs64, just statfs.f_type +my ($FS_IOC_GETFLAGS, $FS_IOC_SETFLAGS); my $SFD_CLOEXEC = 02000000; # Perl does not expose O_CLOEXEC our $no_deprecated = 0; @@ -98,6 +99,8 @@ if ($^O eq "linux") { $SYS_signalfd4 = 327; $SYS_renameat2 //= 353; $SYS_fstatfs = 100; + $FS_IOC_GETFLAGS = 0x80046601; + $FS_IOC_SETFLAGS = 0x40046602; } elsif ($machine eq "x86_64") { $SYS_epoll_create = 213; $SYS_epoll_ctl = 233; @@ -105,6 +108,8 @@ if ($^O eq "linux") { $SYS_signalfd4 = 289; $SYS_renameat2 //= 316; $SYS_fstatfs = 138; + $FS_IOC_GETFLAGS = 0x80086601; + $FS_IOC_SETFLAGS = 0x40086602; } elsif ($machine eq 'x32') { $SYS_epoll_create = 1073742037; $SYS_epoll_ctl = 1073742057; @@ -112,6 +117,8 @@ if ($^O eq "linux") { $SYS_signalfd4 = 1073742113; $SYS_renameat2 //= 0x40000000 + 316; $SYS_fstatfs = 138; + $FS_IOC_GETFLAGS = 0x80046601; + $FS_IOC_SETFLAGS = 0x40046602; } elsif ($machine eq 'sparc64') { $SYS_epoll_create = 193; $SYS_epoll_ctl = 194; @@ -121,6 +128,8 @@ if ($^O eq "linux") { $SYS_renameat2 //= 345; $SFD_CLOEXEC = 020000000; $SYS_fstatfs = 158; + $FS_IOC_GETFLAGS = 0x40086601; + $FS_IOC_SETFLAGS = 0x80086602; } elsif ($machine =~ m/^parisc/) { $SYS_epoll_create = 224; $SYS_epoll_ctl = 225; @@ -135,6 +144,8 @@ if ($^O eq "linux") { $SYS_signalfd4 = 313; $SYS_renameat2 //= 357; $SYS_fstatfs = 100; + $FS_IOC_GETFLAGS = 0x40086601; + $FS_IOC_SETFLAGS = 0x80086602; } elsif ($machine eq "ppc") { $SYS_epoll_create = 236; $SYS_epoll_ctl = 237; @@ -143,6 +154,8 @@ if ($^O eq "linux") { $SYS_signalfd4 = 313; $SYS_renameat2 //= 357; $SYS_fstatfs = 100; + $FS_IOC_GETFLAGS = 0x40086601; + $FS_IOC_SETFLAGS = 0x80086602; } elsif ($machine =~ m/^s390/) { $SYS_epoll_create = 249; $SYS_epoll_ctl = 250; @@ -174,6 +187,8 @@ if ($^O eq "linux") { $SYS_signalfd4 = 74; $SYS_renameat2 //= 276; $SYS_fstatfs = 44; + $FS_IOC_GETFLAGS = 0x80086601; + $FS_IOC_SETFLAGS = 0x40086602; } elsif ($machine =~ m/arm(v\d+)?.*l/) { # ARM OABI $SYS_epoll_create = 250; @@ -191,6 +206,8 @@ if ($^O eq "linux") { $SYS_signalfd4 = 5283; $SYS_renameat2 //= 5311; $SYS_fstatfs = 5135; + $FS_IOC_GETFLAGS = 0x40046601; + $FS_IOC_SETFLAGS = 0x80046602; } elsif ($machine =~ m/^mips/) { $SYS_epoll_create = 4248; $SYS_epoll_ctl = 4249; @@ -199,6 +216,8 @@ if ($^O eq "linux") { $SYS_signalfd4 = 4324; $SYS_renameat2 //= 4351; $SYS_fstatfs = 4100; + $FS_IOC_GETFLAGS = 0x40046601; + $FS_IOC_SETFLAGS = 0x80046602; } else { # as a last resort, try using the *.ph files which may not # exist or may be wrong @@ -345,22 +364,14 @@ sub nodatacow_fh { my $f_type = unpack('l!', $buf); # statfs.f_type is a signed word return if $f_type != 0x9123683E; # BTRFS_SUPER_MAGIC - state ($FS_IOC_GETFLAGS, $FS_IOC_SETFLAGS); - unless (defined $FS_IOC_GETFLAGS) { - if (substr($Config{byteorder}, 0, 4) eq '1234') { - $FS_IOC_GETFLAGS = 0x80086601; - $FS_IOC_SETFLAGS = 0x40086602; - } else { # Big endian - $FS_IOC_GETFLAGS = 0x40086601; - $FS_IOC_SETFLAGS = 0x80086602; - } - } + $FS_IOC_GETFLAGS // + return warn('FS_IOC_GETFLAGS undefined for platform'); ioctl($fh, $FS_IOC_GETFLAGS, $buf) // - return warn("FS_IOC_GET_FLAGS: $!\n"); + return warn("FS_IOC_GETFLAGS: $!\n"); my $attr = unpack('l!', $buf); return if ($attr & 0x00800000); # FS_NOCOW_FL; ioctl($fh, $FS_IOC_SETFLAGS, pack('l', $attr | 0x00800000)) // - warn("FS_IOC_SET_FLAGS: $!\n"); + warn("FS_IOC_SETFLAGS: $!\n"); } sub nodatacow_dir { ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: Test failures with 1.7.0 2021-12-08 10:56 ` Dominique Martinet 2021-12-08 18:22 ` [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] Eric Wong @ 2021-12-09 1:37 ` Julien Moutinho 2021-12-09 2:53 ` Dominique Martinet ` (3 more replies) 1 sibling, 4 replies; 31+ messages in thread From: Julien Moutinho @ 2021-12-09 1:37 UTC (permalink / raw) To: Dominique Martinet; +Cc: Eric Wong, meta Without Inline::C, and without Nix's sandbox -------------------------------------------- I get 3 tests to fail (in addition to those currently disabled in Nixpkgs: https://github.com/NixOS/nixpkgs/pull/104457/files#diff-77474340b38081fea7ce3b112c1bc060d3a95cc5e781915463e2f3fc5556da7bR34-R69 ie. convert-compact, search, v2writable, www_listing, eml, hl_mod, lei-mirror and spawn): $ nix develop -f . public-inbox $ doCheck= bash pkgs/development/perl-modules/generic/builder.sh $ make -C public-inbox-1.7.0 test > [...] > Test Summary Report > ------------------- > t/nntpd-v2.t (Wstat: 256 Tests: 113 Failed: 1) > Failed test: 102 > Non-zero exit status: 1 > t/nntpd.t (Wstat: 256 Tests: 110 Failed: 1) > Failed test: 99 > Non-zero exit status: 1 > t/solver_git.t (Wstat: 1280 Tests: 13 Failed: 5) > Failed tests: 9-13 > Non-zero exit status: 5 > Parse errors: No plan found in TAP output > Files=149, Tests=4658, 296 wallclock secs ( 1.78 usr 0.56 sys + 116.73 cusr 165.82 csys = 284.89 CPU) > Result: FAIL > Failed 3/149 test programs. 7/4658 subtests failed. More precisely: $ (cd public-inbox-1.7.0; perl -I$out/lib/perl5/site_perl t/nntpd-v2.t ) > [...] > ok 101 - new article retrieved after compact > not ok 102 - no deleted files > # Failed test 'no deleted files' > # at ./t/nntpd.t line 363. > # Structures begin differing at: > # $got->[0] = '-nntpd 3130714 julm 4u REG 0,53 77189 2606550 /tmp/nix-shell.D0VGvX (deleted) > # ' > # $expected->[0] = Does not exist > # [ > # '-nntpd 3130714 julm 4u REG 0,53 77189 2606550 /tmp/nix-shell.D0VGvX (deleted) > # ' > # ] $ (cd public-inbox-1.7.0; perl -I$out/lib/perl5/site_perl t/nntpd.t ) > [...] > not ok 99 - no deleted files > # Failed test 'no deleted files' > # at t/nntpd.t line 363. > # Structures begin differing at: > # $got->[0] = '-nntpd 3131133 julm 4u REG 0,53 77189 2606550 /tmp/nix-shell.D0VGvX (deleted) > # ' > # $expected->[0] = Does not exist > # [ > # '-nntpd 3131133 julm 4u REG 0,53 77189 2606550 /tmp/nix-shell.D0VGvX (deleted) > # ' > # ] $ (cd public-inbox-1.7.0; perl -I$out/lib/perl5/site_perl t/solver_git.t ) > ok 1 - use PublicInbox::Inbox; > ok 2 - use PublicInbox::V2Writable; > ok 3 - use PublicInbox::Git; > ok 4 - use PublicInbox::SolverGit; > ok 5 - use PublicInbox::WWW; > ok 6 # skip Inline::C unconfigured/missing (mkdir -p ~/.cache/public-inbox/inline-c) OR Socket::MsgHdr missing > ok 7 # skip Inline::C unconfigured/missing (mkdir -p ~/.cache/public-inbox/inline-c) OR Socket::MsgHdr missing > ok 8 # skip Inline::C unconfigured/missing (mkdir -p ~/.cache/public-inbox/inline-c) OR Socket::MsgHdr missing > not ok 9 - solved a blob! > # Failed test 'solved a blob!' > # at t/solver_git.t line 151. > not ok 10 - got a git object for the blob > # Failed test 'got a git object for the blob' > # at t/solver_git.t line 153. > # got: '' > # expected: 'PublicInbox::Git' > not ok 11 - resolved blob to unabbreviated identifier > # Failed test 'resolved blob to unabbreviated identifier' > # at t/solver_git.t line 154. > # got: undef > # expected: '69df7d565d49fbaaeb0a067910f03dc22cd52bd0' > not ok 12 - type specified > # Failed test 'type specified' > # at t/solver_git.t line 155. > # got: undef > # expected: 'blob' > not ok 13 - size returned > # Failed test 'size returned' > # at t/solver_git.t line 156. > # got: undef > # expected: '4405' > Can't call method "cat_file" on an undefined value at t/solver_git.t line 158. > # Tests were run but no plan was declared and done_testing() was not seen. I can also reproduce Infinisil's test failure with: $ (cd public-inbox-1.7.0; TMPDIR=/var/tmp perl -I$out/lib/perl5/site_perl t/lei_to_mail.t ) > ok 96 - got Maildir callback > Use of uninitialized value in open at t/lei_to_mail.t line 263. > Bail out! No such file or directory $ mount | grep 'on /var/tmp' > losurdo/var/tmp on /var/tmp type zfs (rw,relatime,xattr,posixacl) With Inline::C, and without Nix's sandbox ----------------------------------------- I get one more test to fail: $ export PERL_INLINE_DIRECTORY=$PWD/inline-c; rm -rf $PERL_INLINE_DIRECTORY; mkdir $PERL_INLINE_DIRECTORY $ make -C public-inbox-1.7.0 test > [...] > Test Summary Report > ------------------- > t/lei-q-save.t (Wstat: 512 Tests: 108 Failed: 2) > Failed tests: 95, 97 > Non-zero exit status: 2 More precisely: $ (cd public-inbox-1.7.0; perl -I$out/lib/perl5/site_perl t/lei-q-save.t ) > [...] > ok 93 - lei up $TMPDIR/lei-daemon/s (absolute lei up) > execve: No such file or directory > fork_exec blib/script/perl -i -p -e s/\[/\0/ /tmp/nix-shell.EKTZrS/pi-lei-q-save-3131696-HNqt/lei-daemon/.local/share/lei/saved-searches/after-af0b1d56d93b52910b7172f052d9b9e6c7b3411a060b50006e417c9aa83f5ce2/lei.saved-search failed: No such file or directory Which is an artifact of my testing method, because with an absolute perl: $ (cd public-inbox-1.7.0; $(which perl) -I$out/lib/perl5/site_perl t/lei-q-save.t ) > [...] > not ok 95 - git config parse error shown w/ lei ls-search > # Failed test 'git config parse error shown w/ lei ls-search' > # at t/lei-q-save.t line 228. > # 'fatal: ligne 3 de config incorrecte dans le fichier /tmp/nix-shell.EKTZrS/pi-lei-q-save-3132353-xECZ/lei-daemon/.local/share/lei/saved-searches/after-c8d60f185616bece7957076be4604781d6cae876642a4faa127a2fcfc9647fc8/lei.saved-search > # git config -z -l --includes --file=/tmp/nix-shell.EKTZrS/pi-lei-q-save-3132353-xECZ/lei-daemon/.cache/lei/saved-tmp.3132468.1639007508.config failed: $?=32768 > # ' > # doesn't match '(?^:bad config line.*?\/tmp\/nix\-shell\.EKTZrS\/pi\-lei\-q\-save\-3132353\-xECZ\/lei\-daemon\/\.local\/share\/lei\/saved\-searches\/after\-c8d60f185616bece7957076be4604781d6cae876642a4faa127a2fcfc9647fc8\/lei\.saved\-search)' > ok 96 - lei up --all (up works with bad config) > not ok 97 - git config parse error shown w/ lei up > # Failed test 'git config parse error shown w/ lei up' > # at t/lei-q-save.t line 231. > # 'fatal: ligne 3 de config incorrecte dans le fichier /tmp/nix-shell.EKTZrS/pi-lei-q-save-3132353-xECZ/lei-daemon/.local/share/lei/saved-searches/after-c8d60f185616bece7957076be4604781d6cae876642a4faa127a2fcfc9647fc8/lei.saved-search > # git config -z -l --includes --file=/tmp/nix-shell.EKTZrS/pi-lei-q-save-3132353-xECZ/lei-daemon/.cache/lei/saved-tmp.3132468.1639007508.config failed: $?=32768 > # ' > # doesn't match '(?^:bad config line.*?\/tmp\/nix\-shell\.EKTZrS\/pi\-lei\-q\-save\-3132353\-xECZ\/lei\-daemon\/\.local\/share\/lei\/saved\-searches\/after\-c8d60f185616bece7957076be4604781d6cae876642a4faa127a2fcfc9647fc8\/lei\.saved\-search)' It succeeds if I use LANG=C instead of LANG=fr_FR.UTF-8 With Inline::C, and with Nix's sandbox -------------------------------------- Using (in pkgs/servers/mail/public-inbox/default.nix): > preCheck = '' > perl certs/create-certs.perl > export HOME=$(mktemp -d) > mkdir -p $HOME/.cache/public-inbox/inline-c > ''; $ nix -L build -f . public-inbox > [...] > perl5.34.0-public-inbox> t/lei-refresh-mail-sync.t .... ok Hangs like forever after this line, while on t/lei-sigpipe.t: $ ps auxwwf > root 1755997 0.0 0.0 453172 2800 ? Ssl déc.04 0:03 nix-daemon --daemon > root 1811429 0.0 0.0 453652 1928 ? Ssl déc.04 0:00 \_ nix-daemon 1757028 > root 1811455 0.0 0.0 453652 2004 ? Ssl déc.04 0:00 \_ nix-daemon 1757031 > root 1839837 0.0 0.0 453652 1832 ? Ssl déc.04 0:00 \_ nix-daemon 1757027 > root 1839841 0.0 0.0 453652 1996 ? Ssl déc.04 0:00 \_ nix-daemon 1757030 > root 1839856 0.0 0.0 453652 1832 ? Ssl déc.04 0:00 \_ nix-daemon 1757029 > root 3135741 0.0 0.2 603352 17080 ? Ssl 01:05 0:00 \_ nix-daemon 3135672 > nixbld1 3137110 0.0 0.0 5200 4072 ? Ss 01:05 0:01 \_ bash -e /nix/store/qf3mzpvsmkrw963xchbivcci06078n13-builder.sh > nixbld1 3140619 0.0 0.0 4996 4012 ? S 01:05 0:00 \_ make SHELL=/nix/store/l0wlqpbsvh1pgvhcdhw7qkka3d31si7k-bash-5.1-p8/bin/bash VERBOSE=y test > nixbld1 3140633 0.0 0.1 17188 13352 ? S 01:05 0:01 | \_ /nix/store/vslsa0l17xjcrdgm2knwj0z5hlvf73m7-perl-5.34.0/bin/perl -MExtUtils::Command::MM -MTest::Harness -e undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch') t/admin.t t/altid.t t/altid_v2.t t/cgi.t t/cmd_ipc.t t/content_hash.t t/dir_idle.t t/edit.t t/eml_content_disposition.t t/eml_content_type.t t/epoll.t t/extindex-psgi.t t/extsearch.t t/fake_inotify.t t/feed.t t/gcf2.t t/gcf2_client.t t/httpd.t t/idx_stack.t t/imap.t t/imap_searchqp.t t/imapd-tls.t t/imapd.t t/inbox_idle.t t/index-git-times.t t/indexlevels-mirror.t t/ipc.t t/kqnotify.t t/lei-auto-watch.t t/lei-convert.t t/lei-daemon.t t/lei-export-kw.t t/lei-externals.t t/lei-import-http.t t/lei-import-imap.t t/lei-import-maildir.t t/lei-import-nntp.t t/lei-import.t t/lei-index.t t/lei-inspect.t t/lei-lcat.t t/lei-p2q.t t/lei-q-kw.t t/lei-q-remote-import.t t/lei-q-save.t t/lei-q-thread.t t/lei-refresh-mail-sync.t t/lei-sigpipe.t t/lei-tag.t t/lei-up.t t/lei-watch.t t/lei.t t/lei_dedupe.t t/lei_external.t t/lei_lcat.t t/lei_mail_sync.t t/lei_overview.t t/lei_saved_search.t t/lei_store.t t/lei_to_mail.t t/lei_xsearch.t t/mbox_lock.t t/mbox_reader.t t/mdir_reader.t t/mime.t t/miscsearch.t t/net_reader-imap.t t/nntpd-tls.t t/nntpd.t t/nodatacow.t t/on_destroy.t t/plack.t t/psgi_attach.t t/psgi_bad_mids.t t/psgi_mount.t t/psgi_multipart_not.t t/psgi_scan_all.t t/psgi_search.t t/psgi_v2.t t/rename_noreplace.t t/reply.t t/search-thr-index.t t/shared_kv.t t/solver_git.t t/thread-index-gap.t t/uri_imap.t t/uri_nntps.t t/v2dupindex.t t/www_altid.t t/xcpdb-reshard.t t/www_static.t t/watch_multiple_headers.t t/watch_maildir_v2.t t/watch_maildir.t t/watch_imap.t t/watch_filter_rubylang.t t/view.t t/v2reindex.t t/v2mirror.t t/v2mda.t t/v2index-late-dupe.t t/v2-add-remove-add.t t/v1reindex.t t/v1-add-remove-add.t t/time.t t/thread-cycle.t t/spamcheck_spamc.t t/sigfd.t t/replace.t t/reindex-time-range.t t/qspawn.t t/purge.t t/psgi_text.t t/precheck.t t/over.t t/nulsubject.t t/nntpd-v2.t t/nntp.t t/multi-mid.t t/msgtime.t t/msgmap.t t/msg_iter.t t/mid.t t/mda_filter_rubylang.t t/mda.t t/linkify.t t/init.t t/indexlevels-mirror-v1.t t/inbox.t t/import.t t/imap_tracker.t t/hval.t t/httpd-unix.t t/httpd-https.t t/httpd-corner.t t/gzip_filter.t t/git.t t/filter_vger.t t/filter_subjecttag.t t/filter_rubylang.t t/filter_mirror.t t/filter_base.t t/emergency.t t/ds-poll.t t/ds-leak.t t/ds-kqxs.t t/config_limiter.t t/config.t t/address.t > nixbld1 3145719 0.0 0.4 44576 35640 ? S 01:09 0:01 | \_ t/lei-sigpipe.t > nixbld1 3145743 0.0 0.1 14388 10388 ? S 01:09 0:00 | \_ /nix/store/vslsa0l17xjcrdgm2knwj0z5hlvf73m7-perl-5.34.0/bin/perl -w -I/nix/store/vslsa0l17xjcrdgm2knwj0z5hlvf73m7-perl-5.34.0/lib/perl5/site_perl -I/nix/store/1hdx7bxjwqrfnalalkgbwi32l45h8z7b-perl5.34.0-Mail-IMAPClient-3.42/lib/perl5/site_perl -I/nix/st blib/script/lei q -q -t z:1.. > nixbld1 3145724 0.0 0.5 50504 42160 ? S 01:09 0:00 \_ lei-daemon /build/pi-lei-sigpipe-7522-WoHO/lei-daemon/xdg_run/lei/5.seq.sock > nixbld1 3145730 0.0 0.4 50092 39736 ? S 01:09 0:00 \_ lei/store /build/pi-lei-sigpipe-7522-WoHO/lei-daemon/.local/share t/lei-sigpipe.t is on: > 01:30:29.508334 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7fffffff5090) = 0 <1.000360> > 01:30:30.509837 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7fffffff5090) = 0 <1.000186> > 01:30:31.510259 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7fffffff5090) = 0 <1.000175> > [...] lei-daemon store is on: > 01:31:03.834708 epoll_wait(5, Looks like it is waiting on sysread() in https://public-inbox.org/public-inbox.git/tree/t/lei-sigpipe.t#n35 This test does succeed outside Nix's sandbox: $ (cd public-inbox-1.7.0; export PERL_INLINE_DIRECTORY=$PWD/inline-c; rm -rf $PERL_INLINE_DIRECTORY; mkdir $PERL_INLINE_DIRECTORY; prove -bvw t/lei-sigpipe.t ) > t/lei-sigpipe.t .. > ok 1 - lei import $TMPDIR/lei-daemon/big.eml > ok 2 - read one byte > ok 3 - signaled > ok 4 - got SIGPIPE > ok 5 - quiet after sigpipe > ok 6 - read one byte > ok 7 - signaled -f mboxcl2 > ok 8 - got SIGPIPE -f mboxcl2 > ok 9 - quiet after sigpipe -f mboxcl2 > ok 10 - read one byte > ok 11 - signaled -f text > ok 12 - got SIGPIPE -f text > ok 13 - quiet after sigpipe -f text > ok 14 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:44) > ok 15 - daemon running after t/lei-sigpipe.t:44 > ok 16 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:44) > ok 17 - t/lei-sigpipe.t:44 daemon stopped > ok 18 - t/lei-sigpipe.t:44 daemon XDG_RUNTIME_DIR/lei/errors.log empty > 1..18 > ok > All tests successful. > Files=1, Tests=18, 7 wallclock secs ( 0.06 usr 0.06 sys + 3.44 cusr 2.73 csys = 6.29 CPU) > Result: PASS More surprisingly, it even succeeds when run manually inside the hanging Nix sandbox: $ sudo nsenter --target 3137110 --all -S 1000 -G 100 $(readlink -e $(which bash)) $ . /build/env-vars $ cd /build $ export HOME=$(mktemp -d) $ mkdir -p $HOME/.cache/public-inbox/inline-c $ LANG=C prove -bvw t/lei-sigpipe.t > t/lei-sigpipe.t .. > ok 1 - lei import $TMPDIR/lei-daemon/big.eml > ok 2 - read one byte > ok 3 - signaled > ok 4 - got SIGPIPE > ok 5 - quiet after sigpipe > ok 6 - read one byte > ok 7 - signaled -f mboxcl2 > ok 8 - got SIGPIPE -f mboxcl2 > ok 9 - quiet after sigpipe -f mboxcl2 > ok 10 - read one byte > ok 11 - signaled -f text > ok 12 - got SIGPIPE -f text > ok 13 - quiet after sigpipe -f text > ok 14 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:44) > ok 15 - daemon running after t/lei-sigpipe.t:44 > ok 16 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:44) > ok 17 - t/lei-sigpipe.t:44 daemon stopped > ok 18 - t/lei-sigpipe.t:44 daemon XDG_RUNTIME_DIR/lei/errors.log empty > 1..18 > ok > All tests successful. > Files=1, Tests=18, 4 wallclock secs ( 0.06 usr 0.06 sys + 1.23 cusr 1.48 csys = 2.83 CPU) > Result: PASS Even more strange, Dominique was able to reproduce the hang this morning, but no longer tonight.. Cheers, Julien ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Test failures with 1.7.0 2021-12-09 1:37 ` Test failures with 1.7.0 Julien Moutinho @ 2021-12-09 2:53 ` Dominique Martinet 2022-02-01 9:37 ` Eric Wong 2022-02-01 23:27 ` FD_CLOEXEC w/ nix-shell [was: Test failures with 1.7.0] Eric Wong ` (2 subsequent siblings) 3 siblings, 1 reply; 31+ messages in thread From: Dominique Martinet @ 2021-12-09 2:53 UTC (permalink / raw) To: Julien Moutinho; +Cc: Eric Wong, meta Julien Moutinho wrote on Thu, Dec 09, 2021 at 02:37:43AM +0100: > I can also reproduce Infinisil's test failure with: > $ (cd public-inbox-1.7.0; TMPDIR=/var/tmp perl -I$out/lib/perl5/site_perl t/lei_to_mail.t ) > > ok 96 - got Maildir callback > > Use of uninitialized value in open at t/lei_to_mail.t line 263. > > Bail out! No such file or directory I got curious on this one. strace tells me: ---- 2813384 renameat2(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/tmp/badc0ffee", AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/cur/badc0ffee:2,", RENAME_NOREPLACE) = -1 EINVAL (Invalid argument) 2813384 openat(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/new/", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4 2813384 newfstatat(4, "", {st_mode=S_IFDIR|0755, st_size=2, ...}, AT_EMPTY_PATH) = 0 2813384 brk(0x44f4000) = 0x44f4000 2813384 getdents64(4, 0x44b3e40 /* 2 entries */, 131072) = 48 2813384 getdents64(4, 0x44b3e40 /* 0 entries */, 131072) = 0 2813384 close(4) = 0 2813384 openat(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/cur/", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4 2813384 newfstatat(4, "", {st_mode=S_IFDIR|0755, st_size=2, ...}, AT_EMPTY_PATH) = 0 2813384 getdents64(4, 0x44b3e40 /* 2 entries */, 131072) = 48 2813384 getdents64(4, 0x44b3e40 /* 0 entries */, 131072) = 0 2813384 close(4) = 0 2813384 write(2, "Use of uninitialized value in op"..., 64) = 64 2813384 openat(AT_FDCWD, "", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) 2813384 getpid() = 2813384 2813384 getpid() = 2813384 2813384 getpid() = 2813384 2813384 write(5, "Bail out! No such file or direc"..., 37) = 37 ---- So this one is a real bug, this appears to fix it: ---- From 50a63628d505ca1c8d36f94ab5703f87a2c5e415 Mon Sep 17 00:00:00 2001 From: Dominique Martinet <asmadeus@codewreck.org> Date: Thu, 9 Dec 2021 11:50:51 +0900 Subject: [PATCH] syscall: fallback to rename on renameat2 EINVAL ZFS appears to incorrectly return EINVAL on renameat2 when the operation is not supported: renameat2(AT_FDCWD, "...", AT_FDCWD, "...", RENAME_NOREPLACE) = -1 EINVAL Fall back to the racy rename in this case as well: diff --git a/lib/PublicInbox/Syscall.pm b/lib/PublicInbox/Syscall.pm index c00385b94db8..78f926ac38f0 100644 --- a/lib/PublicInbox/Syscall.pm +++ b/lib/PublicInbox/Syscall.pm @@ -15,7 +15,7 @@ package PublicInbox::Syscall; use strict; use v5.10.1; use parent qw(Exporter); -use POSIX qw(ENOENT EEXIST ENOSYS O_NONBLOCK); +use POSIX qw(ENOENT EEXIST ENOSYS EINVAL O_NONBLOCK); use Config; # $VERSION = '0.25'; # Sys::Syscall version @@ -312,7 +312,7 @@ sub rename_noreplace ($$) { my $ret = syscall($SYS_renameat2, -100, $old, -100, $new, 1); if ($ret == 0) { 1; # like rename() perlop - } elsif ($! == ENOSYS) { + } elsif ($! == ENOSYS || $! == EINVAL) { undef $SYS_renameat2; _rename_noreplace_racy($old, $new); } else { ---- > This test does succeed outside Nix's sandbox: > $ (cd public-inbox-1.7.0; export PERL_INLINE_DIRECTORY=$PWD/inline-c; rm -rf $PERL_INLINE_DIRECTORY; mkdir $PERL_INLINE_DIRECTORY; prove -bvw t/lei-sigpipe.t ) > > t/lei-sigpipe.t .. > > ok 1 - lei import $TMPDIR/lei-daemon/big.eml > > ok 2 - read one byte > > ok 3 - signaled > > ok 4 - got SIGPIPE > > ok 5 - quiet after sigpipe > > ok 6 - read one byte > > ok 7 - signaled -f mboxcl2 > > ok 8 - got SIGPIPE -f mboxcl2 > > ok 9 - quiet after sigpipe -f mboxcl2 > > ok 10 - read one byte > > ok 11 - signaled -f text > > ok 12 - got SIGPIPE -f text > > ok 13 - quiet after sigpipe -f text > > ok 14 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:44) > > ok 15 - daemon running after t/lei-sigpipe.t:44 > > ok 16 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:44) > > ok 17 - t/lei-sigpipe.t:44 daemon stopped > > ok 18 - t/lei-sigpipe.t:44 daemon XDG_RUNTIME_DIR/lei/errors.log empty > > 1..18 > > ok > > All tests successful. > > Files=1, Tests=18, 7 wallclock secs ( 0.06 usr 0.06 sys + 3.44 cusr 2.73 csys = 6.29 CPU) > > Result: PASS > > More surprisingly, it even succeeds when run manually > inside the hanging Nix sandbox: > $ sudo nsenter --target 3137110 --all -S 1000 -G 100 $(readlink -e $(which bash)) > $ . /build/env-vars > $ cd /build > $ export HOME=$(mktemp -d) > $ mkdir -p $HOME/.cache/public-inbox/inline-c > $ LANG=C prove -bvw t/lei-sigpipe.t > > t/lei-sigpipe.t .. > > ok 1 - lei import $TMPDIR/lei-daemon/big.eml > > ok 2 - read one byte > > ok 3 - signaled > > ok 4 - got SIGPIPE > > ok 5 - quiet after sigpipe > > ok 6 - read one byte > > ok 7 - signaled -f mboxcl2 > > ok 8 - got SIGPIPE -f mboxcl2 > > ok 9 - quiet after sigpipe -f mboxcl2 > > ok 10 - read one byte > > ok 11 - signaled -f text > > ok 12 - got SIGPIPE -f text > > ok 13 - quiet after sigpipe -f text > > ok 14 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:44) > > ok 15 - daemon running after t/lei-sigpipe.t:44 > > ok 16 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:44) > > ok 17 - t/lei-sigpipe.t:44 daemon stopped > > ok 18 - t/lei-sigpipe.t:44 daemon XDG_RUNTIME_DIR/lei/errors.log empty > > 1..18 > > ok > > All tests successful. > > Files=1, Tests=18, 4 wallclock secs ( 0.06 usr 0.06 sys + 1.23 cusr 1.48 csys = 2.83 CPU) > > Result: PASS > > Even more strange, Dominique was able to reproduce > the hang this morning, but no longer tonight.. Yes, I don't get it, it hanged once but no longer hangs, so as much as I'd have liked to investigate I'm a bit stuck. With this, I can confirm running with inline-c also makes the tests that failed with the btrfs chattr call also pass. So all that's left is fix the proc mounts parsing there :) -- Dominique ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: Test failures with 1.7.0 2021-12-09 2:53 ` Dominique Martinet @ 2022-02-01 9:37 ` Eric Wong 0 siblings, 0 replies; 31+ messages in thread From: Eric Wong @ 2022-02-01 9:37 UTC (permalink / raw) To: Dominique Martinet; +Cc: Julien Moutinho, meta Dominique Martinet <asmadeus@codewreck.org> wrote: > Subject: [PATCH] syscall: fallback to rename on renameat2 EINVAL Thanks, applied and pushed as d8f1a9f189f0d74887cbb896caacaf7f1f256607 Will dig through more in a bit, slowly... ^ permalink raw reply [flat|nested] 31+ messages in thread
* FD_CLOEXEC w/ nix-shell [was: Test failures with 1.7.0] 2021-12-09 1:37 ` Test failures with 1.7.0 Julien Moutinho 2021-12-09 2:53 ` Dominique Martinet @ 2022-02-01 23:27 ` Eric Wong 2022-02-02 0:23 ` Dominique Martinet 2022-02-01 23:34 ` [PATCH] test_lei: use consistent locale for error messages Eric Wong 2022-02-17 21:02 ` [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test Eric Wong 3 siblings, 1 reply; 31+ messages in thread From: Eric Wong @ 2022-02-01 23:27 UTC (permalink / raw) To: Julien Moutinho; +Cc: Dominique Martinet, meta Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > Without Inline::C, and without Nix's sandbox > -------------------------------------------- > > I get 3 tests to fail (in addition to those currently disabled in Nixpkgs: > https://github.com/NixOS/nixpkgs/pull/104457/files#diff-77474340b38081fea7ce3b112c1bc060d3a95cc5e781915463e2f3fc5556da7bR34-R69 > ie. convert-compact, search, v2writable, www_listing, eml, hl_mod, > lei-mirror and spawn): > $ nix develop -f . public-inbox > $ doCheck= bash pkgs/development/perl-modules/generic/builder.sh > $ make -C public-inbox-1.7.0 test > > [...] > > Test Summary Report > > ------------------- > > t/nntpd-v2.t (Wstat: 256 Tests: 113 Failed: 1) > > Failed test: 102 > > Non-zero exit status: 1 > > t/nntpd.t (Wstat: 256 Tests: 110 Failed: 1) > > Failed test: 99 > > Non-zero exit status: 1 > > t/solver_git.t (Wstat: 1280 Tests: 13 Failed: 5) > > Failed tests: 9-13 > > Non-zero exit status: 5 > > Parse errors: No plan found in TAP output > > Files=149, Tests=4658, 296 wallclock secs ( 1.78 usr 0.56 sys + 116.73 cusr 165.82 csys = 284.89 CPU) > > Result: FAIL > > Failed 3/149 test programs. 7/4658 subtests failed. > > More precisely: > $ (cd public-inbox-1.7.0; perl -I$out/lib/perl5/site_perl t/nntpd-v2.t ) > > [...] > > ok 101 - new article retrieved after compact > > not ok 102 - no deleted files > > # Failed test 'no deleted files' > > # at ./t/nntpd.t line 363. > > # Structures begin differing at: > > # $got->[0] = '-nntpd 3130714 julm 4u REG 0,53 77189 2606550 /tmp/nix-shell.D0VGvX (deleted) OK, so that's a false positive for a deleted file that wasn't created by public-inbox. I suppose our tests can be altered to account for that. Is nix-shell setting FD_CLOEXEC on all FDs (>= 3) it creates? IMNSHO it should. It seems to be standard practice nowadays to set FD_CLOEXEC on all FDs (>=3) created. At least that's my experience with Perl 5.6+ and Ruby 2.0+; I'll let experts in other languages chime in for other stuff... More on the rest in a separate email(s). ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: FD_CLOEXEC w/ nix-shell [was: Test failures with 1.7.0] 2022-02-01 23:27 ` FD_CLOEXEC w/ nix-shell [was: Test failures with 1.7.0] Eric Wong @ 2022-02-02 0:23 ` Dominique Martinet 2022-02-02 2:11 ` Dominique Martinet 0 siblings, 1 reply; 31+ messages in thread From: Dominique Martinet @ 2022-02-02 0:23 UTC (permalink / raw) To: Eric Wong; +Cc: Julien Moutinho, meta Eric Wong wrote on Tue, Feb 01, 2022 at 11:27:40PM +0000: > Is nix-shell setting FD_CLOEXEC on all FDs (>= 3) it creates? > IMNSHO it should. ugh, it definitely should -- in this case it looks like a plain fd leak. $ ls -l /proc/$$/fd/ total 0 lrwx------ 1 asmadeus users 64 Feb 2 09:17 0 -> /dev/pts/28 lrwx------ 1 asmadeus users 64 Feb 2 09:17 1 -> /dev/pts/28 lrwx------ 1 asmadeus users 64 Feb 2 09:17 2 -> /dev/pts/28 lrwx------ 1 asmadeus users 64 Feb 2 09:17 255 -> /dev/pts/28 lrwx------ 1 asmadeus users 64 Feb 2 09:17 4 -> '/tmp/nix-shell.vhDsb9 (deleted)' $ ls -l /proc/self/fd/ total 0 lrwx------ 1 asmadeus users 64 Feb 2 09:17 0 -> /dev/pts/28 lrwx------ 1 asmadeus users 64 Feb 2 09:17 1 -> /dev/pts/28 lrwx------ 1 asmadeus users 64 Feb 2 09:17 2 -> /dev/pts/28 lr-x------ 1 asmadeus users 64 Feb 2 09:17 3 -> /proc/1220800/fd lrwx------ 1 asmadeus users 64 Feb 2 09:17 4 -> '/tmp/nix-shell.vhDsb9 (deleted)' nix-shell/nix shell don't seem to have the problem it's a 'nix develop' bug, I'll follow it up with nix devs and get it fixed there. (I don't have an opinion if you want to make your tests more robust, it might be a good thing in general anyway) Thanks for figuring this out! -- Dominique ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: FD_CLOEXEC w/ nix-shell [was: Test failures with 1.7.0] 2022-02-02 0:23 ` Dominique Martinet @ 2022-02-02 2:11 ` Dominique Martinet 0 siblings, 0 replies; 31+ messages in thread From: Dominique Martinet @ 2022-02-02 2:11 UTC (permalink / raw) To: Eric Wong; +Cc: Julien Moutinho, meta Dominique Martinet wrote on Wed, Feb 02, 2022 at 09:23:38AM +0900: > nix-shell/nix shell don't seem to have the problem it's a 'nix develop' > bug, I'll follow it up with nix devs and get it fixed there. It looks like this has already been fixed upstream: https://github.com/NixOS/nix/issues/5674 https://github.com/NixOS/nix/pull/5683 temporary files have been marked as CLOEXEC so this particular bug shouldn't be a problem anymore. The tests run during package builds don't use the same mechanism, so can only hope this was also fixed by the same swoop. Cheers, -- Dominique ^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH] test_lei: use consistent locale for error messages 2021-12-09 1:37 ` Test failures with 1.7.0 Julien Moutinho 2021-12-09 2:53 ` Dominique Martinet 2022-02-01 23:27 ` FD_CLOEXEC w/ nix-shell [was: Test failures with 1.7.0] Eric Wong @ 2022-02-01 23:34 ` Eric Wong 2022-02-17 21:02 ` [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test Eric Wong 3 siblings, 0 replies; 31+ messages in thread From: Eric Wong @ 2022-02-01 23:34 UTC (permalink / raw) To: Julien Moutinho; +Cc: Dominique Martinet, meta Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > Which is an artifact of my testing method, because with an absolute perl: > $ (cd public-inbox-1.7.0; $(which perl) -I$out/lib/perl5/site_perl t/lei-q-save.t ) > > [...] > > not ok 95 - git config parse error shown w/ lei ls-search > > # Failed test 'git config parse error shown w/ lei ls-search' > > # at t/lei-q-save.t line 228. > > # 'fatal: ligne 3 de config incorrecte dans le fichier /tmp/nix-shell.EKTZrS/pi-lei-q-save-3132353-xECZ/lei-daemon/.local/share/lei/saved-searches/after-c8d60f185616bece7957076be4604781d6cae876642a4faa127a2fcfc9647fc8/lei.saved-search > > # git config -z -l --includes --file=/tmp/nix-shell.EKTZrS/pi-lei-q-save-3132353-xECZ/lei-daemon/.cache/lei/saved-tmp.3132468.1639007508.config failed: $?=32768 > > # ' > > # doesn't match '(?^:bad config line.*?\/tmp\/nix\-shell\.EKTZrS\/pi\-lei\-q\-save\-3132353\-xECZ\/lei\-daemon\/\.local\/share\/lei\/saved\-searches\/after\-c8d60f185616bece7957076be4604781d6cae876642a4faa127a2fcfc9647fc8\/lei\.saved\-search)' > > ok 96 - lei up --all (up works with bad config) > > not ok 97 - git config parse error shown w/ lei up > > # Failed test 'git config parse error shown w/ lei up' > > # at t/lei-q-save.t line 231. > > # 'fatal: ligne 3 de config incorrecte dans le fichier /tmp/nix-shell.EKTZrS/pi-lei-q-save-3132353-xECZ/lei-daemon/.local/share/lei/saved-searches/after-c8d60f185616bece7957076be4604781d6cae876642a4faa127a2fcfc9647fc8/lei.saved-search > > # git config -z -l --includes --file=/tmp/nix-shell.EKTZrS/pi-lei-q-save-3132353-xECZ/lei-daemon/.cache/lei/saved-tmp.3132468.1639007508.config failed: $?=32768 > > # ' > > # doesn't match '(?^:bad config line.*?\/tmp\/nix\-shell\.EKTZrS\/pi\-lei\-q\-save\-3132353\-xECZ\/lei\-daemon\/\.local\/share\/lei\/saved\-searches\/after\-c8d60f185616bece7957076be4604781d6cae876642a4faa127a2fcfc9647fc8\/lei\.saved\-search)' > It succeeds if I use LANG=C instead of LANG=fr_FR.UTF-8 ---------8<--------- Subject: [PATCH] test_lei: use consistent locale for error messages git-config(1) error messages are locale-dependent, so follow the lead taken by git's own test suite and set LC_ALL=C and LANG=C to ensure error messages we check against are not localized. --- lib/PublicInbox/TestCommon.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/PublicInbox/TestCommon.pm b/lib/PublicInbox/TestCommon.pm index 052d6e45..ca732811 100644 --- a/lib/PublicInbox/TestCommon.pm +++ b/lib/PublicInbox/TestCommon.pm @@ -568,6 +568,7 @@ SKIP: { delete $ENV{XDG_CACHE_HOME}; $ENV{GIT_COMMITTER_EMAIL} = 'lei@example.com'; $ENV{GIT_COMMITTER_NAME} = 'lei user'; + $ENV{LANG} = $ENV{LC_ALL} = 'C'; my (undef, $fn, $lineno) = caller(0); my $t = "$fn:$lineno"; state $lei_daemon = PublicInbox::Spawn->can('send_cmd4') || ^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test 2021-12-09 1:37 ` Test failures with 1.7.0 Julien Moutinho ` (2 preceding siblings ...) 2022-02-01 23:34 ` [PATCH] test_lei: use consistent locale for error messages Eric Wong @ 2022-02-17 21:02 ` Eric Wong 2022-02-20 1:38 ` Julien Moutinho 3 siblings, 1 reply; 31+ messages in thread From: Eric Wong @ 2022-02-17 21:02 UTC (permalink / raw) To: Julien Moutinho; +Cc: Dominique Martinet, meta Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > With Inline::C, and with Nix's sandbox > -------------------------------------- > > Using (in pkgs/servers/mail/public-inbox/default.nix): > > preCheck = '' > > perl certs/create-certs.perl > > export HOME=$(mktemp -d) > > mkdir -p $HOME/.cache/public-inbox/inline-c > > ''; > > $ nix -L build -f . public-inbox > > [...] > > perl5.34.0-public-inbox> t/lei-refresh-mail-sync.t .... ok > Hangs like forever after this line, while on t/lei-sigpipe.t: > > $ ps auxwwf > > root 1755997 0.0 0.0 453172 2800 ? Ssl déc.04 0:03 nix-daemon --daemon > > root 1811429 0.0 0.0 453652 1928 ? Ssl déc.04 0:00 \_ nix-daemon 1757028 > > root 1811455 0.0 0.0 453652 2004 ? Ssl déc.04 0:00 \_ nix-daemon 1757031 > > root 1839837 0.0 0.0 453652 1832 ? Ssl déc.04 0:00 \_ nix-daemon 1757027 > > root 1839841 0.0 0.0 453652 1996 ? Ssl déc.04 0:00 \_ nix-daemon 1757030 > > root 1839856 0.0 0.0 453652 1832 ? Ssl déc.04 0:00 \_ nix-daemon 1757029 > > root 3135741 0.0 0.2 603352 17080 ? Ssl 01:05 0:00 \_ nix-daemon 3135672 > > nixbld1 3137110 0.0 0.0 5200 4072 ? Ss 01:05 0:01 \_ bash -e /nix/store/qf3mzpvsmkrw963xchbivcci06078n13-builder.sh > > nixbld1 3140619 0.0 0.0 4996 4012 ? S 01:05 0:00 \_ make SHELL=/nix/store/l0wlqpbsvh1pgvhcdhw7qkka3d31si7k-bash-5.1-p8/bin/bash VERBOSE=y test > > nixbld1 3140633 0.0 0.1 17188 13352 ? S 01:05 0:01 | \_ /nix/store/vslsa0l17xjcrdgm2knwj0z5hlvf73m7-perl-5.34.0/bin/perl -MExtUtils::Command::MM -MTest::Harness -e undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch') t/admin.t t/altid.t t/altid_v2.t t/cgi.t t/cmd_ipc.t t/content_hash.t t/dir_idle.t t/edit.t t/eml_content_disposition.t t/eml_content_type.t t/epoll.t t/extindex-psgi.t t/extsearch.t t/fake_inotify.t t/feed.t t/gcf2.t t/gcf2_client.t t/httpd.t t/idx_stack.t t/imap.t t/imap_searchqp.t t/imapd-tls.t t/imapd.t t/inbox_idle.t t/index-git-times.t t/indexlevels-mirror.t t/ipc.t t/kqnotify.t t/lei-auto-watch.t t/lei-convert.t t/lei-daemon.t t/lei-export-kw.t t/lei-externals.t t/lei-import-http.t t/lei-import-imap.t t/lei-import-maildir.t t/lei-import-nntp.t t/lei-import.t t/lei-index.t t/lei-inspect.t t/lei-lcat.t t/lei-p2q.t t/lei-q-kw.t t/lei-q-remote-import.t t/lei-q-save.t t/lei-q-thread.t t/lei-refresh-mail-sync.t t/lei-sigpipe.t t/lei-tag.t t/lei-up.t t/lei-watch.t t/lei.t t/lei_dedupe.t t/lei_external.t t/lei_lcat.t t/lei_mail_sync.t t/lei_overview.t t/lei_saved_search.t t/lei_store.t t/lei_to_mail.t t/lei_xsearch.t t/mbox_lock.t t/mbox_reader.t t/mdir_reader.t t/mime.t t/miscsearch.t t/net_reader-imap.t t/nntpd-tls.t t/nntpd.t t/nodatacow.t t/on_destroy.t t/plack.t t/psgi_attach.t t/psgi_bad_mids.t t/psgi_mount.t t/psgi_multipart_not.t t/psgi_scan_all.t t/psgi_search.t t/psgi_v2.t t/rename_noreplace.t t/reply.t t/search-thr-index.t t/shared_kv.t t/solver_git.t t/thread-index-gap.t t/uri_imap.t t/uri_nntps.t t/v2dupindex.t t/www_altid.t t/xcpdb-reshard.t t/www_static.t t/watch_multiple_headers.t t/watch_maildir_v2.t t/watch_maildir.t t/watch_imap.t t/watch_filter_rubylang.t t/view.t t/v2reindex.t t/v2mirror.t t/v2mda.t t/v2index-late-dupe.t t/v2-add-remove-add.t t/v1reindex.t t/v1-add-remove-add.t t/time.t t/thread-cycle.t t/spamcheck_spamc.t t/sigfd.t t/replace.t t/reindex-time-range.t t/qspawn.t t/purge.t t/psgi_text.t t/precheck.t t/over.t t/nulsubject.t t/nntpd-v2.t t/nntp.t t/multi-mid.t t/msgtime.t t/msgmap.t t/msg_iter.t t/mid.t t/mda_filter_rubylang.t t/mda.t t/linkify.t t/init.t t/indexlevels-mirror-v1.t t/inbox.t t/import.t t/imap_tracker.t t/hval.t t/httpd-unix.t t/httpd-https.t t/httpd-corner.t t/gzip_filter.t t/git.t t/filter_vger.t t/filter_subjecttag.t t/filter_rubylang.t t/filter_mirror.t t/filter_base.t t/emergency.t t/ds-poll.t t/ds-leak.t t/ds-kqxs.t t/config_limiter.t t/config.t t/address.t > > nixbld1 3145719 0.0 0.4 44576 35640 ? S 01:09 0:01 | \_ t/lei-sigpipe.t > > nixbld1 3145743 0.0 0.1 14388 10388 ? S 01:09 0:00 | \_ /nix/store/vslsa0l17xjcrdgm2knwj0z5hlvf73m7-perl-5.34.0/bin/perl -w -I/nix/store/vslsa0l17xjcrdgm2knwj0z5hlvf73m7-perl-5.34.0/lib/perl5/site_perl -I/nix/store/1hdx7bxjwqrfnalalkgbwi32l45h8z7b-perl5.34.0-Mail-IMAPClient-3.42/lib/perl5/site_perl -I/nix/st blib/script/lei q -q -t z:1.. > > nixbld1 3145724 0.0 0.5 50504 42160 ? S 01:09 0:00 \_ lei-daemon /build/pi-lei-sigpipe-7522-WoHO/lei-daemon/xdg_run/lei/5.seq.sock > > nixbld1 3145730 0.0 0.4 50092 39736 ? S 01:09 0:00 \_ lei/store /build/pi-lei-sigpipe-7522-WoHO/lei-daemon/.local/share > > t/lei-sigpipe.t is on: > > 01:30:29.508334 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7fffffff5090) = 0 <1.000360> > > 01:30:30.509837 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7fffffff5090) = 0 <1.000186> > > 01:30:31.510259 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7fffffff5090) = 0 <1.000175> > > [...] Huh? but it's stuck in sysread() as noted below... I'm not sure where clock_nanosleep(2) calls are coming from with a 1s interval. I doubt I'd ever loop clock_nanosleep(CLOCK_REALTIME no matter how messed up my brain gets :P I do use nanosleep(2) with a 100ms interval on sendmsg ENOBUFS|ENOMEM|ETOOMANYREFS in Spawn.pm and that's a different syscall. From what I can tell, Mail::IMAPClient doesn't clock_nanosleep, nor does Perl itself... Sleeping on CLOCK_REALTIME seems wrong outside of a cron-like scheduler... > lei-daemon store is on: > > 01:31:03.834708 epoll_wait(5, OK, that's normal. > Looks like it is waiting on sysread() in > https://public-inbox.org/public-inbox.git/tree/t/lei-sigpipe.t#n35 OK, that makes more sense... > This test does succeed outside Nix's sandbox: > $ (cd public-inbox-1.7.0; export PERL_INLINE_DIRECTORY=$PWD/inline-c; rm -rf $PERL_INLINE_DIRECTORY; mkdir $PERL_INLINE_DIRECTORY; prove -bvw t/lei-sigpipe.t ) OK, <snip> > More surprisingly, it even succeeds when run manually > inside the hanging Nix sandbox: OK, <snip> > Even more strange, Dominique was able to reproduce > the hang this morning, but no longer tonight.. It's been a while, and I can't reproduce it. Maybe this patch will help with diagnosis: ------------8<--------- Subject: [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test This may help diagnose a difficult-to-reproduce test failure on NixOS. Link: https://public-inbox/meta/20211209013743.okzgim7bbrpahks7@sourcephile.fr/ --- t/lei-sigpipe.t | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/t/lei-sigpipe.t b/t/lei-sigpipe.t index d9738b07..6b2772a6 100644 --- a/t/lei-sigpipe.t +++ b/t/lei-sigpipe.t @@ -27,11 +27,17 @@ EOM } lei_ok(qw(import), $f) if $imported++ == 0; - open my $errfh, '+>', "$ENV{HOME}/stderr.log" or xbail $!; + open my $errfh, '+>>', "$ENV{HOME}/stderr.log" or xbail $!; my $opt = { run_mode => 0, 2 => $errfh, 1 => $w }; my $cmd = [qw(lei q -q -t), @$out, 'z:1..']; my $tp = start_script($cmd, undef, $opt); close $w; + vec(my $rvec = '', fileno($r), 1) = 1; + if (!select($rvec, undef, undef, 30)) { + seek($errfh, 0, 0) or xbail $!; + my $s = do { local $/; <$errfh> }; + xbail "lei q had no output after 30s, stderr=$s"; + } is(sysread($r, my $buf, 1), 1, 'read one byte'); close $r; # trigger SIGPIPE $tp->join; ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test 2022-02-17 21:02 ` [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test Eric Wong @ 2022-02-20 1:38 ` Julien Moutinho 2022-02-22 6:44 ` Eric Wong 0 siblings, 1 reply; 31+ messages in thread From: Julien Moutinho @ 2022-02-20 1:38 UTC (permalink / raw) To: Eric Wong; +Cc: Dominique Martinet, meta [-- Attachment #1: Type: text/plain, Size: 3484 bytes --] Hi Eric, Thank you for following up on this despite your having more important matters on your shoulders :\ Le jeu. 17 févr. 2022 21h02 +0000, Eric Wong a écrit : > > Looks like it is waiting on sysread() in > > https://public-inbox.org/public-inbox.git/tree/t/lei-sigpipe.t#n35 > > OK, that makes more sense... My tracing with print statements shows that sysread is passed and that it is $tp->join which hangs: [nixbld@localhost:/build]$ cat stderr.log > lei-sigpipe: begin > lei-sigpipe: start_script: done > lei-sigpipe: close w: done > lei-sigpipe: sysread > lei-sigpipe: sysread: done > lei-sigpipe: close r: done [nixbld@localhost:/build]$ cat start_script.log > start_script: begin > start_script: fork > start_script: pid!=0 > start_script: pid==0 > start_script: pid!=0: tail > start_script: pid==0: if sub > start_script: pid==0: exec lei ps auxwwf | grep nixbld > nixbld1 491489 0.1 0.0 5312 4036 ? Ss 02:12 0:01 \_ bash -e /nix/store/qf3mzpvsmkrw963xchbivcci06078n13-builder.sh > nixbld1 494307 0.0 0.1 16672 12616 ? S 02:12 0:00 \_ perl -MExtUtils::Command::MM -MTest::Harness -e undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch') t/lei-sigpipe.t > nixbld1 494311 0.2 0.5 49320 40068 ? S 02:12 0:02 | \_ t/lei-sigpipe.t > nixbld1 494445 0.0 0.0 0 0 ? Z 02:12 0:00 | \_ [/nix/store/176g] <defunct> > nixbld1 494447 0.0 0.1 14388 10176 ? S 02:12 0:00 | \_ /nix/store/6wcdxh3h5jp9qgrccq3n71wnalca5mf5-perl-5.34.0/bin/perl -w -I/nix/store/6wcdxh3h5jp9qgrccq3n71wnalca5mf5-perl-5.34.0/lib/perl5/site_perl -I/nix/store/jibsc747pfzmyxp5lsqgzyvrs2lnkws6-perl5.34.0-Mail-IMAPClient-3.42/lib/perl5/site_perl -I/nix/st blib/script/lei q -q -t z:1.. > nixbld1 494427 0.1 0.5 50664 41892 ? S 02:12 0:00 \_ lei-daemon /build/pi-lei-sigpipe-2750-0bv8/lei-daemon/xdg_run/lei/5.seq.sock > nixbld1 494433 0.0 0.4 50260 39580 ? S 02:12 0:00 \_ lei/store /build/pi-lei-sigpipe-2750-0bv8/lei-daemon/.local/share The defunct process with truncated path "/nix/store/176g" is points to /nix/store/176gh50y24c0lx2bnnmsvf9wazb73php-coreutils-9.0 It appears only when I export TAIL=${coreutils}/bin/tail which does not workaround the hang. This deadlock appears only when the test is run by nix, it doesn't when I run it manually from a terminal by entering the sandbox: [nixbld@localhost:/]$ (cd /build/public-inbox-1.7.0; export PERL_INLINE_DIRECTORY=$PWD/inline-c; rm -rf $PERL_INLINE_DIRECTORY; mkdir $PERL_INLINE_DIRECTORY; prove -bvw t/lei-sigpipe.t ) > t/lei-sigpipe.t .. > ok 1 - lei import $TMPDIR/big.eml > ok 2 - read one byte > ok 3 - signaled > ok 4 - got SIGPIPE > ok 5 - read one byte > ok 6 - signaled -f mboxcl2 > ok 7 - got SIGPIPE -f mboxcl2 > ok 8 - read one byte > ok 9 - signaled -f text > ok 10 - got SIGPIPE -f text > ok 11 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:58) > ok 12 - daemon running after t/lei-sigpipe.t:58 > ok 13 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:58) > ok 14 - t/lei-sigpipe.t:58 daemon stopped > ok 15 - t/lei-sigpipe.t:58 daemon XDG_RUNTIME_DIR/lei/errors.log empty > 1..15 > ok > All tests successful. > Files=1, Tests=15, 8 wallclock secs ( 0.06 usr 0.08 sys + 3.55 cusr 2.81 csys = 6.50 CPU) > Result: PASS [-- Attachment #2: 0001-trace-t-lei-sigpipe.t.patch --] [-- Type: text/plain, Size: 3841 bytes --] From d919b3441bdf61e8dfae0c08005b8a77662ddb91 Mon Sep 17 00:00:00 2001 From: Julien Moutinho <julm+public-inbox@sourcephile.fr> Date: Sun, 20 Feb 2022 01:33:04 +0100 Subject: [PATCH] trace t/lei-sigpipe.t --- lib/PublicInbox/TestCommon.pm | 10 ++++++++++ t/lei-sigpipe.t | 22 +++++++++++++++------- 2 files changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/PublicInbox/TestCommon.pm b/lib/PublicInbox/TestCommon.pm index ca732811..d8416f90 100644 --- a/lib/PublicInbox/TestCommon.pm +++ b/lib/PublicInbox/TestCommon.pm @@ -439,6 +439,9 @@ sub start_script { my $run_mode = $ENV{TEST_RUN_MODE} // $opt->{run_mode} // 2; my $sub = $run_mode == 0 ? undef : key2sub($key); my $tail; + open my $errfh, '+>>', "/build/start_script.log" or xbail $!; + $errfh->autoflush(1); + print $errfh "\nstart_script: begin\n"; if ($tail_cmd) { my @paths; for (@argv) { @@ -459,8 +462,10 @@ sub start_script { } $tail = tail_f(@paths); } + print $errfh "\nstart_script: fork\n"; my $pid = fork // die "fork: $!\n"; if ($pid == 0) { + print $errfh "\nstart_script: pid==0\n"; eval { PublicInbox::DS->Reset }; # pretend to be systemd (cf. sd_listen_fds(3)) # 3 == SD_LISTEN_FDS_START @@ -484,18 +489,23 @@ sub start_script { } if ($opt->{-C}) { chdir($opt->{-C}) or die "chdir: $!" } $0 = join(' ', @$cmd); + print $errfh "\nstart_script: pid==0: if sub\n"; if ($sub) { + print $errfh "\nstart_script: pid==0: sub\n"; eval { PublicInbox::DS->Reset }; _run_sub($sub, $key, \@argv); POSIX::_exit($? >> 8); } else { + print $errfh "\nstart_script: pid==0: exec $key\n"; exec(key2script($key), @argv); die "FAIL: ",join(' ', $key, @argv), ": $!\n"; } } + print $errfh "\nstart_script: pid!=0\n"; require PublicInbox::AutoReap; my $td = PublicInbox::AutoReap->new($pid); $td->{-extra} = $tail; + print $errfh "\nstart_script: pid!=0: tail\n"; $td; } diff --git a/t/lei-sigpipe.t b/t/lei-sigpipe.t index 6b2772a6..e5e701dd 100644 --- a/t/lei-sigpipe.t +++ b/t/lei-sigpipe.t @@ -6,7 +6,7 @@ use v5.10.1; use PublicInbox::TestCommon; use POSIX qw(WTERMSIG WIFSIGNALED SIGPIPE); test_lei(sub { - my $f = "$ENV{HOME}/big.eml"; + my $f = "/build/big.eml"; my $imported; for my $out ([], [qw(-f mboxcl2)], [qw(-f text)]) { pipe(my ($r, $w)) or BAIL_OUT $!; @@ -27,25 +27,33 @@ EOM } lei_ok(qw(import), $f) if $imported++ == 0; - open my $errfh, '+>>', "$ENV{HOME}/stderr.log" or xbail $!; + open my $errfh, '+>>', "/build/stderr.log" or xbail $!; + $errfh->autoflush(1); + print $errfh "\nlei-sigpipe: begin\n"; my $opt = { run_mode => 0, 2 => $errfh, 1 => $w }; my $cmd = [qw(lei q -q -t), @$out, 'z:1..']; my $tp = start_script($cmd, undef, $opt); + print $errfh "\nlei-sigpipe: start_script: done\n"; close $w; + print $errfh "\nlei-sigpipe: close w: done\n"; vec(my $rvec = '', fileno($r), 1) = 1; - if (!select($rvec, undef, undef, 30)) { + if (!select($rvec, undef, undef, 3)) { seek($errfh, 0, 0) or xbail $!; my $s = do { local $/; <$errfh> }; - xbail "lei q had no output after 30s, stderr=$s"; + xbail "lei q had no output after 3s, stderr=$s"; } + print $errfh "\nlei-sigpipe: sysread\n"; is(sysread($r, my $buf, 1), 1, 'read one byte'); + print $errfh "\nlei-sigpipe: sysread: done\n"; close $r; # trigger SIGPIPE + print $errfh "\nlei-sigpipe: close r: done\n"; $tp->join; + print $errfh "\nlei-sigpipe: join: done\n"; ok(WIFSIGNALED($?), "signaled @$out"); is(WTERMSIG($?), SIGPIPE, "got SIGPIPE @$out"); - seek($errfh, 0, 0) or xbail $!; - my $s = do { local $/; <$errfh> }; - is($s, '', "quiet after sigpipe @$out"); + #seek($errfh, 0, 0) or xbail $!; + #my $s = do { local $/; <$errfh> }; + #is($s, '', "quiet after sigpipe @$out"); } }); -- 2.34.1 ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test 2022-02-20 1:38 ` Julien Moutinho @ 2022-02-22 6:44 ` Eric Wong 2022-02-27 4:15 ` Julien Moutinho 0 siblings, 1 reply; 31+ messages in thread From: Eric Wong @ 2022-02-22 6:44 UTC (permalink / raw) To: Julien Moutinho; +Cc: Dominique Martinet, meta Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > This deadlock appears only when the test is run by nix, > it doesn't when I run it manually from a terminal > by entering the sandbox: Oh, btw, is this with the FD_CLOEXEC fixes to nix shell itself? https://public-inbox.org/meta/YfnoOVOG3TrbqZFs@codewreck.org/ ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test 2022-02-22 6:44 ` Eric Wong @ 2022-02-27 4:15 ` Julien Moutinho 2022-02-27 6:41 ` Julien Moutinho 0 siblings, 1 reply; 31+ messages in thread From: Julien Moutinho @ 2022-02-27 4:15 UTC (permalink / raw) To: Eric Wong; +Cc: Dominique Martinet, meta Le mar. 22 févr. 2022 06h44 +0000, Eric Wong a écrit : > Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > > This deadlock appears only when the test is run by nix, > > it doesn't when I run it manually from a terminal > > by entering the sandbox: > > Oh, btw, is this with the FD_CLOEXEC fixes to nix shell itself? > https://public-inbox.org/meta/YfnoOVOG3TrbqZFs@codewreck.org/ Yes, I'm using: $ nix-daemon --version > nix-daemon (Nix) 2.7.0pre20220127_558c4ee Which contains the commit from https://github.com/NixOS/nix/pull/5683 : $ git merge-base --is-ancestor 6e0cbc666b60515b5e201dd28855f5fe1de9a107 558c4ee $ echo $? > 0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test 2022-02-27 4:15 ` Julien Moutinho @ 2022-02-27 6:41 ` Julien Moutinho 2022-02-27 7:23 ` Dominique Martinet 0 siblings, 1 reply; 31+ messages in thread From: Julien Moutinho @ 2022-02-27 6:41 UTC (permalink / raw) To: Eric Wong; +Cc: Dominique Martinet, meta Le dim. 27 févr. 2022 05h15 +0100, Julien Moutinho a écrit : > Yes, I'm using: > $ nix-daemon --version > > nix-daemon (Nix) 2.7.0pre20220127_558c4ee Dominique reminded me that that fix was first intended for `nix develop`, so I've asked whether it also applies to `nix build`: https://github.com/NixOS/nix/pull/5683#issuecomment-1053239025 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test 2022-02-27 6:41 ` Julien Moutinho @ 2022-02-27 7:23 ` Dominique Martinet 2022-02-27 8:04 ` Julien Moutinho 0 siblings, 1 reply; 31+ messages in thread From: Dominique Martinet @ 2022-02-27 7:23 UTC (permalink / raw) To: Julien Moutinho; +Cc: Eric Wong, meta Hello, Julien Moutinho wrote on Sun, Feb 27, 2022 at 07:41:44AM +0100: > Le dim. 27 févr. 2022 05h15 +0100, Julien Moutinho a écrit : > > Yes, I'm using: > > $ nix-daemon --version > > > nix-daemon (Nix) 2.7.0pre20220127_558c4ee > Dominique reminded me that that fix was first intended for > `nix develop`, so I've asked whether it also applies to `nix build`: > https://github.com/NixOS/nix/pull/5683#issuecomment-1053239025 I've replied over there, I don't think leaked fd is a problem. However Julien's latest traces gave a good enough hint: the lei process that should die from sigpipe just didn't die... And surely enough, nix adds SIGPIPE to ignores. This can be confirmed by looking at SigIgn in /proc/xxx/status of one of the builder PIDs: # grep SigIgn /proc/3124202/status SigIgn: 0000000000001004 Where 0x1000 is the 13th bit: $ kill -l 13 PIPE Running tests manually won't inherit masked signals and thus won't reproduce. I guess we can now open a nix issue asking to reset signal handlers after fork, before executing its builder?... -- Dominique ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test 2022-02-27 7:23 ` Dominique Martinet @ 2022-02-27 8:04 ` Julien Moutinho 2022-02-27 11:17 ` [PATCH] t/lei-sigpipe: ensure SIGPIPE is unblocked for this test Eric Wong 0 siblings, 1 reply; 31+ messages in thread From: Julien Moutinho @ 2022-02-27 8:04 UTC (permalink / raw) To: Dominique Martinet; +Cc: Eric Wong, meta Le dim. 27 févr. 2022 16h23 +0900, Dominique Martinet a écrit : > I guess we can now open a nix issue asking to reset signal handlers > after fork, before executing its builder?... Well spotted Dominique, thanks! Turns out this is actually systemd which ignores SIGPIPE. I'm able to pass t/lei-sigpipe.t when my `nix build` connects to a nix-daemon run manually outside systemd. There is already a 3yo issue on nix's tracker for that: https://github.com/NixOS/nix/issues/2803 suggesting to add to nix-daemon.service: > [Service] > IgnoreSIGPIPE=no And that indeed also makes t/lei-sigpipe.t succeed. In the meantime I'll disable t/lei-sigpipe.t to finally get public-inbox added to Nixpkgs. Thank you Dominique and Eric! ^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH] t/lei-sigpipe: ensure SIGPIPE is unblocked for this test 2022-02-27 8:04 ` Julien Moutinho @ 2022-02-27 11:17 ` Eric Wong 2022-03-11 10:42 ` [PATCH] t/lei-sigpipe.t: ensure SIGPIPE is not ignored instead of not blocked Julien Moutinho 0 siblings, 1 reply; 31+ messages in thread From: Eric Wong @ 2022-02-27 11:17 UTC (permalink / raw) To: Julien Moutinho; +Cc: Dominique Martinet, meta Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > Le dim. 27 févr. 2022 16h23 +0900, Dominique Martinet a écrit : > > I guess we can now open a nix issue asking to reset signal handlers > > after fork, before executing its builder?... > Well spotted Dominique, thanks! > > Turns out this is actually systemd which ignores SIGPIPE. > I'm able to pass t/lei-sigpipe.t when my `nix build` > connects to a nix-daemon run manually outside systemd. > There is already a 3yo issue on nix's tracker for that: > https://github.com/NixOS/nix/issues/2803 > suggesting to add to nix-daemon.service: > > [Service] > > IgnoreSIGPIPE=no > And that indeed also makes t/lei-sigpipe.t succeed. > > In the meantime I'll disable t/lei-sigpipe.t > to finally get public-inbox added to Nixpkgs. > > Thank you Dominique and Eric! You're welcome and thank you for tracking this down. Yes, blocking SIGPIPE makes sense for most daemons so I think systemd is being reasonable, here. However, this lei test is for simulating an interactive environment, so re-enabling SIGPIPE under t/lei-sigpipe.t seems best... I initially considered adding this for all test_lei(), but most lei tests don't care and I didn't want to waste CPU time. ------8<----- Subject: [PATCH] t/lei-sigpipe: ensure SIGPIPE is unblocked for this test Tests run under systemd (and similar) have SIGPIPE blocked by default. This was causing this SIGPIPE test to get stuck when run by automated builders used by Nix. Thanks to Julien Moutinho and Dominique Martinet for tracking down this failure. Reported-by: Julien Moutinho <julm+public-inbox@sourcephile.fr> Reported-by: Dominique Martinet <asmadeus@codewreck.org> Link: https://public-inbox.org/meta/20220227080422.gyqowrxomzu6gyin@sourcephile.fr/ --- t/lei-sigpipe.t | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/t/lei-sigpipe.t b/t/lei-sigpipe.t index 6b2772a6..7fab9aeb 100644 --- a/t/lei-sigpipe.t +++ b/t/lei-sigpipe.t @@ -1,10 +1,25 @@ #!perl -w -# Copyright (C) 2021 all contributors <meta@public-inbox.org> +# Copyright (C) all contributors <meta@public-inbox.org> # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> use strict; use v5.10.1; use PublicInbox::TestCommon; -use POSIX qw(WTERMSIG WIFSIGNALED SIGPIPE); +use POSIX qw(WTERMSIG WIFSIGNALED SIGPIPE SIG_UNBLOCK SIG_SETMASK sigprocmask); +use PublicInbox::OnDestroy; + +# undo systemd (and similar) blocking SIGPIPE, since lei expects to be run +# from an interactive terminal: +# https://public-inbox.org/meta/20220227080422.gyqowrxomzu6gyin@sourcephile.fr/ +my $set = POSIX::SigSet->new; +my $old = POSIX::SigSet->new; +$set->emptyset or xbail "sigemptyset $!"; +$old->emptyset or xbail "sigemptyset $!"; +$set->addset(SIGPIPE); +sigprocmask(SIG_UNBLOCK, $set, $old) or xbail "SIG_UNBLOCK: $!"; +my $cleanup = PublicInbox::OnDestroy->new($$, sub { + sigprocmask(SIG_SETMASK, $old); +}); + test_lei(sub { my $f = "$ENV{HOME}/big.eml"; my $imported; ^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH] t/lei-sigpipe.t: ensure SIGPIPE is not ignored instead of not blocked 2022-02-27 11:17 ` [PATCH] t/lei-sigpipe: ensure SIGPIPE is unblocked for this test Eric Wong @ 2022-03-11 10:42 ` Julien Moutinho 2022-03-14 22:14 ` Eric Wong 0 siblings, 1 reply; 31+ messages in thread From: Julien Moutinho @ 2022-03-11 10:42 UTC (permalink / raw) To: meta; +Cc: Dominique Martinet, Eric Wong, Julien Moutinho Ignoring a signal is different than blocking a signal. --- t/lei-sigpipe.t | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/t/lei-sigpipe.t b/t/lei-sigpipe.t index 7fab9aeb..55c208e2 100644 --- a/t/lei-sigpipe.t +++ b/t/lei-sigpipe.t @@ -4,20 +4,16 @@ use strict; use v5.10.1; use PublicInbox::TestCommon; -use POSIX qw(WTERMSIG WIFSIGNALED SIGPIPE SIG_UNBLOCK SIG_SETMASK sigprocmask); +use POSIX qw(WTERMSIG WIFSIGNALED SIGPIPE); use PublicInbox::OnDestroy; -# undo systemd (and similar) blocking SIGPIPE, since lei expects to be run +# undo systemd (and similar) ignoring SIGPIPE, since lei expects to be run # from an interactive terminal: # https://public-inbox.org/meta/20220227080422.gyqowrxomzu6gyin@sourcephile.fr/ -my $set = POSIX::SigSet->new; -my $old = POSIX::SigSet->new; -$set->emptyset or xbail "sigemptyset $!"; -$old->emptyset or xbail "sigemptyset $!"; -$set->addset(SIGPIPE); -sigprocmask(SIG_UNBLOCK, $set, $old) or xbail "SIG_UNBLOCK: $!"; +my $oldSIGPIPE = $SIG{PIPE}; +$SIG{PIPE} = 'DEFAULT'; my $cleanup = PublicInbox::OnDestroy->new($$, sub { - sigprocmask(SIG_SETMASK, $old); + $SIG{PIPE} = $oldSIGPIPE; }); test_lei(sub { -- 2.35.1 ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH] t/lei-sigpipe.t: ensure SIGPIPE is not ignored instead of not blocked 2022-03-11 10:42 ` [PATCH] t/lei-sigpipe.t: ensure SIGPIPE is not ignored instead of not blocked Julien Moutinho @ 2022-03-14 22:14 ` Eric Wong 2022-03-15 2:56 ` Julien Moutinho 0 siblings, 1 reply; 31+ messages in thread From: Eric Wong @ 2022-03-14 22:14 UTC (permalink / raw) To: Julien Moutinho; +Cc: meta, Dominique Martinet Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > Ignoring a signal is different than blocking a signal. Thanks, pushed as commit f1e4e14793d155ea7d6ed7a6858b668e97c7e5d8 I wasn't sure why this patch was necessary at first. Thus I took take a closer look at systemd behavior to confirm, and updated the message: Ignoring a signal is different than blocking a signal, and the "IgnoreSIGPIPE" option of systemd ignores. [ew: note systemd behavior] Acked-by: Eric Wong <e@80x24.org> I (and any other project maintainer) would appreciate more complete explanations in the future as to why a patch is needed. Thanks again. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH] t/lei-sigpipe.t: ensure SIGPIPE is not ignored instead of not blocked 2022-03-14 22:14 ` Eric Wong @ 2022-03-15 2:56 ` Julien Moutinho 0 siblings, 0 replies; 31+ messages in thread From: Julien Moutinho @ 2022-03-15 2:56 UTC (permalink / raw) To: Eric Wong; +Cc: meta, Dominique Martinet Le lun. 14 mars 2022 22h14 +0000, Eric Wong a écrit : > I (and any other project maintainer) would appreciate more > complete explanations in the future as to why a patch is needed. > Thanks again. Oh sorry, being immersed into the testing I tought that it was obvious. I should have clarified that the unblocking fix was not working. Thanks for the merge! ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Test failures with 1.7.0 2021-12-08 4:08 ` Eric Wong 2021-12-08 10:56 ` Dominique Martinet @ 2022-03-01 2:30 ` Julien Moutinho 2022-03-01 4:05 ` Eric Wong 1 sibling, 1 reply; 31+ messages in thread From: Julien Moutinho @ 2022-03-01 2:30 UTC (permalink / raw) To: Eric Wong; +Cc: meta Hi Eric, A reviewer reported some more test failures but I cannot reproduce them, any clue at what could cause them? https://github.com/NixOS/nixpkgs/pull/104457#issuecomment-1054886919 Thank you in advance :) > t/lei-convert.t .............. 1/? > # Failed test 'got expected emails in Maildir' > # at t/lei-convert.t line 63. > # got: '0' > # expected: '15' > # # converted 15 messages > > # Failed test 'Maildir output matches mboxrd' > # at t/lei-convert.t line 72. > # Structures begin differing at: > # $got->[0] = Does not exist > # $expected->[0] = PublicInbox::Eml=HASH(0x3384268) > > # Failed test 'mboxrd round-tripped through Maildir w/ flags' > # at t/lei-convert.t line 79. > # Structures begin differing at: > # $got->[0] = PublicInbox::Eml=HASH(0x3505cd0) > # $expected->[0] = Does not exist > > # Failed test '`seen' flag set in Maildir' > # at t/lei-convert.t line 125. > # '/build/pi-lei-convert-5666-Cu05/lei-daemon/md2/tmp/7b1157a4,621d7780,1650,1' > # doesn't match '(?^::2,S\z)' > > # Failed test '`seen' flag preserved' > # at t/lei-convert.t line 127. > # '' > # doesn't match '(?^ms:^Status: RO)' > # Looks like you failed 5 tests of 50. > t/lei-convert.t .............. Dubious, test returned 5 (wstat 1280, 0x500) > Failed 5/50 subtests > t/lei-daemon.t ............... ok > t/lei-export-kw.t ............ All 2 subtests passed > t/lei-externals.t ............ ok > t/lei-import-http.t .......... ok > t/lei-import-imap.t .......... ok > t/lei-import-maildir.t ....... ok > t/lei-import-nntp.t .......... ok > t/lei-import.t ............... ok > t/lei-index.t ................ ok > t/lei-inspect.t .............. ok > t/lei-lcat.t ................. ok > t/lei-p2q.t .................. ok > t/lei-q-kw.t ................. 1/? Bailout called. Further testing stopped: # /build/pi-lei-q-kw-6884-0YKN/lei-daemon/.local/share/lei/store 1/1 > FAILED--Further testing stopped: # /build/pi-lei-q-kw-6884-0YKN/lei-daemon/.local/share/lei/store 1/1 > make: *** [Makefile:1207: test_dynamic] Error 255 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Test failures with 1.7.0 2022-03-01 2:30 ` Test failures with 1.7.0 Julien Moutinho @ 2022-03-01 4:05 ` Eric Wong 0 siblings, 0 replies; 31+ messages in thread From: Eric Wong @ 2022-03-01 4:05 UTC (permalink / raw) To: Julien Moutinho; +Cc: meta Julien Moutinho <julm+public-inbox@sourcephile.fr> wrote: > Hi Eric, > > A reviewer reported some more test failures > but I cannot reproduce them, any clue at what could cause them? > https://github.com/NixOS/nixpkgs/pull/104457#issuecomment-1054886919 > > Thank you in advance :) > > > t/lei-convert.t .............. 1/? > > # Failed test 'got expected emails in Maildir' > > # at t/lei-convert.t line 63. > > # got: '0' > > # expected: '15' > > # # converted 15 messages Odd, it said it converted 15 messages, but nothing on the FS... I wonder if syslog shows anything. Perhaps it's some weird FS/cache issue? I wouldn't expect it on common Linux FSes, though... Setting TEST_LEI_ERR_LOUD=1 in env for tests may help. I'm not sure I've seen lei-convert.t fail, here; or I've forgotten everything :x > > # Failed test 'Maildir output matches mboxrd' > > # at t/lei-convert.t line 72. > > # Structures begin differing at: > > # $got->[0] = Does not exist > > # $expected->[0] = PublicInbox::Eml=HASH(0x3384268) Looks like that's because of the previous failure > > # Failed test 'mboxrd round-tripped through Maildir w/ flags' > > # at t/lei-convert.t line 79. > > # Structures begin differing at: > > # $got->[0] = PublicInbox::Eml=HASH(0x3505cd0) > > # $expected->[0] = Does not exist ditto > > # Failed test '`seen' flag set in Maildir' > > # at t/lei-convert.t line 125. > > # '/build/pi-lei-convert-5666-Cu05/lei-daemon/md2/tmp/7b1157a4,621d7780,1650,1' > > # doesn't match '(?^::2,S\z)' > > > > # Failed test '`seen' flag preserved' > > # at t/lei-convert.t line 127. > > # '' > > # doesn't match '(?^ms:^Status: RO)' > > # Looks like you failed 5 tests of 50. Probably related to above, something seems odd with Maildirs... > > t/lei-convert.t .............. Dubious, test returned 5 (wstat 1280, 0x500) > > Failed 5/50 subtests <snip> > > t/lei-q-kw.t ................. 1/? Bailout called. Further testing stopped: # /build/pi-lei-q-kw-6884-0YKN/lei-daemon/.local/share/lei/store 1/1 > > FAILED--Further testing stopped: # /build/pi-lei-q-kw-6884-0YKN/lei-daemon/.local/share/lei/store 1/1 Hmm.. anything more from that? (ditto for TEST_LEI_ERR_LOUD) Thanks. ^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2022-03-15 2:56 UTC | newest] Thread overview: 31+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-12-08 1:07 Test failures with 1.7.0 Julien Moutinho 2021-12-08 4:08 ` Eric Wong 2021-12-08 10:56 ` Dominique Martinet 2021-12-08 18:22 ` [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] Eric Wong 2021-12-08 21:14 ` Dominique Martinet 2021-12-08 22:01 ` Dominique Martinet 2022-01-30 21:49 ` Eric Wong 2022-01-30 23:18 ` Dominique Martinet 2022-01-31 2:03 ` Eric Wong 2022-01-31 3:34 ` Dominique Martinet 2022-02-01 1:27 ` Eric Wong 2021-12-09 1:37 ` Test failures with 1.7.0 Julien Moutinho 2021-12-09 2:53 ` Dominique Martinet 2022-02-01 9:37 ` Eric Wong 2022-02-01 23:27 ` FD_CLOEXEC w/ nix-shell [was: Test failures with 1.7.0] Eric Wong 2022-02-02 0:23 ` Dominique Martinet 2022-02-02 2:11 ` Dominique Martinet 2022-02-01 23:34 ` [PATCH] test_lei: use consistent locale for error messages Eric Wong 2022-02-17 21:02 ` [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test Eric Wong 2022-02-20 1:38 ` Julien Moutinho 2022-02-22 6:44 ` Eric Wong 2022-02-27 4:15 ` Julien Moutinho 2022-02-27 6:41 ` Julien Moutinho 2022-02-27 7:23 ` Dominique Martinet 2022-02-27 8:04 ` Julien Moutinho 2022-02-27 11:17 ` [PATCH] t/lei-sigpipe: ensure SIGPIPE is unblocked for this test Eric Wong 2022-03-11 10:42 ` [PATCH] t/lei-sigpipe.t: ensure SIGPIPE is not ignored instead of not blocked Julien Moutinho 2022-03-14 22:14 ` Eric Wong 2022-03-15 2:56 ` Julien Moutinho 2022-03-01 2:30 ` Test failures with 1.7.0 Julien Moutinho 2022-03-01 4:05 ` Eric Wong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).