From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS16276 91.121.0.0/16 X-Spam-Status: No, score=-3.3 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,SPF_HELO_NONE, SPF_PASS,UNPARSEABLE_RELAY shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from nautica.notk.org (nautica.notk.org [91.121.71.147]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 78A431F9FC for ; Thu, 9 Dec 2021 02:54:11 +0000 (UTC) Received: by nautica.notk.org (Postfix, from userid 108) id 2641AC01C; Thu, 9 Dec 2021 03:54:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=codewreck.org; s=2; t=1639018450; bh=ALOkv1KlmMgC6petBhWTXRnpBuxjEilYhf8iSEzUPPE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EtyAxSRFaUgSy/08/GwIx1dIo28Dj1j8xYNQsZkoww4CBTqqGm+uGzqg5QPfkWL0i oXi5fJ9EJGgTT/hcxphYEBAtmxDxx0ImkRcSJr8Hw/M80uazqrkv0X2PxK9V/RrZBT QCwc5M0wpt50Wwa2DkmB+QyQVbx6EvCYZ2kg5wnZ6FtWeyE8Ejz1THqaBIYkDQStpA Ye0vih1nVkxFjkC1XjV5c9SGNmbmV6hyQO8RDkUu5BThVF+d3ytw+7SUOMYPRI8/xD DZ8wQUpwJ2DtuxsFIzig61NpyxKQRefkMySjQLtebuX62WPnVwj02yNB5/owfDdYun BkNen+87ZxF0A== Received: from odin.codewreck.org (localhost [127.0.0.1]) by nautica.notk.org (Postfix) with ESMTPS id 3BC50C009; Thu, 9 Dec 2021 03:54:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=codewreck.org; s=2; t=1639018449; bh=ALOkv1KlmMgC6petBhWTXRnpBuxjEilYhf8iSEzUPPE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=NwLS0qcf1Asy74lDff3tAgi9H+jJc+0onqYh1FdS+wf1O4Ahft66aYrpUYXo2728k CMslUleS+VPp3SX0Ba4Ftvzf9pcWLFab9OkQ35I4B7if9/tJJEx9aNPs8f8FUv7Wrf NUzKP1WBU79NOlYtHhVI6wtxjnSh7FZ0YiZMeIfQrNkUNpRTcF/ik2BGu7CofDPl3Y dgFXjX/KymmfdaqQI58gNerkECZkA0oAKlGyY0ywsRksmz/bgQ2dQ8shnem6rzpX/w YMgOe5MQE0q1S8LZ1BZxmncWFdgD+D781NBLTMz580r7rTlx9s1Hs3NZX19z0Fs2B0 UgU98jGSLVlUQ== Received: from localhost (odin.codewreck.org [local]) by odin.codewreck.org (OpenSMTPD) with ESMTPA id 0c009338; Thu, 9 Dec 2021 02:54:04 +0000 (UTC) Date: Thu, 9 Dec 2021 11:53:49 +0900 From: Dominique Martinet To: Julien Moutinho Cc: Eric Wong , meta@public-inbox.org Subject: Re: Test failures with 1.7.0 Message-ID: References: <20211208010730.f47xxgzj53nwgvja@sourcephile.fr> <20211208040836.GA27368@dcvr> <20211209013743.okzgim7bbrpahks7@sourcephile.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20211209013743.okzgim7bbrpahks7@sourcephile.fr> List-Id: Julien Moutinho wrote on Thu, Dec 09, 2021 at 02:37:43AM +0100: > I can also reproduce Infinisil's test failure with: > $ (cd public-inbox-1.7.0; TMPDIR=/var/tmp perl -I$out/lib/perl5/site_perl t/lei_to_mail.t ) > > ok 96 - got Maildir callback > > Use of uninitialized value in open at t/lei_to_mail.t line 263. > > Bail out! No such file or directory I got curious on this one. strace tells me: ---- 2813384 renameat2(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/tmp/badc0ffee", AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/cur/badc0ffee:2,", RENAME_NOREPLACE) = -1 EINVAL (Invalid argument) 2813384 openat(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/new/", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4 2813384 newfstatat(4, "", {st_mode=S_IFDIR|0755, st_size=2, ...}, AT_EMPTY_PATH) = 0 2813384 brk(0x44f4000) = 0x44f4000 2813384 getdents64(4, 0x44b3e40 /* 2 entries */, 131072) = 48 2813384 getdents64(4, 0x44b3e40 /* 0 entries */, 131072) = 0 2813384 close(4) = 0 2813384 openat(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/cur/", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4 2813384 newfstatat(4, "", {st_mode=S_IFDIR|0755, st_size=2, ...}, AT_EMPTY_PATH) = 0 2813384 getdents64(4, 0x44b3e40 /* 2 entries */, 131072) = 48 2813384 getdents64(4, 0x44b3e40 /* 0 entries */, 131072) = 0 2813384 close(4) = 0 2813384 write(2, "Use of uninitialized value in op"..., 64) = 64 2813384 openat(AT_FDCWD, "", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) 2813384 getpid() = 2813384 2813384 getpid() = 2813384 2813384 getpid() = 2813384 2813384 write(5, "Bail out! No such file or direc"..., 37) = 37 ---- So this one is a real bug, this appears to fix it: ---- >From 50a63628d505ca1c8d36f94ab5703f87a2c5e415 Mon Sep 17 00:00:00 2001 From: Dominique Martinet Date: Thu, 9 Dec 2021 11:50:51 +0900 Subject: [PATCH] syscall: fallback to rename on renameat2 EINVAL ZFS appears to incorrectly return EINVAL on renameat2 when the operation is not supported: renameat2(AT_FDCWD, "...", AT_FDCWD, "...", RENAME_NOREPLACE) = -1 EINVAL Fall back to the racy rename in this case as well: diff --git a/lib/PublicInbox/Syscall.pm b/lib/PublicInbox/Syscall.pm index c00385b94db8..78f926ac38f0 100644 --- a/lib/PublicInbox/Syscall.pm +++ b/lib/PublicInbox/Syscall.pm @@ -15,7 +15,7 @@ package PublicInbox::Syscall; use strict; use v5.10.1; use parent qw(Exporter); -use POSIX qw(ENOENT EEXIST ENOSYS O_NONBLOCK); +use POSIX qw(ENOENT EEXIST ENOSYS EINVAL O_NONBLOCK); use Config; # $VERSION = '0.25'; # Sys::Syscall version @@ -312,7 +312,7 @@ sub rename_noreplace ($$) { my $ret = syscall($SYS_renameat2, -100, $old, -100, $new, 1); if ($ret == 0) { 1; # like rename() perlop - } elsif ($! == ENOSYS) { + } elsif ($! == ENOSYS || $! == EINVAL) { undef $SYS_renameat2; _rename_noreplace_racy($old, $new); } else { ---- > This test does succeed outside Nix's sandbox: > $ (cd public-inbox-1.7.0; export PERL_INLINE_DIRECTORY=$PWD/inline-c; rm -rf $PERL_INLINE_DIRECTORY; mkdir $PERL_INLINE_DIRECTORY; prove -bvw t/lei-sigpipe.t ) > > t/lei-sigpipe.t .. > > ok 1 - lei import $TMPDIR/lei-daemon/big.eml > > ok 2 - read one byte > > ok 3 - signaled > > ok 4 - got SIGPIPE > > ok 5 - quiet after sigpipe > > ok 6 - read one byte > > ok 7 - signaled -f mboxcl2 > > ok 8 - got SIGPIPE -f mboxcl2 > > ok 9 - quiet after sigpipe -f mboxcl2 > > ok 10 - read one byte > > ok 11 - signaled -f text > > ok 12 - got SIGPIPE -f text > > ok 13 - quiet after sigpipe -f text > > ok 14 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:44) > > ok 15 - daemon running after t/lei-sigpipe.t:44 > > ok 16 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:44) > > ok 17 - t/lei-sigpipe.t:44 daemon stopped > > ok 18 - t/lei-sigpipe.t:44 daemon XDG_RUNTIME_DIR/lei/errors.log empty > > 1..18 > > ok > > All tests successful. > > Files=1, Tests=18, 7 wallclock secs ( 0.06 usr 0.06 sys + 3.44 cusr 2.73 csys = 6.29 CPU) > > Result: PASS > > More surprisingly, it even succeeds when run manually > inside the hanging Nix sandbox: > $ sudo nsenter --target 3137110 --all -S 1000 -G 100 $(readlink -e $(which bash)) > $ . /build/env-vars > $ cd /build > $ export HOME=$(mktemp -d) > $ mkdir -p $HOME/.cache/public-inbox/inline-c > $ LANG=C prove -bvw t/lei-sigpipe.t > > t/lei-sigpipe.t .. > > ok 1 - lei import $TMPDIR/lei-daemon/big.eml > > ok 2 - read one byte > > ok 3 - signaled > > ok 4 - got SIGPIPE > > ok 5 - quiet after sigpipe > > ok 6 - read one byte > > ok 7 - signaled -f mboxcl2 > > ok 8 - got SIGPIPE -f mboxcl2 > > ok 9 - quiet after sigpipe -f mboxcl2 > > ok 10 - read one byte > > ok 11 - signaled -f text > > ok 12 - got SIGPIPE -f text > > ok 13 - quiet after sigpipe -f text > > ok 14 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:44) > > ok 15 - daemon running after t/lei-sigpipe.t:44 > > ok 16 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:44) > > ok 17 - t/lei-sigpipe.t:44 daemon stopped > > ok 18 - t/lei-sigpipe.t:44 daemon XDG_RUNTIME_DIR/lei/errors.log empty > > 1..18 > > ok > > All tests successful. > > Files=1, Tests=18, 4 wallclock secs ( 0.06 usr 0.06 sys + 1.23 cusr 1.48 csys = 2.83 CPU) > > Result: PASS > > Even more strange, Dominique was able to reproduce > the hang this morning, but no longer tonight.. Yes, I don't get it, it hanged once but no longer hangs, so as much as I'd have liked to investigate I'm a bit stuck. With this, I can confirm running with inline-c also makes the tests that failed with the btrfs chattr call also pass. So all that's left is fix the proc mounts parsing there :) -- Dominique