unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
From: Eric Wong <e@80x24.org>
To: Dominique Martinet <asmadeus@codewreck.org>
Cc: Julien Moutinho <julm+public-inbox@sourcephile.fr>,
	meta@public-inbox.org
Subject: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0]
Date: Wed, 8 Dec 2021 18:22:47 +0000	[thread overview]
Message-ID: <20211208182247.M197857@dcvr> (raw)
In-Reply-To: <YbCPWGaJEkV6eWfo@codewreck.org>

Dominique Martinet <asmadeus@codewreck.org> wrote:
> Eric Wong wrote on Wed, Dec 08, 2021 at 04:08:36AM +0000:
> > > And another reviewer fails 3 times with:
> > > > t/extsearch.t              (Wstat: 2048 Tests: 145 Failed: 8)
> > > >   Failed tests:  68-69, 75-76, 86, 98, 102, 139
> > > >   Non-zero exit status: 8
> > > > t/imapd.t                  (Wstat: 256 Tests: 186 Failed: 1)
> > > >   Failed test:  183
> > > >   Non-zero exit status: 1
> > > > t/nntpd.t                  (Wstat: 256 Tests: 110 Failed: 1)
> > > >   Failed test:  104
> > > >   Non-zero exit status: 1
> 
> (I'm that one)
> 
> So instead of giving the output of prove -bvw t/nntpd.t (I couldn't
> reproduce manually..), I'll go straight to the solution: it looks like
> there's a problem with the chattr fallback when btrfs is present on the
> machine (to disable cow, lib/PublicInbox/NDC_PP.pm)

Hi, thanks for the extra info.

Note: You might want to rm -rf ~/.cache/public-inbox/inline-c or
try a different HOME directory to get a clean repro (more on
this later).

I also forget to note, setting TAIL='tail -F' can dump daemon
output for tests (works better for GNU tail w/ inotify support).

> I had initial messages about chattr not being found; adding chattr to
> the build inputs gave another error message that chattr +C doesn't work
> on tmpfs.
> The problem here seems to be that the fallback here just checks
> /proc/self/mounts and calls chattr "just in case", saying it's ok if it
> fails, but some tests probably rely on stderr output being empty?

Yes on tests requiring stderr to be empty.  Below is a patch
which should fix it; however it should only be calling chattr on
btrfs mounts.

So I wonder if our /proc/self/mounts parsing is broken...
Can you share your contents of /proc/self/mounts?
(maybe spaces/odd chars aren't handled properly...)

You can also try:

  BTRFS_TESTDIR=/path/to/your/btrfs-mount prove -bvw t/nodatacow.t

?

> If so it looks more like a test problem than anything else to me, but
> perhaps the fallback path could be a bit more prudent in its calling of
> chattr (or simply try silencing its output?)

Agreed, the patch below silences stderr.

> For testing, being a bit forceful and removing that chattr call made all
> the tests pass for me.

> Now the $100 question is I don't know why the Inline::C version wasn't
> used; this one properly calls statfs() and does the ioctl directly only
> if required so would have worked.
> As far as I can see we're installing Inline::C in the build chroot, I
> see it in the PERL5LIB paths; and make/gcc are also available so it
> probably should work. Is there a way to check that?

For public-inbox-* stuff, Inline::C isn't enabled by default
even if installed.   Either ~/.cache/public-inbox/inline-c needs
to exist or PERL_INLINE_DIRECTORY must be set in env.

However, lei will auto-create ~/.cache/public-inbox/inline-c
if Socket::MsgHdr is missing (since lei is new w/ no existing users).
Thus running lei will enable Inline::C for public-inbox-* commands.

<snip>

Thanks again for all the extra info.  Hopefully this below is
one step towards making things work more smoothly, but I also
suspect /proc/self/mount parsing is off...

------8<-----
Subject: [PATCH] nodatacow: quiet chattr errors

Maybe this helps when chattr is missing or our /proc/self/mount
parsing is broken, somehow.

Link: https://public-inbox.org/meta/YbCPWGaJEkV6eWfo@codewreck.org/
---
 lib/PublicInbox/NDC_PP.pm | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/lib/PublicInbox/NDC_PP.pm b/lib/PublicInbox/NDC_PP.pm
index 57abccbe0625..a84682d371c8 100644
--- a/lib/PublicInbox/NDC_PP.pm
+++ b/lib/PublicInbox/NDC_PP.pm
@@ -16,11 +16,13 @@ sub nodatacow_dir ($) {
 		# weird chars are escaped as octal
 		$mnt_path =~ s/\\(0[0-9]{2})/chr(oct($1))/egs;
 		$mnt_path .= '/' unless $mnt_path =~ m!/\z!;
-		if (index($path, $mnt_path) == 0) {
-			# error goes to stderr, but non-fatal for us
-			system('chattr', '+C', $path);
-			last;
-		}
+		next if index($path, $mnt_path) < 0;
+
+		open my $olderr, '>&', \*STDERR or die "dup(STDERR): $!";
+		open STDERR, '>', '/dev/null' or die "open(/dev/null): $!";
+		system(qw(chattr +C), $path); # ignore error
+		open STDERR, '>&', $olderr or die "dup2(..,STDERR): $!";
+		last;
 	}
 }
 

  reply	other threads:[~2021-12-08 18:22 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-08  1:07 Test failures with 1.7.0 Julien Moutinho
2021-12-08  4:08 ` Eric Wong
2021-12-08 10:56   ` Dominique Martinet
2021-12-08 18:22     ` Eric Wong [this message]
2021-12-08 21:14       ` [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] Dominique Martinet
2021-12-08 22:01         ` Dominique Martinet
2022-01-30 21:49           ` Eric Wong
2022-01-30 23:18             ` Dominique Martinet
2022-01-31  2:03               ` Eric Wong
2022-01-31  3:34                 ` Dominique Martinet
2022-02-01  1:27                   ` Eric Wong
2021-12-09  1:37     ` Test failures with 1.7.0 Julien Moutinho
2021-12-09  2:53       ` Dominique Martinet
2022-02-01  9:37         ` Eric Wong
2022-02-01 23:27       ` FD_CLOEXEC w/ nix-shell [was: Test failures with 1.7.0] Eric Wong
2022-02-02  0:23         ` Dominique Martinet
2022-02-02  2:11           ` Dominique Martinet
2022-02-01 23:34       ` [PATCH] test_lei: use consistent locale for error messages Eric Wong
2022-02-17 21:02       ` [PATCH] t/lei-sigpipe: attempt to improve diagnostics for stuck test Eric Wong
2022-02-20  1:38         ` Julien Moutinho
2022-02-22  6:44           ` Eric Wong
2022-02-27  4:15             ` Julien Moutinho
2022-02-27  6:41               ` Julien Moutinho
2022-02-27  7:23                 ` Dominique Martinet
2022-02-27  8:04                   ` Julien Moutinho
2022-02-27 11:17                     ` [PATCH] t/lei-sigpipe: ensure SIGPIPE is unblocked for this test Eric Wong
2022-03-11 10:42                       ` [PATCH] t/lei-sigpipe.t: ensure SIGPIPE is not ignored instead of not blocked Julien Moutinho
2022-03-14 22:14                         ` Eric Wong
2022-03-15  2:56                           ` Julien Moutinho
2022-03-01  2:30   ` Test failures with 1.7.0 Julien Moutinho
2022-03-01  4:05     ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211208182247.M197857@dcvr \
    --to=e@80x24.org \
    --cc=asmadeus@codewreck.org \
    --cc=julm+public-inbox@sourcephile.fr \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).