unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
* [PATCH 0/3] lei: diagnostic and reliability fix
@ 2021-05-28  9:45 Eric Wong
  2021-05-28  9:45 ` [PATCH 1/3] t/lei-*: better diagnostics for occasional failures Eric Wong
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Eric Wong @ 2021-05-28  9:45 UTC (permalink / raw)
  To: meta

1/3 may no longer be needed, but doesn't hurt, either.
It was made while tracking down occasional failures for
v3 of "lei: handle a single IMAP message in most places":
https://public-inbox.org/meta/20210528093740.GB18768@dcvr/

2/3 is necessary for parallel invocations from different dirs,
and in multi-tenant situations; and I just noticed 3/3 because
I had a typo somewhere else :x

Eric Wong (3):
  t/lei-*: better diagnostics for occasional failures
  lei: restore working directory in more places
  script/lei: drop leftover message about fallback

 lib/PublicInbox/LEI.pm        | 6 ++++++
 lib/PublicInbox/LeiLcat.pm    | 4 +---
 lib/PublicInbox/LeiQuery.pm   | 4 +---
 lib/PublicInbox/LeiXSearch.pm | 1 +
 script/lei                    | 1 -
 t/lei-q-kw.t                  | 2 +-
 t/lei-q-remote-import.t       | 6 +++---
 t/lei-q-save.t                | 6 ++++--
 8 files changed, 17 insertions(+), 13 deletions(-)


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/3] t/lei-*: better diagnostics for occasional failures
  2021-05-28  9:45 [PATCH 0/3] lei: diagnostic and reliability fix Eric Wong
@ 2021-05-28  9:45 ` Eric Wong
  2021-05-28  9:45 ` [PATCH 2/3] lei: restore working directory in more places Eric Wong
  2021-05-28  9:45 ` [PATCH 3/3] script/lei: drop leftover message about fallback Eric Wong
  2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2021-05-28  9:45 UTC (permalink / raw)
  To: meta

Some of these have been failing occasionally, not sure
how, yet...
---
 t/lei-q-kw.t            | 2 +-
 t/lei-q-remote-import.t | 6 +++---
 t/lei-q-save.t          | 6 ++++--
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/t/lei-q-kw.t b/t/lei-q-kw.t
index 074c573d..528751b4 100644
--- a/t/lei-q-kw.t
+++ b/t/lei-q-kw.t
@@ -116,7 +116,7 @@ for my $sfx ('', '.gz') {
 		}
 		$res{$mid} = $eml;
 	});
-	is_deeply(\%res, $exp, '--augment worked');
+	is_deeply(\%res, $exp, '--augment worked') or diag $lei_err;
 
 	lei_ok(qw(q -o), "mboxrd:/dev/stdout", qw(m:qp@example.com)) or
 		diag $lei_err;
diff --git a/t/lei-q-remote-import.t b/t/lei-q-remote-import.t
index 80067061..7db684d9 100644
--- a/t/lei-q-remote-import.t
+++ b/t/lei-q-remote-import.t
@@ -49,10 +49,10 @@ test_lei({ tmpdir => $tmpdir }, sub {
 
 	open my $fh, '>', "$o.lock";
 	$cmd[-1] = 'm:qp@example.com';
-	unlink $o or BAIL_OUT $!;
+	unlink $o or xbail("unlink $o $! cwd=".Cwd::getcwd());
 	lei_ok(@cmd, '--lock=none');
-	ok(-f $o && -s _, '--lock=none respected');
-	unlink $o or BAIL_OUT $!;
+	ok(-f $o && -s _, '--lock=none respected') or diag $lei_err;
+	unlink $o or xbail("unlink $o $! cwd=".Cwd::getcwd());
 	ok(!lei(@cmd, '--lock=dotlock,timeout=0.000001'), 'dotlock fails');
 	ok(-f $o && !-s _, 'nothing output on lock failure');
 	unlink "$o.lock" or BAIL_OUT $!;
diff --git a/t/lei-q-save.t b/t/lei-q-save.t
index aed38a51..bea65133 100644
--- a/t/lei-q-save.t
+++ b/t/lei-q-save.t
@@ -25,7 +25,8 @@ test_lei(sub {
 	lei_ok [qw(import -q -F eml -)], undef, { 0 => \$in, %$lei_opt };
 	lei_ok qw(q -q --save z:0.. d:last.week..), '-o', "MAILDIR:$home/md/";
 	my %before = map { $_ => 1 } glob("$home/md/cur/*");
-	is_deeply(eml_load((keys %before)[0]), $doc1, 'doc1 matches');
+	my $f = (keys %before)[0] or xbail({before => \%before});
+	is_deeply(eml_load($f), $doc1, 'doc1 matches');
 	lei_ok qw(ls-mail-sync);
 	is($lei_out, "maildir:$home/md\n", 'canonicalized mail sync name');
 
@@ -45,7 +46,8 @@ test_lei(sub {
 	my %after = map { $_ => 1 } glob("$home/md/{new,cur}/*");
 	is(delete $after{(keys(%before))[0]}, 1, 'original message kept');
 	is(scalar(keys %after), 1, 'one new message added');
-	is_deeply(eml_load((keys %after)[0]), $doc2, 'doc2 matches');
+	$f = (keys %after)[0] or xbail({after => \%after});
+	is_deeply(eml_load($f), $doc2, 'doc2 matches');
 
 	# check stdin
 	lei_ok [qw(q --save - -o), "mboxcl2:mbcl2" ],

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 2/3] lei: restore working directory in more places
  2021-05-28  9:45 [PATCH 0/3] lei: diagnostic and reliability fix Eric Wong
  2021-05-28  9:45 ` [PATCH 1/3] t/lei-*: better diagnostics for occasional failures Eric Wong
@ 2021-05-28  9:45 ` Eric Wong
  2021-05-28  9:45 ` [PATCH 3/3] script/lei: drop leftover message about fallback Eric Wong
  2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2021-05-28  9:45 UTC (permalink / raw)
  To: meta

Every tick of the event loop can change the working directory,
so we need to restore it for every client if they operate
in different directories.

This would be easier if we had openat(2) and friends in Perl;
but Inline::C is practically required for lei, now.
---
 lib/PublicInbox/LEI.pm        | 6 ++++++
 lib/PublicInbox/LeiLcat.pm    | 4 +---
 lib/PublicInbox/LeiQuery.pm   | 4 +---
 lib/PublicInbox/LeiXSearch.pm | 1 +
 4 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/lib/PublicInbox/LEI.pm b/lib/PublicInbox/LEI.pm
index 07378ca7..e5ff9e5d 100644
--- a/lib/PublicInbox/LEI.pm
+++ b/lib/PublicInbox/LEI.pm
@@ -1239,6 +1239,12 @@ sub wq_done_wait { # dwaitpid callback
 	$lei->dclose;
 }
 
+sub fchdir {
+	my ($lei) = @_;
+	my $dh = $lei->{3} // die 'BUG: lei->{3} (CWD) gone';
+	chdir($dh) || $lei->fail("fchdir: $!");
+}
+
 sub wq_eof { # EOF callback for main daemon
 	my ($lei) = @_;
 	my $wq1 = delete $lei->{wq1} // return $lei->fail; # already failed
diff --git a/lib/PublicInbox/LeiLcat.pm b/lib/PublicInbox/LeiLcat.pm
index 0f585ff5..f9d9633a 100644
--- a/lib/PublicInbox/LeiLcat.pm
+++ b/lib/PublicInbox/LeiLcat.pm
@@ -89,9 +89,7 @@ sub _stdin { # PublicInbox::InputPipe::consume callback for --stdin
 	my ($lei) = @_; # $_[1] = $rbuf
 	if (defined($_[1])) {
 		$_[1] eq '' and return eval {
-			if (my $dfd = $lei->{3}) {
-				chdir($dfd) or return $lei->fail("fchdir: $!");
-			}
+			$lei->fchdir or return;
 			my @argv = split(/\s+/, $lei->{mset_opt}->{qstr});
 			$lei->{mset_opt}->{qstr} = extract_all($lei, @argv)
 				or return;
diff --git a/lib/PublicInbox/LeiQuery.pm b/lib/PublicInbox/LeiQuery.pm
index 1999a534..0435a516 100644
--- a/lib/PublicInbox/LeiQuery.pm
+++ b/lib/PublicInbox/LeiQuery.pm
@@ -51,9 +51,7 @@ sub qstr_add { # PublicInbox::InputPipe::consume callback for --stdin
 	my ($self) = @_; # $_[1] = $rbuf
 	if (defined($_[1])) {
 		$_[1] eq '' and return eval {
-			if (my $dfd = $self->{3}) {
-				chdir($dfd) or return $self->fail("fchdir: $!");
-			}
+			$self->fchdir or return;
 			$self->{mset_opt}->{q_raw} = $self->{mset_opt}->{qstr};
 			$self->{lse}->query_approxidate($self->{lse}->git,
 						$self->{mset_opt}->{qstr});
diff --git a/lib/PublicInbox/LeiXSearch.pm b/lib/PublicInbox/LeiXSearch.pm
index e2a8e8e3..760f9718 100644
--- a/lib/PublicInbox/LeiXSearch.pm
+++ b/lib/PublicInbox/LeiXSearch.pm
@@ -394,6 +394,7 @@ Error closing $lei->{ovv}->{dst}: $!
 sub do_post_augment {
 	my ($lei) = @_;
 	my $l2m = $lei->{l2m} or return; # client disconnected
+	$lei->fchdir or return;
 	my $err;
 	eval { $l2m->post_augment($lei) };
 	$err = $@;

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 3/3] script/lei: drop leftover message about fallback
  2021-05-28  9:45 [PATCH 0/3] lei: diagnostic and reliability fix Eric Wong
  2021-05-28  9:45 ` [PATCH 1/3] t/lei-*: better diagnostics for occasional failures Eric Wong
  2021-05-28  9:45 ` [PATCH 2/3] lei: restore working directory in more places Eric Wong
@ 2021-05-28  9:45 ` Eric Wong
  2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2021-05-28  9:45 UTC (permalink / raw)
  To: meta

Non-daemon lei isn't implemented, anymore.
---
 script/lei | 1 -
 1 file changed, 1 deletion(-)

diff --git a/script/lei b/script/lei
index 4d752fd8..fce8124a 100755
--- a/script/lei
+++ b/script/lei
@@ -98,7 +98,6 @@ lei-daemon could not start, exited with \$?=$?
 	# try connecting again anyways, unlink+bind may be racy
 	connect($sock, $addr) or die <<"";
 connect($path): $! (after attempted daemon start)
-Falling back to (slow) one-shot mode
 
 }
 # (Socket::MsgHdr|Inline::C), $sock are all available:

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-05-28  9:45 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-28  9:45 [PATCH 0/3] lei: diagnostic and reliability fix Eric Wong
2021-05-28  9:45 ` [PATCH 1/3] t/lei-*: better diagnostics for occasional failures Eric Wong
2021-05-28  9:45 ` [PATCH 2/3] lei: restore working directory in more places Eric Wong
2021-05-28  9:45 ` [PATCH 3/3] script/lei: drop leftover message about fallback Eric Wong

unofficial mirror of meta@public-inbox.org

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://yhetil.org/meta

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 meta meta/ https://yhetil.org/meta \
		meta@public-inbox.org
	public-inbox-index meta

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.yhetil.org/yhetil.mail.public-inbox.meta
	nntp://news.public-inbox.org/inbox.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general


AGPL code for this site: git clone http://ou63pmih66umazou.onion/public-inbox.git