From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 09/12] lei q: show progress on >1s preparation phase
Date: Tue, 21 Sep 2021 07:41:56 +0000 [thread overview]
Message-ID: <20210921074159.20052-10-e@80x24.org> (raw)
In-Reply-To: <20210921074159.20052-1-e@80x24.org>
Overwriting existing destinations safe (but slow) by default,
so show a progress message noting what we're doing while
a user waits.
---
lib/PublicInbox/LeiToMail.pm | 23 ++++++++++++++++++++++-
lib/PublicInbox/LeiXSearch.pm | 2 +-
t/lei-watch.t | 2 +-
3 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/lib/PublicInbox/LeiToMail.pm b/lib/PublicInbox/LeiToMail.pm
index a419b83f..ed609081 100644
--- a/lib/PublicInbox/LeiToMail.pm
+++ b/lib/PublicInbox/LeiToMail.pm
@@ -689,6 +689,8 @@ sub do_augment { # slow, runs in wq worker
# fast (spawn compressor or mkdir), runs in same process as pre_augment
sub post_augment {
my ($self, $lei, @args) = @_;
+ $self->{-au_noted}++ and $lei->qerr("# writing to $self->{dst} ...");
+
my $wait = $lei->{opt}->{'import-before'} ?
$lei->{sto}->wq_do('checkpoint', 1) : 0;
# _post_augment_mbox
@@ -784,9 +786,28 @@ sub wq_atexit_child {
$lei->{pkt_op_p}->pkt_do('l2m_progress', $nr);
}
+# runs on a 1s timer in lei-daemon
+sub augment_inprogress {
+ my ($err, $opt, $dst, $au_noted) = @_;
+ $$au_noted++ and return;
+ print $err '# '.($opt->{'import-before'} ?
+ "importing non-external contents of $dst" : (
+ ($opt->{dedupe} // 'content') ne 'none') ?
+ "scanning old contents of $dst for dedupe" :
+ "removing old contents of $dst")." ...\n";
+}
+
# called in top-level lei-daemon when LeiAuth is done
sub net_merge_all_done {
- my ($self) = @_;
+ my ($self, $lei) = @_;
+ if ($PublicInbox::DS::in_loop &&
+ $self->can("_do_augment_$self->{base_type}") &&
+ !$lei->{opt}->{quiet}) {
+ $self->{-au_noted} = 0;
+ PublicInbox::DS::add_timer(1, \&augment_inprogress,
+ $lei->{2}, $lei->{opt},
+ $self->{dst}, \$self->{-au_noted});
+ }
$self->wq_broadcast('do_post_auth');
$self->wq_close(1);
}
diff --git a/lib/PublicInbox/LeiXSearch.pm b/lib/PublicInbox/LeiXSearch.pm
index 3ce8f32d..2227c2ac 100644
--- a/lib/PublicInbox/LeiXSearch.pm
+++ b/lib/PublicInbox/LeiXSearch.pm
@@ -572,7 +572,7 @@ sub do_query {
$self->{opt_sort} = $lei->{opt}->{'sort'};
$self->{-do_lcat} = !!(delete $lei->{lcat_todo});
if ($l2m) {
- $l2m->net_merge_all_done unless $lei->{auth};
+ $l2m->net_merge_all_done($lei) unless $lei->{auth};
} else {
start_query($self);
}
diff --git a/t/lei-watch.t b/t/lei-watch.t
index a881fbb9..df887a03 100644
--- a/t/lei-watch.t
+++ b/t/lei-watch.t
@@ -91,7 +91,7 @@ test_lei(sub {
$ino_fdinfo or skip 'Linux/inotify-only removal removal', 1;
open my $fh, '<', $ino_fdinfo or xbail "open $ino_fdinfo: $!";
my $cmp = [ <$fh> ];
- is_deeply($cmp, $ino_contents, 'inotify Maildir watches gone');
+ is_xdeeply($cmp, $ino_contents, 'inotify Maildir watches gone');
};
});
next prev parent reply other threads:[~2021-09-21 7:41 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-21 7:41 [PATCH 00/12] lei: fix various annoyances Eric Wong
2021-09-21 7:41 ` [PATCH 01/12] lei inspect: convert to WQ worker Eric Wong
2021-09-21 7:41 ` [PATCH 02/12] lei inspect: support NNTP URLs Eric Wong
2021-09-21 7:41 ` [PATCH 03/12] lei_mail_sync: account for non-unique cases Eric Wong
2021-09-21 7:41 ` [PATCH 04/12] lei: simplify internal arg2folder usage Eric Wong
2021-09-21 7:41 ` [PATCH 05/12] lei lcat: use single queue for ordering Eric Wong
2021-09-21 7:41 ` [PATCH 06/12] doc: lei-security: section for WIP auth methods Eric Wong
2021-09-21 7:41 ` [PATCH 07/12] lei lcat: support NNTP URLs Eric Wong
2021-09-21 7:41 ` [PATCH 08/12] lei: various completion improvements Eric Wong
2021-09-21 7:41 ` Eric Wong [this message]
2021-09-21 7:41 ` [PATCH 10/12] search: drop reopen retry message Eric Wong
2021-09-21 7:41 ` [PATCH 11/12] lei q: update messages to reflect --save default Eric Wong
2021-09-21 7:41 ` [PATCH 12/12] lei q: improve --limit behavior and progress Eric Wong
2021-09-21 9:29 ` [PATCH 0/3] lei: a few more annoyances fixed Eric Wong
2021-09-21 9:29 ` [PATCH 1/3] t/lei-up: use '-q' to silence non-redirected test Eric Wong
2021-09-21 9:29 ` [PATCH 2/3] script/lei: handle SIGTSTP and SIGCONT Eric Wong
2021-09-21 9:29 ` [PATCH 3/3] lei: umask(077) before opening errors.log Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210921074159.20052-10-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).