From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id F03121FA01 for ; Fri, 12 Mar 2021 10:39:43 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 3/3] lei q: mbox*: disable changing parallelism, add --rsyncable Date: Fri, 12 Mar 2021 10:39:43 +0000 Message-Id: <20210312103943.4783-4-e@80x24.org> In-Reply-To: <20210312103943.4783-1-e@80x24.org> References: <20210312103943.4783-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Unfortunately, being mairix-compatible with --threads means we can't change thread-count of gzip, bzip2, or xz when writing to compressed mbox with a --threads= parameter. It's probably not worth changing, anyways, so another switch or additional value for --jobs= won't be added. While we're in the area, add --rsyncable support since most installations of gzip support it nowadays. Fixes: 5beb4a5f6585acd ("lei: replace --thread with --threads") --- lib/PublicInbox/LEI.pm | 2 +- lib/PublicInbox/LeiToMail.pm | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/PublicInbox/LEI.pm b/lib/PublicInbox/LEI.pm index 9bf60ad4..59a3338c 100644 --- a/lib/PublicInbox/LEI.pm +++ b/lib/PublicInbox/LEI.pm @@ -113,7 +113,7 @@ our %CMD = ( # sorted in order of importance/use: qw(save-as=s output|mfolder|o=s format|f=s dedupe|d=s threads|t+ sort|s=s reverse|r offset=i remote! local! external! pretty include|I=s@ exclude=s@ only=s@ jobs|j=s globoff|g augment|a - import-remote! import-before! lock=s@ + import-remote! import-before! lock=s@ rsyncable alert=s@ mua=s no-torsocks torsocks=s verbose|v+ quiet|q C=s@), PublicInbox::LeiQuery::curl_opt(), opt_dash('limit|n=i', '[0-9]+') ], diff --git a/lib/PublicInbox/LeiToMail.pm b/lib/PublicInbox/LeiToMail.pm index 13b4f672..13764d79 100644 --- a/lib/PublicInbox/LeiToMail.pm +++ b/lib/PublicInbox/LeiToMail.pm @@ -170,9 +170,9 @@ sub reap_compress { # dwaitpid callback # { foo => '' } means "--foo" is passed to the command-line, # otherwise { foo => '--bar' } passes "--bar" our %zsfx2cmd = ( - gz => [ qw(GZIP pigz gzip), { rsyncable => '', threads => '-p' } ], + gz => [ qw(GZIP pigz gzip), { rsyncable => '' } ], bz2 => [ 'bzip2', {} ], - xz => [ 'xz', { threads => '-T' } ], + xz => [ 'xz', {} ], # XXX does anybody care for these? I prefer zstd on entire FSes, # so it's probably not necessary on a per-file basis # zst => [ 'zstd', { -default => [ qw(-q) ], # it's noisy by default @@ -203,7 +203,7 @@ sub zsfx2cmd ($$$) { my $switch = $cmd_opt->{rsyncable} // next; push @cmd, '--'.($switch || $bool); } - for my $key (qw(threads)) { # support compression level? + for my $key (qw(rsyncable)) { # support compression level? my $switch = $cmd_opt->{$key} // next; my $val = $lei->{opt}->{$key} // next; push @cmd, $switch, $val;