* [PATCH 0/4] lei: some net-related things
@ 2021-09-10 9:08 Eric Wong
2021-09-10 9:08 ` [PATCH 1/4] lei_query: fix comment about %lei2curl commands Eric Wong
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Eric Wong @ 2021-09-10 9:08 UTC (permalink / raw)
To: meta
After some consideration, ~/.netrc will no longer be read by
default to match the behavior of existing IMAP/NNTP clients.
And lei-index is pretty limited, but still useful for Maildir
users, so it's documented (mainly for its limitations).
Eric Wong (4):
lei_query: fix comment about %lei2curl commands
lei: split out @net_opt for curl/torsocks use
lei: do not read ~/.netrc by default
doc: lei-index manpage
Documentation/lei-index.pod | 69 ++++++++++++++++++++++++++++++++
MANIFEST | 1 +
Makefile.PL | 2 +-
lib/PublicInbox/GitCredential.pm | 8 +++-
lib/PublicInbox/LEI.pm | 34 ++++++++--------
lib/PublicInbox/LeiQuery.pm | 2 +-
lib/PublicInbox/NetReader.pm | 4 +-
7 files changed, 96 insertions(+), 24 deletions(-)
create mode 100644 Documentation/lei-index.pod
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH 1/4] lei_query: fix comment about %lei2curl commands 2021-09-10 9:08 [PATCH 0/4] lei: some net-related things Eric Wong @ 2021-09-10 9:08 ` Eric Wong 2021-09-10 9:08 ` [PATCH 2/4] lei: split out @net_opt for curl/torsocks use Eric Wong ` (2 subsequent siblings) 3 siblings, 0 replies; 5+ messages in thread From: Eric Wong @ 2021-09-10 9:08 UTC (permalink / raw) To: meta Just a typo. --- lib/PublicInbox/LeiQuery.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/PublicInbox/LeiQuery.pm b/lib/PublicInbox/LeiQuery.pm index 962ad49e..d5f132f1 100644 --- a/lib/PublicInbox/LeiQuery.pm +++ b/lib/PublicInbox/LeiQuery.pm @@ -190,7 +190,7 @@ sub _complete_q { # analogues in git(1)? that would reduce likelyhood of conflicts with # our other CLI options # Note: some names are renamed to avoid potential conflicts, -# see %lei2mail in lib/PublicInbox/LeiCurl.pm +# see %lei2curl in lib/PublicInbox/LeiCurl.pm sub curl_opt { qw( curl-config=s@ abstract-unix-socket=s anyauth basic cacert=s capath=s ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/4] lei: split out @net_opt for curl/torsocks use 2021-09-10 9:08 [PATCH 0/4] lei: some net-related things Eric Wong 2021-09-10 9:08 ` [PATCH 1/4] lei_query: fix comment about %lei2curl commands Eric Wong @ 2021-09-10 9:08 ` Eric Wong 2021-09-10 9:08 ` [PATCH 3/4] lei: do not read ~/.netrc by default Eric Wong 2021-09-10 9:08 ` [PATCH 4/4] doc: lei-index manpage Eric Wong 3 siblings, 0 replies; 5+ messages in thread From: Eric Wong @ 2021-09-10 9:08 UTC (permalink / raw) To: meta IMAP and NNTP connections share some curl(1) options for TLS, IPv4/IPv6, or netrc, etc... --- lib/PublicInbox/LEI.pm | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/lib/PublicInbox/LEI.pm b/lib/PublicInbox/LEI.pm index 3dce0236..bbb6ab7e 100644 --- a/lib/PublicInbox/LEI.pm +++ b/lib/PublicInbox/LEI.pm @@ -147,9 +147,9 @@ sub index_opt { } my @c_opt = qw(c=s@ C=s@ quiet|q); -my @lxs_opt = (qw(remote! local! external! include|I=s@ exclude=s@ only=s@ - import-remote! no-torsocks torsocks=s), - PublicInbox::LeiQuery::curl_opt()); +my @net_opt = (qw(no-torsocks torsocks=s), PublicInbox::LeiQuery::curl_opt()); +my @lxs_opt = qw(remote! local! external! include|I=s@ exclude=s@ only=s@ + import-remote!); # we don't support -C as an alias for --find-copies since it's already # used for chdir @@ -174,7 +174,7 @@ our @diff_opt = qw(unified|U=i output-indicator-new=s output-indicator-old=s our %CMD = ( # sorted in order of importance/use: 'q' => [ '--stdin|SEARCH_TERMS...', 'search for messages matching terms', 'stdin|', # /|\z/ must be first for lone dash - @lxs_opt, + @lxs_opt, @net_opt, qw(save! output|mfolder|o=s format|f=s dedupe|d=s threads|t+ sort|s=s reverse|r offset=i pretty jobs|j=s globoff|g augment|a import-before! lock=s@ rsyncable alert=s@ mua=s verbose|v+ @@ -186,26 +186,26 @@ our %CMD = ( # sorted in order of importance/use: 'lcat' => [ '--stdin|MSGID_OR_URL...', 'display local copy of message(s)', 'stdin|', # /|\z/ must be first for lone dash # some of these options are ridiculous for lcat - @lxs_opt, qw(output|mfolder|o=s format|f=s dedupe|d=s threads|t+ + @lxs_opt, @net_opt, + qw(output|mfolder|o=s format|f=s dedupe|d=s threads|t+ sort|s=s reverse|r offset=i jobs|j=s globoff|g augment|a import-before! lock=s@ rsyncable alert=s@ mua=s verbose|v+ color!), @c_opt, opt_dash('limit|n=i', '[0-9]+') ], 'blob' => [ 'OID', 'show a git blob, reconstructing from mail if necessary', qw(git-dir=s@ cwd! verbose|v+ mail! oid-a|A=s path-a|a=s path-b|b=s), - @lxs_opt, @c_opt ], + @lxs_opt, @net_opt, @c_opt ], 'rediff' => [ '--stdin|LOCATION...', 'regenerate a diff with different options', 'stdin|', # /|\z/ must be first for lone dash qw(git-dir=s@ cwd! verbose|v+ color:s no-color), - @diff_opt, @lxs_opt, @c_opt ], + @diff_opt, @lxs_opt, @net_opt, @c_opt ], 'add-external' => [ 'LOCATION', 'add/set priority of a publicinbox|extindex for extra matches', - qw(boost=i mirror=s no-torsocks torsocks=s inbox-version=i - verbose|v+), @c_opt, index_opt(), - PublicInbox::LeiQuery::curl_opt() ], + qw(boost=i mirror=s inbox-version=i verbose|v+), + @c_opt, index_opt(), @net_opt ], 'ls-external' => [ '[FILTER]', 'list publicinbox|extindex locations', qw(format|f=s z|0 globoff|g invert-match|v local remote), @c_opt ], 'ls-label' => [ '', 'list labels', qw(z|0 stats:s), @c_opt ], @@ -226,15 +226,14 @@ our %CMD = ( # sorted in order of importance/use: 'rm' => [ '--stdin|LOCATION...', 'remove a message from the index and prevent reindexing', 'stdin|', # /|\z/ must be first for lone dash - qw(in-format|F=s lock=s@), @c_opt ], + qw(in-format|F=s lock=s@), @net_opt, @c_opt ], 'plonk' => [ '--threads|--from=IDENT', 'exclude mail matching From: or threads from non-Message-ID searches', qw(stdin| threads|t from|f=s mid=s oid=s), @c_opt ], 'tag' => [ 'KEYWORDS...', 'set/unset keywords and/or labels on message(s)', qw(stdin| in-format|F=s input|i=s@ oid=s@ mid=s@), - qw(no-torsocks torsocks=s), PublicInbox::LeiQuery::curl_opt(), @c_opt, - pass_through('-kw:foo for delete') ], + @net_opt, @c_opt, pass_through('-kw:foo for delete') ], 'purge-mailsource' => [ 'LOCATION|--all', 'remove imported messages from IMAP, Maildirs, and MH', @@ -253,25 +252,24 @@ our %CMD = ( # sorted in order of importance/use: 'index' => [ 'LOCATION...', 'one-time index from URL or filesystem', qw(in-format|F=s kw! offset=i recursive|r exclude=s include|I=s - verbose|v+ incremental!), - PublicInbox::LeiQuery::curl_opt(), # mainly for --proxy= + verbose|v+ incremental!), @net_opt, # mainly for --proxy= @c_opt ], 'import' => [ 'LOCATION...|--stdin', 'one-time import/update from URL or filesystem', qw(stdin| offset=i recursive|r exclude=s include|I=s jobs=s new-only lock=s@ in-format|F=s kw! verbose|v+ incremental! mail-sync!), - qw(no-torsocks torsocks=s), PublicInbox::LeiQuery::curl_opt(), @c_opt ], + @net_opt, @c_opt ], 'forget-mail-sync' => [ 'LOCATION...', 'forget sync information for a mail folder', @c_opt ], 'prune-mail-sync' => [ 'LOCATION...|--all', 'prune dangling sync data for a mail folder', 'all:s', @c_opt ], 'export-kw' => [ 'LOCATION...|--all', 'one-time export of keywords of sync sources', - qw(all:s mode=s), @c_opt ], + qw(all:s mode=s), @net_opt, @c_opt ], 'convert' => [ 'LOCATION...|--stdin', 'one-time conversion from URL or filesystem to another format', qw(stdin| in-format|F=s out-format|f=s output|mfolder|o=s lock=s@ kw!), - qw(no-torsocks torsocks=s), PublicInbox::LeiQuery::curl_opt(), @c_opt ], + @net_opt, @c_opt ], 'p2q' => [ 'FILE|COMMIT_OID|--stdin', "use a patch to generate a query for `lei q --stdin'", qw(stdin| want|w=s@ uri debug), @c_opt ], ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/4] lei: do not read ~/.netrc by default 2021-09-10 9:08 [PATCH 0/4] lei: some net-related things Eric Wong 2021-09-10 9:08 ` [PATCH 1/4] lei_query: fix comment about %lei2curl commands Eric Wong 2021-09-10 9:08 ` [PATCH 2/4] lei: split out @net_opt for curl/torsocks use Eric Wong @ 2021-09-10 9:08 ` Eric Wong 2021-09-10 9:08 ` [PATCH 4/4] doc: lei-index manpage Eric Wong 3 siblings, 0 replies; 5+ messages in thread From: Eric Wong @ 2021-09-10 9:08 UTC (permalink / raw) To: meta Since ~/.netrc isn't widely used by most (if any) NNTP and IMAP clients, we won't read it by default for lei. AFAIK, ~/.netrc is mainly by FTP clients (e.g. ftp(1) and lftp(1)). wget uses it by default for HTTP(S) (and FTP), but curl does not. To avoid breaking stable release use cases, public-inbox-watch continues to read ~/.netrc by default. The --netrc switch is supported by all existing lei commands which may use curl. --- lib/PublicInbox/GitCredential.pm | 8 ++++++-- lib/PublicInbox/NetReader.pm | 4 ++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/PublicInbox/GitCredential.pm b/lib/PublicInbox/GitCredential.pm index c83fed43..b18bba1e 100644 --- a/lib/PublicInbox/GitCredential.pm +++ b/lib/PublicInbox/GitCredential.pm @@ -31,8 +31,12 @@ sub run ($$;$) { close $out_r or die "`git credential $op' failed: \$!=$! \$?=$?\n"; } -sub check_netrc ($) { - my ($self) = @_; +sub check_netrc { + my ($self, $lei) = @_; + + # n.b. lei doesn't load ~/.netrc by default, public-inbox-watch does, + # which may've been a mistake, but we have to live with it. + return if ($lei && !$lei->{opt}->{netrc}); # part of the standard library, but distributions may split it out eval { require Net::Netrc }; diff --git a/lib/PublicInbox/NetReader.pm b/lib/PublicInbox/NetReader.pm index a0e52fc5..f0f56431 100644 --- a/lib/PublicInbox/NetReader.pm +++ b/lib/PublicInbox/NetReader.pm @@ -96,7 +96,7 @@ sub mic_for ($$$$) { # mic = Mail::IMAPClient $cred = undef; } if ($cred) { - my $p = $cred->{password} // $cred->check_netrc; + my $p = $cred->{password} // $cred->check_netrc($lei); $cred->fill($lei) unless defined($p); # may prompt user here $mic->User($mic_arg->{User} = $cred->{username}); $mic->Password($mic_arg->{Password} = $cred->{password}); @@ -191,7 +191,7 @@ sub nn_for ($$$$) { # nn = Net::NNTP }, 'PublicInbox::GitCredential'; ($u, $p) = split(/:/, $ui, 2); ($cred->{username}, $cred->{password}) = ($u, $p); - $p //= $cred->check_netrc; + $p //= $cred->check_netrc($lei); } my $common = $nn_common->{$sec} // {}; my $nn_arg = { ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 4/4] doc: lei-index manpage 2021-09-10 9:08 [PATCH 0/4] lei: some net-related things Eric Wong ` (2 preceding siblings ...) 2021-09-10 9:08 ` [PATCH 3/4] lei: do not read ~/.netrc by default Eric Wong @ 2021-09-10 9:08 ` Eric Wong 3 siblings, 0 replies; 5+ messages in thread From: Eric Wong @ 2021-09-10 9:08 UTC (permalink / raw) To: meta It's a pretty incomplete command, so it's important to document its incompleteness. --- Documentation/lei-index.pod | 69 +++++++++++++++++++++++++++++++++++++ MANIFEST | 1 + Makefile.PL | 2 +- 3 files changed, 71 insertions(+), 1 deletion(-) create mode 100644 Documentation/lei-index.pod diff --git a/Documentation/lei-index.pod b/Documentation/lei-index.pod new file mode 100644 index 00000000..bd125bcc --- /dev/null +++ b/Documentation/lei-index.pod @@ -0,0 +1,69 @@ +=head1 NAME + +lei-index - index messages without importing them into lei/store + +=head1 SYNOPSIS + +lei index [OPTIONS] FOLDER + +lei index [OPTIONS] --stdin + +=head1 DESCRIPTION + +Similar to L<lei-import(1)>, but does not store a copy of +messages into C<lei/store>. + +This command only makes sense for messages stored in Maildir +folders. Other folder types may be supported in the future +(they can all be indexed, but the message isn't automatically +retrieved by L<lei-q(1)> or L<lei-lcat(1)>). + +Combined with L<lei-q(1)>, C<lei index> allows Maildir users to +have similar functionality to L<mairix(1)> by not duplicating +messages into C<lei/store>. + +=head1 OPTIONS + +=over + +=item - + +=item --stdin + +Read input from standard input. This is the default if standard +input is a pipe or regular file and there are no arguments on +the command-line. + +=item -F MAIL_FORMAT + +=item --in-format=MAIL_FORMAT + +Message input format: C<eml>, C<maildir>, C<imap>, C<imaps>, C<nntp>, +C<nntps>, C<mboxrd>, C<mboxcl2>, C<mboxcl>, or C<mboxo>. + +Default: C<eml> when reading from stdin + +=item -q + +=item --quiet + +Suppress feedback messages. + +=back + +=head1 CONTACT + +Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org> + +The mail archives are hosted at L<https://public-inbox.org/meta/> and +L<http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/> + +=head1 COPYRIGHT + +Copyright all contributors L<mailto:meta@public-inbox.org> + +License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt> + +=head1 SEE ALSO + +L<lei-store-format(5)>, L<lei-import(1)> diff --git a/MANIFEST b/MANIFEST index 531f8c46..c0e3e855 100644 --- a/MANIFEST +++ b/MANIFEST @@ -31,6 +31,7 @@ Documentation/lei-edit-search.pod Documentation/lei-forget-external.pod Documentation/lei-forget-search.pod Documentation/lei-import.pod +Documentation/lei-index.pod Documentation/lei-init.pod Documentation/lei-lcat.pod Documentation/lei-ls-external.pod diff --git a/Makefile.PL b/Makefile.PL index 82b50543..bfabb171 100644 --- a/Makefile.PL +++ b/Makefile.PL @@ -47,7 +47,7 @@ $v->{-m1} = [ map { qw( lei-add-external lei-blob lei-config lei-convert lei-edit-search lei-daemon-kill lei-daemon-pid lei-forget-external lei-forget-search - lei-import lei-init lei-lcat lei-ls-external lei-ls-label + lei-import lei-index lei-init lei-lcat lei-ls-external lei-ls-label lei-ls-mail-sync lei-ls-search lei-p2q lei-q lei-rediff lei-rm lei-tag lei-up)]; $v->{-m5} = [ qw(public-inbox-config public-inbox-v1-format ^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-09-10 9:08 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-09-10 9:08 [PATCH 0/4] lei: some net-related things Eric Wong 2021-09-10 9:08 ` [PATCH 1/4] lei_query: fix comment about %lei2curl commands Eric Wong 2021-09-10 9:08 ` [PATCH 2/4] lei: split out @net_opt for curl/torsocks use Eric Wong 2021-09-10 9:08 ` [PATCH 3/4] lei: do not read ~/.netrc by default Eric Wong 2021-09-10 9:08 ` [PATCH 4/4] doc: lei-index manpage Eric Wong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).