* Re: RFC: lei searches managed by users in git
2023-09-15 21:08 RFC: lei searches managed by users in git Konstantin Ryabitsev
@ 2023-09-15 22:47 ` Eric Wong
0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2023-09-15 22:47 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: meta
Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> Hello:
>
> I am curious what is the best approach to have a centrally managed set of lei
> searches, for example via config files tracked in git. For example, the file
> could look like this:
I don't have one nor have I thought about it until now...
I wonder if `lei edit-search --all' could be a thing...
As would adding gitconfig output to ls-search
(`lei ls-search -l -f gitconfig').
Maybe `lei edit-search --all=lei.q' could cut down
on non-interesting fields when editing all searches.
I also wouldn't be opposed to supporting non-interactive use
for automation:
lei edit-search --set lei.q=foo OUTPUT
lei edit-search --add lei.q=foo OUTPUT
...but maybe it's not needed (see below on `lei q'):
> mricon.toml:
Any particular reason for toml? I don't have a toml parser
installed and I suspect many don't, either. git-config is
widely available and installed, of course. I've been trying
hard to avoid data format and dependency proliferation.
> [search.torvalds]
> # All mail sent by torvalds
> q = 'f:torvalds@linux-foundation.org'
> [search.floppy]
> # Any messages talking about floppies or touching floppy code
> q = 'dfhh:floppy_* OR dfn:drivers/block/floppy.c OR s:floppy OR ((nq:bug OR nq:regression) AND nq:floppy)'
>
> I could then have a small wrapper maintaining saved searches and making the
> mailboxes available via special newsgroups like:
>
> org.kernel.lei.mricon.torvalds
> org.kernel.lei.mricon.floppy
Sidenote: I think `query' or similar is more appropriate than
using `lei' in a public newsgroup name since it's not `local' :>
> The goal is to make it possible for maintainers to define their own set of
> saved searches and have access to them at kernel.org via imap/pop3/nntp.
>
> It's easy to write a simple wrapper that would invoke lei-edit-search and
> replace the search string when there are updates to the config files, but I'm
> curious if you already have thoughts on how to best implement something like
> this.
Rerunning `lei q' on existing destinations is fine for updating
existing queries. This is especially nice with `-f v2', since
dedupe is faster on v2 than any other output format.
Thus there's no need to even bother with `lei edit-search' if
you're doing this non-interactively with your own files/formats
(e.g. via web UI or something).
> My biggest concern is someone committing an invalid query and not receiving
> any more email as a result -- so having a sane way to validate the query
> before sticking it into the saved search would be handy.
Perhaps `lei q --limit=1' could be appropriate for validating queries
with the below fix. But validating searches can ahead of time is
a TOCTOU problem. There can be added support for per-inbox prefixes
(e.g. altid with gmane: on public-inbox.org/git), and also when mixing
old/new versions via different HTTP(S) externals.
-------8<-------
Subject: [PATCH] lei q: set exit code for invalid Xapian queries
Xapian can't parse every query, so ensure we set the
exit code for the client.
---
lib/PublicInbox/LeiXSearch.pm | 6 ++++--
t/lei.t | 5 +++++
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/lib/PublicInbox/LeiXSearch.pm b/lib/PublicInbox/LeiXSearch.pm
index 5965274c..7f4911b3 100644
--- a/lib/PublicInbox/LeiXSearch.pm
+++ b/lib/PublicInbox/LeiXSearch.pm
@@ -186,7 +186,8 @@ sub query_one_mset { # for --threads and l2m w/o sort
}
my $first_ids;
do {
- $mset = $srch->mset($mo->{qstr}, $mo);
+ $mset = eval { $srch->mset($mo->{qstr}, $mo) };
+ return $lei->child_error(22 << 8, "E: $@") if $@; # 22 from curl
mset_progress($lei, $dir, $mo->{offset} + $mset->size,
$mset->get_matches_estimated);
wait_startq($lei); # wait for keyword updates
@@ -249,7 +250,8 @@ sub query_combined_mset { # non-parallel for non-"--threads" users
}
my $each_smsg = $lei->{ovv}->ovv_each_smsg_cb($lei);
do {
- $mset = $self->mset($mo->{qstr}, $mo);
+ $mset = eval { $self->mset($mo->{qstr}, $mo) };
+ return $lei->child_error(22 << 8, "E: $@") if $@; # 22 from curl
mset_progress($lei, 'xsearch', $mo->{offset} + $mset->size,
$mset->get_matches_estimated);
wait_startq($lei); # wait for keyword updates
diff --git a/t/lei.t b/t/lei.t
index d83bde69..1199ca75 100644
--- a/t/lei.t
+++ b/t/lei.t
@@ -147,6 +147,11 @@ my $test_fail = sub {
lei_ok('q', "foo\n");
like($lei_err, qr/trailing `\\n' removed/s, "noted `\\n' removal");
+ lei(qw(q from:infinity..));
+ is($? >> 8, 22, 'combined query fails on invalid range op');
+ lei(qw(q -t from:infinity..));
+ is($? >> 8, 22, 'single query fails on invalid range op');
+
for my $lk (qw(ei inbox)) {
my $d = "$home/newline\n$lk";
my $all = $lk eq 'ei' ? 'ALL' : 'all';
^ permalink raw reply related [flat|nested] 2+ messages in thread