From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 136A41F55F; Fri, 15 Sep 2023 22:47:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1694818021; bh=cNS7sDlN3P8UR2K5FHs/1rEw9Z34vaxkkQ/08FVwKsI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=vbDQYG2aYFis7yxlb6l8FnbRKkFn9G9YCpju2oC/M3uRC3exf1Djo7x5dG19dNEcK WLf39J2WLKkrroDyWku6zOyPeHuewE5WQlTt6zsxloXuJEJ+NyWyfPcujeGgXndYI6 w6XmwF9W/llHbeHgZmrGfrhWnEfHCy+ItN7W8ePo= Date: Fri, 15 Sep 2023 22:47:00 +0000 From: Eric Wong To: Konstantin Ryabitsev Cc: meta@public-inbox.org Subject: Re: RFC: lei searches managed by users in git Message-ID: <20230915224701.M396@dcvr> References: <20230915-conjure-exuberant-d527e8@meerkat> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20230915-conjure-exuberant-d527e8@meerkat> List-Id: Konstantin Ryabitsev wrote: > Hello: > > I am curious what is the best approach to have a centrally managed set of lei > searches, for example via config files tracked in git. For example, the file > could look like this: I don't have one nor have I thought about it until now... I wonder if `lei edit-search --all' could be a thing... As would adding gitconfig output to ls-search (`lei ls-search -l -f gitconfig'). Maybe `lei edit-search --all=lei.q' could cut down on non-interesting fields when editing all searches. I also wouldn't be opposed to supporting non-interactive use for automation: lei edit-search --set lei.q=foo OUTPUT lei edit-search --add lei.q=foo OUTPUT ...but maybe it's not needed (see below on `lei q'): > mricon.toml: Any particular reason for toml? I don't have a toml parser installed and I suspect many don't, either. git-config is widely available and installed, of course. I've been trying hard to avoid data format and dependency proliferation. > [search.torvalds] > # All mail sent by torvalds > q = 'f:torvalds@linux-foundation.org' > [search.floppy] > # Any messages talking about floppies or touching floppy code > q = 'dfhh:floppy_* OR dfn:drivers/block/floppy.c OR s:floppy OR ((nq:bug OR nq:regression) AND nq:floppy)' > > I could then have a small wrapper maintaining saved searches and making the > mailboxes available via special newsgroups like: > > org.kernel.lei.mricon.torvalds > org.kernel.lei.mricon.floppy Sidenote: I think `query' or similar is more appropriate than using `lei' in a public newsgroup name since it's not `local' :> > The goal is to make it possible for maintainers to define their own set of > saved searches and have access to them at kernel.org via imap/pop3/nntp. > > It's easy to write a simple wrapper that would invoke lei-edit-search and > replace the search string when there are updates to the config files, but I'm > curious if you already have thoughts on how to best implement something like > this. Rerunning `lei q' on existing destinations is fine for updating existing queries. This is especially nice with `-f v2', since dedupe is faster on v2 than any other output format. Thus there's no need to even bother with `lei edit-search' if you're doing this non-interactively with your own files/formats (e.g. via web UI or something). > My biggest concern is someone committing an invalid query and not receiving > any more email as a result -- so having a sane way to validate the query > before sticking it into the saved search would be handy. Perhaps `lei q --limit=1' could be appropriate for validating queries with the below fix. But validating searches can ahead of time is a TOCTOU problem. There can be added support for per-inbox prefixes (e.g. altid with gmane: on public-inbox.org/git), and also when mixing old/new versions via different HTTP(S) externals. -------8<------- Subject: [PATCH] lei q: set exit code for invalid Xapian queries Xapian can't parse every query, so ensure we set the exit code for the client. --- lib/PublicInbox/LeiXSearch.pm | 6 ++++-- t/lei.t | 5 +++++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/lib/PublicInbox/LeiXSearch.pm b/lib/PublicInbox/LeiXSearch.pm index 5965274c..7f4911b3 100644 --- a/lib/PublicInbox/LeiXSearch.pm +++ b/lib/PublicInbox/LeiXSearch.pm @@ -186,7 +186,8 @@ sub query_one_mset { # for --threads and l2m w/o sort } my $first_ids; do { - $mset = $srch->mset($mo->{qstr}, $mo); + $mset = eval { $srch->mset($mo->{qstr}, $mo) }; + return $lei->child_error(22 << 8, "E: $@") if $@; # 22 from curl mset_progress($lei, $dir, $mo->{offset} + $mset->size, $mset->get_matches_estimated); wait_startq($lei); # wait for keyword updates @@ -249,7 +250,8 @@ sub query_combined_mset { # non-parallel for non-"--threads" users } my $each_smsg = $lei->{ovv}->ovv_each_smsg_cb($lei); do { - $mset = $self->mset($mo->{qstr}, $mo); + $mset = eval { $self->mset($mo->{qstr}, $mo) }; + return $lei->child_error(22 << 8, "E: $@") if $@; # 22 from curl mset_progress($lei, 'xsearch', $mo->{offset} + $mset->size, $mset->get_matches_estimated); wait_startq($lei); # wait for keyword updates diff --git a/t/lei.t b/t/lei.t index d83bde69..1199ca75 100644 --- a/t/lei.t +++ b/t/lei.t @@ -147,6 +147,11 @@ my $test_fail = sub { lei_ok('q', "foo\n"); like($lei_err, qr/trailing `\\n' removed/s, "noted `\\n' removal"); + lei(qw(q from:infinity..)); + is($? >> 8, 22, 'combined query fails on invalid range op'); + lei(qw(q -t from:infinity..)); + is($? >> 8, 22, 'single query fails on invalid range op'); + for my $lk (qw(ei inbox)) { my $d = "$home/newline\n$lk"; my $all = $lk eq 'ei' ? 'ALL' : 'all';