From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 21BC51F910; Sun, 30 Oct 2022 23:06:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1667171192; bh=Qq6A2oTW7ClSZB5ZIh1YMGEGh41O1QDKZJKHEDFjhic=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=vdJ+Q1teE59k6fH1ObcdYzfWJhp6vox8z/d66wFmcZ5ciKVdgu4T2kl72xOd0Swv/ NeXM7+Ab3U8th2bdF0Yf0dSbKRrD/yAOH6vv4FrLDX5+DCZhrj+ZXXh1fpCR8niX/L RQZQoojycyk1k+J71dU+32ZH9U020O5sHgO1If+8= Date: Sun, 30 Oct 2022 23:06:31 +0000 From: Eric Wong To: Hangbin Liu Cc: meta@public-inbox.org Subject: Re: [Need Help] lei add quotes at the search Message-ID: <20221030230631.GB24949@dcvr> References: <20221030051333.M329162@dcvr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: List-Id: Hangbin Liu wrote: > Hi Eric, > > Thanks for the help. > > On Sun, Oct 30, 2022 at 05:13:33AM +0000, Eric Wong wrote: > > Hangbin Liu wrote: > > > Hi, > > > > > > I used to use a search like > > > > > > lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' > > > > > > It works on fc35. But after I update to fc36 with lei-1.9.0-1.fc36. It start to > > > add quotes in the search link and make the search never works. e.g. > > > > Are you able to show the curl CLI from fc35? > > Which public-inbox/lei version was it? > > > > I'm actually curious fc35 worked at all, since the quoting would've > > been broken, I think... > > Sorry, I don't have the fc35 environment now. No worries, I dont think fc35 is really a culprit. Were you running a pre-release version of public-inbox or lei before? > > > $ lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' > > > # /home/Liu/.local/share/lei/store 0/0 > > > # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3A%22liuhangbin+AND+rt%3A6.month.ago..)+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au)%22 > > > # 0 written to /home/Liu/Mail/liuhangbin/ (0 matches) > > > > > > Do you think if this is a bug, or I should update my search. > > > > The %22 in fc36 is because your entire query is treated as one > > element in argv and matches expected behavior. > > > > Since '(' and ')' in the shell CLI is special, I suggest either: > > I'm curious about why the quote(%22) is added after "tc", not after "(" It's because Xapian can only handle a phrase after the `tc:' prefix. thus: tc:"foo bar" actually parses `tc:' as a prefix for To/Cc; while: "tc:foo bar" looks for the phrase "tc:foo bar" anywhere in the message, and won't limit to To/Cc headers. This happens in the query_argv_to_string sub: https://public-inbox.org/meta/2feb3e13b49d222bc7bd28430a9cf159692a933f/s/?b=lib/PublicInbox/Search.pm#n358 >From the CLI: lei q "tc:foo bar" is indistinguishable from lei q tc:"foo bar" , so it gets treated as the latter. > > a) using --stdin to enter queries containing '(' and ')' > > > > b) quoting (or escaping) only the '(' and ')': > > > > '('tc:liuhangbin AND rt:6.month.ago..')' NOT ... > > > > or > > > > \(tc:liuhangbin AND rt:6.month.ago..\) NOT ... > > with this way, the cmd line works. And in config file, it would looks like > > [lei] > q = ((tc:liuhangbin > q = AND > q = rt:6.month.ago..) > q = NOT > q = (tc:stable@vger.kernel.org > q = OR > q = f:sfr@canb.auug.org.au) > > But if I have a long search line. This will breaks too much and hard to edit. > e.g. My real previous search is like > > [lei] > q = (tc:liuhangbin OR \ > (dfn:drivers/net/wireguard/ AND rt:6.month.ago..) OR \ > (dfn:tools/testing/selftests/net/ AND rt:1.month.ago..) OR \ > (dfn:drivers/net/team/ AND rt:6.month.ago..) OR \ > (dfn:net/ipv4/igmp.c AND rt:6.month.ago..) OR \ > (dfn:net/ipv6/mcast.c AND rt:6.month.ago..)) \ > NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au) > > If I add "\" on each "(", this will break to a very long config search. > I tried to adjust it to I think that can work if lei.internal.rawstr is set in the config to indicate stdin was used (It's auto-set by --stdin). I guess it also works if it's the only lei.q config entry and the lei.q entry contains "\n" cf. https://public-inbox.org/meta/20211110102837.41721-1-e@80x24.org/ > [lei] > q = (tc:liuhangbin OR \ > (dfn:drivers/net/wireguard/ AND rt:6.month.ago..) OR \ > (dfn:tools/testing/selftests/net/ AND rt:1.month.ago..) OR \ > (dfn:drivers/net/team/ AND rt:6.month.ago..) OR \ > (dfn:net/ipv4/igmp.c AND rt:6.month.ago..) OR \ > (dfn:net/ipv6/mcast.c AND rt:6.month.ago..)) > q = NOT > q = (tc:stable@vger.kernel.org > q = OR > q = f:sfr@canb.auug.org.au) > > And now it works... Sorta... at least for remotes it does: > $ lei up /home/Liu/Mail/gmail/Linux_Kernel > # https://lore.kernel.org/all/ limiting to 2022-09-30 17:00 +0800 and newer > 60927 lei_xsearch 0 wq_worker: query_one_mset: Exception: Unknown range operation at /usr/share/perl5/vendor_perl/PublicInbox/IPC.pm line 254. Note that Exception means it's not handling the first part of the query when hitting the local Xapian DB. It's not doing the approxidate ($X.month.ago) substitution for the local Xapian DB, thus you got the "Unknown range operation". > # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3Aliuhangbin+OR+(dfn%3Adrivers%2Fnet%2Fwireguard%2F+AND+rt%3A6.month.ago..)+OR+(dfn%3Atools%2Ftesting%2Fselftests%2Fnet%2F+AND+rt%3A1.month.ago..)+OR+(dfn%3Adrivers%2Fnet%2Fteam%2F+AND+rt%3A6.month.ago..)+OR+(dfn%3Anet%2Fipv4%2Figmp.c+AND+rt%3A6.month.ago..)+OR+(dfn%3Anet%2Fipv6%2Fmcast.c+AND+rt%3A1651301673..))+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au))+AND+dt%3A20220930090001.. > # https://lore.kernel.org/all/ 43/? Of course, the lack of approxidate parsing there inside lei is fine, since the lore.kernel.org instance will do it remotely... > So I want to know when/why *lei* add the quotes. lei adds quotes since it can't distinguish if the shell user used single or double quotes. Xapian uses double quotes for phrase search, and I wanted: lei q "this is a phrase" to work naturally, which means: lei q 'this is a phrase' (with single quotes) works the same way as with double quotes because the difference is handled by the shell and lei never sees it.