* [Need Help] lei add quotes at the search @ 2022-10-30 4:03 Hangbin Liu 2022-10-30 5:13 ` Eric Wong 0 siblings, 1 reply; 6+ messages in thread From: Hangbin Liu @ 2022-10-30 4:03 UTC (permalink / raw) To: meta Hi, I used to use a search like lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' It works on fc35. But after I update to fc36 with lei-1.9.0-1.fc36. It start to add quotes in the search link and make the search never works. e.g. $ lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' # /home/Liu/.local/share/lei/store 0/0 # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3A%22liuhangbin+AND+rt%3A6.month.ago..)+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au)%22 # 0 written to /home/Liu/Mail/liuhangbin/ (0 matches) Do you think if this is a bug, or I should update my search. Thanks Hangbin ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Need Help] lei add quotes at the search 2022-10-30 4:03 [Need Help] lei add quotes at the search Hangbin Liu @ 2022-10-30 5:13 ` Eric Wong 2022-10-30 7:08 ` Hangbin Liu 0 siblings, 1 reply; 6+ messages in thread From: Eric Wong @ 2022-10-30 5:13 UTC (permalink / raw) To: Hangbin Liu; +Cc: meta Hangbin Liu <liuhangbin@gmail.com> wrote: > Hi, > > I used to use a search like > > lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' > > It works on fc35. But after I update to fc36 with lei-1.9.0-1.fc36. It start to > add quotes in the search link and make the search never works. e.g. Are you able to show the curl CLI from fc35? Which public-inbox/lei version was it? I'm actually curious fc35 worked at all, since the quoting would've been broken, I think... > $ lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' > # /home/Liu/.local/share/lei/store 0/0 > # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3A%22liuhangbin+AND+rt%3A6.month.ago..)+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au)%22 > # 0 written to /home/Liu/Mail/liuhangbin/ (0 matches) > > Do you think if this is a bug, or I should update my search. The %22 in fc36 is because your entire query is treated as one element in argv and matches expected behavior. Since '(' and ')' in the shell CLI is special, I suggest either: a) using --stdin to enter queries containing '(' and ')' b) quoting (or escaping) only the '(' and ')': '('tc:liuhangbin AND rt:6.month.ago..')' NOT ... or \(tc:liuhangbin AND rt:6.month.ago..\) NOT ... Which makes your argv something like: [ "(tc:liuhangbin", "AND", "rt:6.month.ago..)", "NOT", ... ] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Need Help] lei add quotes at the search 2022-10-30 5:13 ` Eric Wong @ 2022-10-30 7:08 ` Hangbin Liu 2022-10-30 23:06 ` Eric Wong 0 siblings, 1 reply; 6+ messages in thread From: Hangbin Liu @ 2022-10-30 7:08 UTC (permalink / raw) To: Eric Wong; +Cc: meta Hi Eric, Thanks for the help. On Sun, Oct 30, 2022 at 05:13:33AM +0000, Eric Wong wrote: > Hangbin Liu <liuhangbin@gmail.com> wrote: > > Hi, > > > > I used to use a search like > > > > lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' > > > > It works on fc35. But after I update to fc36 with lei-1.9.0-1.fc36. It start to > > add quotes in the search link and make the search never works. e.g. > > Are you able to show the curl CLI from fc35? > Which public-inbox/lei version was it? > > I'm actually curious fc35 worked at all, since the quoting would've > been broken, I think... Sorry, I don't have the fc35 environment now. > > > $ lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' > > # /home/Liu/.local/share/lei/store 0/0 > > # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3A%22liuhangbin+AND+rt%3A6.month.ago..)+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au)%22 > > # 0 written to /home/Liu/Mail/liuhangbin/ (0 matches) > > > > Do you think if this is a bug, or I should update my search. > > The %22 in fc36 is because your entire query is treated as one > element in argv and matches expected behavior. > > Since '(' and ')' in the shell CLI is special, I suggest either: I'm curious about why the quote(%22) is added after "tc", not after "(" > > a) using --stdin to enter queries containing '(' and ')' > > b) quoting (or escaping) only the '(' and ')': > > '('tc:liuhangbin AND rt:6.month.ago..')' NOT ... > > or > > \(tc:liuhangbin AND rt:6.month.ago..\) NOT ... with this way, the cmd line works. And in config file, it would looks like [lei] q = ((tc:liuhangbin q = AND q = rt:6.month.ago..) q = NOT q = (tc:stable@vger.kernel.org q = OR q = f:sfr@canb.auug.org.au) But if I have a long search line. This will breaks too much and hard to edit. e.g. My real previous search is like [lei] q = (tc:liuhangbin OR \ (dfn:drivers/net/wireguard/ AND rt:6.month.ago..) OR \ (dfn:tools/testing/selftests/net/ AND rt:1.month.ago..) OR \ (dfn:drivers/net/team/ AND rt:6.month.ago..) OR \ (dfn:net/ipv4/igmp.c AND rt:6.month.ago..) OR \ (dfn:net/ipv6/mcast.c AND rt:6.month.ago..)) \ NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au) If I add "\" on each "(", this will break to a very long config search. I tried to adjust it to [lei] q = (tc:liuhangbin OR \ (dfn:drivers/net/wireguard/ AND rt:6.month.ago..) OR \ (dfn:tools/testing/selftests/net/ AND rt:1.month.ago..) OR \ (dfn:drivers/net/team/ AND rt:6.month.ago..) OR \ (dfn:net/ipv4/igmp.c AND rt:6.month.ago..) OR \ (dfn:net/ipv6/mcast.c AND rt:6.month.ago..)) q = NOT q = (tc:stable@vger.kernel.org q = OR q = f:sfr@canb.auug.org.au) And now it works... $ lei up /home/Liu/Mail/gmail/Linux_Kernel # https://lore.kernel.org/all/ limiting to 2022-09-30 17:00 +0800 and newer 60927 lei_xsearch 0 wq_worker: query_one_mset: Exception: Unknown range operation at /usr/share/perl5/vendor_perl/PublicInbox/IPC.pm line 254. # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3Aliuhangbin+OR+(dfn%3Adrivers%2Fnet%2Fwireguard%2F+AND+rt%3A6.month.ago..)+OR+(dfn%3Atools%2Ftesting%2Fselftests%2Fnet%2F+AND+rt%3A1.month.ago..)+OR+(dfn%3Adrivers%2Fnet%2Fteam%2F+AND+rt%3A6.month.ago..)+OR+(dfn%3Anet%2Fipv4%2Figmp.c+AND+rt%3A6.month.ago..)+OR+(dfn%3Anet%2Fipv6%2Fmcast.c+AND+rt%3A1651301673..))+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au))+AND+dt%3A20220930090001.. # https://lore.kernel.org/all/ 43/? So I want to know when/why *lei* add the quotes. Thanks Hangbin ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Need Help] lei add quotes at the search 2022-10-30 7:08 ` Hangbin Liu @ 2022-10-30 23:06 ` Eric Wong 2022-10-31 7:36 ` Hangbin Liu 2022-10-31 7:47 ` Hangbin Liu 0 siblings, 2 replies; 6+ messages in thread From: Eric Wong @ 2022-10-30 23:06 UTC (permalink / raw) To: Hangbin Liu; +Cc: meta Hangbin Liu <liuhangbin@gmail.com> wrote: > Hi Eric, > > Thanks for the help. > > On Sun, Oct 30, 2022 at 05:13:33AM +0000, Eric Wong wrote: > > Hangbin Liu <liuhangbin@gmail.com> wrote: > > > Hi, > > > > > > I used to use a search like > > > > > > lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' > > > > > > It works on fc35. But after I update to fc36 with lei-1.9.0-1.fc36. It start to > > > add quotes in the search link and make the search never works. e.g. > > > > Are you able to show the curl CLI from fc35? > > Which public-inbox/lei version was it? > > > > I'm actually curious fc35 worked at all, since the quoting would've > > been broken, I think... > > Sorry, I don't have the fc35 environment now. No worries, I dont think fc35 is really a culprit. Were you running a pre-release version of public-inbox or lei before? > > > $ lei q -I https://lore.kernel.org/all/ -o ~/Mail/liuhangbin --threads --dedupe=mid '((tc:liuhangbin AND rt:6.month.ago..) NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au)' > > > # /home/Liu/.local/share/lei/store 0/0 > > > # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3A%22liuhangbin+AND+rt%3A6.month.ago..)+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au)%22 > > > # 0 written to /home/Liu/Mail/liuhangbin/ (0 matches) > > > > > > Do you think if this is a bug, or I should update my search. > > > > The %22 in fc36 is because your entire query is treated as one > > element in argv and matches expected behavior. > > > > Since '(' and ')' in the shell CLI is special, I suggest either: > > I'm curious about why the quote(%22) is added after "tc", not after "(" It's because Xapian can only handle a phrase after the `tc:' prefix. thus: tc:"foo bar" actually parses `tc:' as a prefix for To/Cc; while: "tc:foo bar" looks for the phrase "tc:foo bar" anywhere in the message, and won't limit to To/Cc headers. This happens in the query_argv_to_string sub: https://public-inbox.org/meta/2feb3e13b49d222bc7bd28430a9cf159692a933f/s/?b=lib/PublicInbox/Search.pm#n358 From the CLI: lei q "tc:foo bar" is indistinguishable from lei q tc:"foo bar" , so it gets treated as the latter. > > a) using --stdin to enter queries containing '(' and ')' > > > > b) quoting (or escaping) only the '(' and ')': > > > > '('tc:liuhangbin AND rt:6.month.ago..')' NOT ... > > > > or > > > > \(tc:liuhangbin AND rt:6.month.ago..\) NOT ... > > with this way, the cmd line works. And in config file, it would looks like > > [lei] > q = ((tc:liuhangbin > q = AND > q = rt:6.month.ago..) > q = NOT > q = (tc:stable@vger.kernel.org > q = OR > q = f:sfr@canb.auug.org.au) > > But if I have a long search line. This will breaks too much and hard to edit. > e.g. My real previous search is like > > [lei] > q = (tc:liuhangbin OR \ > (dfn:drivers/net/wireguard/ AND rt:6.month.ago..) OR \ > (dfn:tools/testing/selftests/net/ AND rt:1.month.ago..) OR \ > (dfn:drivers/net/team/ AND rt:6.month.ago..) OR \ > (dfn:net/ipv4/igmp.c AND rt:6.month.ago..) OR \ > (dfn:net/ipv6/mcast.c AND rt:6.month.ago..)) \ > NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au) > > If I add "\" on each "(", this will break to a very long config search. > I tried to adjust it to I think that can work if lei.internal.rawstr is set in the config to indicate stdin was used (It's auto-set by --stdin). I guess it also works if it's the only lei.q config entry and the lei.q entry contains "\n" cf. https://public-inbox.org/meta/20211110102837.41721-1-e@80x24.org/ > [lei] > q = (tc:liuhangbin OR \ > (dfn:drivers/net/wireguard/ AND rt:6.month.ago..) OR \ > (dfn:tools/testing/selftests/net/ AND rt:1.month.ago..) OR \ > (dfn:drivers/net/team/ AND rt:6.month.ago..) OR \ > (dfn:net/ipv4/igmp.c AND rt:6.month.ago..) OR \ > (dfn:net/ipv6/mcast.c AND rt:6.month.ago..)) > q = NOT > q = (tc:stable@vger.kernel.org > q = OR > q = f:sfr@canb.auug.org.au) > > And now it works... Sorta... at least for remotes it does: > $ lei up /home/Liu/Mail/gmail/Linux_Kernel > # https://lore.kernel.org/all/ limiting to 2022-09-30 17:00 +0800 and newer > 60927 lei_xsearch 0 wq_worker: query_one_mset: Exception: Unknown range operation at /usr/share/perl5/vendor_perl/PublicInbox/IPC.pm line 254. Note that Exception means it's not handling the first part of the query when hitting the local Xapian DB. It's not doing the approxidate ($X.month.ago) substitution for the local Xapian DB, thus you got the "Unknown range operation". > # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3Aliuhangbin+OR+(dfn%3Adrivers%2Fnet%2Fwireguard%2F+AND+rt%3A6.month.ago..)+OR+(dfn%3Atools%2Ftesting%2Fselftests%2Fnet%2F+AND+rt%3A1.month.ago..)+OR+(dfn%3Adrivers%2Fnet%2Fteam%2F+AND+rt%3A6.month.ago..)+OR+(dfn%3Anet%2Fipv4%2Figmp.c+AND+rt%3A6.month.ago..)+OR+(dfn%3Anet%2Fipv6%2Fmcast.c+AND+rt%3A1651301673..))+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au))+AND+dt%3A20220930090001.. > # https://lore.kernel.org/all/ 43/? Of course, the lack of approxidate parsing there inside lei is fine, since the lore.kernel.org instance will do it remotely... > So I want to know when/why *lei* add the quotes. lei adds quotes since it can't distinguish if the shell user used single or double quotes. Xapian uses double quotes for phrase search, and I wanted: lei q "this is a phrase" to work naturally, which means: lei q 'this is a phrase' (with single quotes) works the same way as with double quotes because the difference is handled by the shell and lei never sees it. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Need Help] lei add quotes at the search 2022-10-30 23:06 ` Eric Wong @ 2022-10-31 7:36 ` Hangbin Liu 2022-10-31 7:47 ` Hangbin Liu 1 sibling, 0 replies; 6+ messages in thread From: Hangbin Liu @ 2022-10-31 7:36 UTC (permalink / raw) To: Eric Wong; +Cc: meta On Sun, Oct 30, 2022 at 11:06:31PM +0000, Eric Wong wrote: > Hangbin Liu <liuhangbin@gmail.com> wrote: > > Sorry, I don't have the fc35 environment now. > > No worries, I dont think fc35 is really a culprit. Were you > running a pre-release version of public-inbox or lei before? Sorry, I forgot. Maybe I installed via `dnf copr enable icon/b4` because I start using lei after reading blog https://people.kernel.org/monsieuricon/lore-lei-part-1-getting-started > > I'm curious about why the quote(%22) is added after "tc", not after "(" > > It's because Xapian can only handle a phrase after the `tc:' prefix. > thus: tc:"foo bar" actually parses `tc:' as a prefix for To/Cc; > while: "tc:foo bar" looks for the phrase "tc:foo bar" anywhere > in the message, and won't limit to To/Cc headers. > > This happens in the query_argv_to_string sub: > > https://public-inbox.org/meta/2feb3e13b49d222bc7bd28430a9cf159692a933f/s/?b=lib/PublicInbox/Search.pm#n358 > > From the CLI: lei q "tc:foo bar" is indistinguishable > from lei q tc:"foo bar" , so it gets treated as the latter. Thanks for the explanation. > > But if I have a long search line. This will breaks too much and hard to edit. > > e.g. My real previous search is like > > > > [lei] > > q = (tc:liuhangbin OR \ > > (dfn:drivers/net/wireguard/ AND rt:6.month.ago..) OR \ > > (dfn:tools/testing/selftests/net/ AND rt:1.month.ago..) OR \ > > (dfn:drivers/net/team/ AND rt:6.month.ago..) OR \ > > (dfn:net/ipv4/igmp.c AND rt:6.month.ago..) OR \ > > (dfn:net/ipv6/mcast.c AND rt:6.month.ago..)) \ > > NOT (tc:stable@vger.kernel.org OR f:sfr@canb.auug.org.au) > > > > If I add "\" on each "(", this will break to a very long config search. > > I tried to adjust it to > > I think that can work if lei.internal.rawstr is set in the > config to indicate stdin was used (It's auto-set by --stdin). > I guess it also works if it's the only lei.q config entry > and the lei.q entry contains "\n" > > cf. https://public-inbox.org/meta/20211110102837.41721-1-e@80x24.org/ > > > [lei] > > q = (tc:liuhangbin OR \ > > (dfn:drivers/net/wireguard/ AND rt:6.month.ago..) OR \ > > (dfn:tools/testing/selftests/net/ AND rt:1.month.ago..) OR \ > > (dfn:drivers/net/team/ AND rt:6.month.ago..) OR \ > > (dfn:net/ipv4/igmp.c AND rt:6.month.ago..) OR \ > > (dfn:net/ipv6/mcast.c AND rt:6.month.ago..)) > > q = NOT > > q = (tc:stable@vger.kernel.org > > q = OR > > q = f:sfr@canb.auug.org.au) > > > > And now it works... > > Sorta... at least for remotes it does: > > > $ lei up /home/Liu/Mail/gmail/Linux_Kernel > > # https://lore.kernel.org/all/ limiting to 2022-09-30 17:00 +0800 and newer > > 60927 lei_xsearch 0 wq_worker: query_one_mset: Exception: Unknown range operation at /usr/share/perl5/vendor_perl/PublicInbox/IPC.pm line 254. > > Note that Exception means it's not handling the first part of > the query when hitting the local Xapian DB. It's not doing the > approxidate ($X.month.ago) substitution for the local Xapian DB, > thus you got the "Unknown range operation". > > > # /usr/bin/curl -Sf -s -d '' https://lore.kernel.org/all/?x=m&t=1&q=((tc%3Aliuhangbin+OR+(dfn%3Adrivers%2Fnet%2Fwireguard%2F+AND+rt%3A6.month.ago..)+OR+(dfn%3Atools%2Ftesting%2Fselftests%2Fnet%2F+AND+rt%3A1.month.ago..)+OR+(dfn%3Adrivers%2Fnet%2Fteam%2F+AND+rt%3A6.month.ago..)+OR+(dfn%3Anet%2Fipv4%2Figmp.c+AND+rt%3A6.month.ago..)+OR+(dfn%3Anet%2Fipv6%2Fmcast.c+AND+rt%3A1651301673..))+NOT+(tc%3Astable%40vger.kernel.org+OR+f%3Asfr%40canb.auug.org.au))+AND+dt%3A20220930090001.. > > # https://lore.kernel.org/all/ 43/? > > Of course, the lack of approxidate parsing there inside lei is > fine, since the lore.kernel.org instance will do it remotely... > > > So I want to know when/why *lei* add the quotes. > > lei adds quotes since it can't distinguish if the shell user > used single or double quotes. Xapian uses double quotes for > phrase search, and I wanted: lei q "this is a phrase" > to work naturally, which means: lei q 'this is a phrase' > (with single quotes) works the same way as with double quotes > because the difference is handled by the shell and lei never > sees it. Thanks for the help. Hangbin ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Need Help] lei add quotes at the search 2022-10-30 23:06 ` Eric Wong 2022-10-31 7:36 ` Hangbin Liu @ 2022-10-31 7:47 ` Hangbin Liu 1 sibling, 0 replies; 6+ messages in thread From: Hangbin Liu @ 2022-10-31 7:47 UTC (permalink / raw) To: Eric Wong; +Cc: meta On Sun, Oct 30, 2022 at 11:06:31PM +0000, Eric Wong wrote: > > If I add "\" on each "(", this will break to a very long config search. > > I tried to adjust it to > > I think that can work if lei.internal.rawstr is set in the > config to indicate stdin was used (It's auto-set by --stdin). OH, BTW, I will get error fatal: bad config line 4 in file [..snip..]/lei.saved-search if adding '\' in the config file. > I guess it also works if it's the only lei.q config entry > and the lei.q entry contains "\n" But with "\n" in the config file. My previous config will works fine. > > cf. https://public-inbox.org/meta/20211110102837.41721-1-e@80x24.org/ Thanks Hangbin ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-10-31 7:47 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-10-30 4:03 [Need Help] lei add quotes at the search Hangbin Liu 2022-10-30 5:13 ` Eric Wong 2022-10-30 7:08 ` Hangbin Liu 2022-10-30 23:06 ` Eric Wong 2022-10-31 7:36 ` Hangbin Liu 2022-10-31 7:47 ` Hangbin Liu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).