* Apparently, terms with a common prefix are *not* connected by implicit "OR" @ 2019-08-11 17:20 Jorge P. de Morais Neto 2019-08-11 23:08 ` David Bremner 0 siblings, 1 reply; 10+ messages in thread From: Jorge P. de Morais Neto @ 2019-08-11 17:20 UTC (permalink / raw) To: notmuch Hi. The NOTMUCH-SEARCH-TERMS man page says: Each term in the query will be implicitly connected by a logical AND if no explicit operator is provided (except that terms with a common prefix will be implicitly combined with OR). However, in practice I get different results: $ notmuch count '(to:pontodosconcursos.com.br OR to:jorge+cp+concurso@disroot.org)' 66 $ notmuch count '(to:pontodosconcursos.com.br to:jorge+cp+concurso@disroot.org)' 0 I have other examples of this. I currently use notmuch 0.29.1 privately backported to Debian buster according to the Debian Wiki [procedure][], but I already got this problem in earlier notmuch releases, including the official Debian buster package. What gives? [procedure]: https://wiki.debian.org/SimpleBackportCreation My workarund is to always explicit the "OR" operator, as in: (to:concurso OR to:dominandoti OR to:cebraspe OR to:gleyson131 OR to:quadrix.org.br OR to:cathedranet.com.br OR from:cathedranet.com.br OR from:quadrix.org.br OR from:gleyson131 OR from:cebraspe OR from:dominandoti OR from:concurso OR is:lists/itaconcursos) If the OR operator was indeed implicit, the query above would be shorter. Regards -- - I am Brazilian. I hope my English is correct and I welcome feedback - Please adopt free formats like PDF, ODF, Org, LaTeX, Opus, WebM and 7z - Free/libre software for Android: https://f-droid.org/ - [[https://www.gnu.org/philosophy/free-sw.html][What is free software?]] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Apparently, terms with a common prefix are *not* connected by implicit "OR" 2019-08-11 17:20 Apparently, terms with a common prefix are *not* connected by implicit "OR" Jorge P. de Morais Neto @ 2019-08-11 23:08 ` David Bremner 2019-08-15 14:58 ` Jorge P. de Morais Neto 0 siblings, 1 reply; 10+ messages in thread From: David Bremner @ 2019-08-11 23:08 UTC (permalink / raw) To: Jorge P. de Morais Neto, notmuch jorge+list@disroot.org (Jorge P. de Morais Neto) writes: > Hi. The NOTMUCH-SEARCH-TERMS man page says: > > Each term in the query will be implicitly connected by a logical AND > if no explicit operator is provided (except that terms with a common > prefix will be implicitly combined with OR). > > However, in practice I get different results: > $ notmuch count '(to:pontodosconcursos.com.br OR to:jorge+cp+concurso@disroot.org)' > 66 > $ notmuch count '(to:pontodosconcursos.com.br to:jorge+cp+concurso@disroot.org)' > 0 > Thanks for the report. As a test, can you try with $ notmuch count '(to:pontodosconcursos.com.br to:"jorge+cp+concurso@disroot.org")' I suspect that will work around the problem, which I believe is related to the way that notmuch uses the xapian parser (in order to provide regexp matching for some prefixes). In particular, if I try that with NOTMUCH_DEBUG_QUERY=yes in the environment I can see the implicit OR. d ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Apparently, terms with a common prefix are *not* connected by implicit "OR" 2019-08-11 23:08 ` David Bremner @ 2019-08-15 14:58 ` Jorge P. de Morais Neto 2019-08-21 1:39 ` David Bremner 0 siblings, 1 reply; 10+ messages in thread From: Jorge P. de Morais Neto @ 2019-08-15 14:58 UTC (permalink / raw) To: David Bremner, notmuch [ I had replied to David Bremner alone so I didn't have to worry about private information leakage. But now I decided to clean the private information and reply to the list too. ] Em 2019-08-11T20:08:58-0300, David Bremner escreveu: > Thanks for the report. As a test, can you try with > > $ notmuch count '(to:pontodosconcursos.com.br to:"jorge+cp+concurso@disroot.org")' > > I suspect that will work around the problem, which I believe is related > to the way that notmuch uses the xapian parser (in order to provide > regexp matching for some prefixes). In particular, if I try that with > NOTMUCH_DEBUG_QUERY=yes in the environment I can see the implicit OR. > > d Thank you for the instructions, but they did not work. Below are several results from ~notmuch count~. --8<---------------cut here---------------start------------->8--- $ notmuch count '(to:pontodosconcursos.com.br OR to:jorge+cp+concurso@disroot.org)' 70 $ notmuch count '(to:pontodosconcursos.com.br)' 0 $ notmuch count 'to:pontodosconcursos.com.br' 0 $ notmuch count 'to:jorge+cp+concurso@disroot.org' 70 $ NOTMUCH_DEBUG_QUERY=yes notmuch count '(to:pontodosconcursos.com.br OR to:jorge+cp+concurso@disroot.org)' Query string is: (to:pontodosconcursos.com.br OR to:jorge+cp+concurso@disroot.org) Exclude query is: Query(((Kdeleted OR Kspam) OR Ktrash)) Final query is: Query(((Tmail AND ((XTOpontodosconcursos@1 PHRASE 3 XTOcom@2 PHRASE 3 XTObr@3) OR (ZXTOjorg@4 AND (Zcp@5 OR ZGcp@5 OR ZKcp@5 OR ZKcp@5 OR ZQcp@5 OR ZQcp@5 OR ZPcp@5 OR ZXPROPERTYcp@5 OR ZXFOLDER:cp@5 OR ZXFROMcp@5 OR ZXTOcp@5 OR ZXATTACHMENTcp@5 OR ZXMIMETYPEcp@5 OR ZXSUBJECTcp@5) AND ((concurso@6 PHRASE 3 disroot@7 PHRASE 3 org@8) OR (Gconcurso@6 PHRASE 3 Gdisroot@7 PHRASE 3 Gorg@8) OR (Kconcurso@6 PHRASE 3 Kdisroot@7 PHRASE 3 Korg@8) OR (Kconcurso@6 PHRASE 3 Kdisroot@7 PHRASE 3 Korg@8) OR (Qconcurso@6 PHRASE 3 Qdisroot@7 PHRASE 3 Qorg@8) OR (Qconcurso@6 PHRASE 3 Qdisroot@7 PHRASE 3 Qorg@8) OR (Pconcurso@6 PHRASE 3 Pdisroot@7 PHRASE 3 Porg@8) OR (XPROPERTYconcurso@6 PHRASE 3 XPROPERTYdisroot@7 PHRASE 3 XPROPERTYorg@8) OR (XFOLDER:concurso@6 PHRASE 3 XFOLDER:disroot@7 PHRASE 3 XFOLDER:org@8) OR (XFROMconcurso@6 PHRASE 3 XFROMdisroot@7 PHRASE 3 XFROMorg@8) OR (XTOconcurso@6 PHRASE 3 XTOdisroot@7 PHRASE 3 XTOorg@8) OR (XATTACHMENTconcurso@6 PHRASE 3 XATTACHMENTdisroot@7 PHRASE 3 XATTACHMENTorg@8) OR (XMIMETYPEconcurso@6 PHRASE 3 XMIMETYPEdisroot@7 PHRASE 3 XMIMETYPEorg@8) OR (XSUBJECTconcurso@6 PHRASE 3 XSUBJECTdisroot@7 PHRASE 3 XSUBJECTorg@8))))) AND_NOT ((Kdeleted OR Kspam) OR Ktrash))) 70 $ NOTMUCH_DEBUG_QUERY=yes notmuch count '(to:pontodosconcursos.com.br to:jorge+cp+concurso@disroot.org)' Query string is: (to:pontodosconcursos.com.br to:jorge+cp+concurso@disroot.org) Exclude query is: Query(((Kdeleted OR Kspam) OR Ktrash)) Final query is: Query(((Tmail AND ((XTOpontodosconcursos@1 PHRASE 3 XTOcom@2 PHRASE 3 XTObr@3) AND ZXTOjorg@4 AND (Zcp@5 OR ZGcp@5 OR ZKcp@5 OR ZKcp@5 OR ZQcp@5 OR ZQcp@5 OR ZPcp@5 OR ZXPROPERTYcp@5 OR ZXFOLDER:cp@5 OR ZXFROMcp@5 OR ZXTOcp@5 OR ZXATTACHMENTcp@5 OR ZXMIMETYPEcp@5 OR ZXSUBJECTcp@5) AND ((concurso@6 PHRASE 3 disroot@7 PHRASE 3 org@8) OR (Gconcurso@6 PHRASE 3 Gdisroot@7 PHRASE 3 Gorg@8) OR (Kconcurso@6 PHRASE 3 Kdisroot@7 PHRASE 3 Korg@8) OR (Kconcurso@6 PHRASE 3 Kdisroot@7 PHRASE 3 Korg@8) OR (Qconcurso@6 PHRASE 3 Qdisroot@7 PHRASE 3 Qorg@8) OR (Qconcurso@6 PHRASE 3 Qdisroot@7 PHRASE 3 Qorg@8) OR (Pconcurso@6 PHRASE 3 Pdisroot@7 PHRASE 3 Porg@8) OR (XPROPERTYconcurso@6 PHRASE 3 XPROPERTYdisroot@7 PHRASE 3 XPROPERTYorg@8) OR (XFOLDER:concurso@6 PHRASE 3 XFOLDER:disroot@7 PHRASE 3 XFOLDER:org@8) OR (XFROMconcurso@6 PHRASE 3 XFROMdisroot@7 PHRASE 3 XFROMorg@8) OR (XTOconcurso@6 PHRASE 3 XTOdisroot@7 PHRASE 3 XTOorg@8) OR (XATTACHMENTconcurso@6 PHRASE 3 XATTACHMENTdisroot@7 PHRASE 3 XATTACHMENTorg@8) OR (XMIMETYPEconcurso@6 PHRASE 3 XMIMETYPEdisroot@7 PHRASE 3 XMIMETYPEorg@8) OR (XSUBJECTconcurso@6 PHRASE 3 XSUBJECTdisroot@7 PHRASE 3 XSUBJECTorg@8)))) AND_NOT ((Kdeleted OR Kspam) OR Ktrash))) 0 $ NOTMUCH_DEBUG_QUERY=yes notmuch count '(to:pontodosconcursos.com.br to:"jorge+cp+concurso@disroot.org")' Query string is: (to:pontodosconcursos.com.br to:"jorge+cp+concurso@disroot.org") Exclude query is: Query(((Kdeleted OR Kspam) OR Ktrash)) Final query is: Query(((Tmail AND ((XTOpontodosconcursos@1 PHRASE 3 XTOcom@2 PHRASE 3 XTObr@3) AND (XTOjorge@4 PHRASE 5 XTOcp@5 PHRASE 5 XTOconcurso@6 PHRASE 5 XTOdisroot@7 PHRASE 5 XTOorg@8))) AND_NOT ((Kdeleted OR Kspam) OR Ktrash))) 0 $ NOTMUCH_DEBUG_QUERY=yes notmuch count '(to:pontodosconcursos.com.br OR to:"jorge+cp+concurso@disroot.org")' Query string is: (to:pontodosconcursos.com.br OR to:"jorge+cp+concurso@disroot.org") Exclude query is: Query(((Kdeleted OR Kspam) OR Ktrash)) Final query is: Query(((Tmail AND ((XTOpontodosconcursos@1 PHRASE 3 XTOcom@2 PHRASE 3 XTObr@3) OR (XTOjorge@4 PHRASE 5 XTOcp@5 PHRASE 5 XTOconcurso@6 PHRASE 5 XTOdisroot@7 PHRASE 5 XTOorg@8))) AND_NOT ((Kdeleted OR Kspam) OR Ktrash))) 69 $ NOTMUCH_DEBUG_QUERY=yes notmuch count '(to:pontodosconcursos.com.br to:"jorge+cp+concurso@disroot.org")' Query string is: (to:pontodosconcursos.com.br to:"jorge+cp+concurso@disroot.org") Exclude query is: Query(((Kdeleted OR Kspam) OR Ktrash)) Final query is: Query(((Tmail AND ((XTOpontodosconcursos@1 PHRASE 3 XTOcom@2 PHRASE 3 XTObr@3) AND (XTOjorge@4 PHRASE 5 XTOcp@5 PHRASE 5 XTOconcurso@6 PHRASE 5 XTOdisroot@7 PHRASE 5 XTOorg@8))) AND_NOT ((Kdeleted OR Kspam) OR Ktrash))) 0 $ NOTMUCH_DEBUG_QUERY=yes notmuch count '(to:"pontodosconcursos.com.br" to:"jorge+cp+concurso@disroot.org")' Query string is: (to:"pontodosconcursos.com.br" to:"jorge+cp+concurso@disroot.org") Exclude query is: Query(((Kdeleted OR Kspam) OR Ktrash)) Final query is: Query(((Tmail AND ((XTOpontodosconcursos@1 PHRASE 3 XTOcom@2 PHRASE 3 XTObr@3) AND (XTOjorge@4 PHRASE 5 XTOcp@5 PHRASE 5 XTOconcurso@6 PHRASE 5 XTOdisroot@7 PHRASE 5 XTOorg@8))) AND_NOT ((Kdeleted OR Kspam) OR Ktrash))) 0 $ NOTMUCH_DEBUG_QUERY=yes notmuch count 'to:"pontodosconcursos.com.br" to:"jorge+cp+concurso@disroot.org"' Query string is: to:"pontodosconcursos.com.br" to:"jorge+cp+concurso@disroot.org" Exclude query is: Query(((Kdeleted OR Kspam) OR Ktrash)) Final query is: Query(((Tmail AND ((XTOpontodosconcursos@1 PHRASE 3 XTOcom@2 PHRASE 3 XTObr@3) AND (XTOjorge@4 PHRASE 5 XTOcp@5 PHRASE 5 XTOconcurso@6 PHRASE 5 XTOdisroot@7 PHRASE 5 XTOorg@8))) AND_NOT ((Kdeleted OR Kspam) OR Ktrash))) 0 $ NOTMUCH_DEBUG_QUERY=yes notmuch count 'to:jorge+cp+concurso@disroot.org XOR to:"jorge+cp+concurso@disroot.org"' Query string is: to:jorge+cp+concurso@disroot.org XOR to:"jorge+cp+concurso@disroot.org" Exclude query is: Query(((Kdeleted OR Kspam) OR Ktrash)) Final query is: Query(((Tmail AND ((ZXTOjorg@1 AND (Zcp@2 OR ZGcp@2 OR ZKcp@2 OR ZKcp@2 OR ZQcp@2 OR ZQcp@2 OR ZPcp@2 OR ZXPROPERTYcp@2 OR ZXFOLDER:cp@2 OR ZXFROMcp@2 OR ZXTOcp@2 OR ZXATTACHMENTcp@2 OR ZXMIMETYPEcp@2 OR ZXSUBJECTcp@2) AND ((concurso@3 PHRASE 3 disroot@4 PHRASE 3 org@5) OR (Gconcurso@3 PHRASE 3 Gdisroot@4 PHRASE 3 Gorg@5) OR (Kconcurso@3 PHRASE 3 Kdisroot@4 PHRASE 3 Korg@5) OR (Kconcurso@3 PHRASE 3 Kdisroot@4 PHRASE 3 Korg@5) OR (Qconcurso@3 PHRASE 3 Qdisroot@4 PHRASE 3 Qorg@5) OR (Qconcurso@3 PHRASE 3 Qdisroot@4 PHRASE 3 Qorg@5) OR (Pconcurso@3 PHRASE 3 Pdisroot@4 PHRASE 3 Porg@5) OR (XPROPERTYconcurso@3 PHRASE 3 XPROPERTYdisroot@4 PHRASE 3 XPROPERTYorg@5) OR (XFOLDER:concurso@3 PHRASE 3 XFOLDER:disroot@4 PHRASE 3 XFOLDER:org@5) OR (XFROMconcurso@3 PHRASE 3 XFROMdisroot@4 PHRASE 3 XFROMorg@5) OR (XTOconcurso@3 PHRASE 3 XTOdisroot@4 PHRASE 3 XTOorg@5) OR (XATTACHMENTconcurso@3 PHRASE 3 XATTACHMENTdisroot@4 PHRASE 3 XATTACHMENTorg@5) OR (XMIMETYPEconcurso@3 PHRASE 3 XMIMETYPEdisroot@4 PHRASE 3 XMIMETYPEorg@5) OR (XSUBJECTconcurso@3 PHRASE 3 XSUBJECTdisroot@4 PHRASE 3 XSUBJECTorg@5))) XOR (XTOjorge@6 PHRASE 5 XTOcp@7 PHRASE 5 XTOconcurso@8 PHRASE 5 XTOdisroot@9 PHRASE 5 XTOorg@10))) AND_NOT ((Kdeleted OR Kspam) OR Ktrash))) 1 --8<---------------cut here---------------end--------------->8--- Below are the contents of ~/home/jorge/.config/notmuch/config~. --8<---------------cut here---------------start------------->8--- # .notmuch-config - Configuration file for the notmuch mail system # # For more information about notmuch, see https://notmuchmail.org # Database configuration # # The only value supported here is 'path' which should be the top-level # directory where your mail currently exists and to where mail will be # delivered in the future. Files should be individual email messages. # Notmuch will store its database within a sub-directory of the path # configured here named ".notmuch". # [database] path=/home/jorge/offlineimap/Jorge-Disroot # User configuration # # Here is where you can let notmuch know how you would like to be # addressed. Valid settings are # # name Your full name. # primary_email Your primary email address. # other_email A list (separated by ';') of other email addresses # at which you receive email. # # Notmuch will use the various email addresses configured here when # formatting replies. It will avoid including your own addresses in the # recipient list of replies, and will set the From address based on the # address to which the original email was addressed. # [user] name=Jorge P. de Morais Neto primary_email= < jorge AT_SIGN disroot DOT org> other_email=<REDACTED;<REDACTED>;<REDACTED>;<REDACTED>;<jorge PLUS-SIGN list AT_SIGN disroot DOT org;<REDACTED>;<REDACTED>;<REDACTED>;<REDACTED>; # Configuration for "notmuch new" # # The following options are supported here: # # tags A list (separated by ';') of the tags that will be # added to all messages incorporated by "notmuch new". # # ignore A list (separated by ';') of file and directory names # that will not be searched for messages by "notmuch new". # # NOTE: *Every* file/directory that goes by one of those # names will be ignored, independent of its depth/location # in the mail store. # [new] # tags=unread;inbox;new; # http://afew.readthedocs.io/en/latest/quickstart.html#initial-config tags=new ignore= # Search configuration # # The following option is supported here: # # exclude_tags # A ;-separated list of tags that will be excluded from # search results by default. Using an excluded tag in a # query will override that exclusion. # [search] exclude_tags=deleted;spam;trash; # Maildir compatibility configuration # # The following option is supported here: # # synchronize_flags Valid values are true and false. # # If true, then the following maildir flags (in message filenames) # will be synchronized with the corresponding notmuch tags: # # Flag Tag # ---- ------- # D draft # F flagged # P passed # R replied # S unread (added when 'S' flag is not present) # # The "notmuch new" command will notice flag changes in filenames # and update tags, while the "notmuch tag" and "notmuch restore" # commands will notice tag changes and update flags in filenames # [maildir] synchronize_flags=true # Cryptography related configuration # # The following *deprecated* option is currently supported: # # gpg_path # binary name or full path to invoke gpg. # NOTE: In a future build, this option will be ignored. # Setting $PATH is a better approach. # [crypto] gpg_path=gpg --8<---------------cut here---------------end--------------->8--- Regards -- - I am Brazilian. I hope my English is correct and I welcome feedback - Please adopt free formats like PDF, ODF, Org, LaTeX, Opus, WebM and 7z - Free/libre software for Android: https://f-droid.org/ - [[https://www.gnu.org/philosophy/free-sw.html][What is free software?]] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Apparently, terms with a common prefix are *not* connected by implicit "OR" 2019-08-15 14:58 ` Jorge P. de Morais Neto @ 2019-08-21 1:39 ` David Bremner 2019-08-21 5:48 ` Rollins, Jameson 0 siblings, 1 reply; 10+ messages in thread From: David Bremner @ 2019-08-21 1:39 UTC (permalink / raw) To: Jorge P. de Morais Neto, notmuch Jorge P. de Morais Neto <jorge+list@disroot.org> writes: > [ I had replied to David Bremner alone so I didn't have to worry about > private information leakage. But now I decided to clean the private > information and reply to the list too. ] > > Em 2019-08-11T20:08:58-0300, David Bremner escreveu: >> Thanks for the report. As a test, can you try with >> >> $ notmuch count '(to:pontodosconcursos.com.br to:"jorge+cp+concurso@disroot.org")' >> >> I suspect that will work around the problem, which I believe is related >> to the way that notmuch uses the xapian parser (in order to provide >> regexp matching for some prefixes). In particular, if I try that with >> NOTMUCH_DEBUG_QUERY=yes in the environment I can see the implicit OR. Thanks for the detailed report. There are (at least) two different things going on (in addition to the strange expansion that I focussed on before, but seems not to be the most important issue). One is that the combining with implicit-OR was only intended to work for "boolean prefixes" like tag:. So this is a documentation bug. A second thing is due to some implimentation details in notmuch, from: was being treated (for purposes of combining) as a filter. I think it's clear we want from: and to: to behave similarly, so I propose the following patch diff --git a/lib/database.cc b/lib/database.cc index 24b7ec43..4db1b465 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -400,7 +400,7 @@ _setup_query_field (const prefix_t *prefix, notmuch_database_t *notmuch) /* we treat all field-processor fields as boolean in order to get the raw input */ if (prefix->prefix) notmuch->query_parser->add_prefix ("", prefix->prefix); - notmuch->query_parser->add_boolean_prefix (prefix->name, fp); + notmuch->query_parser->add_boolean_prefix (prefix->name, fp, !(prefix->flags & NOTMUCH_FIELD_PROBABILISTIC)); } else { _setup_query_field_default (prefix, notmuch); } This will make to:a to:b and from:a from:b expand as to:a AND to:b and from:a AND from:b I don't think it's possible to have to:a to:b expand to to:a OR to:b without also having a b expand to a OR b which I think most people would find surprising. At the moment I'm not sure I see the benefit of having tag: combine with implicit OR (other than being slightly easier to document). ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: Apparently, terms with a common prefix are *not* connected by implicit "OR" 2019-08-21 1:39 ` David Bremner @ 2019-08-21 5:48 ` Rollins, Jameson 2019-08-21 11:41 ` WIP: fix implicit operators David Bremner 0 siblings, 1 reply; 10+ messages in thread From: Rollins, Jameson @ 2019-08-21 5:48 UTC (permalink / raw) To: David Bremner, Jorge P. de Morais Neto, notmuch@notmuchmail.org On Tue, Aug 20 2019, David Bremner <david@tethera.net> wrote: > This will make > > to:a to:b > > and > > from:a from:b > > expand as > > to:a AND to:b > > and > > from:a AND from:b I can't say if the proposed parser prefix change is correct, but I support this behavior. jamie. ^ permalink raw reply [flat|nested] 10+ messages in thread
* WIP: fix implicit operators 2019-08-21 5:48 ` Rollins, Jameson @ 2019-08-21 11:41 ` David Bremner 2019-08-21 11:41 ` [PATCH 1/3] test: add known broken tests for from: and subject: David Bremner ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: David Bremner @ 2019-08-21 11:41 UTC (permalink / raw) To: Rollins, Jameson, David Bremner, Jorge P. de Morais Neto, notmuch@notmuchmail.org I'm posting this for feedback. I did all the tests that were easy, but things like threads and attachments will take more effort to test. for probabilistic prefixes, we are basically stuck with AND. For boolean prefixes, we have the choice. Generally I tried to follow the Xapian convention that if a field/prefix is "exclusive" (one per message), then we group with OR. Regexp fields like mid:// are tricky since even if the underlying data is exclusive it makes sense to use multiple instances to match one piece of data, e.g. mid:/david/ mid:/tethera/ should probably behave roughly like mid:/david.*tethera/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/3] test: add known broken tests for from: and subject: 2019-08-21 11:41 ` WIP: fix implicit operators David Bremner @ 2019-08-21 11:41 ` David Bremner 2019-08-25 9:33 ` Tomi Ollila 2019-08-21 11:41 ` [PATCH 2/3] lib: introduce N_F_GROUP and use it to fix implicit AND for from: David Bremner 2019-08-21 11:41 ` [PATCH 3/3] WIP/test: extend field grouping tests David Bremner 2 siblings, 1 reply; 10+ messages in thread From: David Bremner @ 2019-08-21 11:41 UTC (permalink / raw) To: Rollins, Jameson, David Bremner, Jorge P. de Morais Neto, notmuch@notmuchmail.org Given we want 'a b' to parse as 'a AND b', then for any probabilistic (free text) prefix foo:, we should also get 'foo:a foo:b' expanding to 'foo:a AND foo:b'. Currently this is not true due to the implimentation of regex fields. --- test/T760-implicit-operators.sh | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 test/T760-implicit-operators.sh diff --git a/test/T760-implicit-operators.sh b/test/T760-implicit-operators.sh new file mode 100755 index 00000000..b79673df --- /dev/null +++ b/test/T760-implicit-operators.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +test_description='implicit operators in query parser' +. $(dirname "$0")/test-lib.sh || exit 1 + +test_AND() { + add_message "[$1]=a@b" + add_message "[$1]=b@c" + + test_begin_subtest "$1: implicitly joined by AND" + $2 + notmuch count $1:a@b > OUTPUT + notmuch count $1:a $1:b >> OUTPUT + notmuch count $1:a@b OR $1:b@c >> OUTPUT + notmuch count $1:a@b $1:b@c >> OUTPUT + cat <<EOF > EXPECTED +1 +1 +2 +0 +EOF + test_expect_equal_file EXPECTED OUTPUT +} + +test_AND from test_subtest_known_broken +test_AND subject test_subtest_known_broken +test_AND to + +test_done -- 2.23.0.rc1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 1/3] test: add known broken tests for from: and subject: 2019-08-21 11:41 ` [PATCH 1/3] test: add known broken tests for from: and subject: David Bremner @ 2019-08-25 9:33 ` Tomi Ollila 0 siblings, 0 replies; 10+ messages in thread From: Tomi Ollila @ 2019-08-25 9:33 UTC (permalink / raw) To: notmuch@notmuchmail.org On Wed, Aug 21 2019, David Bremner wrote: > Given we want 'a b' to parse as 'a AND b', then for any > probabilistic (free text) prefix foo:, we should also get 'foo:a > foo:b' expanding to 'foo:a AND foo:b'. Currently this is not true due > to the implimentation of regex fields. implementation =D > --- > test/T760-implicit-operators.sh | 28 ++++++++++++++++++++++++++++ > 1 file changed, 28 insertions(+) > create mode 100755 test/T760-implicit-operators.sh > > diff --git a/test/T760-implicit-operators.sh b/test/T760-implicit-operators.sh > new file mode 100755 > index 00000000..b79673df > --- /dev/null > +++ b/test/T760-implicit-operators.sh > @@ -0,0 +1,28 @@ > +#!/usr/bin/env bash > +test_description='implicit operators in query parser' > +. $(dirname "$0")/test-lib.sh || exit 1 > + > +test_AND() { > + add_message "[$1]=a@b" > + add_message "[$1]=b@c" > + > + test_begin_subtest "$1: implicitly joined by AND" > + $2 > + notmuch count $1:a@b > OUTPUT > + notmuch count $1:a $1:b >> OUTPUT > + notmuch count $1:a@b OR $1:b@c >> OUTPUT > + notmuch count $1:a@b $1:b@c >> OUTPUT > + cat <<EOF > EXPECTED > +1 > +1 > +2 > +0 > +EOF the above could be done printf %s\\n 1 1 2 0 > EXPECTED (whichever way is "clearer" -- using '%s\n' or even "%s\n" is also possible (just increasingly harder to write ;D)) Tomi > + test_expect_equal_file EXPECTED OUTPUT > +} > + > +test_AND from test_subtest_known_broken > +test_AND subject test_subtest_known_broken > +test_AND to > + > +test_done > -- > 2.23.0.rc1 ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 2/3] lib: introduce N_F_GROUP and use it to fix implicit AND for from: 2019-08-21 11:41 ` WIP: fix implicit operators David Bremner 2019-08-21 11:41 ` [PATCH 1/3] test: add known broken tests for from: and subject: David Bremner @ 2019-08-21 11:41 ` David Bremner 2019-08-21 11:41 ` [PATCH 3/3] WIP/test: extend field grouping tests David Bremner 2 siblings, 0 replies; 10+ messages in thread From: David Bremner @ 2019-08-21 11:41 UTC (permalink / raw) To: Rollins, Jameson, David Bremner, Jorge P. de Morais Neto, notmuch@notmuchmail.org This needs tests for every prefix and documentation --- lib/database-private.h | 1 + lib/database.cc | 23 +++++++++++++++++------ test/T760-implicit-operators.sh | 4 ++-- 3 files changed, 20 insertions(+), 8 deletions(-) diff --git a/lib/database-private.h b/lib/database-private.h index 87ae1bdf..a5eb83cb 100644 --- a/lib/database-private.h +++ b/lib/database-private.h @@ -159,6 +159,7 @@ typedef enum notmuch_field_flags { NOTMUCH_FIELD_EXTERNAL = 1 << 0, NOTMUCH_FIELD_PROBABILISTIC = 1 << 1, NOTMUCH_FIELD_PROCESSOR = 1 << 2, + NOTMUCH_FIELD_GROUP = 1 << 3, } notmuch_field_flag_t; /* diff --git a/lib/database.cc b/lib/database.cc index 24b7ec43..5d6c4ee6 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -272,16 +272,21 @@ prefix_t prefix_table[] = { { "body", "", NOTMUCH_FIELD_EXTERNAL | NOTMUCH_FIELD_PROBABILISTIC }, { "thread", "G", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, + NOTMUCH_FIELD_PROCESSOR | + NOTMUCH_FIELD_GROUP + }, { "tag", "K", NOTMUCH_FIELD_EXTERNAL | NOTMUCH_FIELD_PROCESSOR }, { "is", "K", NOTMUCH_FIELD_EXTERNAL | NOTMUCH_FIELD_PROCESSOR }, - { "id", "Q", NOTMUCH_FIELD_EXTERNAL }, + { "id", "Q", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_GROUP }, { "mid", "Q", NOTMUCH_FIELD_EXTERNAL | NOTMUCH_FIELD_PROCESSOR }, { "path", "P", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, + NOTMUCH_FIELD_PROCESSOR | + NOTMUCH_FIELD_GROUP + }, { "property", "XPROPERTY", NOTMUCH_FIELD_EXTERNAL }, /* * Unconditionally add ':' to reduce potential ambiguity with @@ -290,10 +295,14 @@ prefix_t prefix_table[] = { * discussion. */ { "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, + NOTMUCH_FIELD_PROCESSOR | + NOTMUCH_FIELD_GROUP + }, #if HAVE_XAPIAN_FIELD_PROCESSOR { "date", NULL, NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, + NOTMUCH_FIELD_PROCESSOR | + NOTMUCH_FIELD_GROUP + }, { "query", NULL, NOTMUCH_FIELD_EXTERNAL | NOTMUCH_FIELD_PROCESSOR }, #endif @@ -400,7 +409,9 @@ _setup_query_field (const prefix_t *prefix, notmuch_database_t *notmuch) /* we treat all field-processor fields as boolean in order to get the raw input */ if (prefix->prefix) notmuch->query_parser->add_prefix ("", prefix->prefix); - notmuch->query_parser->add_boolean_prefix (prefix->name, fp); + notmuch->query_parser->add_boolean_prefix (prefix->name, fp, + !(prefix->flags & NOTMUCH_FIELD_PROBABILISTIC) && + (prefix->flags & NOTMUCH_FIELD_GROUP)); } else { _setup_query_field_default (prefix, notmuch); } diff --git a/test/T760-implicit-operators.sh b/test/T760-implicit-operators.sh index b79673df..1a6ba61f 100755 --- a/test/T760-implicit-operators.sh +++ b/test/T760-implicit-operators.sh @@ -21,8 +21,8 @@ EOF test_expect_equal_file EXPECTED OUTPUT } -test_AND from test_subtest_known_broken -test_AND subject test_subtest_known_broken +test_AND from +test_AND subject test_AND to test_done -- 2.23.0.rc1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 3/3] WIP/test: extend field grouping tests 2019-08-21 11:41 ` WIP: fix implicit operators David Bremner 2019-08-21 11:41 ` [PATCH 1/3] test: add known broken tests for from: and subject: David Bremner 2019-08-21 11:41 ` [PATCH 2/3] lib: introduce N_F_GROUP and use it to fix implicit AND for from: David Bremner @ 2019-08-21 11:41 ` David Bremner 2 siblings, 0 replies; 10+ messages in thread From: David Bremner @ 2019-08-21 11:41 UTC (permalink / raw) To: Rollins, Jameson, David Bremner, Jorge P. de Morais Neto, notmuch@notmuchmail.org --- test/T760-implicit-operators.sh | 60 +++++++++++++++++++++++++++------ 1 file changed, 49 insertions(+), 11 deletions(-) diff --git a/test/T760-implicit-operators.sh b/test/T760-implicit-operators.sh index 1a6ba61f..438f766f 100755 --- a/test/T760-implicit-operators.sh +++ b/test/T760-implicit-operators.sh @@ -2,16 +2,16 @@ test_description='implicit operators in query parser' . $(dirname "$0")/test-lib.sh || exit 1 -test_AND() { - add_message "[$1]=a@b" - add_message "[$1]=b@c" +test_prob_AND() { + add_message "[$1]=alpha@beta" + add_message "[$1]=beta@gamma" - test_begin_subtest "$1: implicitly joined by AND" + test_begin_subtest "probabilistic field '$1:' implicitly joined by AND" $2 - notmuch count $1:a@b > OUTPUT - notmuch count $1:a $1:b >> OUTPUT - notmuch count $1:a@b OR $1:b@c >> OUTPUT - notmuch count $1:a@b $1:b@c >> OUTPUT + notmuch count $1:alpha@beta > OUTPUT + notmuch count $1:alpha $1:beta >> OUTPUT + notmuch count $1:alpha@beta OR $1:beta@gamma >> OUTPUT + notmuch count $1:alpha@beta $1:beta@gamma >> OUTPUT cat <<EOF > EXPECTED 1 1 @@ -21,8 +21,46 @@ EOF test_expect_equal_file EXPECTED OUTPUT } -test_AND from -test_AND subject -test_AND to +test_regex_AND() { + test_begin_subtest "regex field '$1:' implicitly joined by AND" + $2 + notmuch count $1:alpha@beta > OUTPUT + notmuch count $1:/alpha/ $1:/beta/ >> OUTPUT + notmuch count $1:alpha@beta OR $1:beta@gamma >> OUTPUT + notmuch count $1:alpha@beta $1:beta@gamma >> OUTPUT + cat <<EOF > EXPECTED +1 +1 +2 +0 +EOF + test_expect_equal_file EXPECTED OUTPUT +} + +test_prob_AND from +test_prob_AND subject +test_prob_AND to + + +add_message "[id]=alpha@beta" +add_message "[id]=beta@gamma" + +test_regex_AND mid + +test_begin_subtest "'id:' implicitly joined by OR" +notmuch count id:alpha@beta > OUTPUT +notmuch count id:alpha@beta OR id:beta@gamma >> OUTPUT +notmuch count id:alpha@beta id:beta@gamma >> OUTPUT +cat <<EOF > EXPECTED +1 +2 +2 +EOF +test_expect_equal_file EXPECTED OUTPUT + +notmuch tag +alpha@beta id:alpha@beta +notmuch tag +beta@gamma id:beta@gamma + +test_regex_AND tag test_done -- 2.23.0.rc1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2019-08-25 9:33 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-08-11 17:20 Apparently, terms with a common prefix are *not* connected by implicit "OR" Jorge P. de Morais Neto 2019-08-11 23:08 ` David Bremner 2019-08-15 14:58 ` Jorge P. de Morais Neto 2019-08-21 1:39 ` David Bremner 2019-08-21 5:48 ` Rollins, Jameson 2019-08-21 11:41 ` WIP: fix implicit operators David Bremner 2019-08-21 11:41 ` [PATCH 1/3] test: add known broken tests for from: and subject: David Bremner 2019-08-25 9:33 ` Tomi Ollila 2019-08-21 11:41 ` [PATCH 2/3] lib: introduce N_F_GROUP and use it to fix implicit AND for from: David Bremner 2019-08-21 11:41 ` [PATCH 3/3] WIP/test: extend field grouping tests David Bremner
Code repositories for project(s) associated with this public inbox https://yhetil.org/notmuch.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).