Query operators

Query operators

[Yuri D'Elia]

Hi everyone,

I'm a bit baffled by the query syntax. Reading the manual:

``Each term in the query will be implicitly connected by a logical AND
if no explicit operator is provided (except that terms with a common
prefix will be implicitly combined with OR).''

I would have assumed that:

term1 (tag:term2 tag:term3)

would be equivalent to:

term1 AND (tag:term2 AND tag:term3)

but if I read carefully (and by looking at the query results), since
tag: is a common prefix between term2 and term3, it is actually:

term1 AND (tag:term2 OR tag:term3)

am I right? Is this a feature of the xapian query syntax? (can it be
tweaked to _unconditionally_ AND all terms?)

For me, the idea that two terms can be ORed if the prefix is the same
can cause queries to return unexpected results.

What about:

term1 xterm2 xterm3 term2

Is this:

  term1 AND (xterm2 OR xterm3) AND term2

or is it:

  term1 OR xterm2 OR xterm3 OR term2

?

Re: Query operators

[Charlie Allom]

You can debug what the real query is by adding NOTMUCH_DEBUG_QUERY=1 to
your search:

  12:29 YELP-CHARLIE~[local@2.7.12] % NOTMUCH_DEBUG_QUERY=1 notmuch search 'term1 (tag:term2 tag:term3)'
  Query string is:
  term1 (tag:term2 tag:term3)
  Exclude query is:
  Xapian::Query((Kdeleted OR Kspam OR Kmuted OR Kkilled OR Kjunk))
  Final query is:
  Xapian::Query(((Tmail AND Zterm1:(pos=1) AND 0 * (Kterm2 OR Kterm3)) AND_NOT (Kdeleted OR Kspam OR Kmuted OR Kkilled OR Kjunk)))
  12:29 YELP-CHARLIE~[local@2.7.12] %

On Mon, Aug 29 2016 at 11:29:03 AM, Yuri D'Elia <wavexx@thregr.org> wrote:

> Hi everyone,
>
> I'm a bit baffled by the query syntax. Reading the manual:
>
> ``Each term in the query will be implicitly connected by a logical AND
> if no explicit operator is provided (except that terms with a common
> prefix will be implicitly combined with OR).''
>
> I would have assumed that:
>
> term1 (tag:term2 tag:term3)
>
> would be equivalent to:
>
> term1 AND (tag:term2 AND tag:term3)
>
> but if I read carefully (and by looking at the query results), since
> tag: is a common prefix between term2 and term3, it is actually:
>
> term1 AND (tag:term2 OR tag:term3)
>
> am I right? Is this a feature of the xapian query syntax? (can it be
> tweaked to _unconditionally_ AND all terms?)
>
> For me, the idea that two terms can be ORed if the prefix is the same
> can cause queries to return unexpected results.
>
> What about:
>
> term1 xterm2 xterm3 term2
>
> Is this:
>
>   term1 AND (xterm2 OR xterm3) AND term2
>
> or is it:
>
>   term1 OR xterm2 OR xterm3 OR term2
>
> ?
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

-- 

Re: Query operators

[Yuri D'Elia]

On Mon, Aug 29 2016, Charlie Allom <charlie@evilforbeginners.com> wrote:
> You can debug what the real query is by adding NOTMUCH_DEBUG_QUERY=1 to
> your search:
>
>   12:29 YELP-CHARLIE~[local@2.7.12] % NOTMUCH_DEBUG_QUERY=1 notmuch search 'term1 (tag:term2 tag:term3)'
>   Query string is:
>   term1 (tag:term2 tag:term3)
>   Exclude query is:
>   Xapian::Query((Kdeleted OR Kspam OR Kmuted OR Kkilled OR Kjunk))
>   Final query is:
>   Xapian::Query(((Tmail AND Zterm1:(pos=1) AND 0 * (Kterm2 OR Kterm3)) AND_NOT (Kdeleted OR Kspam OR Kmuted OR Kkilled OR Kjunk)))
>   12:29 YELP-CHARLIE~[local@2.7.12] %

That was helpful, but confirms that tag terms are combined with OR:

Query string is:
tag:a -(b c)
Final query is:
=> (0 * Ka AND_NOT (Zb:(pos=1) AND Zc:(pos=2)))

(removed the exclusion for brevity)

Query string is:
tag:a -(tag:b tag:c)
Final query is:
=> (0 * Ka AND_NOT 0 * (Kb OR Kc))

is this because they're both prefixed with tag?
Indeed 'tag:a is:b' combines them with AND.

'tag:a is:b tag:c' is actually (tag:a OR tag:c) AND tag:b.

Why this should be helpful?
It's totally unexpected for me.

Re: Query operators

[David Bremner]

Yuri D'Elia <wavexx@thregr.org> writes:

> term1 AND (tag:term2 OR tag:term3)
>
> am I right? Is this a feature of the xapian query syntax? (can it be
> tweaked to _unconditionally_ AND all terms?)

This is most likely a feature of the "grouping" parameter in
QueryParser::add_boolean_prefix. It _might_ be the case that passing an
empty string (in lib/database.cc) will improve this behaviour from your
point of view.

d

Re: Query operators

[Yuri D'Elia]

On Mon, Aug 29 2016, David Bremner <david@tethera.net> wrote:
> Yuri D'Elia <wavexx@thregr.org> writes:
>
>> term1 AND (tag:term2 OR tag:term3)
>>
>> am I right? Is this a feature of the xapian query syntax? (can it be
>> tweaked to _unconditionally_ AND all terms?)
>
> This is most likely a feature of the "grouping" parameter in
> QueryParser::add_boolean_prefix. It _might_ be the case that passing an
> empty string (in lib/database.cc) will improve this behaviour from your
> point of view.

What's your POV on this? Is it expected/useful in some scenario?

I'd expect all terms to follow the same rules for consistency.

Re: Query operators

[David Bremner]

Yuri D'Elia <wavexx@thregr.org> writes:

>
> What's your POV on this? Is it expected/useful in some scenario?
>
> I'd expect all terms to follow the same rules for consistency.

For terms where each document (mail message) has only one term with a
given prefix, OR is a more natural default, since AND can never
match. In notmuch we have id:, path:, and thread: of this type (and
maybe lastmod: and date:, although these are not boolean prefixes),
compared to tag: where each document can have many terms with the same
prefix.  Making tag: behave differently than the others is probably not
that nice either, so mostly I'd suggest explicitly writing what
operators you want.

d