Re: Xapian regex with parentheses fails

Re: Xapian regex with parentheses fails

[Javier Garcia]

[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

Fantastic, it works!

Just as a suggestion, maybe this tip can be included in the man pages.
Something on the lines of:

Use *folder:"/<regex>/"* in case the regex contains special characters
interpretable by Xapian. Double escape as necesary. For instance:

notmuch search 'folder:"/^.*/(Junk|Spam)$/"'

On 06/04/18 17:36, David Bremner wrote:
> Javier Garcia <javiertury@gmail.com> writes:
>
>> Hi,
>>
>> I'm configuring notmuch in a multi-account directory. I want it to tag
>> spam and deleted mails using a post hook. Using regexes with parentheses
>> fails, or perhaps I misunderstood the documentation. The following one
>> liner replicates the problem I face.
>>
>> $ notmuch search 'folder:/^.*/(Junk|Spam)$/'
>> notmuch search: A Xapian exception occurred
>> A Xapian exception occurred parsing query: unmatched regex delimiter in
>> '/^.*/(Junk|Spam'
>> Query string was: folder:/^.*/(Junk|Spam)$/
> Try
>
>  notmuch search 'folder:"/^.*/(Junk|Spam)$/"'
>  


[-- Attachment #2: Type: text/html, Size: 1638 bytes --]

[PATCH] doc: add a section on quoting to notmuch-search-terms(7)

[David Bremner]

I think we've diverged enough from the Xapian query parser
that we can't rely on that syntax description [1]. As far as I can
tell, [1] also only discusses quotes in the context of phrases.

[1]: https://xapian.org/docs/queryparser.html
---
 doc/man7/notmuch-search-terms.rst | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/doc/man7/notmuch-search-terms.rst b/doc/man7/notmuch-search-terms.rst
index b4d784a3..a4e5abfc 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -273,6 +273,28 @@ Both of these will match a subject "Free Delicious Pizza" while
 
 will not.
 
+Quoting
+-------
+
+Double quotes are also used by the notmuch query parser to protect
+boolean terms or regular expressions containing spaces or other
+special characters, e.g.
+
+::
+
+   tag:"a tag"
+
+::
+
+   folder:"/^.*/(Junk|Spam)$/"
+
+As with phrases, you will probably have to protect the double quotes
+from the shell e.g.
+
+::
+
+   % notmuch search 'folder:"/^.*/(Junk|Spam)$/"'
+
 DATE AND TIME SEARCH
 ====================
 
-- 
2.16.3

Re: [PATCH] doc: add a section on quoting to notmuch-search-terms(7)

[Daniel Kahn Gillmor]

On Sat 2018-04-07 19:10:51 -0300, David Bremner wrote:
> I think we've diverged enough from the Xapian query parser
> that we can't rely on that syntax description [1]. As far as I can
> tell, [1] also only discusses quotes in the context of phrases.
>
> [1]: https://xapian.org/docs/queryparser.html
> ---
>  doc/man7/notmuch-search-terms.rst | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
>
> diff --git a/doc/man7/notmuch-search-terms.rst b/doc/man7/notmuch-search-terms.rst
> index b4d784a3..a4e5abfc 100644
> --- a/doc/man7/notmuch-search-terms.rst
> +++ b/doc/man7/notmuch-search-terms.rst
> @@ -273,6 +273,28 @@ Both of these will match a subject "Free Delicious Pizza" while
>  
>  will not.
>  
> +Quoting
> +-------
> +
> +Double quotes are also used by the notmuch query parser to protect
> +boolean terms or regular expressions containing spaces or other
> +special characters, e.g.
> +
> +::
> +
> +   tag:"a tag"
> +
> +::
> +
> +   folder:"/^.*/(Junk|Spam)$/"
> +
> +As with phrases, you will probably have to protect the double quotes
> +from the shell e.g.
> +
> +::
> +
> +   % notmuch search 'folder:"/^.*/(Junk|Spam)$/"'
> +
>  DATE AND TIME SEARCH
>  ====================

LGTM.

        --dkg

Re: [PATCH] doc: add a section on quoting to notmuch-search-terms(7)

[Tomi Ollila]

On Sat, Apr 07 2018, David Bremner wrote:

> I think we've diverged enough from the Xapian query parser
> that we can't rely on that syntax description [1]. As far as I can
> tell, [1] also only discusses quotes in the context of phrases.
>
> [1]: https://xapian.org/docs/queryparser.html
> ---
>  doc/man7/notmuch-search-terms.rst | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
>
> diff --git a/doc/man7/notmuch-search-terms.rst b/doc/man7/notmuch-search-terms.rst
> index b4d784a3..a4e5abfc 100644
> --- a/doc/man7/notmuch-search-terms.rst
> +++ b/doc/man7/notmuch-search-terms.rst
> @@ -273,6 +273,28 @@ Both of these will match a subject "Free Delicious Pizza" while
>  
>  will not.
>  
> +Quoting
> +-------
> +
> +Double quotes are also used by the notmuch query parser to protect
> +boolean terms or regular expressions containing spaces or other
> +special characters, e.g.
> +
> +::
> +
> +   tag:"a tag"
> +
> +::
> +
> +   folder:"/^.*/(Junk|Spam)$/"
> +
> +As with phrases, you will probably have to protect the double quotes

The 'probably' gives somewhat uncertain answer -- perhaps s/will probably //

Also, perhaps the example in line 45 could be in format:

    notmuch search 'from:"/bob@.*[.]example[.]com/"'

-- format that should be always safe to use -- good for 
   e.g. tldr people like me.

Btw, the ones who have notmuch email since 2011-10-18 can try the
following:
 
    notmuch count 'from:/Richard Stallman/'
    3
    notmuch count 'from:/Richard Stallma/' 
    0
    notmuch count 'from:"/Richard Stallman/"'
    3
    notmuch count 'from:"/Richard Stallma/"'
    0

I find all but 3rd odd (1st one "works" even there are spaces, and 
second and last don't find matches...).

Tomi

> +from the shell e.g.
> +
> +::
> +
> +   % notmuch search 'folder:"/^.*/(Junk|Spam)$/"'
> +
>  DATE AND TIME SEARCH
>  ====================
>  
> -- 
> 2.16.3
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] doc: add a section on quoting to notmuch-search-terms(7)

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:


> Btw, the ones who have notmuch email since 2011-10-18 can try the
> following:
>  
>     notmuch count 'from:/Richard Stallman/'
>     3


A Xapian exception occurred parsing query: unmatched regex delimiter in
'/Richard'

>     notmuch count 'from:/Richard Stallma/' 
>     0

A Xapian exception occurred parsing query: unmatched regex delimiter in '/Richard'

>     notmuch count 'from:"/Richard Stallman/"'
>     3

Well, I have 7 messages, but I report Emacs bugs...

>     notmuch count 'from:"/Richard Stallma/"'
>     0
>

Also 7

> I find all but 3rd odd (1st one "works" even there are spaces, and 
> second and last don't find matches...).

I don't what's going on with your results. Maybe old Xapian?

d

Re: [PATCH] doc: add a section on quoting to notmuch-search-terms(7)

[Tomi Ollila]

On Tue, Apr 24 2018, David Bremner wrote:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>
>
>> Btw, the ones who have notmuch email since 2011-10-18 can try the
>> following:
>>  
>>     notmuch count 'from:/Richard Stallman/'
>>     3
>
>
> A Xapian exception occurred parsing query: unmatched regex delimiter in
> '/Richard'

Interesting...

>
>>     notmuch count 'from:/Richard Stallma/' 
>>     0
>
> A Xapian exception occurred parsing query: unmatched regex delimiter in '/Richard'
>
>>     notmuch count 'from:"/Richard Stallman/"'
>>     3
>
> Well, I have 7 messages, but I report Emacs bugs...
>
>>     notmuch count 'from:"/Richard Stallma/"'
>>     0
>>
>
> Also 7

Looks better than mine :D

>> I find all but 3rd odd (1st one "works" even there are spaces, and 
>> second and last don't find matches...).
>
> I don't what's going on with your results. Maybe old Xapian?

 xapian-core/current -> 1.2.24

Did our documentation mentioned regexp searches require Xapian 1.4+.

>
> d

Tomi

Re: [PATCH] doc: add a section on quoting to notmuch-search-terms(7)

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

>
> Did our documentation mentioned regexp searches require Xapian 1.4+.
>

Yes, in the sense that it mentions field processors. That's what
"notmuch config get built_with.field_processor" exists.

d

Re: [PATCH] doc: add a section on quoting to notmuch-search-terms(7)

[David Bremner]

David Bremner <david@tethera.net> writes:

> I think we've diverged enough from the Xapian query parser
> that we can't rely on that syntax description [1]. As far as I can
> tell, [1] also only discusses quotes in the context of phrases.

pushed, amended per Tomi's comments.

d