[RFC] lib: add support for date:<query>..! to mean date:<query>..<query>

[RFC] lib: add support for date:<query>..! to mean date:<query>..<query>

[Jani Nikula]

Up to debate:

1) Is something like this useful at all as an intermediate step before
we can have support for date:<query>? (This can be done with a future
version of Xapian, or with a custom query query parser.)

2) If yes, are there better alternatives to "!" as the end point? (Or
should the special case be the start point?) Also "@" and "same" have
been suggested. Examples: date:yesterday..! date:today..@
date:@..monday date:january..same.

Idea from Mark Walters <markwalters1009@gmail.com>.
---
 lib/parse-time-vrp.cc    | 5 +++++
 test/T500-search-date.sh | 4 ++++
 2 files changed, 9 insertions(+)

diff --git a/lib/parse-time-vrp.cc b/lib/parse-time-vrp.cc
index 33f07db3410e..03804cf50fa8 100644
--- a/lib/parse-time-vrp.cc
+++ b/lib/parse-time-vrp.cc
@@ -31,6 +31,7 @@ Xapian::valueno
 ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
 {
     time_t t, now;
+    std::string b;
 
     /* Require date: prefix in start of the range... */
     if (STRNCMP_LITERAL (begin.c_str (), PREFIX))
@@ -38,6 +39,7 @@ ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
 
     /* ...and remove it. */
     begin.erase (0, sizeof (PREFIX) - 1);
+    b = begin;
 
     /* Use the same 'now' for begin and end. */
     if (time (&now) == (time_t) -1)
@@ -51,6 +53,9 @@ ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
     }
 
     if (!end.empty ()) {
+	if (end == "!" && ! b.empty ())
+	    end = b;
+
 	if (parse_time_string (end.c_str (), &t, &now, PARSE_TIME_ROUND_UP_INCLUSIVE))
 	    return Xapian::BAD_VALUENO;
 
diff --git a/test/T500-search-date.sh b/test/T500-search-date.sh
index 70bcf344b4f7..18a47b114fa9 100755
--- a/test/T500-search-date.sh
+++ b/test/T500-search-date.sh
@@ -8,6 +8,10 @@ test_begin_subtest "Absolute date range"
 output=$(notmuch search date:2010-12-16..12/16/2010 | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; Essai accentué (inbox unread)"
 
+test_begin_subtest "Absolute date range with 'same' operator"
+output=$(notmuch search date:2010-12-16..! | notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; Essai accentué (inbox unread)"
+
 test_begin_subtest "Absolute time range with TZ"
 notmuch search date:18-Nov-2009_02:19:26-0800..2009-11-18_04:49:52-06:00 | notmuch_search_sanitize > OUTPUT
 cat <<EOF >EXPECTED
-- 
2.1.4

Re: [RFC] lib: add support for date:<query>..! to mean date:<query>..<query>

[Mark Walters]

Hi

On Sat, 07 Mar 2015, Jani Nikula <jani@nikula.org> wrote:
> Up to debate:
>
> 1) Is something like this useful at all as an intermediate step before
> we can have support for date:<query>? (This can be done with a future
> version of Xapian, or with a custom query query parser.)

This looks good to me. Yes it may not be needed in the future but the
patch is very small. It passes all tests. +1 from me.

> 2) If yes, are there better alternatives to "!" as the end point? (Or
> should the special case be the start point?) Also "@" and "same" have
> been suggested. Examples: date:yesterday..! date:today..@
> date:@..monday date:january..same.

I would be happy with any of these.

Best wishes

Mark

>
> Idea from Mark Walters <markwalters1009@gmail.com>.
> ---
>  lib/parse-time-vrp.cc    | 5 +++++
>  test/T500-search-date.sh | 4 ++++
>  2 files changed, 9 insertions(+)
>
> diff --git a/lib/parse-time-vrp.cc b/lib/parse-time-vrp.cc
> index 33f07db3410e..03804cf50fa8 100644
> --- a/lib/parse-time-vrp.cc
> +++ b/lib/parse-time-vrp.cc
> @@ -31,6 +31,7 @@ Xapian::valueno
>  ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
>  {
>      time_t t, now;
> +    std::string b;
>  
>      /* Require date: prefix in start of the range... */
>      if (STRNCMP_LITERAL (begin.c_str (), PREFIX))
> @@ -38,6 +39,7 @@ ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
>  
>      /* ...and remove it. */
>      begin.erase (0, sizeof (PREFIX) - 1);
> +    b = begin;
>  
>      /* Use the same 'now' for begin and end. */
>      if (time (&now) == (time_t) -1)
> @@ -51,6 +53,9 @@ ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
>      }
>  
>      if (!end.empty ()) {
> +	if (end == "!" && ! b.empty ())
> +	    end = b;
> +
>  	if (parse_time_string (end.c_str (), &t, &now, PARSE_TIME_ROUND_UP_INCLUSIVE))
>  	    return Xapian::BAD_VALUENO;
>  
> diff --git a/test/T500-search-date.sh b/test/T500-search-date.sh
> index 70bcf344b4f7..18a47b114fa9 100755
> --- a/test/T500-search-date.sh
> +++ b/test/T500-search-date.sh
> @@ -8,6 +8,10 @@ test_begin_subtest "Absolute date range"
>  output=$(notmuch search date:2010-12-16..12/16/2010 | notmuch_search_sanitize)
>  test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; Essai accentué (inbox unread)"
>  
> +test_begin_subtest "Absolute date range with 'same' operator"
> +output=$(notmuch search date:2010-12-16..! | notmuch_search_sanitize)
> +test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; Essai accentué (inbox unread)"
> +
>  test_begin_subtest "Absolute time range with TZ"
>  notmuch search date:18-Nov-2009_02:19:26-0800..2009-11-18_04:49:52-06:00 | notmuch_search_sanitize > OUTPUT
>  cat <<EOF >EXPECTED
> -- 
> 2.1.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [RFC] lib: add support for date:<query>..! to mean date:<query>..<query>

[Suvayu Ali]

Hi,

On Fri, Apr 17, 2015 at 07:57:27PM +0100, Mark Walters wrote:
> 
> > 2) If yes, are there better alternatives to "!" as the end point? (Or
> > should the special case be the start point?) Also "@" and "same" have
> > been suggested. Examples: date:yesterday..! date:today..@
> > date:@..monday date:january..same.
> 
> I would be happy with any of these.

Many shells use ! for history expansion; maybe one of the other
characters would be better (no need to escape)?

I like @, purely aesthetic preference :).

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: [RFC] lib: add support for date:<query>..! to mean date:<query>..<query>

[Tomi Ollila]

On Sat, Mar 07 2015, Jani Nikula <jani@nikula.org> wrote:

> Up to debate:
>
> 1) Is something like this useful at all as an intermediate step before
> we can have support for date:<query>? (This can be done with a future
> version of Xapian, or with a custom query query parser.)
>
> 2) If yes, are there better alternatives to "!" as the end point? (Or
> should the special case be the start point?) Also "@" and "same" have
> been suggested. Examples: date:yesterday..! date:today..@
> date:@..monday date:january..same.


I'd rather see a temporary ugly hack in implementation (to expand
date:<date> to date:<date>..<date>) than permanent interface feature.
But this looks simple enough... and I'd go just with this implementation
(i.e. <date>..!)


Tomi


> diff --git a/lib/parse-time-vrp.cc b/lib/parse-time-vrp.cc
> index 33f07db3410e..03804cf50fa8 100644
> --- a/lib/parse-time-vrp.cc
> +++ b/lib/parse-time-vrp.cc
> @@ -31,6 +31,7 @@ Xapian::valueno
>  ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
>  {
>      time_t t, now;
> +    std::string b;
>  
>      /* Require date: prefix in start of the range... */
>      if (STRNCMP_LITERAL (begin.c_str (), PREFIX))
> @@ -38,6 +39,7 @@ ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
>  
>      /* ...and remove it. */
>      begin.erase (0, sizeof (PREFIX) - 1);
> +    b = begin;
>  
>      /* Use the same 'now' for begin and end. */
>      if (time (&now) == (time_t) -1)
> @@ -51,6 +53,9 @@ ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
>      }
>  
>      if (!end.empty ()) {
> +	if (end == "!" && ! b.empty ())
> +	    end = b;
> +
>  	if (parse_time_string (end.c_str (), &t, &now, PARSE_TIME_ROUND_UP_INCLUSIVE))
>  	    return Xapian::BAD_VALUENO;
>  

Re: [RFC] lib: add support for date:<query>..! to mean date:<query>..<query>

[Gaute Hope]

Excerpts from Mark Walters's message of April 17, 2015 20:57:
> 
> Hi
> 
> On Sat, 07 Mar 2015, Jani Nikula <jani@nikula.org> wrote:
>> Up to debate:
>>
>> 1) Is something like this useful at all as an intermediate step before
>> we can have support for date:<query>? (This can be done with a future
>> version of Xapian, or with a custom query query parser.)
> 
> This looks good to me. Yes it may not be needed in the future but the
> patch is very small. It passes all tests. +1 from me.
> 
>> 2) If yes, are there better alternatives to "!" as the end point? (Or
>> should the special case be the start point?) Also "@" and "same" have
>> been suggested. Examples: date:yesterday..! date:today..@
>> date:@..monday date:january..same.
> 
> I would be happy with any of these.

I am probably missing something, but why do you need the end
point?

  date:2015-04-16 mean date:2015-04-16..2015-04-16?
  date:<query> -> date:<query>..<query>

- gaute

Re: [RFC] lib: add support for date:<query>..! to mean date:<query>..<query>

[David Bremner]

Gaute Hope <eg@gaute.vetsj.com> writes:

>
> I am probably missing something, but why do you need the end
> point?
>
>   date:2015-04-16 mean date:2015-04-16..2015-04-16?
>   date:<query> -> date:<query>..<query>

means according to humans yes, but not according to the xapian query
parser we use.

d

Re: [RFC] lib: add support for date:<query>..! to mean date:<query>..<query>

[David Bremner]

Jani Nikula <jani@nikula.org> writes:

> Up to debate:
>
> 1) Is something like this useful at all as an intermediate step before
> we can have support for date:<query>? (This can be done with a future
> version of Xapian, or with a custom query query parser.)
>
> 2) If yes, are there better alternatives to "!" as the end point? (Or
> should the special case be the start point?) Also "@" and "same" have
> been suggested. Examples: date:yesterday..! date:today..@
> date:@..monday date:january..same.
>

From the comments I guess it just needs a small man page update.

d

[PATCH] lib: add support for date:<expr>..! to mean date:<expr>..<expr>

[Jani Nikula]

It doesn't seem likely we can support simple date:<expr> expanding to
date:<expr>..<expr> any time soon. (This can be done with a future
version of Xapian, or with a custom query query parser.) In the mean
time, provide shorthand date:<expr>..! to mean the same. This is
useful, as the expansion takes place before interpetation, and we can
use, for example, date:yesterday..! to match from beginning of
yesterday to end of yesterday.

Idea from Mark Walters <markwalters1009@gmail.com>.
---
 doc/man7/notmuch-search-terms.rst | 12 +++++++-----
 lib/parse-time-vrp.cc             |  5 +++++
 test/T500-search-date.sh          |  4 ++++
 3 files changed, 16 insertions(+), 5 deletions(-)

diff --git a/doc/man7/notmuch-search-terms.rst b/doc/man7/notmuch-search-terms.rst
index 1d27ac1e6de5..3b35a32b7d23 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -270,6 +270,13 @@ In this case, <since> is taken as the earliest time it could describe
 could describe (the end of yesterday). Similarly, date:january..february
 matches from the beginning of January to the end of February.
 
+date:<expr>..! can be used as a shorthand for date:<expr>..<expr>. The
+expansion takes place before interpretation, and thus, for example,
+date:monday..! matches from the beginning of Monday until the end of
+Monday. (Note that entering date:<expr> without "..", for example
+date:yesterday, won't work, as it's not interpreted as a range
+expression at all. Again, use date:yesterday..!)
+
 Currently, we do not support spaces in range expressions. You can
 replace the spaces with '\_', or (in most cases) '-', or (in some cases)
 leave the spaces out altogether. Examples in this man page use spaces
@@ -280,11 +287,6 @@ to specify date:..<until> or date:<since>.. to not limit the start or
 end time, respectively. Pre-1.2.1 Xapian does not report an error on
 open ended ranges, but it does not work as expected either.
 
-Entering date:expr without ".." (for example date:yesterday) won't work,
-as it's not interpreted as a range expression at all. You can achieve
-the expected result by duplicating the expr both sides of ".." (for
-example date:yesterday..yesterday).
-
 Relative date and time
 ----------------------
 
diff --git a/lib/parse-time-vrp.cc b/lib/parse-time-vrp.cc
index 33f07db3410e..03804cf50fa8 100644
--- a/lib/parse-time-vrp.cc
+++ b/lib/parse-time-vrp.cc
@@ -31,6 +31,7 @@ Xapian::valueno
 ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
 {
     time_t t, now;
+    std::string b;
 
     /* Require date: prefix in start of the range... */
     if (STRNCMP_LITERAL (begin.c_str (), PREFIX))
@@ -38,6 +39,7 @@ ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
 
     /* ...and remove it. */
     begin.erase (0, sizeof (PREFIX) - 1);
+    b = begin;
 
     /* Use the same 'now' for begin and end. */
     if (time (&now) == (time_t) -1)
@@ -51,6 +53,9 @@ ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end)
     }
 
     if (!end.empty ()) {
+	if (end == "!" && ! b.empty ())
+	    end = b;
+
 	if (parse_time_string (end.c_str (), &t, &now, PARSE_TIME_ROUND_UP_INCLUSIVE))
 	    return Xapian::BAD_VALUENO;
 
diff --git a/test/T500-search-date.sh b/test/T500-search-date.sh
index b28496b78e2e..f5cea421d785 100755
--- a/test/T500-search-date.sh
+++ b/test/T500-search-date.sh
@@ -8,6 +8,10 @@ test_begin_subtest "Absolute date range"
 output=$(notmuch search date:2010-12-16..12/16/2010 | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; Essai accentué (inbox unread)"
 
+test_begin_subtest "Absolute date range with 'same' operator"
+output=$(notmuch search date:2010-12-16..! | notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; Essai accentué (inbox unread)"
+
 test_begin_subtest "Absolute time range with TZ"
 notmuch search date:18-Nov-2009_02:19:26-0800..2009-11-18_04:49:52-06:00 | notmuch_search_sanitize > OUTPUT
 cat <<EOF >EXPECTED
-- 
2.1.4

Re: [PATCH] lib: add support for date:<expr>..! to mean date:<expr>..<expr>

[David Bremner]

Jani Nikula <jani@nikula.org> writes:

> It doesn't seem likely we can support simple date:<expr> expanding to
> date:<expr>..<expr> any time soon. (This can be done with a future
> version of Xapian, or with a custom query query parser.) In the mean
> time, provide shorthand date:<expr>..! to mean the same. This is
> useful, as the expansion takes place before interpetation, and we can
> use, for example, date:yesterday..! to match from beginning of
> yesterday to end of yesterday.

pushed. At some point we probably want a NEWS item.

d

Re: [PATCH] lib: add support for date:<expr>..! to mean date:<expr>..<expr>

[Tomi Ollila]

On Sat, Sep 26 2015, David Bremner <david@tethera.net> wrote:

> Jani Nikula <jani@nikula.org> writes:
>
>> It doesn't seem likely we can support simple date:<expr> expanding to
>> date:<expr>..<expr> any time soon. (This can be done with a future
>> version of Xapian, or with a custom query query parser.) In the mean
>> time, provide shorthand date:<expr>..! to mean the same. This is
>> useful, as the expansion takes place before interpetation, and we can
>> use, for example, date:yesterday..! to match from beginning of
>> yesterday to end of yesterday.
>
> pushed. At some point we probably want a NEWS item.

If it is in the NEWS, it is perpetual, otoh if only in documentation we
could theoretically deprecate and remove it easier in the future ... ;p

> d

Tomi