[Wishlist] Add equivalent of lastmod:nnn.. to sexp query parser

[Wishlist] Add equivalent of lastmod:nnn.. to sexp query parser

[David Bremner]

This is mainly a reminder to myself, but also an opportunity for query
language bikeshedding.

Currently the sexp query parser parses '(lastmod 1234)' as equivalent to
lastmod:1234..1234. This is reasonable enough, but leaves no easy way to
do the equivalent to lastmod:1234.. (the other one sided range
lastmod:..1234 is easy since 0 works on the left e.g. (lastmod 0 1234).

This could be fixed by

1) changing the semantics of the 1 argument (lastmod 1234)
2) re-using * as in (lastmod 1234 *)
3) introducing a new reserved atom, e.g. "max" (lastmod 1234 max)

[PATCH 1/4] test/sexp: one sided range searches.

[David Bremner]

The date range parsing machinery already knows how to do something
appropriate with an empty string, but the lastmod parsing blindly
tries to parse each atom as a number.
---
 test/T081-sexpr-search.sh | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh
index da819190..31169fb5 100755
--- a/test/T081-sexpr-search.sh
+++ b/test/T081-sexpr-search.sh
@@ -854,6 +854,16 @@ notmuch search date:2009-11-17..2009-11-18 and from:keithp | notmuch_search_sani
 notmuch search --query=sexp  '(and (date 2009-11-17 2009-11-18) (from keithp))' | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "date query, lower bound only"
+notmuch search date:2009-11-18.. and from:keithp | notmuch_search_sanitize > EXPECTED
+notmuch search --query=sexp  '(and (date 2009-11-18 "") (from keithp))' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
+test_begin_subtest "date query, upper bound only"
+notmuch search date:..2009-11-17 and from:keithp | notmuch_search_sanitize > EXPECTED
+notmuch search --query=sexp  '(and (date "" 2009-11-17) (from keithp))' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
 test_begin_subtest "date query, illegal nesting 1"
 notmuch search --query=sexp '(to (date))' > OUTPUT 2>&1
 cat <<EOF > EXPECTED
@@ -921,6 +931,18 @@ notmuch search lastmod:$revision..$revision2 | notmuch_search_sanitize > EXPECTE
 notmuch search --query=sexp  "(and (lastmod $revision $revision2))" | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "lastmod query, lower bound only"
+test_subtest_known_broken
+notmuch search lastmod:$revision.. | notmuch_search_sanitize > EXPECTED
+notmuch search --query=sexp  "(lastmod $revision \"\")" | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
+test_begin_subtest "lastmod query, upper bound only"
+test_subtest_known_broken
+notmuch search lastmod:..$revision2 | notmuch_search_sanitize > EXPECTED
+notmuch search --query=sexp  "(lastmod \"\" $revision2)" | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
 test_begin_subtest "lastmod query, illegal nesting 1"
 notmuch search --query=sexp '(to (lastmod))' > OUTPUT 2>&1
 cat <<EOF > EXPECTED
-- 
2.35.2

[PATCH 2/4] lib/sexp: special case "" as an argument in lastmod ranges.

[David Bremner]

Support this syntax for constincy with (data from to) ranges.
---
 lib/parse-sexp.cc         | 10 ++++++++--
 test/T081-sexpr-search.sh |  2 --
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/lib/parse-sexp.cc b/lib/parse-sexp.cc
index 08fd7037..6282a456 100644
--- a/lib/parse-sexp.cc
+++ b/lib/parse-sexp.cc
@@ -504,14 +504,20 @@ _sexp_parse_range (notmuch_database_t *notmuch,  const _sexp_prefix_t *prefix,
 	long from_idx, to_idx;
 
 	try {
-	    from_idx = std::stol (from);
+	    if (EMPTY_STRING (from))
+		from_idx = 0L;
+	    else
+		from_idx = std::stol (from);
 	} catch (std::logic_error &e) {
 	    _notmuch_database_log (notmuch, "bad 'from' revision: '%s'\n", from);
 	    return NOTMUCH_STATUS_BAD_QUERY_SYNTAX;
 	}
 
 	try {
-	    to_idx = std::stol (to);
+	    if (EMPTY_STRING (to))
+		to_idx = LONG_MAX;
+	    else
+		to_idx = std::stol (to);
 	} catch (std::logic_error &e) {
 	    _notmuch_database_log (notmuch, "bad 'to' revision: '%s'\n", to);
 	    return NOTMUCH_STATUS_BAD_QUERY_SYNTAX;
diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh
index 31169fb5..896ffe5d 100755
--- a/test/T081-sexpr-search.sh
+++ b/test/T081-sexpr-search.sh
@@ -932,13 +932,11 @@ notmuch search --query=sexp  "(and (lastmod $revision $revision2))" | notmuch_se
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "lastmod query, lower bound only"
-test_subtest_known_broken
 notmuch search lastmod:$revision.. | notmuch_search_sanitize > EXPECTED
 notmuch search --query=sexp  "(lastmod $revision \"\")" | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file_nonempty EXPECTED OUTPUT
 
 test_begin_subtest "lastmod query, upper bound only"
-test_subtest_known_broken
 notmuch search lastmod:..$revision2 | notmuch_search_sanitize > EXPECTED
 notmuch search --query=sexp  "(lastmod \"\" $revision2)" | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file_nonempty EXPECTED OUTPUT
-- 
2.35.2

[PATCH 3/4] lib/sexp: allow * as alias for "" in range searches.

[David Bremner]

It can be tedious to use "" inside of a string, e.g. in a shell script.
---
 lib/parse-sexp.cc         |  5 +++++
 test/T081-sexpr-search.sh | 20 ++++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/lib/parse-sexp.cc b/lib/parse-sexp.cc
index 6282a456..6814c9fc 100644
--- a/lib/parse-sexp.cc
+++ b/lib/parse-sexp.cc
@@ -473,6 +473,9 @@ _sexp_parse_range (notmuch_database_t *notmuch,  const _sexp_prefix_t *prefix,
     }
 
     from = sx->val;
+    if (strcmp (from, "*") == 0)
+	from = "";
+
     to = from;
 
     if (sx->next) {
@@ -488,6 +491,8 @@ _sexp_parse_range (notmuch_database_t *notmuch,  const _sexp_prefix_t *prefix,
 	}
 
 	to = sx->next->val;
+	if (strcmp (to, "*") == 0)
+	    to = "";
     }
 
     if (strcmp (prefix->name, "date") == 0) {
diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh
index 896ffe5d..1b036a0d 100755
--- a/test/T081-sexpr-search.sh
+++ b/test/T081-sexpr-search.sh
@@ -864,6 +864,16 @@ notmuch search date:..2009-11-17 and from:keithp | notmuch_search_sanitize > EXP
 notmuch search --query=sexp  '(and (date "" 2009-11-17) (from keithp))' | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file_nonempty EXPECTED OUTPUT
 
+test_begin_subtest "date query, lower bound only, using *"
+notmuch search date:2009-11-18.. and from:keithp | notmuch_search_sanitize > EXPECTED
+notmuch search --query=sexp  '(and (date 2009-11-18 *) (from keithp))' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
+test_begin_subtest "date query, upper bound only, using *"
+notmuch search date:..2009-11-17 and from:keithp | notmuch_search_sanitize > EXPECTED
+notmuch search --query=sexp  '(and (date * 2009-11-17) (from keithp))' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
 test_begin_subtest "date query, illegal nesting 1"
 notmuch search --query=sexp '(to (date))' > OUTPUT 2>&1
 cat <<EOF > EXPECTED
@@ -941,6 +951,16 @@ notmuch search lastmod:..$revision2 | notmuch_search_sanitize > EXPECTED
 notmuch search --query=sexp  "(lastmod \"\" $revision2)" | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file_nonempty EXPECTED OUTPUT
 
+test_begin_subtest "lastmod query, lower bound only, using *"
+notmuch search lastmod:$revision.. | notmuch_search_sanitize > EXPECTED
+notmuch search --query=sexp  "(lastmod $revision *)" | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
+test_begin_subtest "lastmod query, upper bound only, using *"
+notmuch search lastmod:..$revision2 | notmuch_search_sanitize > EXPECTED
+notmuch search --query=sexp  "(lastmod * $revision2)" | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
 test_begin_subtest "lastmod query, illegal nesting 1"
 notmuch search --query=sexp '(to (lastmod))' > OUTPUT 2>&1
 cat <<EOF > EXPECTED
-- 
2.35.2

[PATCH 4/4] doc/sexp: document range queries.

[David Bremner]

Give examples for date fields, as these are commonly useful for user
queries.
---
 doc/man7/notmuch-sexp-queries.rst | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/doc/man7/notmuch-sexp-queries.rst b/doc/man7/notmuch-sexp-queries.rst
index 1d7e0ae9..be589f98 100644
--- a/doc/man7/notmuch-sexp-queries.rst
+++ b/doc/man7/notmuch-sexp-queries.rst
@@ -119,6 +119,12 @@ a message has one such attribute, and ``and`` otherwise.
 Term or phrase fields can contain arbitrarily complex queries made up
 from terms, operators, and modifiers, but not other fields.
 
+Range fields take one or two arguments specifying lower and upper
+bounds.  One argument is interpreted as identical upper and lower
+bounds. Either upper or lower bound may be specified as ``""`` or
+``*`` to specify the lowest possible lower bound or highest possible
+upper bound.
+
 .. _field-table:
 
 .. table:: Fields with supported modifiers
@@ -240,6 +246,18 @@ EXAMPLES
 
     Match messages in the given date range with tag unread.
 
+``(and (date 2009-11-18 2009-11-18) (tag unread))``
+
+    Match messages in the given date range with tag unread.
+
+``(and (date 2009-11-18 *) (tag unread))``
+
+    Match messages from 2009-11-18 or later with tag unread.
+
+``(and (date * 2009-11-18) (tag unread))``
+
+    Match messages from 2009-11-18 or earlier with tag unread.
+    
 ``(starts-with prelim)``
 
     Match any words starting with "prelim".
-- 
2.35.2

Re: [PATCH 1/4] test/sexp: one sided range searches.

[David Bremner]

David Bremner <david@tethera.net> writes:

> The date range parsing machinery already knows how to do something
> appropriate with an empty string, but the lastmod parsing blindly
> tries to parse each atom as a number.

Series applied to master.

d