unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* sexp and strings
@ 2022-06-12 23:11 erik colson
  2022-06-13  9:12 ` Michael J Gruber
  2022-06-13 18:39 ` David Bremner
  0 siblings, 2 replies; 9+ messages in thread
From: erik colson @ 2022-06-12 23:11 UTC (permalink / raw)
  To: notmuch

Hi,

I would like to define a squery in my notmuch configuration which would
ease a query I often use.  The query is

  tag:/ddddd/

where ddddd are decimal numbers.
Now I would like to shorten this to

  D ddddd

wherefor I was thinking of using a macro like:

  D=(macro (dossier) ((tag (regex ,dossier))))

But this doesn't seem to do the job.
Any ideas how I can achieve this ?

thx
-- 
erik colson

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: sexp and strings
  2022-06-12 23:11 sexp and strings erik colson
@ 2022-06-13  9:12 ` Michael J Gruber
  2022-06-13  9:31   ` erik colson
  2022-06-13 18:39 ` David Bremner
  1 sibling, 1 reply; 9+ messages in thread
From: Michael J Gruber @ 2022-06-13  9:12 UTC (permalink / raw)
  To: erik colson; +Cc: notmuch

Am Mo., 13. Juni 2022 um 01:20 Uhr schrieb erik colson <eco@ecocode.net>:
>
> Hi,
>
> I would like to define a squery in my notmuch configuration which would
> ease a query I often use.  The query is
>
>   tag:/ddddd/
>
> where ddddd are decimal numbers.
> Now I would like to shorten this to
>
>   D ddddd
>
> wherefor I was thinking of using a macro like:
>
>   D=(macro (dossier) ((tag (regex ,dossier))))
>
> But this doesn't seem to do the job.
> Any ideas how I can achieve this ?

That search works without the macro, but not as a macro: notmuch
computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`).
I'm not sure whether this is intended or an artefact of the
implementation, since both the macro and the regex need
expansion/evaluation before being fed to xapian, and the order
matters.

Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with
`(D (rx ddddd))` works, btw (but is not what you want, obviously), so
something tells me lazy evaluation of macros is not completely lazy ;)

Michael

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: sexp and strings
  2022-06-13  9:12 ` Michael J Gruber
@ 2022-06-13  9:31   ` erik colson
  2022-06-13 10:11     ` Michael J Gruber
  0 siblings, 1 reply; 9+ messages in thread
From: erik colson @ 2022-06-13  9:31 UTC (permalink / raw)
  To: Michael J Gruber; +Cc: notmuch

Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes:

> That search works without the macro, but not as a macro: notmuch
> computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`).
> I'm not sure whether this is intended or an artefact of the
> implementation, since both the macro and the regex need
> expansion/evaluation before being fed to xapian, and the order
> matters.
>
> Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with
> `(D (rx ddddd))` works, btw (but is not what you want, obviously), so
> something tells me lazy evaluation of macros is not completely lazy ;)

Thanks for checking this out Michael.  I decided to write an emacs lisp
function which I keybind and which prompts for the variable, and then
launches notmuch-search with that:

(defun ec/notmuch-search-dossier ()
  "Zoek mails van een dossier"
  (interactive
   (let* ((dossier (read-no-blanks-input "Dossier:"))
          (zoek (concat "tag:/" dossier "/")))
          (notmuch-search zoek nil nil nil nil))))

This works like a charm ;) Well, I am of course open to enhancement
suggestions!

Also I upgraded my OS to fedora36 and now I use the standard notmuch
package instead of compiling it myself.  So I don't have sexp support
anymore and therefor I am moving the squeries I added to my notmuch
config into emacs lisp functions.

best
-- 
erik colson

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: sexp and strings
  2022-06-13  9:31   ` erik colson
@ 2022-06-13 10:11     ` Michael J Gruber
  2022-06-13 15:10       ` erik colson
  0 siblings, 1 reply; 9+ messages in thread
From: Michael J Gruber @ 2022-06-13 10:11 UTC (permalink / raw)
  To: erik colson; +Cc: notmuch

Am Mo., 13. Juni 2022 um 11:31 Uhr schrieb erik colson <eco@ecocode.net>:
>
> Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes:
>
> > That search works without the macro, but not as a macro: notmuch
> > computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`).
> > I'm not sure whether this is intended or an artefact of the
> > implementation, since both the macro and the regex need
> > expansion/evaluation before being fed to xapian, and the order
> > matters.
> >
> > Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with
> > `(D (rx ddddd))` works, btw (but is not what you want, obviously), so
> > something tells me lazy evaluation of macros is not completely lazy ;)
>
> Thanks for checking this out Michael.  I decided to write an emacs lisp
> function which I keybind and which prompts for the variable, and then
> launches notmuch-search with that:
>
> (defun ec/notmuch-search-dossier ()
>   "Zoek mails van een dossier"
>   (interactive
>    (let* ((dossier (read-no-blanks-input "Dossier:"))
>           (zoek (concat "tag:/" dossier "/")))
>           (notmuch-search zoek nil nil nil nil))))
>
> This works like a charm ;) Well, I am of course open to enhancement
> suggestions!
>
> Also I upgraded my OS to fedora36 and now I use the standard notmuch
> package instead of compiling it myself.  So I don't have sexp support
> anymore and therefor I am moving the squeries I added to my notmuch
> config into emacs lisp functions.

FYI: I submitted sfsexp to fedora (review pending), and as soon as
that is in, I will adjust the notmuch package.
(Until then there is copr mjg/notmuch-sfsexp at your own risk ;))

Cheers
Michael

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: sexp and strings
  2022-06-13 10:11     ` Michael J Gruber
@ 2022-06-13 15:10       ` erik colson
  0 siblings, 0 replies; 9+ messages in thread
From: erik colson @ 2022-06-13 15:10 UTC (permalink / raw)
  To: Michael J Gruber; +Cc: notmuch

Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes:

> (Until then there is copr mjg/notmuch-sfsexp at your own risk ;))

how could I possibly overlook that repo !
thanks a lot !!!
-- 
erik colson

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: sexp and strings
  2022-06-12 23:11 sexp and strings erik colson
  2022-06-13  9:12 ` Michael J Gruber
@ 2022-06-13 18:39 ` David Bremner
  2022-06-14 11:22   ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner
  2022-07-01 11:41   ` sexp and strings David Bremner
  1 sibling, 2 replies; 9+ messages in thread
From: David Bremner @ 2022-06-13 18:39 UTC (permalink / raw)
  To: erik colson, notmuch

erik colson <eco@ecocode.net> writes:

> Hi,
>
> I would like to define a squery in my notmuch configuration which would
> ease a query I often use.  The query is
>
>   tag:/ddddd/
>
> where ddddd are decimal numbers.
> Now I would like to shorten this to
>
>   D ddddd
>
> wherefor I was thinking of using a macro like:
>
>   D=(macro (dossier) ((tag (regex ,dossier))))
>
> But this doesn't seem to do the job.
> Any ideas how I can achieve this ?
>

Although you have one too many sets of (), there is still a bug here.

         $ notmuch config set squery.D '(macro (dossier) (tag (regex ,dossier))'

also doesn't work.  Apparently both for regex and wildcard expansion I
did not do macro parameter expansion. In the regex case the regex
matching for tags actually happens at query construction time, so that's
why it ends up as an empty query (presumably none of your tags starts
with ,dossier).

Anyway, I think I know how to fix this, it will mean that in the corner
case of a regex starting with a comma, we'll have to use (regex ",foo").

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard
  2022-06-13 18:39 ` David Bremner
@ 2022-06-14 11:22   ` David Bremner
  2022-06-14 11:22     ` [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard David Bremner
  2022-07-01 11:41   ` sexp and strings David Bremner
  1 sibling, 1 reply; 9+ messages in thread
From: David Bremner @ 2022-06-14 11:22 UTC (permalink / raw)
  To: David Bremner, erik colson, notmuch

These tests replicate the problem reported by Eric Colson [1] (for the
regex case).

[1]: id:87o7yxqxy6.fsf@code.pm
---
 test/T081-sexpr-search.sh | 50 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh
index da819190..d28e5b76 100755
--- a/test/T081-sexpr-search.sh
+++ b/test/T081-sexpr-search.sh
@@ -1115,6 +1115,26 @@ too many arguments to macro
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Saved Search: bad parameter syntax 5"
+test_subtest_known_broken
+notmuch config set squery.Bad5 '(macro (thing) (tag (rx ,thing)))'
+notmuch search --query=sexp '(Bad5 (1 2))' >OUTPUT 2>&1
+cat <<EOF > EXPECTED
+notmuch search: Syntax error in query
+'rx' expects single atom as argument
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "Saved Search: bad parameter syntax 6"
+test_subtest_known_broken
+notmuch config set squery.Bad6 '(macro (thing) (tag (starts-with ,thing)))'
+notmuch search --query=sexp '(Bad6 (1 2))' >OUTPUT 2>&1
+cat <<EOF > EXPECTED
+notmuch search: Syntax error in query
+'starts-with' expects single atom as argument
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "Saved Search: macro without body"
 notmuch config set squery.Bad3  '(macro (a b))'
 notmuch search --query=sexp '(Bad3)' >OUTPUT 2>&1
@@ -1166,6 +1186,20 @@ notmuch config set squery.TagSubject2  '(macro (tagname subj) (and (tag ,tagname
 notmuch search --query=sexp '(TagSubject2 inbox maildir)' | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "macro in regex"
+test_subtest_known_broken
+notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED
+notmuch config set squery.D  '(macro (tagname)  (and (date 2009-11-17) (tag (rx ,tagname))))'
+notmuch search --query=sexp '(D inbo)' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
+test_begin_subtest "macro in wildcard"
+test_subtest_known_broken
+notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED
+notmuch config set squery.W  '(macro (tagname)  (and (date 2009-11-17) (tag (starts-with ,tagname))))'
+notmuch search --query=sexp '(W inbo)' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
 test_begin_subtest "nested macros (shadowing)"
 notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED
 notmuch config set squery.Inner '(macro (x) (subject ,x))'
@@ -1183,6 +1217,22 @@ undefined parameter y
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "nested macros (shadowing, regex)"
+test_subtest_known_broken
+notmuch search tag:/inbo/ and subject:/Maildi/ | notmuch_search_sanitize > EXPECTED
+notmuch config set squery.Inner3 '(macro (x) (subject (rx ,x)))'
+notmuch config set squery.Outer3  '(macro (x y) (and (tag (rx ,x)) (Inner3 ,y)))'
+notmuch search --query=sexp '(Outer3 inbo Maildi)' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
+test_begin_subtest "nested macros (shadowing, wildcard)"
+test_subtest_known_broken
+notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED
+notmuch config set squery.Inner4 '(macro (x) (subject (starts-with ,x)))'
+notmuch config set squery.Outer4  '(macro (x y) (and (tag (starts-with ,x)) (Inner4 ,y)))'
+notmuch search --query=sexp '(Outer4 inbo maildi)' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
 test_begin_subtest "combine macro and user defined header"
 notmuch config set squery.About '(macro (name) (or (subject ,name) (List ,name)))'
 notmuch search subject:notmuch or List:notmuch | notmuch_search_sanitize > EXPECTED
-- 
2.35.2

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard
  2022-06-14 11:22   ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner
@ 2022-06-14 11:22     ` David Bremner
  0 siblings, 0 replies; 9+ messages in thread
From: David Bremner @ 2022-06-14 11:22 UTC (permalink / raw)
  To: David Bremner, erik colson, notmuch

Fix the bug reported at [1].

The parameter expansion for regex and wildcard modifiers has to be
done a bit differently, because their arguments are not s-expressions
defining complete Xapian queries.

[1]: id:87o7yxqxy6.fsf@code.pm
---
 lib/parse-sexp.cc         | 55 ++++++++++++++++++++++++++++++++++-----
 test/T081-sexpr-search.sh |  6 -----
 2 files changed, 49 insertions(+), 12 deletions(-)

diff --git a/lib/parse-sexp.cc b/lib/parse-sexp.cc
index 08fd7037..d0c9f145 100644
--- a/lib/parse-sexp.cc
+++ b/lib/parse-sexp.cc
@@ -187,6 +187,37 @@ _sexp_parse_phrase (std::string term_prefix, const char *phrase, Xapian::Query &
     return NOTMUCH_STATUS_SUCCESS;
 }
 
+notmuch_status_t
+_sexp_expand_term (notmuch_database_t *notmuch,
+		   const _sexp_prefix_t *prefix,
+		   const _sexp_binding_t *env,
+		   const sexp_t *sx,
+		   const char **out)
+{
+    if (! out)
+	return NOTMUCH_STATUS_NULL_POINTER;
+
+    while (sx->ty == SEXP_VALUE && sx->aty == SEXP_BASIC && sx->val[0] == ',') {
+	const char *name = sx->val + 1;
+	for (; env; env = env->next) {
+	    if (strcmp (name, env->name) == 0) {
+		sx = env->sx;
+		env = env->context;
+		break;
+	    }
+	}
+    }
+    if (sx->ty != SEXP_VALUE) {
+	_notmuch_database_log (notmuch, "'%s' expects single atom as argument\n",
+			       prefix->name);
+	return NOTMUCH_STATUS_BAD_QUERY_SYNTAX;
+    }
+
+    *out = sx->val;
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 static notmuch_status_t
 _sexp_parse_wildcard (notmuch_database_t *notmuch,
 		      const _sexp_prefix_t *parent,
@@ -227,8 +258,8 @@ _sexp_parse_one_term (notmuch_database_t *notmuch, std::string term_prefix, cons
 notmuch_status_t
 _sexp_parse_regex (notmuch_database_t *notmuch,
 		   const _sexp_prefix_t *prefix, const _sexp_prefix_t *parent,
-		   unused(const _sexp_binding_t *env),
-		   std::string val, Xapian::Query &output)
+		   const _sexp_binding_t *env,
+		   const sexp_t *term, Xapian::Query &output)
 {
     if (! parent) {
 	_notmuch_database_log (notmuch, "illegal '%s' outside field\n",
@@ -243,9 +274,15 @@ _sexp_parse_regex (notmuch_database_t *notmuch,
     }
 
     std::string msg; /* ignored */
+    const char *str;
+    notmuch_status_t status;
+
+    status = _sexp_expand_term (notmuch, prefix, env, term, &str);
+    if (status)
+	return status;
 
     return _notmuch_regexp_to_query (notmuch, Xapian::BAD_VALUENO, parent->name,
-				     val, output, msg);
+				     str, output, msg);
 }
 
 
@@ -638,11 +675,17 @@ _sexp_to_xapian_query (notmuch_database_t *notmuch, const _sexp_prefix_t *parent
 		return _notmuch_query_name_to_query (notmuch, sx->list->next->val, output);
 	    }
 
-	    if (prefix->xapian_op == Xapian::Query::OP_WILDCARD)
-		return _sexp_parse_wildcard (notmuch, parent, env, sx->list->next->val, output);
+	    if (prefix->xapian_op == Xapian::Query::OP_WILDCARD) {
+		const char *str;
+		status = _sexp_expand_term (notmuch, prefix, env, sx->list->next, &str);
+		if (status)
+		    return status;
+
+		return _sexp_parse_wildcard (notmuch, parent, env, str, output);
+	    }
 
 	    if (prefix->flags & SEXP_FLAG_DO_REGEX) {
-		return _sexp_parse_regex (notmuch, prefix, parent, env, sx->list->next->val, output);
+		return _sexp_parse_regex (notmuch, prefix, parent, env, sx->list->next, output);
 	    }
 
 	    if (prefix->flags & SEXP_FLAG_DO_EXPAND) {
diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh
index d28e5b76..de9cbe7b 100755
--- a/test/T081-sexpr-search.sh
+++ b/test/T081-sexpr-search.sh
@@ -1116,7 +1116,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Saved Search: bad parameter syntax 5"
-test_subtest_known_broken
 notmuch config set squery.Bad5 '(macro (thing) (tag (rx ,thing)))'
 notmuch search --query=sexp '(Bad5 (1 2))' >OUTPUT 2>&1
 cat <<EOF > EXPECTED
@@ -1126,7 +1125,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Saved Search: bad parameter syntax 6"
-test_subtest_known_broken
 notmuch config set squery.Bad6 '(macro (thing) (tag (starts-with ,thing)))'
 notmuch search --query=sexp '(Bad6 (1 2))' >OUTPUT 2>&1
 cat <<EOF > EXPECTED
@@ -1187,14 +1185,12 @@ notmuch search --query=sexp '(TagSubject2 inbox maildir)' | notmuch_search_sanit
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "macro in regex"
-test_subtest_known_broken
 notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED
 notmuch config set squery.D  '(macro (tagname)  (and (date 2009-11-17) (tag (rx ,tagname))))'
 notmuch search --query=sexp '(D inbo)' | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file_nonempty EXPECTED OUTPUT
 
 test_begin_subtest "macro in wildcard"
-test_subtest_known_broken
 notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED
 notmuch config set squery.W  '(macro (tagname)  (and (date 2009-11-17) (tag (starts-with ,tagname))))'
 notmuch search --query=sexp '(W inbo)' | notmuch_search_sanitize > OUTPUT
@@ -1218,7 +1214,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "nested macros (shadowing, regex)"
-test_subtest_known_broken
 notmuch search tag:/inbo/ and subject:/Maildi/ | notmuch_search_sanitize > EXPECTED
 notmuch config set squery.Inner3 '(macro (x) (subject (rx ,x)))'
 notmuch config set squery.Outer3  '(macro (x y) (and (tag (rx ,x)) (Inner3 ,y)))'
@@ -1226,7 +1221,6 @@ notmuch search --query=sexp '(Outer3 inbo Maildi)' | notmuch_search_sanitize > O
 test_expect_equal_file_nonempty EXPECTED OUTPUT
 
 test_begin_subtest "nested macros (shadowing, wildcard)"
-test_subtest_known_broken
 notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED
 notmuch config set squery.Inner4 '(macro (x) (subject (starts-with ,x)))'
 notmuch config set squery.Outer4  '(macro (x y) (and (tag (starts-with ,x)) (Inner4 ,y)))'
-- 
2.35.2

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: sexp and strings
  2022-06-13 18:39 ` David Bremner
  2022-06-14 11:22   ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner
@ 2022-07-01 11:41   ` David Bremner
  1 sibling, 0 replies; 9+ messages in thread
From: David Bremner @ 2022-07-01 11:41 UTC (permalink / raw)
  To: erik colson, notmuch

David Bremner <david@tethera.net> writes:

>
> Although you have one too many sets of (), there is still a bug here.
>
>          $ notmuch config set squery.D '(macro (dossier) (tag (regex ,dossier))'
>
> also doesn't work.  Apparently both for regex and wildcard expansion I
> did not do macro parameter expansion. In the regex case the regex
> matching for tags actually happens at query construction time, so that's
> why it ends up as an empty query (presumably none of your tags starts
> with ,dossier).
>
> Anyway, I think I know how to fix this, it will mean that in the corner
> case of a regex starting with a comma, we'll have to use (regex ",foo").

This bug should be fixed in master as of

     6a9ae990990848ec99107f2e2e6b6ae12dcd33df


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-07-01 11:49 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-12 23:11 sexp and strings erik colson
2022-06-13  9:12 ` Michael J Gruber
2022-06-13  9:31   ` erik colson
2022-06-13 10:11     ` Michael J Gruber
2022-06-13 15:10       ` erik colson
2022-06-13 18:39 ` David Bremner
2022-06-14 11:22   ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner
2022-06-14 11:22     ` [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard David Bremner
2022-07-01 11:41   ` sexp and strings David Bremner

Code repositories for project(s) associated with this inbox:

	notmuch.git.git (no URL configured)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).