* sexp and strings
@ 2022-06-12 23:11 erik colson
2022-06-13 9:12 ` Michael J Gruber
2022-06-13 18:39 ` David Bremner
0 siblings, 2 replies; 9+ messages in thread
From: erik colson @ 2022-06-12 23:11 UTC (permalink / raw)
To: notmuch
Hi,
I would like to define a squery in my notmuch configuration which would
ease a query I often use. The query is
tag:/ddddd/
where ddddd are decimal numbers.
Now I would like to shorten this to
D ddddd
wherefor I was thinking of using a macro like:
D=(macro (dossier) ((tag (regex ,dossier))))
But this doesn't seem to do the job.
Any ideas how I can achieve this ?
thx
--
erik colson
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings
2022-06-12 23:11 sexp and strings erik colson
@ 2022-06-13 9:12 ` Michael J Gruber
2022-06-13 9:31 ` erik colson
2022-06-13 18:39 ` David Bremner
1 sibling, 1 reply; 9+ messages in thread
From: Michael J Gruber @ 2022-06-13 9:12 UTC (permalink / raw)
To: erik colson; +Cc: notmuch
Am Mo., 13. Juni 2022 um 01:20 Uhr schrieb erik colson <eco@ecocode.net>:
>
> Hi,
>
> I would like to define a squery in my notmuch configuration which would
> ease a query I often use. The query is
>
> tag:/ddddd/
>
> where ddddd are decimal numbers.
> Now I would like to shorten this to
>
> D ddddd
>
> wherefor I was thinking of using a macro like:
>
> D=(macro (dossier) ((tag (regex ,dossier))))
>
> But this doesn't seem to do the job.
> Any ideas how I can achieve this ?
That search works without the macro, but not as a macro: notmuch
computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`).
I'm not sure whether this is intended or an artefact of the
implementation, since both the macro and the regex need
expansion/evaluation before being fed to xapian, and the order
matters.
Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with
`(D (rx ddddd))` works, btw (but is not what you want, obviously), so
something tells me lazy evaluation of macros is not completely lazy ;)
Michael
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings
2022-06-13 9:12 ` Michael J Gruber
@ 2022-06-13 9:31 ` erik colson
2022-06-13 10:11 ` Michael J Gruber
0 siblings, 1 reply; 9+ messages in thread
From: erik colson @ 2022-06-13 9:31 UTC (permalink / raw)
To: Michael J Gruber; +Cc: notmuch
Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes:
> That search works without the macro, but not as a macro: notmuch
> computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`).
> I'm not sure whether this is intended or an artefact of the
> implementation, since both the macro and the regex need
> expansion/evaluation before being fed to xapian, and the order
> matters.
>
> Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with
> `(D (rx ddddd))` works, btw (but is not what you want, obviously), so
> something tells me lazy evaluation of macros is not completely lazy ;)
Thanks for checking this out Michael. I decided to write an emacs lisp
function which I keybind and which prompts for the variable, and then
launches notmuch-search with that:
(defun ec/notmuch-search-dossier ()
"Zoek mails van een dossier"
(interactive
(let* ((dossier (read-no-blanks-input "Dossier:"))
(zoek (concat "tag:/" dossier "/")))
(notmuch-search zoek nil nil nil nil))))
This works like a charm ;) Well, I am of course open to enhancement
suggestions!
Also I upgraded my OS to fedora36 and now I use the standard notmuch
package instead of compiling it myself. So I don't have sexp support
anymore and therefor I am moving the squeries I added to my notmuch
config into emacs lisp functions.
best
--
erik colson
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings
2022-06-13 9:31 ` erik colson
@ 2022-06-13 10:11 ` Michael J Gruber
2022-06-13 15:10 ` erik colson
0 siblings, 1 reply; 9+ messages in thread
From: Michael J Gruber @ 2022-06-13 10:11 UTC (permalink / raw)
To: erik colson; +Cc: notmuch
Am Mo., 13. Juni 2022 um 11:31 Uhr schrieb erik colson <eco@ecocode.net>:
>
> Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes:
>
> > That search works without the macro, but not as a macro: notmuch
> > computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`).
> > I'm not sure whether this is intended or an artefact of the
> > implementation, since both the macro and the regex need
> > expansion/evaluation before being fed to xapian, and the order
> > matters.
> >
> > Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with
> > `(D (rx ddddd))` works, btw (but is not what you want, obviously), so
> > something tells me lazy evaluation of macros is not completely lazy ;)
>
> Thanks for checking this out Michael. I decided to write an emacs lisp
> function which I keybind and which prompts for the variable, and then
> launches notmuch-search with that:
>
> (defun ec/notmuch-search-dossier ()
> "Zoek mails van een dossier"
> (interactive
> (let* ((dossier (read-no-blanks-input "Dossier:"))
> (zoek (concat "tag:/" dossier "/")))
> (notmuch-search zoek nil nil nil nil))))
>
> This works like a charm ;) Well, I am of course open to enhancement
> suggestions!
>
> Also I upgraded my OS to fedora36 and now I use the standard notmuch
> package instead of compiling it myself. So I don't have sexp support
> anymore and therefor I am moving the squeries I added to my notmuch
> config into emacs lisp functions.
FYI: I submitted sfsexp to fedora (review pending), and as soon as
that is in, I will adjust the notmuch package.
(Until then there is copr mjg/notmuch-sfsexp at your own risk ;))
Cheers
Michael
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings
2022-06-13 10:11 ` Michael J Gruber
@ 2022-06-13 15:10 ` erik colson
0 siblings, 0 replies; 9+ messages in thread
From: erik colson @ 2022-06-13 15:10 UTC (permalink / raw)
To: Michael J Gruber; +Cc: notmuch
Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes:
> (Until then there is copr mjg/notmuch-sfsexp at your own risk ;))
how could I possibly overlook that repo !
thanks a lot !!!
--
erik colson
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings
2022-06-12 23:11 sexp and strings erik colson
2022-06-13 9:12 ` Michael J Gruber
@ 2022-06-13 18:39 ` David Bremner
2022-06-14 11:22 ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner
2022-07-01 11:41 ` sexp and strings David Bremner
1 sibling, 2 replies; 9+ messages in thread
From: David Bremner @ 2022-06-13 18:39 UTC (permalink / raw)
To: erik colson, notmuch
erik colson <eco@ecocode.net> writes:
> Hi,
>
> I would like to define a squery in my notmuch configuration which would
> ease a query I often use. The query is
>
> tag:/ddddd/
>
> where ddddd are decimal numbers.
> Now I would like to shorten this to
>
> D ddddd
>
> wherefor I was thinking of using a macro like:
>
> D=(macro (dossier) ((tag (regex ,dossier))))
>
> But this doesn't seem to do the job.
> Any ideas how I can achieve this ?
>
Although you have one too many sets of (), there is still a bug here.
$ notmuch config set squery.D '(macro (dossier) (tag (regex ,dossier))'
also doesn't work. Apparently both for regex and wildcard expansion I
did not do macro parameter expansion. In the regex case the regex
matching for tags actually happens at query construction time, so that's
why it ends up as an empty query (presumably none of your tags starts
with ,dossier).
Anyway, I think I know how to fix this, it will mean that in the corner
case of a regex starting with a comma, we'll have to use (regex ",foo").
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard
2022-06-13 18:39 ` David Bremner
@ 2022-06-14 11:22 ` David Bremner
2022-06-14 11:22 ` [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard David Bremner
2022-07-01 11:41 ` sexp and strings David Bremner
1 sibling, 1 reply; 9+ messages in thread
From: David Bremner @ 2022-06-14 11:22 UTC (permalink / raw)
To: David Bremner, erik colson, notmuch
These tests replicate the problem reported by Eric Colson [1] (for the
regex case).
[1]: id:87o7yxqxy6.fsf@code.pm
---
test/T081-sexpr-search.sh | 50 +++++++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh
index da819190..d28e5b76 100755
--- a/test/T081-sexpr-search.sh
+++ b/test/T081-sexpr-search.sh
@@ -1115,6 +1115,26 @@ too many arguments to macro
EOF
test_expect_equal_file EXPECTED OUTPUT
+test_begin_subtest "Saved Search: bad parameter syntax 5"
+test_subtest_known_broken
+notmuch config set squery.Bad5 '(macro (thing) (tag (rx ,thing)))'
+notmuch search --query=sexp '(Bad5 (1 2))' >OUTPUT 2>&1
+cat <<EOF > EXPECTED
+notmuch search: Syntax error in query
+'rx' expects single atom as argument
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "Saved Search: bad parameter syntax 6"
+test_subtest_known_broken
+notmuch config set squery.Bad6 '(macro (thing) (tag (starts-with ,thing)))'
+notmuch search --query=sexp '(Bad6 (1 2))' >OUTPUT 2>&1
+cat <<EOF > EXPECTED
+notmuch search: Syntax error in query
+'starts-with' expects single atom as argument
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
test_begin_subtest "Saved Search: macro without body"
notmuch config set squery.Bad3 '(macro (a b))'
notmuch search --query=sexp '(Bad3)' >OUTPUT 2>&1
@@ -1166,6 +1186,20 @@ notmuch config set squery.TagSubject2 '(macro (tagname subj) (and (tag ,tagname
notmuch search --query=sexp '(TagSubject2 inbox maildir)' | notmuch_search_sanitize > OUTPUT
test_expect_equal_file EXPECTED OUTPUT
+test_begin_subtest "macro in regex"
+test_subtest_known_broken
+notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED
+notmuch config set squery.D '(macro (tagname) (and (date 2009-11-17) (tag (rx ,tagname))))'
+notmuch search --query=sexp '(D inbo)' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
+test_begin_subtest "macro in wildcard"
+test_subtest_known_broken
+notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED
+notmuch config set squery.W '(macro (tagname) (and (date 2009-11-17) (tag (starts-with ,tagname))))'
+notmuch search --query=sexp '(W inbo)' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
test_begin_subtest "nested macros (shadowing)"
notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED
notmuch config set squery.Inner '(macro (x) (subject ,x))'
@@ -1183,6 +1217,22 @@ undefined parameter y
EOF
test_expect_equal_file EXPECTED OUTPUT
+test_begin_subtest "nested macros (shadowing, regex)"
+test_subtest_known_broken
+notmuch search tag:/inbo/ and subject:/Maildi/ | notmuch_search_sanitize > EXPECTED
+notmuch config set squery.Inner3 '(macro (x) (subject (rx ,x)))'
+notmuch config set squery.Outer3 '(macro (x y) (and (tag (rx ,x)) (Inner3 ,y)))'
+notmuch search --query=sexp '(Outer3 inbo Maildi)' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
+test_begin_subtest "nested macros (shadowing, wildcard)"
+test_subtest_known_broken
+notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED
+notmuch config set squery.Inner4 '(macro (x) (subject (starts-with ,x)))'
+notmuch config set squery.Outer4 '(macro (x y) (and (tag (starts-with ,x)) (Inner4 ,y)))'
+notmuch search --query=sexp '(Outer4 inbo maildi)' | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file_nonempty EXPECTED OUTPUT
+
test_begin_subtest "combine macro and user defined header"
notmuch config set squery.About '(macro (name) (or (subject ,name) (List ,name)))'
notmuch search subject:notmuch or List:notmuch | notmuch_search_sanitize > EXPECTED
--
2.35.2
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard
2022-06-14 11:22 ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner
@ 2022-06-14 11:22 ` David Bremner
0 siblings, 0 replies; 9+ messages in thread
From: David Bremner @ 2022-06-14 11:22 UTC (permalink / raw)
To: David Bremner, erik colson, notmuch
Fix the bug reported at [1].
The parameter expansion for regex and wildcard modifiers has to be
done a bit differently, because their arguments are not s-expressions
defining complete Xapian queries.
[1]: id:87o7yxqxy6.fsf@code.pm
---
lib/parse-sexp.cc | 55 ++++++++++++++++++++++++++++++++++-----
test/T081-sexpr-search.sh | 6 -----
2 files changed, 49 insertions(+), 12 deletions(-)
diff --git a/lib/parse-sexp.cc b/lib/parse-sexp.cc
index 08fd7037..d0c9f145 100644
--- a/lib/parse-sexp.cc
+++ b/lib/parse-sexp.cc
@@ -187,6 +187,37 @@ _sexp_parse_phrase (std::string term_prefix, const char *phrase, Xapian::Query &
return NOTMUCH_STATUS_SUCCESS;
}
+notmuch_status_t
+_sexp_expand_term (notmuch_database_t *notmuch,
+ const _sexp_prefix_t *prefix,
+ const _sexp_binding_t *env,
+ const sexp_t *sx,
+ const char **out)
+{
+ if (! out)
+ return NOTMUCH_STATUS_NULL_POINTER;
+
+ while (sx->ty == SEXP_VALUE && sx->aty == SEXP_BASIC && sx->val[0] == ',') {
+ const char *name = sx->val + 1;
+ for (; env; env = env->next) {
+ if (strcmp (name, env->name) == 0) {
+ sx = env->sx;
+ env = env->context;
+ break;
+ }
+ }
+ }
+ if (sx->ty != SEXP_VALUE) {
+ _notmuch_database_log (notmuch, "'%s' expects single atom as argument\n",
+ prefix->name);
+ return NOTMUCH_STATUS_BAD_QUERY_SYNTAX;
+ }
+
+ *out = sx->val;
+
+ return NOTMUCH_STATUS_SUCCESS;
+}
+
static notmuch_status_t
_sexp_parse_wildcard (notmuch_database_t *notmuch,
const _sexp_prefix_t *parent,
@@ -227,8 +258,8 @@ _sexp_parse_one_term (notmuch_database_t *notmuch, std::string term_prefix, cons
notmuch_status_t
_sexp_parse_regex (notmuch_database_t *notmuch,
const _sexp_prefix_t *prefix, const _sexp_prefix_t *parent,
- unused(const _sexp_binding_t *env),
- std::string val, Xapian::Query &output)
+ const _sexp_binding_t *env,
+ const sexp_t *term, Xapian::Query &output)
{
if (! parent) {
_notmuch_database_log (notmuch, "illegal '%s' outside field\n",
@@ -243,9 +274,15 @@ _sexp_parse_regex (notmuch_database_t *notmuch,
}
std::string msg; /* ignored */
+ const char *str;
+ notmuch_status_t status;
+
+ status = _sexp_expand_term (notmuch, prefix, env, term, &str);
+ if (status)
+ return status;
return _notmuch_regexp_to_query (notmuch, Xapian::BAD_VALUENO, parent->name,
- val, output, msg);
+ str, output, msg);
}
@@ -638,11 +675,17 @@ _sexp_to_xapian_query (notmuch_database_t *notmuch, const _sexp_prefix_t *parent
return _notmuch_query_name_to_query (notmuch, sx->list->next->val, output);
}
- if (prefix->xapian_op == Xapian::Query::OP_WILDCARD)
- return _sexp_parse_wildcard (notmuch, parent, env, sx->list->next->val, output);
+ if (prefix->xapian_op == Xapian::Query::OP_WILDCARD) {
+ const char *str;
+ status = _sexp_expand_term (notmuch, prefix, env, sx->list->next, &str);
+ if (status)
+ return status;
+
+ return _sexp_parse_wildcard (notmuch, parent, env, str, output);
+ }
if (prefix->flags & SEXP_FLAG_DO_REGEX) {
- return _sexp_parse_regex (notmuch, prefix, parent, env, sx->list->next->val, output);
+ return _sexp_parse_regex (notmuch, prefix, parent, env, sx->list->next, output);
}
if (prefix->flags & SEXP_FLAG_DO_EXPAND) {
diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh
index d28e5b76..de9cbe7b 100755
--- a/test/T081-sexpr-search.sh
+++ b/test/T081-sexpr-search.sh
@@ -1116,7 +1116,6 @@ EOF
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "Saved Search: bad parameter syntax 5"
-test_subtest_known_broken
notmuch config set squery.Bad5 '(macro (thing) (tag (rx ,thing)))'
notmuch search --query=sexp '(Bad5 (1 2))' >OUTPUT 2>&1
cat <<EOF > EXPECTED
@@ -1126,7 +1125,6 @@ EOF
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "Saved Search: bad parameter syntax 6"
-test_subtest_known_broken
notmuch config set squery.Bad6 '(macro (thing) (tag (starts-with ,thing)))'
notmuch search --query=sexp '(Bad6 (1 2))' >OUTPUT 2>&1
cat <<EOF > EXPECTED
@@ -1187,14 +1185,12 @@ notmuch search --query=sexp '(TagSubject2 inbox maildir)' | notmuch_search_sanit
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "macro in regex"
-test_subtest_known_broken
notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED
notmuch config set squery.D '(macro (tagname) (and (date 2009-11-17) (tag (rx ,tagname))))'
notmuch search --query=sexp '(D inbo)' | notmuch_search_sanitize > OUTPUT
test_expect_equal_file_nonempty EXPECTED OUTPUT
test_begin_subtest "macro in wildcard"
-test_subtest_known_broken
notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED
notmuch config set squery.W '(macro (tagname) (and (date 2009-11-17) (tag (starts-with ,tagname))))'
notmuch search --query=sexp '(W inbo)' | notmuch_search_sanitize > OUTPUT
@@ -1218,7 +1214,6 @@ EOF
test_expect_equal_file EXPECTED OUTPUT
test_begin_subtest "nested macros (shadowing, regex)"
-test_subtest_known_broken
notmuch search tag:/inbo/ and subject:/Maildi/ | notmuch_search_sanitize > EXPECTED
notmuch config set squery.Inner3 '(macro (x) (subject (rx ,x)))'
notmuch config set squery.Outer3 '(macro (x y) (and (tag (rx ,x)) (Inner3 ,y)))'
@@ -1226,7 +1221,6 @@ notmuch search --query=sexp '(Outer3 inbo Maildi)' | notmuch_search_sanitize > O
test_expect_equal_file_nonempty EXPECTED OUTPUT
test_begin_subtest "nested macros (shadowing, wildcard)"
-test_subtest_known_broken
notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED
notmuch config set squery.Inner4 '(macro (x) (subject (starts-with ,x)))'
notmuch config set squery.Outer4 '(macro (x y) (and (tag (starts-with ,x)) (Inner4 ,y)))'
--
2.35.2
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: sexp and strings
2022-06-13 18:39 ` David Bremner
2022-06-14 11:22 ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner
@ 2022-07-01 11:41 ` David Bremner
1 sibling, 0 replies; 9+ messages in thread
From: David Bremner @ 2022-07-01 11:41 UTC (permalink / raw)
To: erik colson, notmuch
David Bremner <david@tethera.net> writes:
>
> Although you have one too many sets of (), there is still a bug here.
>
> $ notmuch config set squery.D '(macro (dossier) (tag (regex ,dossier))'
>
> also doesn't work. Apparently both for regex and wildcard expansion I
> did not do macro parameter expansion. In the regex case the regex
> matching for tags actually happens at query construction time, so that's
> why it ends up as an empty query (presumably none of your tags starts
> with ,dossier).
>
> Anyway, I think I know how to fix this, it will mean that in the corner
> case of a regex starting with a comma, we'll have to use (regex ",foo").
This bug should be fixed in master as of
6a9ae990990848ec99107f2e2e6b6ae12dcd33df
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2022-07-01 11:49 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-12 23:11 sexp and strings erik colson
2022-06-13 9:12 ` Michael J Gruber
2022-06-13 9:31 ` erik colson
2022-06-13 10:11 ` Michael J Gruber
2022-06-13 15:10 ` erik colson
2022-06-13 18:39 ` David Bremner
2022-06-14 11:22 ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner
2022-06-14 11:22 ` [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard David Bremner
2022-07-01 11:41 ` sexp and strings David Bremner
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).