* sexp and strings @ 2022-06-12 23:11 erik colson 2022-06-13 9:12 ` Michael J Gruber 2022-06-13 18:39 ` David Bremner 0 siblings, 2 replies; 9+ messages in thread From: erik colson @ 2022-06-12 23:11 UTC (permalink / raw) To: notmuch Hi, I would like to define a squery in my notmuch configuration which would ease a query I often use. The query is tag:/ddddd/ where ddddd are decimal numbers. Now I would like to shorten this to D ddddd wherefor I was thinking of using a macro like: D=(macro (dossier) ((tag (regex ,dossier)))) But this doesn't seem to do the job. Any ideas how I can achieve this ? thx -- erik colson ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings 2022-06-12 23:11 sexp and strings erik colson @ 2022-06-13 9:12 ` Michael J Gruber 2022-06-13 9:31 ` erik colson 2022-06-13 18:39 ` David Bremner 1 sibling, 1 reply; 9+ messages in thread From: Michael J Gruber @ 2022-06-13 9:12 UTC (permalink / raw) To: erik colson; +Cc: notmuch Am Mo., 13. Juni 2022 um 01:20 Uhr schrieb erik colson <eco@ecocode.net>: > > Hi, > > I would like to define a squery in my notmuch configuration which would > ease a query I often use. The query is > > tag:/ddddd/ > > where ddddd are decimal numbers. > Now I would like to shorten this to > > D ddddd > > wherefor I was thinking of using a macro like: > > D=(macro (dossier) ((tag (regex ,dossier)))) > > But this doesn't seem to do the job. > Any ideas how I can achieve this ? That search works without the macro, but not as a macro: notmuch computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`). I'm not sure whether this is intended or an artefact of the implementation, since both the macro and the regex need expansion/evaluation before being fed to xapian, and the order matters. Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with `(D (rx ddddd))` works, btw (but is not what you want, obviously), so something tells me lazy evaluation of macros is not completely lazy ;) Michael ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings 2022-06-13 9:12 ` Michael J Gruber @ 2022-06-13 9:31 ` erik colson 2022-06-13 10:11 ` Michael J Gruber 0 siblings, 1 reply; 9+ messages in thread From: erik colson @ 2022-06-13 9:31 UTC (permalink / raw) To: Michael J Gruber; +Cc: notmuch Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes: > That search works without the macro, but not as a macro: notmuch > computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`). > I'm not sure whether this is intended or an artefact of the > implementation, since both the macro and the regex need > expansion/evaluation before being fed to xapian, and the order > matters. > > Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with > `(D (rx ddddd))` works, btw (but is not what you want, obviously), so > something tells me lazy evaluation of macros is not completely lazy ;) Thanks for checking this out Michael. I decided to write an emacs lisp function which I keybind and which prompts for the variable, and then launches notmuch-search with that: (defun ec/notmuch-search-dossier () "Zoek mails van een dossier" (interactive (let* ((dossier (read-no-blanks-input "Dossier:")) (zoek (concat "tag:/" dossier "/"))) (notmuch-search zoek nil nil nil nil)))) This works like a charm ;) Well, I am of course open to enhancement suggestions! Also I upgraded my OS to fedora36 and now I use the standard notmuch package instead of compiling it myself. So I don't have sexp support anymore and therefor I am moving the squeries I added to my notmuch config into emacs lisp functions. best -- erik colson ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings 2022-06-13 9:31 ` erik colson @ 2022-06-13 10:11 ` Michael J Gruber 2022-06-13 15:10 ` erik colson 0 siblings, 1 reply; 9+ messages in thread From: Michael J Gruber @ 2022-06-13 10:11 UTC (permalink / raw) To: erik colson; +Cc: notmuch Am Mo., 13. Juni 2022 um 11:31 Uhr schrieb erik colson <eco@ecocode.net>: > > Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes: > > > That search works without the macro, but not as a macro: notmuch > > computes an empty `Query()` for this (as per `NOTMUCH_DEBUG_QUERY=1`). > > I'm not sure whether this is intended or an artefact of the > > implementation, since both the macro and the regex need > > expansion/evaluation before being fed to xapian, and the order > > matters. > > > > Defining `D=(macro (dossier) ((tag ,dossier)))` and calling it with > > `(D (rx ddddd))` works, btw (but is not what you want, obviously), so > > something tells me lazy evaluation of macros is not completely lazy ;) > > Thanks for checking this out Michael. I decided to write an emacs lisp > function which I keybind and which prompts for the variable, and then > launches notmuch-search with that: > > (defun ec/notmuch-search-dossier () > "Zoek mails van een dossier" > (interactive > (let* ((dossier (read-no-blanks-input "Dossier:")) > (zoek (concat "tag:/" dossier "/"))) > (notmuch-search zoek nil nil nil nil)))) > > This works like a charm ;) Well, I am of course open to enhancement > suggestions! > > Also I upgraded my OS to fedora36 and now I use the standard notmuch > package instead of compiling it myself. So I don't have sexp support > anymore and therefor I am moving the squeries I added to my notmuch > config into emacs lisp functions. FYI: I submitted sfsexp to fedora (review pending), and as soon as that is in, I will adjust the notmuch package. (Until then there is copr mjg/notmuch-sfsexp at your own risk ;)) Cheers Michael ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings 2022-06-13 10:11 ` Michael J Gruber @ 2022-06-13 15:10 ` erik colson 0 siblings, 0 replies; 9+ messages in thread From: erik colson @ 2022-06-13 15:10 UTC (permalink / raw) To: Michael J Gruber; +Cc: notmuch Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes: > (Until then there is copr mjg/notmuch-sfsexp at your own risk ;)) how could I possibly overlook that repo ! thanks a lot !!! -- erik colson ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: sexp and strings 2022-06-12 23:11 sexp and strings erik colson 2022-06-13 9:12 ` Michael J Gruber @ 2022-06-13 18:39 ` David Bremner 2022-06-14 11:22 ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner 2022-07-01 11:41 ` sexp and strings David Bremner 1 sibling, 2 replies; 9+ messages in thread From: David Bremner @ 2022-06-13 18:39 UTC (permalink / raw) To: erik colson, notmuch erik colson <eco@ecocode.net> writes: > Hi, > > I would like to define a squery in my notmuch configuration which would > ease a query I often use. The query is > > tag:/ddddd/ > > where ddddd are decimal numbers. > Now I would like to shorten this to > > D ddddd > > wherefor I was thinking of using a macro like: > > D=(macro (dossier) ((tag (regex ,dossier)))) > > But this doesn't seem to do the job. > Any ideas how I can achieve this ? > Although you have one too many sets of (), there is still a bug here. $ notmuch config set squery.D '(macro (dossier) (tag (regex ,dossier))' also doesn't work. Apparently both for regex and wildcard expansion I did not do macro parameter expansion. In the regex case the regex matching for tags actually happens at query construction time, so that's why it ends up as an empty query (presumably none of your tags starts with ,dossier). Anyway, I think I know how to fix this, it will mean that in the corner case of a regex starting with a comma, we'll have to use (regex ",foo"). ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard 2022-06-13 18:39 ` David Bremner @ 2022-06-14 11:22 ` David Bremner 2022-06-14 11:22 ` [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard David Bremner 2022-07-01 11:41 ` sexp and strings David Bremner 1 sibling, 1 reply; 9+ messages in thread From: David Bremner @ 2022-06-14 11:22 UTC (permalink / raw) To: David Bremner, erik colson, notmuch These tests replicate the problem reported by Eric Colson [1] (for the regex case). [1]: id:87o7yxqxy6.fsf@code.pm --- test/T081-sexpr-search.sh | 50 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh index da819190..d28e5b76 100755 --- a/test/T081-sexpr-search.sh +++ b/test/T081-sexpr-search.sh @@ -1115,6 +1115,26 @@ too many arguments to macro EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "Saved Search: bad parameter syntax 5" +test_subtest_known_broken +notmuch config set squery.Bad5 '(macro (thing) (tag (rx ,thing)))' +notmuch search --query=sexp '(Bad5 (1 2))' >OUTPUT 2>&1 +cat <<EOF > EXPECTED +notmuch search: Syntax error in query +'rx' expects single atom as argument +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: bad parameter syntax 6" +test_subtest_known_broken +notmuch config set squery.Bad6 '(macro (thing) (tag (starts-with ,thing)))' +notmuch search --query=sexp '(Bad6 (1 2))' >OUTPUT 2>&1 +cat <<EOF > EXPECTED +notmuch search: Syntax error in query +'starts-with' expects single atom as argument +EOF +test_expect_equal_file EXPECTED OUTPUT + test_begin_subtest "Saved Search: macro without body" notmuch config set squery.Bad3 '(macro (a b))' notmuch search --query=sexp '(Bad3)' >OUTPUT 2>&1 @@ -1166,6 +1186,20 @@ notmuch config set squery.TagSubject2 '(macro (tagname subj) (and (tag ,tagname notmuch search --query=sexp '(TagSubject2 inbox maildir)' | notmuch_search_sanitize > OUTPUT test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "macro in regex" +test_subtest_known_broken +notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED +notmuch config set squery.D '(macro (tagname) (and (date 2009-11-17) (tag (rx ,tagname))))' +notmuch search --query=sexp '(D inbo)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "macro in wildcard" +test_subtest_known_broken +notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED +notmuch config set squery.W '(macro (tagname) (and (date 2009-11-17) (tag (starts-with ,tagname))))' +notmuch search --query=sexp '(W inbo)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + test_begin_subtest "nested macros (shadowing)" notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED notmuch config set squery.Inner '(macro (x) (subject ,x))' @@ -1183,6 +1217,22 @@ undefined parameter y EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "nested macros (shadowing, regex)" +test_subtest_known_broken +notmuch search tag:/inbo/ and subject:/Maildi/ | notmuch_search_sanitize > EXPECTED +notmuch config set squery.Inner3 '(macro (x) (subject (rx ,x)))' +notmuch config set squery.Outer3 '(macro (x y) (and (tag (rx ,x)) (Inner3 ,y)))' +notmuch search --query=sexp '(Outer3 inbo Maildi)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "nested macros (shadowing, wildcard)" +test_subtest_known_broken +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.Inner4 '(macro (x) (subject (starts-with ,x)))' +notmuch config set squery.Outer4 '(macro (x y) (and (tag (starts-with ,x)) (Inner4 ,y)))' +notmuch search --query=sexp '(Outer4 inbo maildi)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + test_begin_subtest "combine macro and user defined header" notmuch config set squery.About '(macro (name) (or (subject ,name) (List ,name)))' notmuch search subject:notmuch or List:notmuch | notmuch_search_sanitize > EXPECTED -- 2.35.2 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard 2022-06-14 11:22 ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner @ 2022-06-14 11:22 ` David Bremner 0 siblings, 0 replies; 9+ messages in thread From: David Bremner @ 2022-06-14 11:22 UTC (permalink / raw) To: David Bremner, erik colson, notmuch Fix the bug reported at [1]. The parameter expansion for regex and wildcard modifiers has to be done a bit differently, because their arguments are not s-expressions defining complete Xapian queries. [1]: id:87o7yxqxy6.fsf@code.pm --- lib/parse-sexp.cc | 55 ++++++++++++++++++++++++++++++++++----- test/T081-sexpr-search.sh | 6 ----- 2 files changed, 49 insertions(+), 12 deletions(-) diff --git a/lib/parse-sexp.cc b/lib/parse-sexp.cc index 08fd7037..d0c9f145 100644 --- a/lib/parse-sexp.cc +++ b/lib/parse-sexp.cc @@ -187,6 +187,37 @@ _sexp_parse_phrase (std::string term_prefix, const char *phrase, Xapian::Query & return NOTMUCH_STATUS_SUCCESS; } +notmuch_status_t +_sexp_expand_term (notmuch_database_t *notmuch, + const _sexp_prefix_t *prefix, + const _sexp_binding_t *env, + const sexp_t *sx, + const char **out) +{ + if (! out) + return NOTMUCH_STATUS_NULL_POINTER; + + while (sx->ty == SEXP_VALUE && sx->aty == SEXP_BASIC && sx->val[0] == ',') { + const char *name = sx->val + 1; + for (; env; env = env->next) { + if (strcmp (name, env->name) == 0) { + sx = env->sx; + env = env->context; + break; + } + } + } + if (sx->ty != SEXP_VALUE) { + _notmuch_database_log (notmuch, "'%s' expects single atom as argument\n", + prefix->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + *out = sx->val; + + return NOTMUCH_STATUS_SUCCESS; +} + static notmuch_status_t _sexp_parse_wildcard (notmuch_database_t *notmuch, const _sexp_prefix_t *parent, @@ -227,8 +258,8 @@ _sexp_parse_one_term (notmuch_database_t *notmuch, std::string term_prefix, cons notmuch_status_t _sexp_parse_regex (notmuch_database_t *notmuch, const _sexp_prefix_t *prefix, const _sexp_prefix_t *parent, - unused(const _sexp_binding_t *env), - std::string val, Xapian::Query &output) + const _sexp_binding_t *env, + const sexp_t *term, Xapian::Query &output) { if (! parent) { _notmuch_database_log (notmuch, "illegal '%s' outside field\n", @@ -243,9 +274,15 @@ _sexp_parse_regex (notmuch_database_t *notmuch, } std::string msg; /* ignored */ + const char *str; + notmuch_status_t status; + + status = _sexp_expand_term (notmuch, prefix, env, term, &str); + if (status) + return status; return _notmuch_regexp_to_query (notmuch, Xapian::BAD_VALUENO, parent->name, - val, output, msg); + str, output, msg); } @@ -638,11 +675,17 @@ _sexp_to_xapian_query (notmuch_database_t *notmuch, const _sexp_prefix_t *parent return _notmuch_query_name_to_query (notmuch, sx->list->next->val, output); } - if (prefix->xapian_op == Xapian::Query::OP_WILDCARD) - return _sexp_parse_wildcard (notmuch, parent, env, sx->list->next->val, output); + if (prefix->xapian_op == Xapian::Query::OP_WILDCARD) { + const char *str; + status = _sexp_expand_term (notmuch, prefix, env, sx->list->next, &str); + if (status) + return status; + + return _sexp_parse_wildcard (notmuch, parent, env, str, output); + } if (prefix->flags & SEXP_FLAG_DO_REGEX) { - return _sexp_parse_regex (notmuch, prefix, parent, env, sx->list->next->val, output); + return _sexp_parse_regex (notmuch, prefix, parent, env, sx->list->next, output); } if (prefix->flags & SEXP_FLAG_DO_EXPAND) { diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh index d28e5b76..de9cbe7b 100755 --- a/test/T081-sexpr-search.sh +++ b/test/T081-sexpr-search.sh @@ -1116,7 +1116,6 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Saved Search: bad parameter syntax 5" -test_subtest_known_broken notmuch config set squery.Bad5 '(macro (thing) (tag (rx ,thing)))' notmuch search --query=sexp '(Bad5 (1 2))' >OUTPUT 2>&1 cat <<EOF > EXPECTED @@ -1126,7 +1125,6 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Saved Search: bad parameter syntax 6" -test_subtest_known_broken notmuch config set squery.Bad6 '(macro (thing) (tag (starts-with ,thing)))' notmuch search --query=sexp '(Bad6 (1 2))' >OUTPUT 2>&1 cat <<EOF > EXPECTED @@ -1187,14 +1185,12 @@ notmuch search --query=sexp '(TagSubject2 inbox maildir)' | notmuch_search_sanit test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "macro in regex" -test_subtest_known_broken notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED notmuch config set squery.D '(macro (tagname) (and (date 2009-11-17) (tag (rx ,tagname))))' notmuch search --query=sexp '(D inbo)' | notmuch_search_sanitize > OUTPUT test_expect_equal_file_nonempty EXPECTED OUTPUT test_begin_subtest "macro in wildcard" -test_subtest_known_broken notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED notmuch config set squery.W '(macro (tagname) (and (date 2009-11-17) (tag (starts-with ,tagname))))' notmuch search --query=sexp '(W inbo)' | notmuch_search_sanitize > OUTPUT @@ -1218,7 +1214,6 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "nested macros (shadowing, regex)" -test_subtest_known_broken notmuch search tag:/inbo/ and subject:/Maildi/ | notmuch_search_sanitize > EXPECTED notmuch config set squery.Inner3 '(macro (x) (subject (rx ,x)))' notmuch config set squery.Outer3 '(macro (x y) (and (tag (rx ,x)) (Inner3 ,y)))' @@ -1226,7 +1221,6 @@ notmuch search --query=sexp '(Outer3 inbo Maildi)' | notmuch_search_sanitize > O test_expect_equal_file_nonempty EXPECTED OUTPUT test_begin_subtest "nested macros (shadowing, wildcard)" -test_subtest_known_broken notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED notmuch config set squery.Inner4 '(macro (x) (subject (starts-with ,x)))' notmuch config set squery.Outer4 '(macro (x y) (and (tag (starts-with ,x)) (Inner4 ,y)))' -- 2.35.2 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: sexp and strings 2022-06-13 18:39 ` David Bremner 2022-06-14 11:22 ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner @ 2022-07-01 11:41 ` David Bremner 1 sibling, 0 replies; 9+ messages in thread From: David Bremner @ 2022-07-01 11:41 UTC (permalink / raw) To: erik colson, notmuch David Bremner <david@tethera.net> writes: > > Although you have one too many sets of (), there is still a bug here. > > $ notmuch config set squery.D '(macro (dossier) (tag (regex ,dossier))' > > also doesn't work. Apparently both for regex and wildcard expansion I > did not do macro parameter expansion. In the regex case the regex > matching for tags actually happens at query construction time, so that's > why it ends up as an empty query (presumably none of your tags starts > with ,dossier). > > Anyway, I think I know how to fix this, it will mean that in the corner > case of a regex starting with a comma, we'll have to use (regex ",foo"). This bug should be fixed in master as of 6a9ae990990848ec99107f2e2e6b6ae12dcd33df ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2022-07-01 11:49 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-06-12 23:11 sexp and strings erik colson 2022-06-13 9:12 ` Michael J Gruber 2022-06-13 9:31 ` erik colson 2022-06-13 10:11 ` Michael J Gruber 2022-06-13 15:10 ` erik colson 2022-06-13 18:39 ` David Bremner 2022-06-14 11:22 ` [PATCH 1/2] test/sexp: add known broken tests for macro param inside rx/wildcard David Bremner 2022-06-14 11:22 ` [PATCH 2/2] lib/sexp: add parameter expansion for regex and wildcard David Bremner 2022-07-01 11:41 ` sexp and strings David Bremner
Code repositories for project(s) associated with this public inbox https://yhetil.org/notmuch.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).