From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qKmMOg6V82C5FwAAgWs5BA (envelope-from ) for ; Sun, 18 Jul 2021 04:42:22 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id OAI/Ng6V82D1XQAA1q6Kng (envelope-from ) for ; Sun, 18 Jul 2021 02:42:22 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 757AF1E82F for ; Sun, 18 Jul 2021 04:42:22 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id BB59D2A153; Sat, 17 Jul 2021 22:41:35 -0400 (EDT) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by mail.notmuchmail.org (Postfix) with ESMTP id C1508293FF for ; Sat, 17 Jul 2021 22:40:51 -0400 (EDT) Received: by fethera.tethera.net (Postfix, from userid 1001) id B05BC5FD21; Sat, 17 Jul 2021 22:40:51 -0400 (EDT) Received: (nullmailer pid 3853971 invoked by uid 1000); Sun, 18 Jul 2021 02:40:27 -0000 From: David Bremner To: notmuch@notmuchmail.org Cc: David Bremner Subject: [PATCH 25/25] lib/parse-sexp: parse user headers Date: Sat, 17 Jul 2021 23:40:21 -0300 Message-Id: <20210718024021.3850340-26-david@tethera.net> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210718024021.3850340-1-david@tethera.net> References: <20210718024021.3850340-1-david@tethera.net> MIME-Version: 1.0 Message-ID-Hash: CLJDRIT3O3VXS3EJZGJKEPQOZHVSN7HB X-Message-ID-Hash: CLJDRIT3O3VXS3EJZGJKEPQOZHVSN7HB X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1626576142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=9mQBb6+Bx8CDDOTPHoO0oF7TrG3bLx064fXPZIPUAAY=; b=j7zYR35ewnZYTDgezW6dzyjvhef3HESw/KIYR1wyBchlGRoA24NIc/3+XxDMjjh7pYIKLN 0a/0Hq38A+mkDmCLiN1kzlSAyZqdQGsCr2UZjEa4aeTK2QS3ZOBsJ1iBU6/lmsaQIOGUM9 /duxRpIhMUkgBmc0340EasVcvmbC/DUuHJjyvgb2+9LPPthXdDmGzSr9RtpttfXx9DIjOJ KONyfLRuy9Ci8eadtNAlxF+xb6YkwvfkrShXQ73iNNB0qIf98Ct1Y6KP4Kr0yt9ZDCAXip jX1d//a4i4IXgOzrj/3na+chVjbGhlnXC8+AXkBNfHlS09Y94r5Z+IAZtvhc3g== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1626576142; a=rsa-sha256; cv=none; b=q03zoGcY+2rbozGYIwD8g2DsukAHgvUv6FLQBGcllU7ce9UFeewXQeWyjb8OC6hMsT3nV4 R/J0ledzCuJjU06FlzaP8qs/Dw4LJ4/xBG82T8a3g7LkEchSJC2b/sZrMMhvWXaSEfKDQX G9qABFASXdUi5Lkyq5kw9FJCRO/PmFDJvR0Oc3XbruI/Q9a7vDe9u4cv/u/AAtPKB19yWV b9Leua+66csmX9rE4R89OafX5LyWq92Ua6Lm6EhyYi3P5kiTfm2qMgF//EwldNh9PuzZpi 5UTp1JRtjtPEGrUUp3XSzdZO4mUculKupgiqRSjO0eg7Nf6pm8ivXR0nFeFyUw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -1.07 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 757AF1E82F X-Spam-Score: -1.07 X-Migadu-Scanner: scn0.migadu.com X-TUID: UqrwBlhHR9az Rather than adding one prefix per user header, we create a single 'header' prefix that takes the configured name of the prefix as a parameter. --- lib/parse-sexp.cc | 50 +++++++++++++++++++++++ test/T081-sexpr-search.sh | 84 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 134 insertions(+) diff --git a/lib/parse-sexp.cc b/lib/parse-sexp.cc index 502b1be0..207403a5 100644 --- a/lib/parse-sexp.cc +++ b/lib/parse-sexp.cc @@ -55,6 +55,7 @@ static _sexp_field_t fields[] = { "date", Xapian::Query::OP_INVALID, SEXP_FLAG_NONE }, { "from", Xapian::Query::OP_PHRASE, SEXP_FLAG_REGEXP }, { "folder", Xapian::Query::OP_OR, SEXP_FLAG_REGEXP }, + { "header", Xapian::Query::OP_INVALID, SEXP_FLAG_WILDCARD }, { "id", Xapian::Query::OP_OR, SEXP_FLAG_REGEXP }, { "infix", Xapian::Query::OP_INVALID, SEXP_FLAG_NONE }, { "is", Xapian::Query::OP_AND, SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEXP }, @@ -265,6 +266,52 @@ _sexp_parse_infix (notmuch_database_t *notmuch, const sexp_t *sx, Xapian::Query return NOTMUCH_STATUS_SUCCESS; } +static notmuch_status_t +_sexp_parse_header (notmuch_database_t *notmuch, _sexp_flag_t flags, const sexp_t *sx, + Xapian::Query &output) +{ + const char *term_prefix; + _sexp_flag_t new_flags; + const sexp_t *rest; + notmuch_status_t status; + + if (! sx) { + _notmuch_database_log (notmuch, "missing header name\n"); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + if (sx->ty != SEXP_VALUE) { + _notmuch_database_log (notmuch, "header name must be atom, not list\n"); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + term_prefix = _notmuch_string_map_get (notmuch->user_prefix, sx->val); + if (! term_prefix) { + _notmuch_database_log (notmuch, "unknown header name %s\n", sx->val); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + status = _sexp_parse_keywords (notmuch, "header", sx->next, SEXP_FLAG_WILDCARD, new_flags, rest); + if (status) + return status; + + if ((new_flags | flags) & SEXP_FLAG_WILDCARD) { + if (rest) { + _notmuch_database_log (notmuch, "extra term(s) after wildcard\n", sx->val); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + output = Xapian::Query (Xapian::Query::OP_WILDCARD, term_prefix); + return NOTMUCH_STATUS_SUCCESS; + } + + if (! rest) { + _notmuch_database_log (notmuch, "missing header terms\n"); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + return _sexp_combine_field (term_prefix, Xapian::Query::OP_PHRASE, rest, output); +} + /* Here we expect the s-expression to be a proper list, with first * element defining and operation, or as a special case the empty * list */ @@ -317,6 +364,9 @@ _notmuch_sexp_to_xapian_query (notmuch_database_t *notmuch, const sexp_t *sx, Xa if (strcasecmp (field->name, "date") == 0) { return _sexp_parse_date (notmuch, sx, output); } + if (strcasecmp (field->name, "header") == 0) { + return _sexp_parse_header (notmuch, flags, rest, output); + } if (strcasecmp (field->name, "query") == 0) { if (! sx->list->next || ! sx->list->next->val) { _notmuch_database_log (notmuch, "missing query name\n"); diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh index 45568ba5..ae76a6e6 100755 --- a/test/T081-sexpr-search.sh +++ b/test/T081-sexpr-search.sh @@ -506,4 +506,88 @@ notmuch search date:2009-11-18..2009-11-18 and tag:unread > EXPECTED notmuch search --query-syntax=sexp '(and (infix "date:2009-11-18..2009-11-18") (infix "tag:unread"))' > OUTPUT test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "user header (no name)" +notmuch search --query-syntax=sexp '(header)' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +missing header name +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "user header (illegal name)" +notmuch search --query-syntax=sexp '(header (what?))' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +header name must be atom, not list +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "user header (unknown header)" +notmuch search --query-syntax=sexp '(header FooBar)' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +unknown header name FooBar +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "adding user header" +test_expect_code 0 "notmuch config set index.header.List \"List-Id\"" + +test_begin_subtest "user header (no terms)" +notmuch search --query-syntax=sexp '(header List)' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +missing header terms +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "reindexing" +test_expect_code 0 'notmuch reindex "*"' + +test_begin_subtest "wildcard search for user header" +grep -Ril List-Id ${MAIL_DIR} | sort | notmuch_dir_sanitize > EXPECTED +notmuch search --output=files --query-syntax=sexp '(header :any List)' | sort | notmuch_dir_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard search for user header: extra tokens" +notmuch search --query-syntax=sexp '(header :any List trailing-garbage)' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +extra term(s) after wildcard +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard search for user header: extra tokens (2)" +notmuch search --query-syntax=sexp '(header List :any trailing-garbage)' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +extra term(s) after wildcard +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard search for user header (post header flag)" +grep -Ril List-Id ${MAIL_DIR} | sort | notmuch_dir_sanitize > EXPECTED +notmuch search --output=files --query-syntax=sexp '(header List :any)' | sort | notmuch_dir_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search for user header" +notmuch search List:notmuch | notmuch_search_sanitize > EXPECTED +notmuch search --query-syntax=sexp '(header List notmuch)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search for user header (list token)" +notmuch search List:notmuch | notmuch_search_sanitize > EXPECTED +notmuch search --query-syntax=sexp '(header List notmuch.notmuchmail.org)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search for user header (quoted string)" +notmuch search 'List:"notmuch notmuchmail org"' | notmuch_search_sanitize > EXPECTED +notmuch search --query-syntax=sexp '(header List "notmuch notmuchmail org")' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search for user header (atoms)" +notmuch search 'List:"notmuch notmuchmail org"' | notmuch_search_sanitize > EXPECTED +notmuch search --query-syntax=sexp '(header List notmuch notmuchmail org)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + test_done -- 2.30.2