From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id AOF2L/MOJWEs5QAAgWs5BA (envelope-from ) for ; Tue, 24 Aug 2021 17:23:31 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id cLEIK/MOJWHIYQAAbx9fmQ (envelope-from ) for ; Tue, 24 Aug 2021 15:23:31 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 51773BC3F for ; Tue, 24 Aug 2021 17:23:31 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id C115E26C36; Tue, 24 Aug 2021 11:21:42 -0400 (EDT) Received: from fethera.tethera.net (fethera.tethera.net [IPv6:2607:5300:60:c5::1]) by mail.notmuchmail.org (Postfix) with ESMTP id 5A70326C12 for ; Tue, 24 Aug 2021 11:21:39 -0400 (EDT) Received: by fethera.tethera.net (Postfix, from userid 1001) id 541205FD5C; Tue, 24 Aug 2021 11:21:39 -0400 (EDT) Received: (nullmailer pid 2942865 invoked by uid 1000); Tue, 24 Aug 2021 15:17:51 -0000 From: David Bremner To: notmuch@notmuchmail.org Cc: David Bremner Subject: [PATCH 18/36] lib/query: generalize exclude handling to s-expression queries Date: Tue, 24 Aug 2021 08:17:27 -0700 Message-Id: <20210824151745.2941868-19-david@tethera.net> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210824151745.2941868-1-david@tethera.net> References: <20210824151745.2941868-1-david@tethera.net> MIME-Version: 1.0 Message-ID-Hash: YMSRWANO7PFVZWZ3TU5FXPVLURMOO35T X-Message-ID-Hash: YMSRWANO7PFVZWZ3TU5FXPVLURMOO35T X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1629818611; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=0YfidD2IVJdCKIjemZS8AlcQUnVa1Qm9Bxf5yU8RBXI=; b=EjIigF5o998hNEuDuKpRQ8144nWiUt3iPn+gq8X04i3OulMXNy64JUKavs+a1O0ku/S9Ly w/T4OLIyYaRcN6tgh/TE7dOvPgqeF0KODOAuYGg9BIWCTeEeqUC8tnf63BFOpJBSEElz0q c5Nia5wB5OAXaJOZlGRAFa4XKPEtqV6GlTtN+UgTAzA6/FacUDcRvWemdGyIVeWz4IVd3/ KwTFcKslDDavBCqwUkmWokUOm3M5YCR8k7lqzqVNXStCYrPnyeWYzYXro7wpS4feIs+0UB QEHdaoxABw+6k4RV3oemUuJyxaF5vVABdlgGvgDeiRZ8v6Olj+DjZEfGBlPoIw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1629818611; a=rsa-sha256; cv=none; b=HixHt1HP0gNvIKG8DfbIi7qeP3Z6LXdvzgYgW5pMu4MuW5V+/zAmH11mJBa37yoRZoeac3 1gTHqyYGLGoaW1ZI/ZRCPCvdeYb71T4lijvyaD10xLHAoWRFsOHki0WKB4W8Kk1EuDkkSU AEqZ7PLNnJXe4+vL79KjG2Vc1M+he8CJyGhxacWUKMZPSa9rNPoj05Avv7vRpl7otb3iEk wg9IlRoBcEfp0yDWZx8ilMhInVw/D1KF0/Wn5MVJxSlhxaPNMa0zqsIc3Otw6k+oicsVZs bnezaFG/vADZrdzDwPHzlXrwOdkt1G4WX7dJ5Ns2SWHmtGToNsure/TzayT/cg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -0.93 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 51773BC3F X-Spam-Score: -0.93 X-Migadu-Scanner: scn0.migadu.com X-TUID: zJVQ9AR6MW6H In fact most of the code path is in common, only the caching of terms in the query needs to be added for s-expression queries. --- lib/query.cc | 34 ++++++++++++++++++++++----------- test/T081-sexpr-search.sh | 40 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 11 deletions(-) diff --git a/lib/query.cc b/lib/query.cc index 435f7229..56f90e1c 100644 --- a/lib/query.cc +++ b/lib/query.cc @@ -165,6 +165,19 @@ notmuch_query_create_with_syntax (notmuch_database_t *notmuch, return NOTMUCH_STATUS_SUCCESS; } +static void +_notmuch_query_cache_terms (notmuch_query_t *query) +{ + /* Xapian doesn't support skip_to on terms from a query since + * they are unordered, so cache a copy of all terms in + * something searchable. + */ + + for (Xapian::TermIterator t = query->xapian_query.get_terms_begin (); + t != query->xapian_query.get_terms_end (); ++t) + query->terms.insert (*t); +} + static notmuch_status_t _notmuch_query_ensure_parsed_xapian (notmuch_query_t *query) { @@ -173,15 +186,7 @@ _notmuch_query_ensure_parsed_xapian (notmuch_query_t *query) query->notmuch->query_parser-> parse_query (query->query_string, NOTMUCH_QUERY_PARSER_FLAGS); - /* Xapian doesn't support skip_to on terms from a query since - * they are unordered, so cache a copy of all terms in - * something searchable. - */ - - for (Xapian::TermIterator t = query->xapian_query.get_terms_begin (); - t != query->xapian_query.get_terms_end (); ++t) - query->terms.insert (*t); - + _notmuch_query_cache_terms (query); query->parsed = true; } catch (const Xapian::Error &error) { @@ -203,11 +208,18 @@ _notmuch_query_ensure_parsed_xapian (notmuch_query_t *query) static notmuch_status_t _notmuch_query_ensure_parsed_sexpr (notmuch_query_t *query) { + notmuch_status_t status; + if (query->parsed) return NOTMUCH_STATUS_SUCCESS; - return _notmuch_sexp_string_to_xapian_query (query->notmuch, query->query_string, - query->xapian_query); + status = _notmuch_sexp_string_to_xapian_query (query->notmuch, query->query_string, + query->xapian_query); + if (status) + return status; + + _notmuch_query_cache_terms (query); + return NOTMUCH_STATUS_SUCCESS; } static notmuch_status_t diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh index 44cb681f..be243fc0 100755 --- a/test/T081-sexpr-search.sh +++ b/test/T081-sexpr-search.sh @@ -525,4 +525,44 @@ notmuch search: Syntax error in query EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "Search, exclude \"deleted\" messages from search" +notmuch config set search.exclude_tags deleted +generate_message '[subject]="Not deleted"' +not_deleted_id=$gen_msg_id +generate_message '[subject]="Deleted"' +notmuch new > /dev/null +notmuch tag +deleted id:$gen_msg_id +deleted_id=$gen_msg_id +output=$(notmuch search --query=sexp '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)" + +test_begin_subtest "Search, exclude \"deleted\" messages from message search --exclude=false" +output=$(notmuch search --query=sexp --exclude=false --output=messages '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "id:$not_deleted_id +id:$deleted_id" + +test_begin_subtest "Search, exclude \"deleted\" messages from search, overridden" +notmuch search --query=sexp '(and (subject deleted) (tag deleted))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Deleted (deleted inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search, exclude \"deleted\" messages from threads" +add_message '[subject]="Not deleted reply"' '[in-reply-to]="<$gen_msg_id>"' +output=$(notmuch search --query=sexp '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread) +thread:XXX 2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inbox unread)" + +test_begin_subtest "Search, don't exclude \"deleted\" messages when --exclude=flag specified" +output=$(notmuch search --query=sexp --exclude=flag '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread) +thread:XXX 2001-01-05 [1/2] Notmuch Test Suite; Deleted (deleted inbox unread)" + +test_begin_subtest "Search, don't exclude \"deleted\" messages from search if not configured" +notmuch config set search.exclude_tags +output=$(notmuch search --query=sexp '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread) +thread:XXX 2001-01-05 [2/2] Notmuch Test Suite; Deleted (deleted inbox unread)" + test_done -- 2.32.0