From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id IAENO2AOJWFm4gAAgWs5BA (envelope-from ) for ; Tue, 24 Aug 2021 17:21:04 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 8NfENmAOJWERHAAA1q6Kng (envelope-from ) for ; Tue, 24 Aug 2021 15:21:04 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 63E748470 for ; Tue, 24 Aug 2021 17:21:04 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 837FD2053B; Tue, 24 Aug 2021 11:20:34 -0400 (EDT) Received: from fethera.tethera.net (fethera.tethera.net [IPv6:2607:5300:60:c5::1]) by mail.notmuchmail.org (Postfix) with ESMTP id 72C772052A for ; Tue, 24 Aug 2021 11:20:25 -0400 (EDT) Received: by fethera.tethera.net (Postfix, from userid 1001) id 6B3545FD5C; Tue, 24 Aug 2021 11:20:25 -0400 (EDT) Received: (nullmailer pid 2942897 invoked by uid 1000); Tue, 24 Aug 2021 15:17:52 -0000 From: David Bremner To: notmuch@notmuchmail.org Cc: David Bremner Subject: [PATCH 34/36] CLI/{count, dump, reindex, reply, show}: enable sexp queries Date: Tue, 24 Aug 2021 08:17:43 -0700 Message-Id: <20210824151745.2941868-35-david@tethera.net> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210824151745.2941868-1-david@tethera.net> References: <20210824151745.2941868-1-david@tethera.net> MIME-Version: 1.0 Message-ID-Hash: WR7BQYDNDB3MJHI2JAR6NYZEYVUWDIAY X-Message-ID-Hash: WR7BQYDNDB3MJHI2JAR6NYZEYVUWDIAY X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1629818464; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=U5+IqtoZGe7QOYcApAeYZDirkzJXm4W1nJ+xygUpEdY=; b=GKmFKmruoCcFCdLMcsOFP7qs4zjx0VuYW0Xjvxa+7Et3FTO1HssR5fYK2KYSPqVBpu2Vf5 8jhxzqPbKQSsSCMMFTBn8lhkSojR742c4vxHpl8myvSCZjbKJWHEVd4LNQ+RA5XauKSfEo PwpGhoUiNyuJBdPoqDKQlmIFP9n304esLSr8ehBhsDcTI0HdhNIil6NEZ6MVgWMmXJ6J+f g8uAR05a4xoxyntu6NPpSJXGWXQ37yy7MlaPsbn3Esl4Sh5oSUhwa0MaRV9knOZMcjLuC+ UCGY8j0uwXrccwOCreKwSs+wHrTo0RikCcgV3gs8ZY+y07r1sYS0xOK9BJXz7w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1629818464; a=rsa-sha256; cv=none; b=m2tNDTE6AAEK8Dph/dfHlMsVEwhrLGTxymKCU32X1bM5IQpppm2RHEcahg57ICdXPYHc6+ lmv8U3r37Tq86Rmt0smfrKYvGlcMAcfc+6/9q/TKIhFlClGpBIv0ruErqieQGKGQ7rfrOu BF1LED/p6XWPOeNOm6yCMG7smHreg3xN5bFnAVGcWWPsczoBEPcpVizPkAZvMTMHYDu+DA OaLkBOF/0ggmr+pqrD9+sIxp5t/QamDef5ozo7mVdohz3SWYefJBzE2OH8ixRfhKTo82L7 fqIszX5ThodeyyCUwzBhX9hyTDQRrCVt09v5qD3HWj/T7AtqNzppVFv/slaRcA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -1.00 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 63E748470 X-Spam-Score: -1.00 X-Migadu-Scanner: scn0.migadu.com X-TUID: ipIcwvxFRA3v The change in each case is to call notmuch_query_create_with_syntax, relying on the already inherited shared options. As a bonus we get improved error handling from the new query creation API. The remaining subcommand is 'tag', which is a bit trickier. --- notmuch-count.c | 10 ++++++---- notmuch-dump.c | 9 +++++---- notmuch-reindex.c | 8 ++++---- notmuch-reply.c | 9 +++++---- notmuch-show.c | 8 ++++---- test/T060-count.sh | 24 ++++++++++++++++++++++++ test/T220-reply.sh | 19 +++++++++++++++---- test/T240-dump-restore.sh | 13 +++++++++++++ test/T520-show.sh | 18 ++++++++++++++++++ test/T700-reindex.sh | 29 +++++++++++++++++++++++++++++ 10 files changed, 123 insertions(+), 24 deletions(-) diff --git a/notmuch-count.c b/notmuch-count.c index e8c545e3..0d9046a8 100644 --- a/notmuch-count.c +++ b/notmuch-count.c @@ -74,10 +74,12 @@ print_count (notmuch_database_t *notmuch, const char *query_str, int ret = 0; notmuch_status_t status; - query = notmuch_query_create (notmuch, query_str); - if (query == NULL) { - fprintf (stderr, "Out of memory\n"); - return -1; + status = notmuch_query_create_with_syntax (notmuch, query_str, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch count", notmuch, status)) { + ret = -1; + goto DONE; } for (notmuch_config_values_start (exclude_tags); diff --git a/notmuch-dump.c b/notmuch-dump.c index 5c8213be..cb82d61f 100644 --- a/notmuch-dump.c +++ b/notmuch-dump.c @@ -232,11 +232,12 @@ database_dump_file (notmuch_database_t *notmuch, gzFile output, if (! query_str) query_str = ""; - query = notmuch_query_create (notmuch, query_str); - if (query == NULL) { - fprintf (stderr, "Out of memory\n"); + status = notmuch_query_create_with_syntax (notmuch, query_str, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch dump", notmuch, status)) return EXIT_FAILURE; - } + /* Don't ask xapian to sort by Message-ID. Xapian optimizes returning the * first results quickly at the expense of total time. */ diff --git a/notmuch-reindex.c b/notmuch-reindex.c index b40edbb6..49eacd47 100644 --- a/notmuch-reindex.c +++ b/notmuch-reindex.c @@ -49,11 +49,11 @@ reindex_query (notmuch_database_t *notmuch, const char *query_string, notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS; - query = notmuch_query_create (notmuch, query_string); - if (query == NULL) { - fprintf (stderr, "Out of memory.\n"); + status = notmuch_query_create_with_syntax (notmuch, query_string, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch reindex", notmuch, status)) return 1; - } /* reindexing is not interested in any special sort order */ notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED); diff --git a/notmuch-reply.c b/notmuch-reply.c index 5d5f95a3..2fb26cbc 100644 --- a/notmuch-reply.c +++ b/notmuch-reply.c @@ -716,6 +716,7 @@ notmuch_reply_command (notmuch_database_t *notmuch, int argc, char *argv[]) }; int format = FORMAT_DEFAULT; int reply_all = true; + notmuch_status_t status; notmuch_opt_desc_t options[] = { { .opt_keyword = &format, .name = "format", .keywords = @@ -758,11 +759,11 @@ notmuch_reply_command (notmuch_database_t *notmuch, int argc, char *argv[]) return EXIT_FAILURE; } - query = notmuch_query_create (notmuch, query_string); - if (query == NULL) { - fprintf (stderr, "Out of memory\n"); + status = notmuch_query_create_with_syntax (notmuch, query_string, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch reply", notmuch, status)) return EXIT_FAILURE; - } if (do_reply (notmuch, query, ¶ms, format, reply_all) != 0) return EXIT_FAILURE; diff --git a/notmuch-show.c b/notmuch-show.c index 667fbee8..2848c9c3 100644 --- a/notmuch-show.c +++ b/notmuch-show.c @@ -1364,11 +1364,11 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[]) return EXIT_FAILURE; } - query = notmuch_query_create (notmuch, query_string); - if (query == NULL) { - fprintf (stderr, "Out of memory\n"); + status = notmuch_query_create_with_syntax (notmuch, query_string, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch show", notmuch, status)) return EXIT_FAILURE; - } notmuch_query_set_sort (query, sort); diff --git a/test/T060-count.sh b/test/T060-count.sh index a1ebf8ba..6ad80df9 100755 --- a/test/T060-count.sh +++ b/test/T060-count.sh @@ -154,4 +154,28 @@ print("4: {} messages".format(query.count_messages())) EOF test_expect_equal_file EXPECTED OUTPUT +if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then + + test_begin_subtest "and of exact terms (query=sexp)" + output=$(notmuch count --query=sexp '(and "wonderful" "wizard")') + test_expect_equal "$output" 1 + + test_begin_subtest "or of exact terms (query=sexp)" + output=$(notmuch count --query=sexp '(or "php" "wizard")') + test_expect_equal "$output" 2 + + test_begin_subtest "starts-with, case-insensitive (query=sexp)" + output=$(notmuch count --query=sexp '(starts-with FreeB)') + test_expect_equal "$output" 5 + + test_begin_subtest "query that matches no messages (query=sexp)" + count=$(notmuch count --query=sexp '(and (from keithp) (to keithp))') + test_expect_equal 0 "$count" + + test_begin_subtest "Compound subquery (query=sexp)" + output=$(notmuch count --query=sexp '(thread (of (from keithp) (subject Maildir)))') + test_expect_equal "$output" 7 + +fi + test_done diff --git a/test/T220-reply.sh b/test/T220-reply.sh index b6d8f42a..6d64cbd7 100755 --- a/test/T220-reply.sh +++ b/test/T220-reply.sh @@ -2,15 +2,14 @@ test_description="\"notmuch reply\" in several variations" . $(dirname "$0")/test-lib.sh || exit 1 -test_begin_subtest "Basic reply" add_message '[from]="Sender "' \ [to]=test_suite@notmuchmail.org \ [subject]=notmuch-reply-test \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="basic reply test"' -output=$(notmuch reply id:${gen_msg_id} 2>&1 && echo OK) -test_expect_equal "$output" "From: Notmuch Test Suite +cat < basic.expected +From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Sender In-Reply-To: <${gen_msg_id}> @@ -18,7 +17,19 @@ References: <${gen_msg_id}> On Tue, 05 Jan 2010 15:43:56 -0000, Sender wrote: > basic reply test -OK" +OK +EOF + +test_begin_subtest "Basic reply" +notmuch reply id:${gen_msg_id} >OUTPUT 2>&1 && echo OK >> OUTPUT +test_expect_equal_file basic.expected OUTPUT + +if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then + + test_begin_subtest "Basic reply (query=sexp)" + notmuch reply --query=sexp "(id ${gen_msg_id})" >OUTPUT 2>&1 && echo OK >> OUTPUT + test_expect_equal_file basic.expected OUTPUT +fi test_begin_subtest "Multiple recipients" add_message '[from]="Sender "' \ diff --git a/test/T240-dump-restore.sh b/test/T240-dump-restore.sh index 105de130..a86f0fb7 100755 --- a/test/T240-dump-restore.sh +++ b/test/T240-dump-restore.sh @@ -117,6 +117,19 @@ test_begin_subtest "dump -- from:cworth" notmuch dump -- from:cworth > dump-dash-cworth.actual test_expect_equal_file dump-cworth.expected dump-dash-cworth.actual + +if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then + + test_begin_subtest "dump --query=sexp -- '(from cworth)'" + notmuch dump --query=sexp -- '(from cworth)' > dump-dash-cworth.actual2 + test_expect_equal_file_nonempty dump-cworth.expected dump-dash-cworth.actual2 + + test_begin_subtest "dump --query=sexp --output=outfile '(from cworth)'" + notmuch dump --output=dump-outfile-cworth.actual2 --query=sexp '(from cworth)' + test_expect_equal_file dump-cworth.expected dump-outfile-cworth.actual2 + +fi + test_begin_subtest "dump --output=outfile from:cworth" notmuch dump --output=dump-outfile-cworth.actual from:cworth test_expect_equal_file dump-cworth.expected dump-outfile-cworth.actual diff --git a/test/T520-show.sh b/test/T520-show.sh index 6f42ca12..12bde6c7 100755 --- a/test/T520-show.sh +++ b/test/T520-show.sh @@ -3,6 +3,13 @@ test_description='"notmuch show"' . $(dirname "$0")/test-lib.sh || exit 1 +test_query_syntax () { + test_begin_subtest "sexpr query: $1" + sexp=$(notmuch show --format=json --query=sexp "$1") + infix=$(notmuch show --format=json "$2") + test_expect_equal_json "$sexp" "$infix" +} + add_email_corpus test_begin_subtest "exit code for show invalid query" @@ -27,4 +34,15 @@ notmuch show --entire-thread=true --sort=newest-first $QUERY > EXPECTED notmuch show --entire-thread=true --sort=oldest-first $QUERY > OUTPUT test_expect_equal_file EXPECTED OUTPUT + +if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then + + test_query_syntax '(and "wonderful" "wizard")' 'wonderful and wizard' + test_query_syntax '(or "php" "wizard")' 'php or wizard' + test_query_syntax 'wizard' 'wizard' + test_query_syntax 'Wizard' 'Wizard' + test_query_syntax '(attachment notmuch-help.patch)' 'attachment:notmuch-help.patch' + +fi + test_done diff --git a/test/T700-reindex.sh b/test/T700-reindex.sh index bac43dc5..347f8483 100755 --- a/test/T700-reindex.sh +++ b/test/T700-reindex.sh @@ -4,6 +4,21 @@ test_description='reindexing messages' add_email_corpus + +if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then + + count=$(notmuch count --lastmod '*' | cut -f 3) + for query in '()' '(not)' '(and)' '(or ())' '(or (not))' '(or (and))' \ + '(or (and) (or) (not (and)))'; do + test_begin_subtest "reindex all messages: $query" + notmuch reindex --query=sexp "$query" + output=$(notmuch count --lastmod '*' | cut -f 3) + count=$((count + 1)) + test_expect_equal "$output" "$count" + done + +fi + notmuch tag +usertag1 '*' notmuch search '*' 2>1 | notmuch_search_sanitize > initial-threads @@ -41,6 +56,7 @@ notmuch dump > OUTPUT notmuch tag -attachment2 -encrypted2 -signed2 '*' test_expect_equal_file EXPECTED OUTPUT +backup_database test_begin_subtest 'reindex moves a message between threads' notmuch search --output=threads id:87iqd9rn3l.fsf@vertex.dottedmag > EXPECTED # re-parent @@ -48,7 +64,19 @@ sed -i 's/1258471718-6781-1-git-send-email-dottedmag@dottedmag.net/87iqd9rn3l.fs notmuch reindex id:1258471718-6781-2-git-send-email-dottedmag@dottedmag.net notmuch search --output=threads id:1258471718-6781-2-git-send-email-dottedmag@dottedmag.net > OUTPUT test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest 'reindex detects removal of all files' +notmuch search --output=messages not id:20091117232137.GA7669@griffis1.net> EXPECTED +# remove both copies +mv $MAIL_DIR/cur/51:2,* duplicate-message-2.eml +notmuch reindex id:20091117232137.GA7669@griffis1.net +notmuch search --output=messages '*' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT +restore_database +backup_database test_begin_subtest 'reindex detects removal of all files' notmuch search --output=messages not id:20091117232137.GA7669@griffis1.net> EXPECTED # remove both copies @@ -56,6 +84,7 @@ mv $MAIL_DIR/cur/51:2,* duplicate-message-2.eml notmuch reindex id:20091117232137.GA7669@griffis1.net notmuch search --output=messages '*' > OUTPUT test_expect_equal_file EXPECTED OUTPUT +restore_database test_begin_subtest "reindex preserves properties" cat < prop-dump -- 2.32.0