From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <amdragon@mit.edu>
Received: from localhost (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id 48D69431FAF
	for <notmuch@notmuchmail.org>; Sun, 30 Dec 2012 22:43:09 -0800 (PST)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
X-Spam-Flag: NO
X-Spam-Score: -0.7
X-Spam-Level: 
X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5
	tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled
Received: from olra.theworths.org ([127.0.0.1])
	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id Ndzt8uX1o5uJ for <notmuch@notmuchmail.org>;
	Sun, 30 Dec 2012 22:43:07 -0800 (PST)
Received: from dmz-mailsec-scanner-3.mit.edu (DMZ-MAILSEC-SCANNER-3.MIT.EDU
	[18.9.25.14])
	by olra.theworths.org (Postfix) with ESMTP id 04768429E34
	for <notmuch@notmuchmail.org>; Sun, 30 Dec 2012 22:43:01 -0800 (PST)
X-AuditID: 1209190e-b7fa16d000001402-35-50e133f5c5af
Received: from mailhub-auth-2.mit.edu ( [18.7.62.36])
	by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP
	id 7A.88.05122.5F331E05; Mon, 31 Dec 2012 01:43:01 -0500 (EST)
Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])
	by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id qBV6gosg027842; 
	Mon, 31 Dec 2012 01:42:50 -0500
Received: from drake.dyndns.org (c-76-21-105-205.hsd1.ca.comcast.net
	[76.21.105.205]) (authenticated bits=0)
	(User authenticated as amdragon@ATHENA.MIT.EDU)
	by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id qBV6gkSg009882
	(version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);
	Mon, 31 Dec 2012 01:42:49 -0500 (EST)
Received: from amthrax by drake.dyndns.org with local (Exim 4.77)
	(envelope-from <amdragon@mit.edu>)
	id 1TpZ5C-0000il-6B; Mon, 31 Dec 2012 01:42:46 -0500
From: Austin Clements <amdragon@MIT.EDU>
To: notmuch@notmuchmail.org
Subject: [PATCH v4 4/5] dump/restore: Use Xapian queries for batch-tag format
Date: Mon, 31 Dec 2012 01:42:41 -0500
Message-Id: <1356936162-2589-5-git-send-email-amdragon@mit.edu>
X-Mailer: git-send-email 1.7.10.4
In-Reply-To: <1356936162-2589-1-git-send-email-amdragon@mit.edu>
References: <1356936162-2589-1-git-send-email-amdragon@mit.edu>
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrJIsWRmVeSWpSXmKPExsUixG6novvV+GGAwYWP8hY3WrsZLZqmO1us
	nstjcf3mTGaLNyvnsTqweuycdZfd4/DXhSwet+6/Zvd4tuoWs8eWQ++ZA1ijuGxSUnMyy1KL
	9O0SuDImzVrEWjBRveLgvW/MDYw75boYOTgkBEwkFn907GLkBDLFJC7cW8/WxcjFISSwj1Hi
	6L67jBDOBkaJxo6t7CBVQgIXmSS+7UyFSMxllNi36DpYgk1AQ2Lb/uWMILaIgLTEzruzWUGK
	mAWaGCWubZrLBpIQFvCV6F32gBnEZhFQlWjafZkVxOYVsJd4P6uRBeIORYnuZxPA6jkFHCR2
	PpvFArHZXmL33vOMExj5FzAyrGKUTcmt0s1NzMwpTk3WLU5OzMtLLdI11svNLNFLTSndxAgK
	QU5Jvh2MXw8qHWIU4GBU4uHlYHoYIMSaWFZcmXuIUZKDSUmUVwkYwEJ8SfkplRmJxRnxRaU5
	qcWHGCU4mJVEeN9qAeV4UxIrq1KL8mFS0hwsSuK8V1Ju+gsJpCeWpGanphakFsFkZTg4lCR4
	U0CGChalpqdWpGXmlCCkmTg4QYbzAA3PA6nhLS5IzC3OTIfIn2JUlBLnDQFJCIAkMkrz4Hph
	KeIVozjQK8K8biBVPMD0Atf9CmgwE9BgLYYHIINLEhFSUg2Mlsks30sPnhR+PlViQZhkD7Og
	RgjTp5fqXxRvfbB8+9tesOzmnYoPic8aOkLKbUritl3Mju8Nu2q87bZPsMTN7+s+WHXynpu4
	KrNkQ3Hp277M4uPT1CrCNSzP35STi8z1uOu5++Nhj5Wxp8v2cH/VyeWbcIzX81vtyvk9u45a
	8rVX5SuU/NmgxFKckWioxVxUnAgAZCK7dOwCAAA=
Cc: tomi.ollila@iki.fi
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: "Use and development of the notmuch mail system."
	<notmuch.notmuchmail.org>
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Mon, 31 Dec 2012 06:43:09 -0000

This switches the new batch-tag format away from using a home-grown
hex-encoding scheme for message IDs in the dump to simply using Xapian
queries with Xapian quoting syntax.

This has a variety of advantages beyond presenting a cleaner and more
consistent interface.  Foremost is that it will dramatically simplify
the quoting for batch tagging, which shares the same input format.
While the hex-encoding is no better or worse for the simple ID queries
used by dump/restore, it becomes onerous for general-purpose queries
used in batch tagging.  It also better handles strange cases like
"id:foo and bar", since this is no longer syntactically valid.
---
 notmuch-dump.c    |    9 +++++----
 notmuch-restore.c |   22 ++++++++++------------
 tag-util.c        |    6 ------
 test/dump-restore |   14 ++++++++------
 4 files changed, 23 insertions(+), 28 deletions(-)

diff --git a/notmuch-dump.c b/notmuch-dump.c
index 29d79da..bf01a39 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -20,6 +20,7 @@
 
 #include "notmuch-client.h"
 #include "dump-restore-private.h"
+#include "string-util.h"
 
 int
 notmuch_dump_command (unused (void *ctx), int argc, char *argv[])
@@ -141,13 +142,13 @@ notmuch_dump_command (unused (void *ctx), int argc, char *argv[])
 		fprintf (stderr, "Error: cannot dump message id containing line break: %s\n", message_id);
 		return 1;
 	    }
-	    if (hex_encode (notmuch, message_id,
-			    &buffer, &buffer_size) != HEX_SUCCESS) {
-		    fprintf (stderr, "Error: failed to hex-encode msg-id %s\n",
+	    if (make_boolean_term (notmuch, "id", message_id,
+				   &buffer, &buffer_size)) {
+		    fprintf (stderr, "Error: failed to quote message id %s\n",
 			     message_id);
 		    return 1;
 	    }
-	    fprintf (output, " -- id:%s\n", buffer);
+	    fprintf (output, " -- %s\n", buffer);
 	}
 
 	notmuch_message_destroy (message);
diff --git a/notmuch-restore.c b/notmuch-restore.c
index 9ed9b51..77a4c27 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -207,7 +207,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
 	    INTERNAL_ERROR ("compile time constant regex failed.");
 
     do {
-	char *query_string;
+	char *query_string, *prefix, *term;
 
 	if (line_ctx != NULL)
 	    talloc_free (line_ctx);
@@ -220,19 +220,17 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
 				  &query_string, tag_ops);
 
 	    if (ret == 0) {
-		if (strncmp ("id:", query_string, 3) != 0) {
-		    fprintf (stderr, "Warning: unsupported query: %s\n", query_string);
+		ret = parse_boolean_term (line_ctx, query_string,
+					  &prefix, &term);
+		if (ret) {
+		    fprintf (stderr, "Warning: cannot parse query: %s\n",
+			     query_string);
+		    continue;
+		} else if (strcmp ("id", prefix) != 0) {
+		    fprintf (stderr, "Warning: not an id query: %s\n", query_string);
 		    continue;
 		}
-		/* delete id: from front of string; tag_message
-		 * expects a raw message-id.
-		 *
-		 * XXX: Note that query string id:foo and bar will be
-		 * interpreted as a message id "foo and bar". This
-		 * should eventually be fixed to give a better error
-		 * message.
-		 */
-		query_string = query_string + 3;
+		query_string = term;
 	    }
 	}
 
diff --git a/tag-util.c b/tag-util.c
index 705b7ba..e4e5dda 100644
--- a/tag-util.c
+++ b/tag-util.c
@@ -124,12 +124,6 @@ parse_tag_line (void *ctx, char *line,
     }
 
     /* tok now points to the query string */
-    if (hex_decode_inplace (tok) != HEX_SUCCESS) {
-	ret = line_error (TAG_PARSE_INVALID, line_for_error,
-			  "hex decoding of query %s failed", tok);
-	goto DONE;
-    }
-
     *query_string = tok;
 
   DONE:
diff --git a/test/dump-restore b/test/dump-restore
index 6a989b6..f9ae5b3 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -195,23 +195,25 @@ a
 
 # the previous line was blank; also no yelling please
 +%zz -- id:whatever
-+e +f id:%yy
++e +f id:"
++e +f tag:abc
 # the next non-comment line should report an an empty tag error for
 # batch tagging, but not for restore
 + +e -- id:20091117232137.GA7669@griffis1.net
-# highlight the sketchy id parsing; this should be last
-+g -- id:foo and bar
+# valid id, but warning about missing message
++e id:missing_message_id
 EOF
 
 cat <<EOF > EXPECTED
-Warning: unsupported query: a
+Warning: cannot parse query: a
 Warning: no query string [+0]
 Warning: no query string [+a +b]
 Warning: missing query string [+a +b ]
 Warning: no query string after -- [+c +d --]
 Warning: hex decoding of tag %zz failed [+%zz -- id:whatever]
-Warning: hex decoding of query id:%yy failed [+e +f id:%yy]
-Warning: cannot apply tags to missing message: foo and bar
+Warning: cannot parse query: id:"
+Warning: not an id query: tag:abc
+Warning: cannot apply tags to missing message: missing_message_id
 EOF
 
 test_expect_equal_file EXPECTED OUTPUT
-- 
1.7.10.4