test infrastructure for new dump/restore

test infrastructure for new dump/restore

[david]

This implements an old suggestion of Mark's to get wacky message-ids
into the database directly without relying on underdefined behaviour
of the gmime parser; the previous effort relied on gmime passing
literally through message-ids not delimitted according RFC.

Also compared to the previous version
(id:"1326591624-15493-10-git-send-email-david@tethera.net"), this now
uses valid UTF-8 text, rather than just ascii, although it is a bit
biased towards ascii because most of the characters that cause
problems are there.

There is a fair amount of code here, but I hope the generation of
random messages may be more useful in the future.

The high level goal here is to (re)-introduce a hex-encoding based
dump-restore that can pass this roundtrip test, and probably some
batch tagging facility that shares code.

If people don't mind things broken up into mini-series (without
obvious gain in new features) like this, I'll probably post the
hex-encoding infrastructure next.

[PATCH 1/3] test: add database routines for testing.

[david]

From: David Bremner <bremner@debian.org>

Initially, provide a way to create "stub" messages in the notmuch
database without corresponding files.  This is essentially cut and
paste from lib/database.cc. This is a seperate file since we don't
want to export these symbols from libnotmuch or bloat the library with
non-exported code.
---
 test/database-test.c |   72 ++++++++++++++++++++++++++++++++++++++++++++++++++
 test/database-test.h |   21 +++++++++++++++
 2 files changed, 93 insertions(+)
 create mode 100644 test/database-test.c
 create mode 100644 test/database-test.h

diff --git a/test/database-test.c b/test/database-test.c
new file mode 100644
index 0000000..f0f1c8e
--- /dev/null
+++ b/test/database-test.c
@@ -0,0 +1,72 @@
+/*
+ * Database routines intended only for testing, not exported from
+ * library.
+ *
+ * Copyright (c) 2012 David Bremner
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: David Bremner <david@tethera.net>
+ */
+
+#include "notmuch-private.h"
+#include "database-test.h"
+
+notmuch_status_t
+notmuch_database_add_stub_message (notmuch_database_t *notmuch,
+				   const char *message_id,
+				   const char **tags)
+{
+    const char **tag;
+    notmuch_status_t ret;
+    notmuch_private_status_t private_status;
+    notmuch_message_t *message;
+
+    ret = _notmuch_database_ensure_writable (notmuch);
+    if (ret)
+	return ret;
+
+    message = _notmuch_message_create_for_message_id (notmuch,
+						      message_id,
+						      &private_status);
+    if (message == NULL) {
+	return COERCE_STATUS (private_status,
+			      "Unexpected status value from _notmuch_message_create_for_message_id");
+
+    }
+
+    if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
+	_notmuch_message_add_term (message, "type", "mail");
+    } else {
+	return NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
+    }
+
+    if (tags) {
+	ret = notmuch_message_freeze (message);
+	if (ret)
+	    return ret;
+
+	for (tag = tags; *tag; tag++) {
+	    ret = notmuch_message_add_tag (message, *tag);
+	    if (ret)
+		return ret;
+	}
+    }
+
+    ret = notmuch_message_thaw (message);
+    if (ret)
+	return ret;
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
diff --git a/test/database-test.h b/test/database-test.h
new file mode 100644
index 0000000..84f7988
--- /dev/null
+++ b/test/database-test.h
@@ -0,0 +1,21 @@
+#ifndef _DATABASE_TEST_H
+#define _DATABASE_TEST_H
+/* Add a new stub message to the given notmuch database.
+ *
+ * At least the following return values are possible:
+ *
+ * NOTMUCH_STATUS_SUCCESS: Message successfully added to database.
+ *
+ * NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: Message has the same message
+ *	ID as another message already in the database.
+ *
+ * NOTMUCH_STATUS_READ_ONLY_DATABASE: Database was opened in read-only
+ *	mode so no message can be added.
+ */
+
+notmuch_status_t
+notmuch_database_add_stub_message (notmuch_database_t *database,
+				   const char *message_id,
+				   const char **tag_list);
+
+#endif
-- 
1.7.10.4

[PATCH 2/3] test: add generator for random "stub" messages

[david]

From: David Bremner <bremner@debian.org>

Initial use case is testing dump and restore, so we only have
message-ids and tags.

The message ID's are nothing like RFC compliant, but it doesn't seem
any harder to roundtrip random UTF-8 strings than RFC-compliant ones.

Tags are UTF-8, even though notmuch is in principle more generous than
that.
---
 test/.gitignore      |    1 +
 test/Makefile.local  |   14 +++-
 test/basic           |    2 +-
 test/random-corpus.c |  201 ++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 216 insertions(+), 2 deletions(-)
 create mode 100644 test/random-corpus.c

diff --git a/test/.gitignore b/test/.gitignore
index e63c689..e23017b 100644
--- a/test/.gitignore
+++ b/test/.gitignore
@@ -3,4 +3,5 @@ corpus.mail
 smtp-dummy
 symbol-test
 arg-test
+random-corpus
 tmp.*
diff --git a/test/Makefile.local b/test/Makefile.local
index c7f1435..586efc6 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -13,6 +13,13 @@ smtp_dummy_modules = $(smtp_dummy_srcs:.c=.o)
 $(dir)/arg-test: $(dir)/arg-test.o command-line-arguments.o util/libutil.a
 	$(call quiet,CC) -I. $^ -o $@
 
+random_corpus_deps =  $(dir)/random-corpus.o  $(dir)/database-test.o \
+			notmuch-config.o command-line-arguments.o \
+			lib/libnotmuch.a util/libutil.a
+
+$(dir)/random-corpus: $(random_corpus_deps)
+	$(call quiet,CC) $(CFLAGS_FINAL) $^ -o $@ $(CONFIGURE_LDFLAGS)
+
 $(dir)/smtp-dummy: $(smtp_dummy_modules)
 	$(call quiet,CC) $^ -o $@
 
@@ -21,7 +28,12 @@ $(dir)/symbol-test: $(dir)/symbol-test.o
 
 .PHONY: test check
 
-test-binaries: $(dir)/arg-test $(dir)/smtp-dummy $(dir)/symbol-test
+TEST_BINARIES=$(dir)/arg-test \
+	      $(dir)/random-corpus \
+	      $(dir)/smtp-dummy \
+	      $(dir)/symbol-test
+
+test-binaries: $(TEST_BINARIES)
 
 test:	all test-binaries
 	@${dir}/notmuch-test $(OPTIONS)
diff --git a/test/basic b/test/basic
index d6aed24..589c4e2 100755
--- a/test/basic
+++ b/test/basic
@@ -54,7 +54,7 @@ test_begin_subtest 'Ensure that all available tests will be run by notmuch-test'
 eval $(sed -n -e '/^TESTS="$/,/^"$/p' $TEST_DIRECTORY/notmuch-test)
 tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
 available=$(find "$TEST_DIRECTORY" -maxdepth 1 -type f -executable -printf '%f\n' | \
-    sed -r -e "/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test)$/d" | \
+    sed -r -e "/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test|random-corpus)$/d" | \
     sort)
 test_expect_equal "$tests_in_suite" "$available"
 
diff --git a/test/random-corpus.c b/test/random-corpus.c
new file mode 100644
index 0000000..ae900a6
--- /dev/null
+++ b/test/random-corpus.c
@@ -0,0 +1,201 @@
+/*
+ * Generate a random corpus of stub messages.
+ *
+ * Initial use case is testing dump and restore, so we only have
+ * message-ids and tags.
+ *
+ * Generated message-id's and tags are intentionally nasty.
+ *
+ * Copyright (c) 2012 David Bremner
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: David Bremner <david@tethera.net>
+ */
+
+#include <stdlib.h>
+#include <assert.h>
+#include <talloc.h>
+#include <string.h>
+#include <glib.h>
+#include <math.h>
+
+#include "notmuch-client.h"
+#include "command-line-arguments.h"
+#include "database-test.h"
+
+/* Current largest UTF-32 value defined. Note that most of these will
+ * be printed as boxes in most fonts.
+ */
+
+#define GLYPH_MAX 0x10FFFE
+
+static gunichar
+random_unichar ()
+{
+    int start=1, stop=GLYPH_MAX;
+    int class = random() % 4;
+
+    switch (class) {
+    case 0:
+	/* control */
+	start=0x01;
+	stop=0x20;
+	break;
+    case 1:
+	start=0x21;
+	stop=0x7E;
+	break;
+    case 2:
+	start=0x41;
+	stop=0x7a;
+	break;
+    case 3:
+	start=0x7F;
+	stop=GLYPH_MAX;
+    }
+
+    return start + (random() % (start - stop + 1));
+}
+
+static char *
+random_utf8_string (void *ctx, size_t char_count)
+{
+
+    gchar *buf = NULL;
+    size_t buf_size = 0;
+
+    size_t offset = 0;
+
+    size_t i;
+
+    buf = talloc_realloc (ctx, NULL, gchar, char_count);
+    buf_size = char_count;
+
+    for (i = 0; i < char_count; i++) {
+	gunichar randomchar;
+	size_t written;
+
+	/* 6 for one glyph, one for null */
+	if (buf_size - offset < 8) {
+	    buf_size += 16;
+	    buf = talloc_realloc (ctx, buf, gchar, buf_size);
+	}
+
+	randomchar = random_unichar();
+
+	written = g_unichar_to_utf8 (randomchar, buf + offset);
+
+	if (written <= 0) {
+	    fprintf (stderr, "error converting to utf8\n");
+	    exit (1);
+	}
+
+	offset += written;
+
+    }
+    buf[offset] = 0;
+    return buf;
+}
+
+
+int
+main (int argc, char **argv)
+{
+
+    void *ctx = talloc_new (NULL);
+
+    char *config_path  = NULL;
+    notmuch_config_t *config;
+    notmuch_database_t *notmuch;
+
+    int num_messages = 500;
+    int max_tags = 10;
+    // leave room for UTF-8 encoding.
+    int tag_len = NOTMUCH_TAG_MAX / 6;
+    // NOTMUCH_MESSAGE_ID_MAX is not exported, so we make a
+    // conservative guess.
+    int message_id_len = (NOTMUCH_TAG_MAX - 20) / 6;
+
+    int seed = 734569;
+
+    notmuch_opt_desc_t options[] = {
+	{ NOTMUCH_OPT_STRING, &config_path, "config-path", 'c', 0 },
+	{ NOTMUCH_OPT_INT, &num_messages, "num-messages", 'n', 0 },
+	{ NOTMUCH_OPT_INT, &max_tags, "max-tags", 'm', 0 },
+	{ NOTMUCH_OPT_INT, &message_id_len, "message-id-len", 'M', 0 },
+	{ NOTMUCH_OPT_INT, &tag_len, "tag-len", 't', 0 },
+	{ NOTMUCH_OPT_INT, &seed, "seed", 's', 0 },
+	{ 0, 0, 0, 0, 0 }
+    };
+
+    int opt_index = parse_arguments (argc, argv, options, 1);
+
+    if (opt_index < 0)
+	exit (1);
+
+    if (config_path == NULL) {
+	fprintf (stderr, "configuration path must be specified");
+	exit (1);
+    }
+
+    config = notmuch_config_open (ctx, config_path, NULL);
+    if (config == NULL)
+	return 1;
+
+    if (notmuch_database_open (notmuch_config_get_database_path (config),
+			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
+	return 1;
+
+    srandom (seed);
+
+    int count;
+    for (count = 0; count < num_messages; count++) {
+	int j;
+	int num_tags = random () % (max_tags + 1);
+	int this_mid_len = random () % message_id_len + 1;
+	const char **tag_list;
+	char *mid;
+	notmuch_status_t status;
+
+	do {
+	    mid = random_utf8_string (ctx, this_mid_len);
+
+	    tag_list = talloc_realloc (ctx, NULL, const char *, num_tags + 2);
+
+	    tag_list[0] = "random-corpus";
+
+	    for (j = 0; j < num_tags; j++) {
+		int this_tag_len = random () % tag_len + 1;
+
+		tag_list[j + 1] = random_utf8_string (ctx, this_tag_len);
+	    }
+
+	    tag_list[j + 1] = NULL;
+
+	    status = notmuch_database_add_stub_message (notmuch, mid, tag_list);
+	} while (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID);
+
+	if (status != NOTMUCH_STATUS_SUCCESS) {
+	    fprintf (stderr, "error %d adding message", status);
+	    exit (status);
+	}
+    }
+
+    notmuch_database_destroy (notmuch);
+
+    talloc_free (ctx);
+
+    return 0;
+}
-- 
1.7.10.4

[PATCH 3/3] test: add broken roundtrip test

[david]

From: David Bremner <bremner@debian.org>

The output of test_cmp is redirected because it is pretty horrible,
and tends to mess up terminals. When the test is no longer marked as
broken, this redirection should be removed.
---
 test/dump-restore |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/test/dump-restore b/test/dump-restore
index 439e998..7979ebf 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -82,4 +82,13 @@ test_begin_subtest "dump outfile -- from:cworth"
 notmuch dump dump-outfile-dash-inbox.actual -- from:cworth
 test_expect_equal_file dump-cworth.expected dump-outfile-dash-inbox.actual
 
+test_expect_success 'roundtripping random message-ids and tags' \
+    'test_subtest_known_broken &&
+    ${TEST_DIRECTORY}/random-corpus --config-path=${NOTMUCH_CONFIG} &&
+    notmuch dump > EXPECTED.$test_count &&
+    notmuch tag -random-corpus tag:random-corpus &&
+    notmuch restore < EXPECTED.$test_count 2>/dev/null &&
+    notmuch dump > OUTPUT.$test_count &&
+    test_cmp EXPECTED.$test_count OUTPUT.$test_count 1>/dev/null'
+
 test_done
-- 
1.7.10.4

Re: [PATCH 2/3] test: add generator for random "stub" messages

[Mark Walters]

Hi I don't think I know enough Xapian to sensibly review the first patch
in this series.

On Sun, 05 Aug 2012, david@tethera.net wrote:
> From: David Bremner <bremner@debian.org>
>
> Initial use case is testing dump and restore, so we only have
> message-ids and tags.
>
> The message ID's are nothing like RFC compliant, but it doesn't seem
> any harder to roundtrip random UTF-8 strings than RFC-compliant ones.
>
> Tags are UTF-8, even though notmuch is in principle more generous than
> that.
> ---
>  test/.gitignore      |    1 +
>  test/Makefile.local  |   14 +++-
>  test/basic           |    2 +-
>  test/random-corpus.c |  201 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 216 insertions(+), 2 deletions(-)
>  create mode 100644 test/random-corpus.c
>
> diff --git a/test/.gitignore b/test/.gitignore
> index e63c689..e23017b 100644
> --- a/test/.gitignore
> +++ b/test/.gitignore
> @@ -3,4 +3,5 @@ corpus.mail
>  smtp-dummy
>  symbol-test
>  arg-test
> +random-corpus
>  tmp.*
> diff --git a/test/Makefile.local b/test/Makefile.local
> index c7f1435..586efc6 100644
> --- a/test/Makefile.local
> +++ b/test/Makefile.local
> @@ -13,6 +13,13 @@ smtp_dummy_modules = $(smtp_dummy_srcs:.c=.o)
>  $(dir)/arg-test: $(dir)/arg-test.o command-line-arguments.o util/libutil.a
>  	$(call quiet,CC) -I. $^ -o $@
>  
> +random_corpus_deps =  $(dir)/random-corpus.o  $(dir)/database-test.o \
> +			notmuch-config.o command-line-arguments.o \
> +			lib/libnotmuch.a util/libutil.a
> +
> +$(dir)/random-corpus: $(random_corpus_deps)
> +	$(call quiet,CC) $(CFLAGS_FINAL) $^ -o $@ $(CONFIGURE_LDFLAGS)
> +
>  $(dir)/smtp-dummy: $(smtp_dummy_modules)
>  	$(call quiet,CC) $^ -o $@
>  
> @@ -21,7 +28,12 @@ $(dir)/symbol-test: $(dir)/symbol-test.o
>  
>  .PHONY: test check
>  
> -test-binaries: $(dir)/arg-test $(dir)/smtp-dummy $(dir)/symbol-test
> +TEST_BINARIES=$(dir)/arg-test \
> +	      $(dir)/random-corpus \
> +	      $(dir)/smtp-dummy \
> +	      $(dir)/symbol-test
> +
> +test-binaries: $(TEST_BINARIES)
>  
>  test:	all test-binaries
>  	@${dir}/notmuch-test $(OPTIONS)
> diff --git a/test/basic b/test/basic
> index d6aed24..589c4e2 100755
> --- a/test/basic
> +++ b/test/basic
> @@ -54,7 +54,7 @@ test_begin_subtest 'Ensure that all available tests will be run by notmuch-test'
>  eval $(sed -n -e '/^TESTS="$/,/^"$/p' $TEST_DIRECTORY/notmuch-test)
>  tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
>  available=$(find "$TEST_DIRECTORY" -maxdepth 1 -type f -executable -printf '%f\n' | \
> -    sed -r -e "/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test)$/d" | \
> +    sed -r -e "/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test|random-corpus)$/d" | \
>      sort)
>  test_expect_equal "$tests_in_suite" "$available"
>  
> diff --git a/test/random-corpus.c b/test/random-corpus.c
> new file mode 100644
> index 0000000..ae900a6
> --- /dev/null
> +++ b/test/random-corpus.c
> @@ -0,0 +1,201 @@
> +/*
> + * Generate a random corpus of stub messages.
> + *
> + * Initial use case is testing dump and restore, so we only have
> + * message-ids and tags.
> + *
> + * Generated message-id's and tags are intentionally nasty.
> + *
> + * Copyright (c) 2012 David Bremner
> + *
> + * This program is free software: you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation, either version 3 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see http://www.gnu.org/licenses/ .
> + *
> + * Author: David Bremner <david@tethera.net>
> + */
> +
> +#include <stdlib.h>
> +#include <assert.h>
> +#include <talloc.h>
> +#include <string.h>
> +#include <glib.h>
> +#include <math.h>
> +
> +#include "notmuch-client.h"
> +#include "command-line-arguments.h"
> +#include "database-test.h"
> +
> +/* Current largest UTF-32 value defined. Note that most of these will
> + * be printed as boxes in most fonts.
> + */
> +
> +#define GLYPH_MAX 0x10FFFE
> +
> +static gunichar
> +random_unichar ()
> +{
> +    int start=1, stop=GLYPH_MAX;
> +    int class = random() % 4;
> +
> +    switch (class) {
> +    case 0:
> +	/* control */
> +	start=0x01;
> +	stop=0x20;
> +	break;
> +    case 1:
> +	start=0x21;
> +	stop=0x7E;
> +	break;
> +    case 2:
> +	start=0x41;
> +	stop=0x7a;
> +	break;
> +    case 3:
> +	start=0x7F;
> +	stop=GLYPH_MAX;
> +    }

I think comments on other the classes might be helpful: I think Case 2
is Ascii A-z but that has a few characters between Z and a. Of course
since you are generating a random character it doesn't "matter".

> +
> +    return start + (random() % (start - stop + 1));
> +}

I think you mean % (stop - start + 1)?

Otherwise this looks fine (I haven't actually run it yet).

Best wishes

Mark

> +static char *
> +random_utf8_string (void *ctx, size_t char_count)
> +{
> +
> +    gchar *buf = NULL;
> +    size_t buf_size = 0;
> +
> +    size_t offset = 0;
> +
> +    size_t i;
> +
> +    buf = talloc_realloc (ctx, NULL, gchar, char_count);
> +    buf_size = char_count;
> +
> +    for (i = 0; i < char_count; i++) {
> +	gunichar randomchar;
> +	size_t written;
> +
> +	/* 6 for one glyph, one for null */
> +	if (buf_size - offset < 8) {
> +	    buf_size += 16;
> +	    buf = talloc_realloc (ctx, buf, gchar, buf_size);
> +	}
> +
> +	randomchar = random_unichar();
> +
> +	written = g_unichar_to_utf8 (randomchar, buf + offset);
> +
> +	if (written <= 0) {
> +	    fprintf (stderr, "error converting to utf8\n");
> +	    exit (1);
> +	}
> +
> +	offset += written;
> +
> +    }
> +    buf[offset] = 0;
> +    return buf;
> +}
> +
> +
> +int
> +main (int argc, char **argv)
> +{
> +
> +    void *ctx = talloc_new (NULL);
> +
> +    char *config_path  = NULL;
> +    notmuch_config_t *config;
> +    notmuch_database_t *notmuch;
> +
> +    int num_messages = 500;
> +    int max_tags = 10;
> +    // leave room for UTF-8 encoding.
> +    int tag_len = NOTMUCH_TAG_MAX / 6;
> +    // NOTMUCH_MESSAGE_ID_MAX is not exported, so we make a
> +    // conservative guess.
> +    int message_id_len = (NOTMUCH_TAG_MAX - 20) / 6;
> +
> +    int seed = 734569;
> +
> +    notmuch_opt_desc_t options[] = {
> +	{ NOTMUCH_OPT_STRING, &config_path, "config-path", 'c', 0 },
> +	{ NOTMUCH_OPT_INT, &num_messages, "num-messages", 'n', 0 },
> +	{ NOTMUCH_OPT_INT, &max_tags, "max-tags", 'm', 0 },
> +	{ NOTMUCH_OPT_INT, &message_id_len, "message-id-len", 'M', 0 },
> +	{ NOTMUCH_OPT_INT, &tag_len, "tag-len", 't', 0 },
> +	{ NOTMUCH_OPT_INT, &seed, "seed", 's', 0 },
> +	{ 0, 0, 0, 0, 0 }
> +    };
> +
> +    int opt_index = parse_arguments (argc, argv, options, 1);
> +
> +    if (opt_index < 0)
> +	exit (1);
> +
> +    if (config_path == NULL) {
> +	fprintf (stderr, "configuration path must be specified");
> +	exit (1);
> +    }
> +
> +    config = notmuch_config_open (ctx, config_path, NULL);
> +    if (config == NULL)
> +	return 1;
> +
> +    if (notmuch_database_open (notmuch_config_get_database_path (config),
> +			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
> +	return 1;
> +
> +    srandom (seed);
> +
> +    int count;
> +    for (count = 0; count < num_messages; count++) {
> +	int j;
> +	int num_tags = random () % (max_tags + 1);
> +	int this_mid_len = random () % message_id_len + 1;
> +	const char **tag_list;
> +	char *mid;
> +	notmuch_status_t status;
> +
> +	do {
> +	    mid = random_utf8_string (ctx, this_mid_len);
> +
> +	    tag_list = talloc_realloc (ctx, NULL, const char *, num_tags + 2);
> +
> +	    tag_list[0] = "random-corpus";
> +
> +	    for (j = 0; j < num_tags; j++) {
> +		int this_tag_len = random () % tag_len + 1;
> +
> +		tag_list[j + 1] = random_utf8_string (ctx, this_tag_len);
> +	    }
> +
> +	    tag_list[j + 1] = NULL;
> +
> +	    status = notmuch_database_add_stub_message (notmuch, mid, tag_list);
> +	} while (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID);
> +
> +	if (status != NOTMUCH_STATUS_SUCCESS) {
> +	    fprintf (stderr, "error %d adding message", status);
> +	    exit (status);
> +	}
> +    }
> +
> +    notmuch_database_destroy (notmuch);
> +
> +    talloc_free (ctx);
> +
> +    return 0;
> +}
> -- 
> 1.7.10.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 3/3] test: add broken roundtrip test

[Mark Walters]

On Sun, 05 Aug 2012, david@tethera.net wrote:
> From: David Bremner <bremner@debian.org>
>
> The output of test_cmp is redirected because it is pretty horrible,
> and tends to mess up terminals. When the test is no longer marked as
> broken, this redirection should be removed.

How about piping the output to hexdump or even cat -v or something?

> ---
>  test/dump-restore |    9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/test/dump-restore b/test/dump-restore
> index 439e998..7979ebf 100755
> --- a/test/dump-restore
> +++ b/test/dump-restore
> @@ -82,4 +82,13 @@ test_begin_subtest "dump outfile -- from:cworth"
>  notmuch dump dump-outfile-dash-inbox.actual -- from:cworth
>  test_expect_equal_file dump-cworth.expected dump-outfile-dash-inbox.actual
>  
> +test_expect_success 'roundtripping random message-ids and tags' \
> +    'test_subtest_known_broken &&
> +    ${TEST_DIRECTORY}/random-corpus --config-path=${NOTMUCH_CONFIG} &&
> +    notmuch dump > EXPECTED.$test_count &&
> +    notmuch tag -random-corpus tag:random-corpus &&
> +    notmuch restore < EXPECTED.$test_count 2>/dev/null &&
> +    notmuch dump > OUTPUT.$test_count &&
> +    test_cmp EXPECTED.$test_count OUTPUT.$test_count 1>/dev/null'

Are the single quotes at the start and end of the main block meant to be
there? And with them deleted this seems to pass (but there is lots of
diff if the redirection is removed).  I am not familiar with
test_expect_success/test_cmp so don't know what to expect.

Best wishes 

Mark

> +
>  test_done
> -- 
> 1.7.10.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 3/3] test: add broken roundtrip test

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

On Wed, Aug 08 2012, Mark Walters <markwalters1009@gmail.com> wrote:
>>  test/dump-restore |    9 +++++++++
>>  1 file changed, 9 insertions(+)
>>
>> diff --git a/test/dump-restore b/test/dump-restore
>> index 439e998..7979ebf 100755
>> --- a/test/dump-restore
>> +++ b/test/dump-restore
>> @@ -82,4 +82,13 @@ test_begin_subtest "dump outfile -- from:cworth"
>>  notmuch dump dump-outfile-dash-inbox.actual -- from:cworth
>>  test_expect_equal_file dump-cworth.expected dump-outfile-dash-inbox.actual
>>  
>> +test_expect_success 'roundtripping random message-ids and tags' \
>> +    'test_subtest_known_broken &&
>> +    ${TEST_DIRECTORY}/random-corpus --config-path=${NOTMUCH_CONFIG} &&
>> +    notmuch dump > EXPECTED.$test_count &&
>> +    notmuch tag -random-corpus tag:random-corpus &&
>> +    notmuch restore < EXPECTED.$test_count 2>/dev/null &&
>> +    notmuch dump > OUTPUT.$test_count &&
>> +    test_cmp EXPECTED.$test_count OUTPUT.$test_count 1>/dev/null'
>
> Are the single quotes at the start and end of the main block meant to be
> there? And with them deleted this seems to pass (but there is lots of
> diff if the redirection is removed).  I am not familiar with
> test_expect_success/test_cmp so don't know what to expect.

I don't understand what's going on here either.  This seems like a
strange way to run these tests, as a command string to
test_expect_success.  Why not just run them directly?

I'm also worried about the test output blowing away the users terminal.
I think that should be avoided, even if we expect failures to be rare.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]