update performance tests for notmuch insert

update performance tests for notmuch insert

[David Bremner]

The first two patches are housekeeping, and the third adds some simple
memory leak testing for notmuch insert.

[PATCH 1/3] test: move generate_message, add_message into test-lib-common.sh

[David Bremner]

The plan is to use at least the former in the perf test suite.
---
 test/test-lib-common.sh | 177 ++++++++++++++++++++++++++++++++++++++++++++++++
 test/test-lib.sh        | 177 ------------------------------------------------
 2 files changed, 177 insertions(+), 177 deletions(-)

diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index ef409171..984a7c1a 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -71,6 +71,183 @@ if [ -e ./test-lib-$PLATFORM.sh ]; then
 	. ./test-lib-$PLATFORM.sh || exit 1
 fi
 
+# Generate a new message in the mail directory, with a unique message
+# ID and subject. The message is not added to the index.
+#
+# After this function returns, the filename of the generated message
+# is available as $gen_msg_filename and the message ID is available as
+# $gen_msg_id .
+#
+# This function supports named parameters with the bash syntax for
+# assigning a value to an associative array ([name]=value). The
+# supported parameters are:
+#
+#  [dir]=directory/of/choice
+#
+#	Generate the message in directory 'directory/of/choice' within
+#	the mail store. The directory will be created if necessary.
+#
+#  [filename]=name
+#
+#	Store the message in file 'name'. The default is to store it
+#	in 'msg-<count>', where <count> is three-digit number of the
+#	message.
+#
+#  [body]=text
+#
+#	Text to use as the body of the email message
+#
+#  '[from]="Some User <user@example.com>"'
+#  '[to]="Some User <user@example.com>"'
+#  '[subject]="Subject of email message"'
+#  '[date]="RFC 822 Date"'
+#
+#	Values for email headers. If not provided, default values will
+#	be generated instead.
+#
+#  '[cc]="Some User <user@example.com>"'
+#  [reply-to]=some-address
+#  [in-reply-to]=<message-id>
+#  [references]=<message-id>
+#  [content-type]=content-type-specification
+#  '[header]=full header line, including keyword'
+#
+#	Additional values for email headers. If these are not provided
+#	then the relevant headers will simply not appear in the
+#	message.
+#
+#  '[id]=message-id'
+#
+#	Controls the message-id of the created message.
+gen_msg_cnt=0
+gen_msg_filename=""
+gen_msg_id=""
+generate_message ()
+{
+    # This is our (bash-specific) magic for doing named parameters
+    local -A template="($@)"
+    local additional_headers
+
+    gen_msg_cnt=$((gen_msg_cnt + 1))
+    if [ -z "${template[filename]}" ]; then
+	gen_msg_name="msg-$(printf "%03d" $gen_msg_cnt)"
+    else
+	gen_msg_name=${template[filename]}
+    fi
+
+    if [ -z "${template[id]}" ]; then
+	gen_msg_id="${gen_msg_name%:2,*}@notmuch-test-suite"
+    else
+	gen_msg_id="${template[id]}"
+    fi
+
+    if [ -z "${template[dir]}" ]; then
+	gen_msg_filename="${MAIL_DIR}/$gen_msg_name"
+    else
+	gen_msg_filename="${MAIL_DIR}/${template[dir]}/$gen_msg_name"
+	mkdir -p "$(dirname "$gen_msg_filename")"
+    fi
+
+    if [ -z "${template[body]}" ]; then
+	template[body]="This is just a test message (#${gen_msg_cnt})"
+    fi
+
+    if [ -z "${template[from]}" ]; then
+	template[from]="Notmuch Test Suite <test_suite@notmuchmail.org>"
+    fi
+
+    if [ -z "${template[to]}" ]; then
+	template[to]="Notmuch Test Suite <test_suite@notmuchmail.org>"
+    fi
+
+    if [ -z "${template[subject]}" ]; then
+	if [ -n "$test_subtest_name" ]; then
+	    template[subject]="$test_subtest_name"
+	else
+	    template[subject]="Test message #${gen_msg_cnt}"
+	fi
+    elif [ "${template[subject]}" = "@FORCE_EMPTY" ]; then
+	template[subject]=""
+    fi
+
+    if [ -z "${template[date]}" ]; then
+	# we use decreasing timestamps here for historical reasons;
+	# the existing test suite when we converted to unique timestamps just
+	# happened to have signicantly fewer failures with that choice.
+	local date_secs=$((978709437 - gen_msg_cnt))
+	# printf %(..)T is bash 4.2+ feature. use perl fallback if needed...
+	TZ=UTC printf -v template[date] "%(%a, %d %b %Y %T %z)T" $date_secs 2>/dev/null ||
+	    template[date]=`perl -le 'use POSIX "strftime";
+				@time = gmtime '"$date_secs"';
+				print strftime "%a, %d %b %Y %T +0000", @time'`
+    fi
+
+    additional_headers=""
+    if [ ! -z "${template[header]}" ]; then
+	additional_headers="${template[header]}
+${additional_headers}"
+    fi
+
+    if [ ! -z "${template[reply-to]}" ]; then
+	additional_headers="Reply-To: ${template[reply-to]}
+${additional_headers}"
+    fi
+
+    if [ ! -z "${template[in-reply-to]}" ]; then
+	additional_headers="In-Reply-To: ${template[in-reply-to]}
+${additional_headers}"
+    fi
+
+    if [ ! -z "${template[cc]}" ]; then
+	additional_headers="Cc: ${template[cc]}
+${additional_headers}"
+    fi
+
+    if [ ! -z "${template[bcc]}" ]; then
+	additional_headers="Bcc: ${template[bcc]}
+${additional_headers}"
+    fi
+
+    if [ ! -z "${template[references]}" ]; then
+	additional_headers="References: ${template[references]}
+${additional_headers}"
+    fi
+
+    if [ ! -z "${template[content-type]}" ]; then
+	additional_headers="Content-Type: ${template[content-type]}
+${additional_headers}"
+    fi
+
+    if [ ! -z "${template[content-transfer-encoding]}" ]; then
+	additional_headers="Content-Transfer-Encoding: ${template[content-transfer-encoding]}
+${additional_headers}"
+    fi
+
+    # Note that in the way we're setting it above and using it below,
+    # `additional_headers' will also serve as the header / body separator
+    # (empty line in between).
+
+    cat <<EOF >"$gen_msg_filename"
+From: ${template[from]}
+To: ${template[to]}
+Message-Id: <${gen_msg_id}>
+Subject: ${template[subject]}
+Date: ${template[date]}
+${additional_headers}
+${template[body]}
+EOF
+}
+
+# Generate a new message and add it to the database.
+#
+# All of the arguments and return values supported by generate_message
+# are also supported here, so see that function for details.
+add_message ()
+{
+    generate_message "$@" &&
+    notmuch new > /dev/null
+}
+
 if test -n "$valgrind"
 then
 	make_symlink () {
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 7ae96c69..c505e764 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -276,183 +276,6 @@ export GNUPGHOME="${TEST_TMPDIR}/gnupg"
 trap 'trap_exit' EXIT
 trap 'trap_signal' HUP INT TERM
 
-# Generate a new message in the mail directory, with a unique message
-# ID and subject. The message is not added to the index.
-#
-# After this function returns, the filename of the generated message
-# is available as $gen_msg_filename and the message ID is available as
-# $gen_msg_id .
-#
-# This function supports named parameters with the bash syntax for
-# assigning a value to an associative array ([name]=value). The
-# supported parameters are:
-#
-#  [dir]=directory/of/choice
-#
-#	Generate the message in directory 'directory/of/choice' within
-#	the mail store. The directory will be created if necessary.
-#
-#  [filename]=name
-#
-#	Store the message in file 'name'. The default is to store it
-#	in 'msg-<count>', where <count> is three-digit number of the
-#	message.
-#
-#  [body]=text
-#
-#	Text to use as the body of the email message
-#
-#  '[from]="Some User <user@example.com>"'
-#  '[to]="Some User <user@example.com>"'
-#  '[subject]="Subject of email message"'
-#  '[date]="RFC 822 Date"'
-#
-#	Values for email headers. If not provided, default values will
-#	be generated instead.
-#
-#  '[cc]="Some User <user@example.com>"'
-#  [reply-to]=some-address
-#  [in-reply-to]=<message-id>
-#  [references]=<message-id>
-#  [content-type]=content-type-specification
-#  '[header]=full header line, including keyword'
-#
-#	Additional values for email headers. If these are not provided
-#	then the relevant headers will simply not appear in the
-#	message.
-#
-#  '[id]=message-id'
-#
-#	Controls the message-id of the created message.
-gen_msg_cnt=0
-gen_msg_filename=""
-gen_msg_id=""
-generate_message ()
-{
-    # This is our (bash-specific) magic for doing named parameters
-    local -A template="($@)"
-    local additional_headers
-
-    gen_msg_cnt=$((gen_msg_cnt + 1))
-    if [ -z "${template[filename]}" ]; then
-	gen_msg_name="msg-$(printf "%03d" $gen_msg_cnt)"
-    else
-	gen_msg_name=${template[filename]}
-    fi
-
-    if [ -z "${template[id]}" ]; then
-	gen_msg_id="${gen_msg_name%:2,*}@notmuch-test-suite"
-    else
-	gen_msg_id="${template[id]}"
-    fi
-
-    if [ -z "${template[dir]}" ]; then
-	gen_msg_filename="${MAIL_DIR}/$gen_msg_name"
-    else
-	gen_msg_filename="${MAIL_DIR}/${template[dir]}/$gen_msg_name"
-	mkdir -p "$(dirname "$gen_msg_filename")"
-    fi
-
-    if [ -z "${template[body]}" ]; then
-	template[body]="This is just a test message (#${gen_msg_cnt})"
-    fi
-
-    if [ -z "${template[from]}" ]; then
-	template[from]="Notmuch Test Suite <test_suite@notmuchmail.org>"
-    fi
-
-    if [ -z "${template[to]}" ]; then
-	template[to]="Notmuch Test Suite <test_suite@notmuchmail.org>"
-    fi
-
-    if [ -z "${template[subject]}" ]; then
-	if [ -n "$test_subtest_name" ]; then
-	    template[subject]="$test_subtest_name"
-	else
-	    template[subject]="Test message #${gen_msg_cnt}"
-	fi
-    elif [ "${template[subject]}" = "@FORCE_EMPTY" ]; then
-	template[subject]=""
-    fi
-
-    if [ -z "${template[date]}" ]; then
-	# we use decreasing timestamps here for historical reasons;
-	# the existing test suite when we converted to unique timestamps just
-	# happened to have signicantly fewer failures with that choice.
-	local date_secs=$((978709437 - gen_msg_cnt))
-	# printf %(..)T is bash 4.2+ feature. use perl fallback if needed...
-	TZ=UTC printf -v template[date] "%(%a, %d %b %Y %T %z)T" $date_secs 2>/dev/null ||
-	    template[date]=`perl -le 'use POSIX "strftime";
-				@time = gmtime '"$date_secs"';
-				print strftime "%a, %d %b %Y %T +0000", @time'`
-    fi
-
-    additional_headers=""
-    if [ ! -z "${template[header]}" ]; then
-	additional_headers="${template[header]}
-${additional_headers}"
-    fi
-
-    if [ ! -z "${template[reply-to]}" ]; then
-	additional_headers="Reply-To: ${template[reply-to]}
-${additional_headers}"
-    fi
-
-    if [ ! -z "${template[in-reply-to]}" ]; then
-	additional_headers="In-Reply-To: ${template[in-reply-to]}
-${additional_headers}"
-    fi
-
-    if [ ! -z "${template[cc]}" ]; then
-	additional_headers="Cc: ${template[cc]}
-${additional_headers}"
-    fi
-
-    if [ ! -z "${template[bcc]}" ]; then
-	additional_headers="Bcc: ${template[bcc]}
-${additional_headers}"
-    fi
-
-    if [ ! -z "${template[references]}" ]; then
-	additional_headers="References: ${template[references]}
-${additional_headers}"
-    fi
-
-    if [ ! -z "${template[content-type]}" ]; then
-	additional_headers="Content-Type: ${template[content-type]}
-${additional_headers}"
-    fi
-
-    if [ ! -z "${template[content-transfer-encoding]}" ]; then
-	additional_headers="Content-Transfer-Encoding: ${template[content-transfer-encoding]}
-${additional_headers}"
-    fi
-
-    # Note that in the way we're setting it above and using it below,
-    # `additional_headers' will also serve as the header / body separator
-    # (empty line in between).
-
-    cat <<EOF >"$gen_msg_filename"
-From: ${template[from]}
-To: ${template[to]}
-Message-Id: <${gen_msg_id}>
-Subject: ${template[subject]}
-Date: ${template[date]}
-${additional_headers}
-${template[body]}
-EOF
-}
-
-# Generate a new message and add it to the database.
-#
-# All of the arguments and return values supported by generate_message
-# are also supported here, so see that function for details.
-add_message ()
-{
-    generate_message "$@" &&
-    notmuch new > /dev/null
-}
-
 # Deliver a message with emacs and add it to the database
 #
 # Uses emacs to generate and deliver a message to the mail store.
-- 
2.14.1

[PATCH 2/3] perf-test: renumber tests

[David Bremner]

One test per number so ordering is clear.
---
 performance-test/{M04-reindex.sh => M05-reindex.sh} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename performance-test/{M04-reindex.sh => M05-reindex.sh} (100%)

diff --git a/performance-test/M04-reindex.sh b/performance-test/M05-reindex.sh
similarity index 100%
rename from performance-test/M04-reindex.sh
rename to performance-test/M05-reindex.sh
-- 
2.14.1

[PATCH 3/3] perf-test: add memory test for notmuch-insert

[David Bremner]

In the future it might be worthwhile selecting corpus messages to
insert, but that seems a bit overcomplicated for now
---
 performance-test/M06-insert.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100755 performance-test/M06-insert.sh

diff --git a/performance-test/M06-insert.sh b/performance-test/M06-insert.sh
new file mode 100755
index 00000000..9bca199e
--- /dev/null
+++ b/performance-test/M06-insert.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+test_description='search'
+
+. ./perf-test-lib.sh || exit 1
+
+memory_start
+
+mkdir -p "$MAIL_DIR"/{cur,new,tmp}
+
+for count in $(seq 1 20); do
+    generate_message "[file]=\"insert-$count\"" "[dir]='tmp/'"
+    memory_run "insert $count" "notmuch insert < $gen_msg_filename"
+done
+
+memory_done
-- 
2.14.1

Re: [PATCH 3/3] perf-test: add memory test for notmuch-insert

[Tomi Ollila]

On Fri, Aug 18 2017, David Bremner wrote:

> In the future it might be worthwhile selecting corpus messages to
> insert, but that seems a bit overcomplicated for now
> ---
>  performance-test/M06-insert.sh | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
>  create mode 100755 performance-test/M06-insert.sh
>
> diff --git a/performance-test/M06-insert.sh b/performance-test/M06-insert.sh
> new file mode 100755
> index 00000000..9bca199e
> --- /dev/null
> +++ b/performance-test/M06-insert.sh
> @@ -0,0 +1,16 @@
> +#!/bin/bash
> +
> +test_description='search'
> +
> +. ./perf-test-lib.sh || exit 1
> +
> +memory_start
> +
> +mkdir -p "$MAIL_DIR"/{cur,new,tmp}
> +
> +for count in $(seq 1 20); do

Since we use bash, {1..20} works. And IIRC some systems did not have
seq(1) (but some alternative).

LGTM (whole series) in any case.

Tomi


> +    generate_message "[file]=\"insert-$count\"" "[dir]='tmp/'"
> +    memory_run "insert $count" "notmuch insert < $gen_msg_filename"
> +done
> +
> +memory_done
> -- 
> 2.14.1

Re: [PATCH 3/3] perf-test: add memory test for notmuch-insert

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

>
> Since we use bash, {1..20} works. And IIRC some systems did not have
> seq(1) (but some alternative).
>
> LGTM (whole series) in any case.
>

pushed the bashy version,

d