Give test messages unique timestamps

Give test messages unique timestamps

[David Bremner]

It turns out that having all messages with the same timestamp leads to
some nondeterministic behaviour of the test suite (see e.g.https://buildd.debian.org/status/fetch.php?pkg=notmuch&arch=i386&ver=0.17%7Erc3-1&stamp=1386412394) 

Decreasing timestamps are chosen to reduce the number of changes
needed to the test suite.  The first three patches are independent of
this choice.


 test/emacs           | 12 +++++++-----
 test/emacs-show      | 10 ++++++----
 test/encoding        |  8 ++++----
 test/excludes        |  4 ++--
 test/maildir-sync    | 15 +++++----------
 test/missing-headers |  8 +++++---
 test/multipart       |  2 +-
 test/raw             |  8 ++++----
 test/test-lib.sh     | 15 ++++++++++++---
 9 files changed, 46 insertions(+), 36 deletions(-)

[PATCH 1/4] test: pass expected output through json_sanitize in 2 places

[David Bremner]

This makes the tests more robust against changes in the
sanitization rules.
---
 test/missing-headers | 8 +++++---
 test/multipart       | 2 +-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/test/missing-headers b/test/missing-headers
index 43e861b..cb38301 100755
--- a/test/missing-headers
+++ b/test/missing-headers
@@ -95,7 +95,7 @@ Body
 
 test_begin_subtest "Show: json"
 output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
-test_expect_equal_json "$output" '
+expected=$(notmuch_json_show_sanitize <<EOF
 [
     [
         [
@@ -156,7 +156,9 @@ test_expect_equal_json "$output" '
             []
         ]
     ]
-]'
-
+]
+EOF
+)
+test_expect_equal_json "$output" "$expected"
 
 test_done
diff --git a/test/multipart b/test/multipart
index b40fa2c..73f8fdd 100755
--- a/test/multipart
+++ b/test/multipart
@@ -594,7 +594,7 @@ test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "'notmuch reply' to a multipart message with json format"
 notmuch reply --format=json 'id:87liy5ap00.fsf@yoom.home.cworth.org' | notmuch_json_show_sanitize >OUTPUT
-cat <<EOF >EXPECTED
+cat <<EOF | notmuch_json_show_sanitize >EXPECTED
 {"reply-headers": {"Subject": "Re: Multipart message",
  "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
  "To": "Carl Worth <cworth@cworth.org>, cworth@cworth.org",
-- 
1.8.4.3

[PATCH 2/4] test: sanitize Date and timestamp fields in json

[David Bremner]

Eventually we want test messages to have distinct dates to avoid
reproducability problems. This sanitization will prevent some test
failures when that change is made.

Replace the use of a local function in maildir-sync with
notmuch_json_show_sanitize
---
 test/maildir-sync | 15 +++++----------
 test/test-lib.sh  |  4 +++-
 2 files changed, 8 insertions(+), 11 deletions(-)

diff --git a/test/maildir-sync b/test/maildir-sync
index 33d2c58..3186e70 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -4,11 +4,6 @@ test_description="maildir synchronization"
 
 . ./test-lib.sh
 
-# Avoid including the local value of MAIL_DIR in the result.
-filter_show_json() {
-    sed -e "s|${MAIL_DIR}/|MAIL_DIR/|"
-}
-
 # Create the expected maildir structure
 mkdir $MAIL_DIR/cur
 mkdir $MAIL_DIR/new
@@ -40,18 +35,18 @@ output=$(cd ${MAIL_DIR}/cur; ls -1 adding-replied*)
 test_expect_equal "$output" "adding-replied-tag:2,RS"
 
 test_begin_subtest "notmuch show works with renamed file (without notmuch new)"
-output=$(notmuch show --format=json id:${gen_msg_id} | filter_show_json)
-test_expect_equal_json "$output" '[[[{"id": "adding-replied-tag@notmuch-test-suite",
+output=$(notmuch show --format=json id:${gen_msg_id} | notmuch_json_show_sanitize)
+test_expect_equal_json "$output" '[[[{"id": "XXXXX",
 "match": true,
 "excluded": false,
-"filename": "MAIL_DIR/cur/adding-replied-tag:2,RS",
-"timestamp": 978709437,
+"filename": "YYYYY",
+"timestamp": 42,
 "date_relative": "2001-01-05",
 "tags": ["inbox","replied"],
 "headers": {"Subject": "Adding replied tag",
 "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
 "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
-"Date": "Fri, 05 Jan 2001 15:43:57 +0000"},
+"Date": "GENERATED_DATE"},
 "body": [{"id": 1,
 "content-type": "text/plain",
 "content": "This is just a test message (#3)\n"}]},
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 8611ba5..9d2f805 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -625,7 +625,9 @@ notmuch_json_show_sanitize ()
 {
     sed \
 	-e 's|"id": "[^"]*",|"id": "XXXXX",|g' \
-	-e 's|"filename": "/[^"]*",|"filename": "YYYYY",|g'
+	-e 's|"Date": "Fri[^"]*0000"|"Date": "GENERATED_DATE"|g' \
+	-e 's|"filename": "/[^"]*",|"filename": "YYYYY",|g' \
+	-e 's|"timestamp": 97.......|"timestamp": 42|g'
 }
 
 notmuch_emacs_error_sanitize ()
-- 
1.8.4.3

[PATCH 3/4] test: sanitize dates in emacs, raw, and text output

[David Bremner]

add a new function notmuch_date_sanitize for rfc822-ish things. Add
date sanitization to notmuch_show_sanitize_all and use it more places.

This is all in aid of a transition to unique timestamps on messages.
---
 test/emacs       | 12 +++++++-----
 test/emacs-show  | 10 ++++++----
 test/encoding    |  8 ++++----
 test/raw         |  8 ++++----
 test/test-lib.sh |  9 ++++++++-
 5 files changed, 29 insertions(+), 18 deletions(-)

diff --git a/test/emacs b/test/emacs
index 456435c..1b1ebe3 100755
--- a/test/emacs
+++ b/test/emacs
@@ -86,15 +86,16 @@ add_message "[subject]=\"message-with-invalid-from\"" \
 	    "[from]=\"\\\"Invalid \\\" From\\\" <test_suite@notmuchmail.org>\""
 thread=$(notmuch search --output=threads subject:message-with-invalid-from)
 test_emacs "(notmuch-show \"$thread\")
-	    (test-output)"
+	    (test-output \"OUTPUT.raw\")"
 cat <<EOF >EXPECTED
 "Invalid " (2001-01-05) (inbox)
 Subject: message-with-invalid-from
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 This is just a test message (#1)
 EOF
+notmuch_date_sanitize < OUTPUT.raw > OUTPUT
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Navigation of notmuch-search to thread view"
@@ -605,11 +606,11 @@ Q: Why is top-posting such a bad thing?
 A: Top-posting.
 Q: What is the most annoying thing in e-mail?"'
 test_emacs "(notmuch-show \"top-posting\")
-	    (test-visible-output)"
+	    (test-visible-output \"OUTPUT.raw\")"
 echo "Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
 Subject: The problem with top-posting
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 A: Because it messes up the order in which people normally read text.
 Q: Why is top-posting such a bad thing?
@@ -618,13 +619,14 @@ Q: What is the most annoying thing in e-mail?
 Top Poster <top@poster.com> (2001-01-05) (inbox unread)
 Subject: Re: The problem with top-posting
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 Thanks for the advice! I will be sure to put it to good use.
 
 -Top Poster
 
 [ 9-line hidden original message. Click/Enter to show. ]" > EXPECTED
+notmuch_date_sanitize < OUTPUT.raw > OUTPUT
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Hiding message in notmuch-show view"
diff --git a/test/emacs-show b/test/emacs-show
index ec86333..2a3a535 100755
--- a/test/emacs-show
+++ b/test/emacs-show
@@ -19,13 +19,14 @@ cat <<EOF >EXPECTED
 Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
 Subject: Hiding Original Message region at beginning of a message
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 [ 2-line hidden original message. Click/Enter to show. ]
 EOF
 
 test_emacs "(notmuch-show \"id:$message_id\")
-	    (test-visible-output)"
+	    (test-visible-output \"OUTPUT.raw\")"
+notmuch_date_sanitize < OUTPUT.raw > OUTPUT
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Bare subject #1"
@@ -130,12 +131,12 @@ mid:abc%20def
 mid:abc. mid:abc, mid:abc;"'
 test_emacs '(notmuch-show "id:'$gen_msg_id'")
 	(notmuch-test-mark-links)
-	(test-visible-output)'
+	(test-visible-output "OUTPUT.raw")'
 cat <<EOF >EXPECTED
 Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
 Subject: id buttonization
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 <<id:abc>>
 <<id:abc.def>>. <<id:abc,def>>, <<id:abc;def>>; <<id:abc:def>>:
@@ -159,6 +160,7 @@ cid:xxx
 <<mid:abc%20def>>
 <<mid:abc>>. <<mid:abc>>, <<mid:abc>>;
 EOF
+notmuch_date_sanitize < OUTPUT.raw > OUTPUT
 test_expect_equal_file OUTPUT EXPECTED
 
 
diff --git a/test/encoding b/test/encoding
index 8609652..b6c86bf 100755
--- a/test/encoding
+++ b/test/encoding
@@ -5,14 +5,14 @@ test_description="encoding issues"
 test_begin_subtest "Message with text of unknown charset"
 add_message '[content-type]="text/plain; charset=unknown-8bit"' \
 	    "[body]=irrelevant"
-output=$(notmuch show id:${gen_msg_id} 2>&1 | notmuch_show_sanitize)
-test_expect_equal "$output" "\fmessage{ id:msg-001@notmuch-test-suite depth:0 match:1 excluded:0 filename:/XXX/mail/msg-001
+output=$(notmuch show id:${gen_msg_id} 2>&1 | notmuch_show_sanitize_all)
+test_expect_equal "$output" "\fmessage{ id:XXXXX depth:0 match:1 excluded:0 filename:XXXXX
 \fheader{
 Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox unread)
 Subject: Message with text of unknown charset
 From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 \fheader}
 \fbody{
 \fpart{ ID: 1, Content-type: text/plain
@@ -26,7 +26,7 @@ add_message '[content-type]="text/plain; charset=iso-8859-2"' \
             '[content-transfer-encoding]=8bit' \
             '[subject]="ISO-8859-2 encoded message"' \
             "[body]=$'Czech word tu\350\362\341\350\350\355 means pinguin\'s.'" # ISO-8859-2 characters are generated by shell's escape sequences
-output=$(notmuch search tučňáččí 2>&1 | notmuch_show_sanitize)
+output=$(notmuch search tučňáččí 2>&1 | notmuch_show_sanitize_all)
 test_expect_equal "$output" "thread:0000000000000002   2001-01-05 [1/1] Notmuch Test Suite; ISO-8859-2 encoded message (inbox unread)"
 
 test_begin_subtest "RFC 2047 encoded word with spaces"
diff --git a/test/raw b/test/raw
index de0b867..daf5735 100755
--- a/test/raw
+++ b/test/raw
@@ -11,22 +11,22 @@ output=$(notmuch show --format=raw "*" 2>&1)
 test_expect_equal "$output" "Error: search term did not match precisely one message."
 
 test_begin_subtest "Show a raw message"
-output=$(notmuch show --format=raw id:msg-001@notmuch-test-suite)
+output=$(notmuch show --format=raw id:msg-001@notmuch-test-suite | notmuch_date_sanitize)
 test_expect_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
 Message-Id: <msg-001@notmuch-test-suite>
 Subject: Test message #1
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 This is just a test message (#1)"
 
 test_begin_subtest "Show another raw message"
-output=$(notmuch show --format=raw id:msg-002@notmuch-test-suite)
+output=$(notmuch show --format=raw id:msg-002@notmuch-test-suite | notmuch_date_sanitize)
 test_expect_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
 Message-Id: <msg-002@notmuch-test-suite>
 Subject: Test message #2
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 This is just a test message (#2)"
 
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 9d2f805..d238c66 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -618,7 +618,8 @@ notmuch_show_sanitize_all ()
 {
     sed \
 	-e 's| filename:.*| filename:XXXXX|' \
-	-e 's| id:[^ ]* | id:XXXXX |'
+	-e 's| id:[^ ]* | id:XXXXX |' \
+	-e 's|Date: Fri, 05 Jan 2001.*|Date: GENERATED_DATE|'
 }
 
 notmuch_json_show_sanitize ()
@@ -641,6 +642,12 @@ notmuch_emacs_error_sanitize ()
 	-e 's/^\[.*\]$/[XXX]/' \
 	-e "s|^\(command: \)\{0,1\}/.*/$command|\1YYY/$command|"
 }
+
+notmuch_date_sanitize ()
+{
+    sed \
+	-e 's/^Date: Fri, 05 Jan 2001 .*/Date: GENERATED_DATE/'
+}
 # End of notmuch helper functions
 
 # Use test_set_prereq to tell that a particular prerequisite is available.
-- 
1.8.4.3

[PATCH 4/4] test: give unique timestamps to messages

[David Bremner]

The choice of decreasing timestamps is a hack which reduces the number
of existing tests which fail.  This can be changed to increasing
if/when somebody wants update another 47 tests.
---
 test/excludes    | 4 ++--
 test/test-lib.sh | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/test/excludes b/test/excludes
index f1ae9ea..8bbbc2d 100755
--- a/test/excludes
+++ b/test/excludes
@@ -67,7 +67,7 @@ thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inb
 test_begin_subtest "Search, don't exclude \"deleted\" messages when --exclude=flag specified"
 output=$(notmuch search --exclude=flag subject:deleted | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
-thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inbox unread)"
+thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Deleted (deleted inbox unread)"
 
 test_begin_subtest "Search, don't exclude \"deleted\" messages from search if not configured"
 notmuch config set search.exclude_tags
@@ -152,7 +152,7 @@ ${matching_message_ids[5]}"
 test_begin_subtest "Search, exclude=flag (thread summary)"
 output=$(notmuch search --exclude=flag tag:test | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: single match: reply 2 (deleted inbox test unread)
-thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 4 (deleted inbox test unread)
+thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 2 (deleted inbox test unread)
 thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; Some messages excluded: single excluded match: reply 3 (deleted inbox test unread)
 thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; Some messages excluded: single non-excluded match: reply 4 (deleted inbox test unread)
 thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; No messages excluded: single match: reply 3 (inbox test unread)"
diff --git a/test/test-lib.sh b/test/test-lib.sh
index d238c66..792e477 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -356,7 +356,7 @@ generate_message ()
     fi
 
     if [ -z "${template[date]}" ]; then
-	template[date]="Fri, 05 Jan 2001 15:43:57 +0000"
+	template[date]=$(date -Ru -d@$((978709437 - gen_msg_cnt)))
     fi
 
     additional_headers=""
-- 
1.8.4.3

Re: [PATCH 1/4] test: pass expected output through json_sanitize in 2 places

[Tomi Ollila]

On Sun, Dec 08 2013, David Bremner <david@tethera.net> wrote:

> This makes the tests more robust against changes in the
> sanitization rules.
> ---
>  test/missing-headers | 8 +++++---
>  test/multipart       | 2 +-
>  2 files changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/test/missing-headers b/test/missing-headers
> index 43e861b..cb38301 100755
> --- a/test/missing-headers
> +++ b/test/missing-headers
> @@ -95,7 +95,7 @@ Body
>  
>  test_begin_subtest "Show: json"
>  output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
> -test_expect_equal_json "$output" '
> +expected=$(notmuch_json_show_sanitize <<EOF
>  [
>      [
>          [
> @@ -156,7 +156,9 @@ test_expect_equal_json "$output" '
>              []
>          ]
>      ]
> -]'
> -
> +]
> +EOF
> +)
> +test_expect_equal_json "$output" "$expected"
>  
>  test_done
> diff --git a/test/multipart b/test/multipart
> index b40fa2c..73f8fdd 100755
> --- a/test/multipart
> +++ b/test/multipart
> @@ -594,7 +594,7 @@ test_expect_equal_file OUTPUT EXPECTED
>  
>  test_begin_subtest "'notmuch reply' to a multipart message with json format"
>  notmuch reply --format=json 'id:87liy5ap00.fsf@yoom.home.cworth.org' | notmuch_json_show_sanitize >OUTPUT
> -cat <<EOF >EXPECTED
> +cat <<EOF | notmuch_json_show_sanitize >EXPECTED

notmuch_json_show_sanitize <<EOF >EXPECTED

would do it.

>  {"reply-headers": {"Subject": "Re: Multipart message",
>   "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
>   "To": "Carl Worth <cworth@cworth.org>, cworth@cworth.org",
> -- 
> 1.8.4.3
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 2/4] test: sanitize Date and timestamp fields in json

[Tomi Ollila]

On Sun, Dec 08 2013, David Bremner <david@tethera.net> wrote:

> Eventually we want test messages to have distinct dates to avoid
> reproducability problems. This sanitization will prevent some test
> failures when that change is made.
>
> Replace the use of a local function in maildir-sync with
> notmuch_json_show_sanitize
> ---
>  test/maildir-sync | 15 +++++----------
>  test/test-lib.sh  |  4 +++-
>  2 files changed, 8 insertions(+), 11 deletions(-)
>
> diff --git a/test/maildir-sync b/test/maildir-sync
> index 33d2c58..3186e70 100755
> --- a/test/maildir-sync
> +++ b/test/maildir-sync
> @@ -4,11 +4,6 @@ test_description="maildir synchronization"
>  
>  . ./test-lib.sh
>  
> -# Avoid including the local value of MAIL_DIR in the result.
> -filter_show_json() {
> -    sed -e "s|${MAIL_DIR}/|MAIL_DIR/|"
> -}
> -
>  # Create the expected maildir structure
>  mkdir $MAIL_DIR/cur
>  mkdir $MAIL_DIR/new
> @@ -40,18 +35,18 @@ output=$(cd ${MAIL_DIR}/cur; ls -1 adding-replied*)
>  test_expect_equal "$output" "adding-replied-tag:2,RS"
>  
>  test_begin_subtest "notmuch show works with renamed file (without notmuch new)"
> -output=$(notmuch show --format=json id:${gen_msg_id} | filter_show_json)
> -test_expect_equal_json "$output" '[[[{"id": "adding-replied-tag@notmuch-test-suite",
> +output=$(notmuch show --format=json id:${gen_msg_id} | notmuch_json_show_sanitize)
> +test_expect_equal_json "$output" '[[[{"id": "XXXXX",
>  "match": true,
>  "excluded": false,
> -"filename": "MAIL_DIR/cur/adding-replied-tag:2,RS",
> -"timestamp": 978709437,
> +"filename": "YYYYY",
> +"timestamp": 42,
>  "date_relative": "2001-01-05",
>  "tags": ["inbox","replied"],
>  "headers": {"Subject": "Adding replied tag",
>  "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
>  "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> -"Date": "Fri, 05 Jan 2001 15:43:57 +0000"},
> +"Date": "GENERATED_DATE"},
>  "body": [{"id": 1,
>  "content-type": "text/plain",
>  "content": "This is just a test message (#3)\n"}]},
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 8611ba5..9d2f805 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -625,7 +625,9 @@ notmuch_json_show_sanitize ()
>  {
>      sed \
>  	-e 's|"id": "[^"]*",|"id": "XXXXX",|g' \
> -	-e 's|"filename": "/[^"]*",|"filename": "YYYYY",|g'
> +	-e 's|"Date": "Fri[^"]*0000"|"Date": "GENERATED_DATE"|g' \

Minor nit: The RE for changing to GENERATED_DATE is more different than
required here and elsewhere in this patch series. Both could use 
'Fri, 05 Jan 2001' and '0000' parts...

> +	-e 's|"filename": "/[^"]*",|"filename": "YYYYY",|g' \
> +	-e 's|"timestamp": 97.......|"timestamp": 42|g'
>  }
>  
>  notmuch_emacs_error_sanitize ()
> -- 
> 1.8.4.3
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/4] test: give unique timestamps to messages

[Tomi Ollila]

On Sun, Dec 08 2013, David Bremner <david@tethera.net> wrote:

> The choice of decreasing timestamps is a hack which reduces the number
> of existing tests which fail.  This can be changed to increasing
> if/when somebody wants update another 47 tests.
> ---
>  test/excludes    | 4 ++--
>  test/test-lib.sh | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/test/excludes b/test/excludes
> index f1ae9ea..8bbbc2d 100755
> --- a/test/excludes
> +++ b/test/excludes
> @@ -67,7 +67,7 @@ thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inb
>  test_begin_subtest "Search, don't exclude \"deleted\" messages when --exclude=flag specified"
>  output=$(notmuch search --exclude=flag subject:deleted | notmuch_search_sanitize)
>  test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
> -thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inbox unread)"
> +thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Deleted (deleted inbox unread)"
>  
>  test_begin_subtest "Search, don't exclude \"deleted\" messages from search if not configured"
>  notmuch config set search.exclude_tags
> @@ -152,7 +152,7 @@ ${matching_message_ids[5]}"
>  test_begin_subtest "Search, exclude=flag (thread summary)"
>  output=$(notmuch search --exclude=flag tag:test | notmuch_search_sanitize)
>  test_expect_equal "$output" "thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: single match: reply 2 (deleted inbox test unread)
> -thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 4 (deleted inbox test unread)
> +thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 2 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; Some messages excluded: single excluded match: reply 3 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; Some messages excluded: single non-excluded match: reply 4 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; No messages excluded: single match: reply 3 (inbox test unread)"
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index d238c66..792e477 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -356,7 +356,7 @@ generate_message ()
>      fi
>  
>      if [ -z "${template[date]}" ]; then
> -	template[date]="Fri, 05 Jan 2001 15:43:57 +0000"
> +	template[date]=$(date -Ru -d@$((978709437 - gen_msg_cnt)))

I am afraid this is not portable (although not sure). This could work:

_s=$((3599 - gen_msg_cnt))
template[date]="Fri, 05 Jan 2001 15:$((_s / 60)):$((_s % 60)) +0000"

>      fi
>  
>      additional_headers=""

Tomi (comments in hurry, after more careful examination done earlyer :)

Re: [PATCH 3/4] test: sanitize dates in emacs, raw, and text output

[Austin Clements]

Quoth David Bremner on Dec 08 at 11:52 pm:
> add a new function notmuch_date_sanitize for rfc822-ish things. Add
> date sanitization to notmuch_show_sanitize_all and use it more places.
> 
> This is all in aid of a transition to unique timestamps on messages.
> ---
>  test/emacs       | 12 +++++++-----
>  test/emacs-show  | 10 ++++++----
>  test/encoding    |  8 ++++----
>  test/raw         |  8 ++++----
>  test/test-lib.sh |  9 ++++++++-
>  5 files changed, 29 insertions(+), 18 deletions(-)
> 
> diff --git a/test/emacs b/test/emacs
> index 456435c..1b1ebe3 100755
> --- a/test/emacs
> +++ b/test/emacs
> @@ -86,15 +86,16 @@ add_message "[subject]=\"message-with-invalid-from\"" \
>  	    "[from]=\"\\\"Invalid \\\" From\\\" <test_suite@notmuchmail.org>\""
>  thread=$(notmuch search --output=threads subject:message-with-invalid-from)
>  test_emacs "(notmuch-show \"$thread\")
> -	    (test-output)"
> +	    (test-output \"OUTPUT.raw\")"
>  cat <<EOF >EXPECTED
>  "Invalid " (2001-01-05) (inbox)
>  Subject: message-with-invalid-from
>  To: Notmuch Test Suite <test_suite@notmuchmail.org>
> -Date: Fri, 05 Jan 2001 15:43:57 +0000
> +Date: GENERATED_DATE
>  
>  This is just a test message (#1)
>  EOF
> +notmuch_date_sanitize < OUTPUT.raw > OUTPUT
>  test_expect_equal_file OUTPUT EXPECTED
>  
>  test_begin_subtest "Navigation of notmuch-search to thread view"
> @@ -605,11 +606,11 @@ Q: Why is top-posting such a bad thing?
>  A: Top-posting.
>  Q: What is the most annoying thing in e-mail?"'
>  test_emacs "(notmuch-show \"top-posting\")
> -	    (test-visible-output)"
> +	    (test-visible-output \"OUTPUT.raw\")"
>  echo "Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
>  Subject: The problem with top-posting
>  To: Notmuch Test Suite <test_suite@notmuchmail.org>
> -Date: Fri, 05 Jan 2001 15:43:57 +0000
> +Date: GENERATED_DATE
>  
>  A: Because it messes up the order in which people normally read text.
>  Q: Why is top-posting such a bad thing?
> @@ -618,13 +619,14 @@ Q: What is the most annoying thing in e-mail?
>  Top Poster <top@poster.com> (2001-01-05) (inbox unread)
>  Subject: Re: The problem with top-posting
>  To: Notmuch Test Suite <test_suite@notmuchmail.org>
> -Date: Fri, 05 Jan 2001 15:43:57 +0000
> +Date: GENERATED_DATE
>  
>  Thanks for the advice! I will be sure to put it to good use.
>  
>  -Top Poster
>  
>  [ 9-line hidden original message. Click/Enter to show. ]" > EXPECTED
> +notmuch_date_sanitize < OUTPUT.raw > OUTPUT
>  test_expect_equal_file OUTPUT EXPECTED
>  
>  test_begin_subtest "Hiding message in notmuch-show view"
> diff --git a/test/emacs-show b/test/emacs-show
> index ec86333..2a3a535 100755
> --- a/test/emacs-show
> +++ b/test/emacs-show
> @@ -19,13 +19,14 @@ cat <<EOF >EXPECTED
>  Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
>  Subject: Hiding Original Message region at beginning of a message
>  To: Notmuch Test Suite <test_suite@notmuchmail.org>
> -Date: Fri, 05 Jan 2001 15:43:57 +0000
> +Date: GENERATED_DATE
>  
>  [ 2-line hidden original message. Click/Enter to show. ]
>  EOF
>  
>  test_emacs "(notmuch-show \"id:$message_id\")
> -	    (test-visible-output)"
> +	    (test-visible-output \"OUTPUT.raw\")"
> +notmuch_date_sanitize < OUTPUT.raw > OUTPUT
>  test_expect_equal_file OUTPUT EXPECTED
>  
>  test_begin_subtest "Bare subject #1"
> @@ -130,12 +131,12 @@ mid:abc%20def
>  mid:abc. mid:abc, mid:abc;"'
>  test_emacs '(notmuch-show "id:'$gen_msg_id'")
>  	(notmuch-test-mark-links)
> -	(test-visible-output)'
> +	(test-visible-output "OUTPUT.raw")'
>  cat <<EOF >EXPECTED
>  Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
>  Subject: id buttonization
>  To: Notmuch Test Suite <test_suite@notmuchmail.org>
> -Date: Fri, 05 Jan 2001 15:43:57 +0000
> +Date: GENERATED_DATE
>  
>  <<id:abc>>
>  <<id:abc.def>>. <<id:abc,def>>, <<id:abc;def>>; <<id:abc:def>>:
> @@ -159,6 +160,7 @@ cid:xxx
>  <<mid:abc%20def>>
>  <<mid:abc>>. <<mid:abc>>, <<mid:abc>>;
>  EOF
> +notmuch_date_sanitize < OUTPUT.raw > OUTPUT
>  test_expect_equal_file OUTPUT EXPECTED
>  
>  
> diff --git a/test/encoding b/test/encoding
> index 8609652..b6c86bf 100755
> --- a/test/encoding
> +++ b/test/encoding
> @@ -5,14 +5,14 @@ test_description="encoding issues"
>  test_begin_subtest "Message with text of unknown charset"
>  add_message '[content-type]="text/plain; charset=unknown-8bit"' \
>  	    "[body]=irrelevant"
> -output=$(notmuch show id:${gen_msg_id} 2>&1 | notmuch_show_sanitize)
> -test_expect_equal "$output" "\fmessage{ id:msg-001@notmuch-test-suite depth:0 match:1 excluded:0 filename:/XXX/mail/msg-001
> +output=$(notmuch show id:${gen_msg_id} 2>&1 | notmuch_show_sanitize_all)

This may be orthogonal to this series, but I don't get why we have
both notmuch_show_sanitize and notmuch_show_sanitize_all (especially
when the "all" version is not obviously a superset of the non-"all"
version!)  Do you have a sense for whether these two functions could
be easily merged?

> +test_expect_equal "$output" "\fmessage{ id:XXXXX depth:0 match:1 excluded:0 filename:XXXXX
>  \fheader{
>  Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox unread)
>  Subject: Message with text of unknown charset
>  From: Notmuch Test Suite <test_suite@notmuchmail.org>
>  To: Notmuch Test Suite <test_suite@notmuchmail.org>
> -Date: Fri, 05 Jan 2001 15:43:57 +0000
> +Date: GENERATED_DATE
>  \fheader}
>  \fbody{
>  \fpart{ ID: 1, Content-type: text/plain
> @@ -26,7 +26,7 @@ add_message '[content-type]="text/plain; charset=iso-8859-2"' \
>              '[content-transfer-encoding]=8bit' \
>              '[subject]="ISO-8859-2 encoded message"' \
>              "[body]=$'Czech word tu\350\362\341\350\350\355 means pinguin\'s.'" # ISO-8859-2 characters are generated by shell's escape sequences
> -output=$(notmuch search tučňáččí 2>&1 | notmuch_show_sanitize)
> +output=$(notmuch search tučňáččí 2>&1 | notmuch_show_sanitize_all)
>  test_expect_equal "$output" "thread:0000000000000002   2001-01-05 [1/1] Notmuch Test Suite; ISO-8859-2 encoded message (inbox unread)"
>  
>  test_begin_subtest "RFC 2047 encoded word with spaces"
> diff --git a/test/raw b/test/raw
> index de0b867..daf5735 100755
> --- a/test/raw
> +++ b/test/raw
> @@ -11,22 +11,22 @@ output=$(notmuch show --format=raw "*" 2>&1)
>  test_expect_equal "$output" "Error: search term did not match precisely one message."
>  
>  test_begin_subtest "Show a raw message"
> -output=$(notmuch show --format=raw id:msg-001@notmuch-test-suite)
> +output=$(notmuch show --format=raw id:msg-001@notmuch-test-suite | notmuch_date_sanitize)
>  test_expect_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
>  To: Notmuch Test Suite <test_suite@notmuchmail.org>
>  Message-Id: <msg-001@notmuch-test-suite>
>  Subject: Test message #1
> -Date: Fri, 05 Jan 2001 15:43:57 +0000
> +Date: GENERATED_DATE
>  
>  This is just a test message (#1)"
>  
>  test_begin_subtest "Show another raw message"
> -output=$(notmuch show --format=raw id:msg-002@notmuch-test-suite)
> +output=$(notmuch show --format=raw id:msg-002@notmuch-test-suite | notmuch_date_sanitize)
>  test_expect_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
>  To: Notmuch Test Suite <test_suite@notmuchmail.org>
>  Message-Id: <msg-002@notmuch-test-suite>
>  Subject: Test message #2
> -Date: Fri, 05 Jan 2001 15:43:57 +0000
> +Date: GENERATED_DATE
>  
>  This is just a test message (#2)"
>  
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 9d2f805..d238c66 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -618,7 +618,8 @@ notmuch_show_sanitize_all ()
>  {
>      sed \
>  	-e 's| filename:.*| filename:XXXXX|' \
> -	-e 's| id:[^ ]* | id:XXXXX |'
> +	-e 's| id:[^ ]* | id:XXXXX |' \
> +	-e 's|Date: Fri, 05 Jan 2001.*|Date: GENERATED_DATE|'

Rather than (nearly) duplicating the regexp, does it work to pipe this
through notmuch_date_sanitize?  Or do the minor differences matter?

>  }
>  
>  notmuch_json_show_sanitize ()
> @@ -641,6 +642,12 @@ notmuch_emacs_error_sanitize ()
>  	-e 's/^\[.*\]$/[XXX]/' \
>  	-e "s|^\(command: \)\{0,1\}/.*/$command|\1YYY/$command|"
>  }
> +
> +notmuch_date_sanitize ()
> +{
> +    sed \
> +	-e 's/^Date: Fri, 05 Jan 2001 .*/Date: GENERATED_DATE/'
> +}
>  # End of notmuch helper functions
>  
>  # Use test_set_prereq to tell that a particular prerequisite is available.

Re: [PATCH 4/4] test: give unique timestamps to messages

[Austin Clements]

Quoth David Bremner on Dec 08 at 11:52 pm:
> The choice of decreasing timestamps is a hack which reduces the number
> of existing tests which fail.  This can be changed to increasing
> if/when somebody wants update another 47 tests.
> ---
>  test/excludes    | 4 ++--
>  test/test-lib.sh | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/test/excludes b/test/excludes
> index f1ae9ea..8bbbc2d 100755
> --- a/test/excludes
> +++ b/test/excludes
> @@ -67,7 +67,7 @@ thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inb
>  test_begin_subtest "Search, don't exclude \"deleted\" messages when --exclude=flag specified"
>  output=$(notmuch search --exclude=flag subject:deleted | notmuch_search_sanitize)
>  test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
> -thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inbox unread)"
> +thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Deleted (deleted inbox unread)"
>  
>  test_begin_subtest "Search, don't exclude \"deleted\" messages from search if not configured"
>  notmuch config set search.exclude_tags
> @@ -152,7 +152,7 @@ ${matching_message_ids[5]}"
>  test_begin_subtest "Search, exclude=flag (thread summary)"
>  output=$(notmuch search --exclude=flag tag:test | notmuch_search_sanitize)
>  test_expect_equal "$output" "thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: single match: reply 2 (deleted inbox test unread)
> -thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 4 (deleted inbox test unread)
> +thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 2 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; Some messages excluded: single excluded match: reply 3 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; Some messages excluded: single non-excluded match: reply 4 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; No messages excluded: single match: reply 3 (inbox test unread)"
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index d238c66..792e477 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -356,7 +356,7 @@ generate_message ()
>      fi
>  
>      if [ -z "${template[date]}" ]; then
> -	template[date]="Fri, 05 Jan 2001 15:43:57 +0000"
> +	template[date]=$(date -Ru -d@$((978709437 - gen_msg_cnt)))

It would be worth adding a comment here to explain that the decreasing
timestamp is a historical artifact, to save people the trouble of
digging back in the log to figure out why it goes down.

>      fi
>  
>      additional_headers=""

Re: [PATCH 3/4] test: sanitize dates in emacs, raw, and text output

[David Bremner]

Austin Clements <amdragon@MIT.EDU> writes:

> This may be orthogonal to this series, but I don't get why we have
> both notmuch_show_sanitize and notmuch_show_sanitize_all (especially
> when the "all" version is not obviously a superset of the non-"all"
> version!)  Do you have a sense for whether these two functions could
> be easily merged?

Yeah, I don't really understand the reason for these two functions
either. I also discovered (and deleted) another duplicate file name
sanitization function elsewhere in the series, so it wouldn't surprise
me if it's just collective sloppiness.

>
> Rather than (nearly) duplicating the regexp, does it work to pipe this
> through notmuch_date_sanitize?  Or do the minor differences matter?
>

I'll try this out.

d

Re: [PATCH 4/4] test: give unique timestamps to messages

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

>>      if [ -z "${template[date]}" ]; then
>> -	template[date]="Fri, 05 Jan 2001 15:43:57 +0000"
>> +	template[date]=$(date -Ru -d@$((978709437 - gen_msg_cnt)))
>
> I am afraid this is not portable (although not sure). This could work:
>
> _s=$((3599 - gen_msg_cnt))
> template[date]="Fri, 05 Jan 2001 15:$((_s / 60)):$((_s % 60)) +0000"
>

How about this instead?

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 792e477..22a4d7f 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -356,7 +356,8 @@ generate_message ()
     fi
 
     if [ -z "${template[date]}" ]; then
-	template[date]=$(date -Ru -d@$((978709437 - gen_msg_cnt)))
+	template[date]=$(TZ=UTC printf "%(%a, %d %b %Y %T %z)T\n" \
+			$((978709437 - gen_msg_cnt)))
     fi
 
     additional_headers=""

Re: [PATCH 4/4] test: give unique timestamps to messages

[Tomi Ollila]

On Mon, Dec 09 2013, David Bremner <david@tethera.net> wrote:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>
>>>      if [ -z "${template[date]}" ]; then
>>> -	template[date]="Fri, 05 Jan 2001 15:43:57 +0000"
>>> +	template[date]=$(date -Ru -d@$((978709437 - gen_msg_cnt)))
>>
>> I am afraid this is not portable (although not sure). This could work:
>>
>> _s=$((3599 - gen_msg_cnt))
>> template[date]="Fri, 05 Jan 2001 15:$((_s / 60)):$((_s % 60)) +0000"
>>
>
> How about this instead?
>
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 792e477..22a4d7f 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -356,7 +356,8 @@ generate_message ()
>      fi
>  
>      if [ -z "${template[date]}" ]; then
> -	template[date]=$(date -Ru -d@$((978709437 - gen_msg_cnt)))
> +	template[date]=$(TZ=UTC printf "%(%a, %d %b %Y %T %z)T\n" \
> +			$((978709437 - gen_msg_cnt)))
>      fi

$ printf "%(%a, %d %b %Y %T %z)T\n" 978709437
printf: %(: invalid directive
zsh: exit 1

$ bash -c 'printf "%(%a, %d %b %Y %T %z)T\n" 978709437'
Fri, 05 Jan 2001 17:43:57 +0200


Using bash builtin is fine by me! :D

BTW: this works too:

bash -c 'TZ=UTC printf -v foo[bar] "%(%a, %d %b %Y %T %z)T\n" 978709437; echo ${foo[bar]}'


Tomi

[Patch v2 1/4] test: pass expected output through json_sanitize in 2 places

[David Bremner]

This makes the tests more robust against changes in the
sanitization rules.
---
 test/missing-headers | 8 +++++---
 test/multipart       | 2 +-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/test/missing-headers b/test/missing-headers
index 43e861b..cb38301 100755
--- a/test/missing-headers
+++ b/test/missing-headers
@@ -95,7 +95,7 @@ Body
 
 test_begin_subtest "Show: json"
 output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
-test_expect_equal_json "$output" '
+expected=$(notmuch_json_show_sanitize <<EOF
 [
     [
         [
@@ -156,7 +156,9 @@ test_expect_equal_json "$output" '
             []
         ]
     ]
-]'
-
+]
+EOF
+)
+test_expect_equal_json "$output" "$expected"
 
 test_done
diff --git a/test/multipart b/test/multipart
index b40fa2c..85cbf67 100755
--- a/test/multipart
+++ b/test/multipart
@@ -594,7 +594,7 @@ test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "'notmuch reply' to a multipart message with json format"
 notmuch reply --format=json 'id:87liy5ap00.fsf@yoom.home.cworth.org' | notmuch_json_show_sanitize >OUTPUT
-cat <<EOF >EXPECTED
+notmuch_json_show_sanitize <<EOF >EXPECTED
 {"reply-headers": {"Subject": "Re: Multipart message",
  "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
  "To": "Carl Worth <cworth@cworth.org>, cworth@cworth.org",
-- 
1.8.4.3

[Patch v2 2/4] test: sanitize Date and timestamp fields in json

[David Bremner]

Eventually we want test messages to have distinct dates to avoid
reproducability problems. This sanitization will prevent some test
failures when that change is made.

Replace the use of a local function in maildir-sync with
notmuch_json_show_sanitize
---
 test/maildir-sync | 15 +++++----------
 test/test-lib.sh  |  4 +++-
 2 files changed, 8 insertions(+), 11 deletions(-)

diff --git a/test/maildir-sync b/test/maildir-sync
index 33d2c58..3186e70 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -4,11 +4,6 @@ test_description="maildir synchronization"
 
 . ./test-lib.sh
 
-# Avoid including the local value of MAIL_DIR in the result.
-filter_show_json() {
-    sed -e "s|${MAIL_DIR}/|MAIL_DIR/|"
-}
-
 # Create the expected maildir structure
 mkdir $MAIL_DIR/cur
 mkdir $MAIL_DIR/new
@@ -40,18 +35,18 @@ output=$(cd ${MAIL_DIR}/cur; ls -1 adding-replied*)
 test_expect_equal "$output" "adding-replied-tag:2,RS"
 
 test_begin_subtest "notmuch show works with renamed file (without notmuch new)"
-output=$(notmuch show --format=json id:${gen_msg_id} | filter_show_json)
-test_expect_equal_json "$output" '[[[{"id": "adding-replied-tag@notmuch-test-suite",
+output=$(notmuch show --format=json id:${gen_msg_id} | notmuch_json_show_sanitize)
+test_expect_equal_json "$output" '[[[{"id": "XXXXX",
 "match": true,
 "excluded": false,
-"filename": "MAIL_DIR/cur/adding-replied-tag:2,RS",
-"timestamp": 978709437,
+"filename": "YYYYY",
+"timestamp": 42,
 "date_relative": "2001-01-05",
 "tags": ["inbox","replied"],
 "headers": {"Subject": "Adding replied tag",
 "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
 "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
-"Date": "Fri, 05 Jan 2001 15:43:57 +0000"},
+"Date": "GENERATED_DATE"},
 "body": [{"id": 1,
 "content-type": "text/plain",
 "content": "This is just a test message (#3)\n"}]},
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 8611ba5..8f18e73 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -625,7 +625,9 @@ notmuch_json_show_sanitize ()
 {
     sed \
 	-e 's|"id": "[^"]*",|"id": "XXXXX",|g' \
-	-e 's|"filename": "/[^"]*",|"filename": "YYYYY",|g'
+	-e 's|"Date": "Fri, 05 Jan 2001 [^"]*0000"|"Date": "GENERATED_DATE"|g' \
+	-e 's|"filename": "/[^"]*",|"filename": "YYYYY",|g' \
+	-e 's|"timestamp": 97.......|"timestamp": 42|g'
 }
 
 notmuch_emacs_error_sanitize ()
-- 
1.8.4.3

[Patch v2 3/4] test: sanitize dates in emacs, raw, and text output

[David Bremner]

add a new function notmuch_date_sanitize for rfc822-ish things. Add
date sanitization to notmuch_show_sanitize_all and use it more places.

This is all in aid of a transition to unique timestamps on messages.
---
 test/emacs       | 12 +++++++-----
 test/emacs-show  | 10 ++++++----
 test/encoding    |  8 ++++----
 test/raw         |  8 ++++----
 test/test-lib.sh |  9 ++++++++-
 5 files changed, 29 insertions(+), 18 deletions(-)

diff --git a/test/emacs b/test/emacs
index 456435c..1b1ebe3 100755
--- a/test/emacs
+++ b/test/emacs
@@ -86,15 +86,16 @@ add_message "[subject]=\"message-with-invalid-from\"" \
 	    "[from]=\"\\\"Invalid \\\" From\\\" <test_suite@notmuchmail.org>\""
 thread=$(notmuch search --output=threads subject:message-with-invalid-from)
 test_emacs "(notmuch-show \"$thread\")
-	    (test-output)"
+	    (test-output \"OUTPUT.raw\")"
 cat <<EOF >EXPECTED
 "Invalid " (2001-01-05) (inbox)
 Subject: message-with-invalid-from
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 This is just a test message (#1)
 EOF
+notmuch_date_sanitize < OUTPUT.raw > OUTPUT
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Navigation of notmuch-search to thread view"
@@ -605,11 +606,11 @@ Q: Why is top-posting such a bad thing?
 A: Top-posting.
 Q: What is the most annoying thing in e-mail?"'
 test_emacs "(notmuch-show \"top-posting\")
-	    (test-visible-output)"
+	    (test-visible-output \"OUTPUT.raw\")"
 echo "Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
 Subject: The problem with top-posting
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 A: Because it messes up the order in which people normally read text.
 Q: Why is top-posting such a bad thing?
@@ -618,13 +619,14 @@ Q: What is the most annoying thing in e-mail?
 Top Poster <top@poster.com> (2001-01-05) (inbox unread)
 Subject: Re: The problem with top-posting
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 Thanks for the advice! I will be sure to put it to good use.
 
 -Top Poster
 
 [ 9-line hidden original message. Click/Enter to show. ]" > EXPECTED
+notmuch_date_sanitize < OUTPUT.raw > OUTPUT
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Hiding message in notmuch-show view"
diff --git a/test/emacs-show b/test/emacs-show
index ec86333..2a3a535 100755
--- a/test/emacs-show
+++ b/test/emacs-show
@@ -19,13 +19,14 @@ cat <<EOF >EXPECTED
 Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
 Subject: Hiding Original Message region at beginning of a message
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 [ 2-line hidden original message. Click/Enter to show. ]
 EOF
 
 test_emacs "(notmuch-show \"id:$message_id\")
-	    (test-visible-output)"
+	    (test-visible-output \"OUTPUT.raw\")"
+notmuch_date_sanitize < OUTPUT.raw > OUTPUT
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Bare subject #1"
@@ -130,12 +131,12 @@ mid:abc%20def
 mid:abc. mid:abc, mid:abc;"'
 test_emacs '(notmuch-show "id:'$gen_msg_id'")
 	(notmuch-test-mark-links)
-	(test-visible-output)'
+	(test-visible-output "OUTPUT.raw")'
 cat <<EOF >EXPECTED
 Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox)
 Subject: id buttonization
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 <<id:abc>>
 <<id:abc.def>>. <<id:abc,def>>, <<id:abc;def>>; <<id:abc:def>>:
@@ -159,6 +160,7 @@ cid:xxx
 <<mid:abc%20def>>
 <<mid:abc>>. <<mid:abc>>, <<mid:abc>>;
 EOF
+notmuch_date_sanitize < OUTPUT.raw > OUTPUT
 test_expect_equal_file OUTPUT EXPECTED
 
 
diff --git a/test/encoding b/test/encoding
index 8609652..b6c86bf 100755
--- a/test/encoding
+++ b/test/encoding
@@ -5,14 +5,14 @@ test_description="encoding issues"
 test_begin_subtest "Message with text of unknown charset"
 add_message '[content-type]="text/plain; charset=unknown-8bit"' \
 	    "[body]=irrelevant"
-output=$(notmuch show id:${gen_msg_id} 2>&1 | notmuch_show_sanitize)
-test_expect_equal "$output" "\fmessage{ id:msg-001@notmuch-test-suite depth:0 match:1 excluded:0 filename:/XXX/mail/msg-001
+output=$(notmuch show id:${gen_msg_id} 2>&1 | notmuch_show_sanitize_all)
+test_expect_equal "$output" "\fmessage{ id:XXXXX depth:0 match:1 excluded:0 filename:XXXXX
 \fheader{
 Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox unread)
 Subject: Message with text of unknown charset
 From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 \fheader}
 \fbody{
 \fpart{ ID: 1, Content-type: text/plain
@@ -26,7 +26,7 @@ add_message '[content-type]="text/plain; charset=iso-8859-2"' \
             '[content-transfer-encoding]=8bit' \
             '[subject]="ISO-8859-2 encoded message"' \
             "[body]=$'Czech word tu\350\362\341\350\350\355 means pinguin\'s.'" # ISO-8859-2 characters are generated by shell's escape sequences
-output=$(notmuch search tučňáččí 2>&1 | notmuch_show_sanitize)
+output=$(notmuch search tučňáččí 2>&1 | notmuch_show_sanitize_all)
 test_expect_equal "$output" "thread:0000000000000002   2001-01-05 [1/1] Notmuch Test Suite; ISO-8859-2 encoded message (inbox unread)"
 
 test_begin_subtest "RFC 2047 encoded word with spaces"
diff --git a/test/raw b/test/raw
index de0b867..daf5735 100755
--- a/test/raw
+++ b/test/raw
@@ -11,22 +11,22 @@ output=$(notmuch show --format=raw "*" 2>&1)
 test_expect_equal "$output" "Error: search term did not match precisely one message."
 
 test_begin_subtest "Show a raw message"
-output=$(notmuch show --format=raw id:msg-001@notmuch-test-suite)
+output=$(notmuch show --format=raw id:msg-001@notmuch-test-suite | notmuch_date_sanitize)
 test_expect_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
 Message-Id: <msg-001@notmuch-test-suite>
 Subject: Test message #1
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 This is just a test message (#1)"
 
 test_begin_subtest "Show another raw message"
-output=$(notmuch show --format=raw id:msg-002@notmuch-test-suite)
+output=$(notmuch show --format=raw id:msg-002@notmuch-test-suite | notmuch_date_sanitize)
 test_expect_equal "$output" "From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
 Message-Id: <msg-002@notmuch-test-suite>
 Subject: Test message #2
-Date: Fri, 05 Jan 2001 15:43:57 +0000
+Date: GENERATED_DATE
 
 This is just a test message (#2)"
 
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 8f18e73..d22408d 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -618,7 +618,8 @@ notmuch_show_sanitize_all ()
 {
     sed \
 	-e 's| filename:.*| filename:XXXXX|' \
-	-e 's| id:[^ ]* | id:XXXXX |'
+	-e 's| id:[^ ]* | id:XXXXX |' | \
+	notmuch_date_sanitize
 }
 
 notmuch_json_show_sanitize ()
@@ -641,6 +642,12 @@ notmuch_emacs_error_sanitize ()
 	-e 's/^\[.*\]$/[XXX]/' \
 	-e "s|^\(command: \)\{0,1\}/.*/$command|\1YYY/$command|"
 }
+
+notmuch_date_sanitize ()
+{
+    sed \
+	-e 's/^Date: Fri, 05 Jan 2001 .*0000/Date: GENERATED_DATE/'
+}
 # End of notmuch helper functions
 
 # Use test_set_prereq to tell that a particular prerequisite is available.
-- 
1.8.4.3

[Patch v2 4/4] test: give unique timestamps to messages

[David Bremner]

The choice of decreasing timestamps is a hack which reduces the number
of existing tests which fail.  This can be changed to increasing
if/when somebody wants update another 47 tests.
---
 test/excludes    | 4 ++--
 test/test-lib.sh | 6 +++++-
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/test/excludes b/test/excludes
index f1ae9ea..8bbbc2d 100755
--- a/test/excludes
+++ b/test/excludes
@@ -67,7 +67,7 @@ thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inb
 test_begin_subtest "Search, don't exclude \"deleted\" messages when --exclude=flag specified"
 output=$(notmuch search --exclude=flag subject:deleted | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
-thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inbox unread)"
+thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Deleted (deleted inbox unread)"
 
 test_begin_subtest "Search, don't exclude \"deleted\" messages from search if not configured"
 notmuch config set search.exclude_tags
@@ -152,7 +152,7 @@ ${matching_message_ids[5]}"
 test_begin_subtest "Search, exclude=flag (thread summary)"
 output=$(notmuch search --exclude=flag tag:test | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: single match: reply 2 (deleted inbox test unread)
-thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 4 (deleted inbox test unread)
+thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 2 (deleted inbox test unread)
 thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; Some messages excluded: single excluded match: reply 3 (deleted inbox test unread)
 thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; Some messages excluded: single non-excluded match: reply 4 (deleted inbox test unread)
 thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; No messages excluded: single match: reply 3 (inbox test unread)"
diff --git a/test/test-lib.sh b/test/test-lib.sh
index d22408d..53cb947 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -356,7 +356,11 @@ generate_message ()
     fi
 
     if [ -z "${template[date]}" ]; then
-	template[date]="Fri, 05 Jan 2001 15:43:57 +0000"
+	# we use decreasing timestamps here for historical reasons;
+	# the existing test suite when we converted to unique timestamps just
+	# happened to have signicantly fewer failures with that choice.
+	template[date]=$(TZ=UTC printf "%(%a, %d %b %Y %T %z)T\n" \
+			$((978709437 - gen_msg_cnt)))
     fi
 
     additional_headers=""
-- 
1.8.4.3

Re: [Patch v2 4/4] test: give unique timestamps to messages

[Tomi Ollila]

On Mon, Dec 09 2013, David Bremner <david@tethera.net> wrote:

> The choice of decreasing timestamps is a hack which reduces the number
> of existing tests which fail.  This can be changed to increasing
> if/when somebody wants update another 47 tests.
> ---

series LGTM.

Tomi


>  test/excludes    | 4 ++--
>  test/test-lib.sh | 6 +++++-
>  2 files changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/test/excludes b/test/excludes
> index f1ae9ea..8bbbc2d 100755
> --- a/test/excludes
> +++ b/test/excludes
> @@ -67,7 +67,7 @@ thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inb
>  test_begin_subtest "Search, don't exclude \"deleted\" messages when --exclude=flag specified"
>  output=$(notmuch search --exclude=flag subject:deleted | notmuch_search_sanitize)
>  test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
> -thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inbox unread)"
> +thread:XXX   2001-01-05 [1/2] Notmuch Test Suite; Deleted (deleted inbox unread)"
>  
>  test_begin_subtest "Search, don't exclude \"deleted\" messages from search if not configured"
>  notmuch config set search.exclude_tags
> @@ -152,7 +152,7 @@ ${matching_message_ids[5]}"
>  test_begin_subtest "Search, exclude=flag (thread summary)"
>  output=$(notmuch search --exclude=flag tag:test | notmuch_search_sanitize)
>  test_expect_equal "$output" "thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: single match: reply 2 (deleted inbox test unread)
> -thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 4 (deleted inbox test unread)
> +thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; All messages excluded: double match: reply 2 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [0/6] Notmuch Test Suite; Some messages excluded: single excluded match: reply 3 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; Some messages excluded: single non-excluded match: reply 4 (deleted inbox test unread)
>  thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; No messages excluded: single match: reply 3 (inbox test unread)"
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index d22408d..53cb947 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -356,7 +356,11 @@ generate_message ()
>      fi
>  
>      if [ -z "${template[date]}" ]; then
> -	template[date]="Fri, 05 Jan 2001 15:43:57 +0000"
> +	# we use decreasing timestamps here for historical reasons;
> +	# the existing test suite when we converted to unique timestamps just
> +	# happened to have signicantly fewer failures with that choice.
> +	template[date]=$(TZ=UTC printf "%(%a, %d %b %Y %T %z)T\n" \
> +			$((978709437 - gen_msg_cnt)))
>      fi
>  
>      additional_headers=""
> -- 
> 1.8.4.3
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [Patch v2 4/4] test: give unique timestamps to messages

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Mon, Dec 09 2013, David Bremner <david@tethera.net> wrote:
>
>> The choice of decreasing timestamps is a hack which reduces the number
>> of existing tests which fail.  This can be changed to increasing
>> if/when somebody wants update another 47 tests.
>> ---
>
> series LGTM.
>

Pushed to release. I'll merge to master after a bit.

d