* test/emacs: reduce use of smtp-dummy, notmuch-hello
@ 2013-12-15 1:15 David Bremner
2013-12-15 1:15 ` [PATCH 1/2] test: add emacs_fcc_message that does not use smtp-dummy David Bremner
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: David Bremner @ 2013-12-15 1:15 UTC (permalink / raw)
To: notmuch
In addition to the timestamp related failures in the test suite,
there were some intermittent problems [1] that turned out to be failure of
the emacs_deliver_message function. In this series we try to make
mail delivery from emacs more robust (mainly replace actual delivery
with fcc)
[1]: https://buildd.debian.org/fetch.cgi?pkg=notmuch&arch=i386&ver=0.17%7Erc3-1&stamp=1386412394&file=log
As I write this, the crypto tests have run successfully for 9155
times. Of course that doesn't really prove anything, but it is twice
as many as the previously longest interval before failure.
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 1/2] test: add emacs_fcc_message that does not use smtp-dummy 2013-12-15 1:15 test/emacs: reduce use of smtp-dummy, notmuch-hello David Bremner @ 2013-12-15 1:15 ` David Bremner 2013-12-22 11:39 ` Mark Walters 2013-12-15 1:15 ` [PATCH 2/2] test: remove call to notmuch-hello from emacs_deliver_message David Bremner 2013-12-15 10:35 ` test/emacs: reduce use of smtp-dummy, notmuch-hello Tomi Ollila 2 siblings, 1 reply; 6+ messages in thread From: David Bremner @ 2013-12-15 1:15 UTC (permalink / raw) To: notmuch Most of the tests previously using emacs_deliver_message do not use the actual transmitted message, so we replace it with a simpler (and presumably more reliable function) that only saves (and indexes) an fcc copy of the message. --- test/crypto | 10 +++------- test/emacs | 4 ++-- test/json | 2 +- test/sexp | 2 +- test/test-lib.sh | 29 +++++++++++++++++++++++++++++ 5 files changed, 36 insertions(+), 11 deletions(-) diff --git a/test/crypto b/test/crypto index 9e5ff4f..477b397 100755 --- a/test/crypto +++ b/test/crypto @@ -28,12 +28,8 @@ add_gnupg_home # get key fingerprint FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10) -# for some reason this is needed for emacs_deliver_message to work, -# although I can't figure out why -add_email_corpus - test_expect_success 'emacs delivery of signed message' \ -'emacs_deliver_message \ +'emacs_fcc_message \ "test signed message 001" \ "This is a test signed message." \ "(mml-secure-message-sign)"' @@ -143,7 +139,7 @@ cat <<EOF >TESTATTACHMENT This is a test file. EOF test_expect_success 'emacs delivery of encrypted message with attachment' \ -'emacs_deliver_message \ +'emacs_fcc_message \ "test encrypted message 001" \ "This is a test encrypted message.\n" \ "(mml-attach-file \"TESTATTACHMENT\") (mml-secure-message-encrypt)"' @@ -270,7 +266,7 @@ test_expect_equal_json \ mv "${GNUPGHOME}"{.bak,} test_expect_success 'emacs delivery of encrypted + signed message' \ -'emacs_deliver_message \ +'emacs_fcc_message \ "test encrypted message 002" \ "This is another test encrypted message.\n" \ "(mml-secure-message-sign-encrypt)"' diff --git a/test/emacs b/test/emacs index 1b1ebe3..863219d 100755 --- a/test/emacs +++ b/test/emacs @@ -771,7 +771,7 @@ test_expect_equal_file OUTPUT EXPECTED test_begin_subtest "Do not call notmuch for non-inlinable application/mpeg parts" id='message-with-application/mpeg-attachment@notmuchmail.org' -emacs_deliver_message \ +emacs_fcc_message \ 'Message with application/mpeg attachment' \ '' \ "(message-goto-eoh) @@ -786,7 +786,7 @@ test_expect_equal $(notmuch_counter_value) 1 test_begin_subtest "Do not call notmuch for non-inlinable audio/mpeg parts" id='message-with-audio/mpeg-attachment@notmuchmail.org' -emacs_deliver_message \ +emacs_fcc_message \ 'Message with audio/mpeg attachment' \ '' \ "(message-goto-eoh) diff --git a/test/json b/test/json index e07a290..c1cf649 100755 --- a/test/json +++ b/test/json @@ -38,7 +38,7 @@ test_expect_equal_json "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true test_begin_subtest "Show message: json, inline attachment filename" subject='json-show-inline-attachment-filename' id="json-show-inline-attachment-filename@notmuchmail.org" -emacs_deliver_message \ +emacs_fcc_message \ "$subject" \ 'This is a test message with inline attachment with a filename' \ "(mml-attach-file \"$TEST_DIRECTORY/README\" nil nil \"inline\") diff --git a/test/sexp b/test/sexp index be815e1..667e319 100755 --- a/test/sexp +++ b/test/sexp @@ -29,7 +29,7 @@ test_expect_equal "$output" "((((:id \"${gen_msg_id}\" :match t :excluded nil :f test_begin_subtest "Show message: sexp, inline attachment filename" subject='sexp-show-inline-attachment-filename' id="sexp-show-inline-attachment-filename@notmuchmail.org" -emacs_deliver_message \ +emacs_fcc_message \ "$subject" \ 'This is a test message with inline attachment with a filename' \ "(mml-attach-file \"$TEST_DIRECTORY/README\" nil nil \"inline\") diff --git a/test/test-lib.sh b/test/test-lib.sh index 53cb947..66c0720 100644 --- a/test/test-lib.sh +++ b/test/test-lib.sh @@ -470,6 +470,35 @@ emacs_deliver_message () notmuch new >/dev/null } +# Pretend to deliver a message with emacs. Really save it to a file +# and add it to the database +# +# Uses emacs to generate and deliver a message to the mail store. +# Accepts arbitrary extra emacs/elisp functions to modify the message +# before sending, which is useful to doing things like attaching files +# to the message and encrypting/signing. +emacs_fcc_message () +{ + local subject="$1" + local body="$2" + shift 2 + # before we can send a message, we have to prepare the FCC maildir + mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp} + + test_emacs \ + "(let ((message-send-mail-function (lambda () t))) + (notmuch-mua-mail) + (message-goto-to) + (insert \"test_suite@notmuchmail.org\nDate: 01 Jan 2000 12:00:00 -0000\") + (message-goto-subject) + (insert \"${subject}\") + (message-goto-body) + (insert \"${body}\") + $@ + (message-send-and-exit))" || return 1 + notmuch new >/dev/null +} + # Generate a corpus of email and add it to the database. # # This corpus is fixed, (it happens to be 50 messages from early in -- 1.8.4.3 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] test: add emacs_fcc_message that does not use smtp-dummy 2013-12-15 1:15 ` [PATCH 1/2] test: add emacs_fcc_message that does not use smtp-dummy David Bremner @ 2013-12-22 11:39 ` Mark Walters 2013-12-23 1:11 ` David Bremner 0 siblings, 1 reply; 6+ messages in thread From: Mark Walters @ 2013-12-22 11:39 UTC (permalink / raw) To: David Bremner, notmuch Hi > Most of the tests previously using emacs_deliver_message do not use > the actual transmitted message, so we replace it with a simpler (and > presumably more reliable function) that only saves (and indexes) an > fcc copy of the message. I get a test failure with this patch, but I have a tentative fix. It may be a mis-configuration in some other part of my setup. My computer has a hostname of e4300 (which perhaps should be a FQDN?) Anyway when running with this patch I get an error for the crypto test as FAIL reply to encrypted message --- crypto.13.expected 2013-12-22 11:32:24.117104674 +0000 +++ crypto.13.output 2013-12-22 11:32:24.117104674 +0000 @@ -1,5 +1,7 @@ From: Notmuch Test Suite <test_suite@notmuchmail.org> Subject: Re: test encrypted message 002 + <871u15xgs8.fsf@e4300.i-did-not-set--mail-host-address--so-tickle-me> + <871u15xgs8.fsf@e4300.i-did-not-set--mail-host-address--so-tickle-me> On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote: > This is another test encrypted message. If I add the line (mail-host-address \"example.com\")) inside the let in emacs_fcc_message below then it passes. All tests also pass on current master. Best wishes Mark > test/crypto | 10 +++------- > test/emacs | 4 ++-- > test/json | 2 +- > test/sexp | 2 +- > test/test-lib.sh | 29 +++++++++++++++++++++++++++++ > 5 files changed, 36 insertions(+), 11 deletions(-) > > diff --git a/test/crypto b/test/crypto > index 9e5ff4f..477b397 100755 > --- a/test/crypto > +++ b/test/crypto > @@ -28,12 +28,8 @@ add_gnupg_home > # get key fingerprint > FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10) > > -# for some reason this is needed for emacs_deliver_message to work, > -# although I can't figure out why > -add_email_corpus > - > test_expect_success 'emacs delivery of signed message' \ > -'emacs_deliver_message \ > +'emacs_fcc_message \ > "test signed message 001" \ > "This is a test signed message." \ > "(mml-secure-message-sign)"' > @@ -143,7 +139,7 @@ cat <<EOF >TESTATTACHMENT > This is a test file. > EOF > test_expect_success 'emacs delivery of encrypted message with attachment' \ > -'emacs_deliver_message \ > +'emacs_fcc_message \ > "test encrypted message 001" \ > "This is a test encrypted message.\n" \ > "(mml-attach-file \"TESTATTACHMENT\") (mml-secure-message-encrypt)"' > @@ -270,7 +266,7 @@ test_expect_equal_json \ > mv "${GNUPGHOME}"{.bak,} > > test_expect_success 'emacs delivery of encrypted + signed message' \ > -'emacs_deliver_message \ > +'emacs_fcc_message \ > "test encrypted message 002" \ > "This is another test encrypted message.\n" \ > "(mml-secure-message-sign-encrypt)"' > diff --git a/test/emacs b/test/emacs > index 1b1ebe3..863219d 100755 > --- a/test/emacs > +++ b/test/emacs > @@ -771,7 +771,7 @@ test_expect_equal_file OUTPUT EXPECTED > > test_begin_subtest "Do not call notmuch for non-inlinable application/mpeg parts" > id='message-with-application/mpeg-attachment@notmuchmail.org' > -emacs_deliver_message \ > +emacs_fcc_message \ > 'Message with application/mpeg attachment' \ > '' \ > "(message-goto-eoh) > @@ -786,7 +786,7 @@ test_expect_equal $(notmuch_counter_value) 1 > > test_begin_subtest "Do not call notmuch for non-inlinable audio/mpeg parts" > id='message-with-audio/mpeg-attachment@notmuchmail.org' > -emacs_deliver_message \ > +emacs_fcc_message \ > 'Message with audio/mpeg attachment' \ > '' \ > "(message-goto-eoh) > diff --git a/test/json b/test/json > index e07a290..c1cf649 100755 > --- a/test/json > +++ b/test/json > @@ -38,7 +38,7 @@ test_expect_equal_json "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true > test_begin_subtest "Show message: json, inline attachment filename" > subject='json-show-inline-attachment-filename' > id="json-show-inline-attachment-filename@notmuchmail.org" > -emacs_deliver_message \ > +emacs_fcc_message \ > "$subject" \ > 'This is a test message with inline attachment with a filename' \ > "(mml-attach-file \"$TEST_DIRECTORY/README\" nil nil \"inline\") > diff --git a/test/sexp b/test/sexp > index be815e1..667e319 100755 > --- a/test/sexp > +++ b/test/sexp > @@ -29,7 +29,7 @@ test_expect_equal "$output" "((((:id \"${gen_msg_id}\" :match t :excluded nil :f > test_begin_subtest "Show message: sexp, inline attachment filename" > subject='sexp-show-inline-attachment-filename' > id="sexp-show-inline-attachment-filename@notmuchmail.org" > -emacs_deliver_message \ > +emacs_fcc_message \ > "$subject" \ > 'This is a test message with inline attachment with a filename' \ > "(mml-attach-file \"$TEST_DIRECTORY/README\" nil nil \"inline\") > diff --git a/test/test-lib.sh b/test/test-lib.sh > index 53cb947..66c0720 100644 > --- a/test/test-lib.sh > +++ b/test/test-lib.sh > @@ -470,6 +470,35 @@ emacs_deliver_message () > notmuch new >/dev/null > } > > +# Pretend to deliver a message with emacs. Really save it to a file > +# and add it to the database > +# > +# Uses emacs to generate and deliver a message to the mail store. > +# Accepts arbitrary extra emacs/elisp functions to modify the message > +# before sending, which is useful to doing things like attaching files > +# to the message and encrypting/signing. > +emacs_fcc_message () > +{ > + local subject="$1" > + local body="$2" > + shift 2 > + # before we can send a message, we have to prepare the FCC maildir > + mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp} > + > + test_emacs \ > + "(let ((message-send-mail-function (lambda () t))) > + (notmuch-mua-mail) > + (message-goto-to) > + (insert \"test_suite@notmuchmail.org\nDate: 01 Jan 2000 12:00:00 -0000\") > + (message-goto-subject) > + (insert \"${subject}\") > + (message-goto-body) > + (insert \"${body}\") > + $@ > + (message-send-and-exit))" || return 1 > + notmuch new >/dev/null > +} > + > # Generate a corpus of email and add it to the database. > # > # This corpus is fixed, (it happens to be 50 messages from early in > -- > 1.8.4.3 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] test: add emacs_fcc_message that does not use smtp-dummy 2013-12-22 11:39 ` Mark Walters @ 2013-12-23 1:11 ` David Bremner 0 siblings, 0 replies; 6+ messages in thread From: David Bremner @ 2013-12-23 1:11 UTC (permalink / raw) To: Mark Walters, notmuch Mark Walters <markwalters1009@gmail.com> writes: > > If I add the line > > (mail-host-address \"example.com\")) > > inside the let in emacs_fcc_message below then it passes. All tests also > pass on current master. I pushed the version suggested by Mark to the release branch. Depending on the response to id:87bo089y0t.fsf@maritornes.cs.unb.ca I'll either wait for a fix for the other intmittent test failure, or tag another release candidate. d ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/2] test: remove call to notmuch-hello from emacs_deliver_message 2013-12-15 1:15 test/emacs: reduce use of smtp-dummy, notmuch-hello David Bremner 2013-12-15 1:15 ` [PATCH 1/2] test: add emacs_fcc_message that does not use smtp-dummy David Bremner @ 2013-12-15 1:15 ` David Bremner 2013-12-15 10:35 ` test/emacs: reduce use of smtp-dummy, notmuch-hello Tomi Ollila 2 siblings, 0 replies; 6+ messages in thread From: David Bremner @ 2013-12-15 1:15 UTC (permalink / raw) To: notmuch There is an obscure bug in notmuch-hello that very occasionally causes emacs_deliver_message to fail. Since it it doesn't serve any actual purpose in the function we delete it, and leave tracking down the the bug for another day. --- test/test-lib.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/test/test-lib.sh b/test/test-lib.sh index 66c0720..1bb5610 100644 --- a/test/test-lib.sh +++ b/test/test-lib.sh @@ -452,7 +452,6 @@ emacs_deliver_message () (mail-host-address \"example.com\") (smtpmail-smtp-server \"localhost\") (smtpmail-smtp-service \"25025\")) - (notmuch-hello) (notmuch-mua-mail) (message-goto-to) (insert \"test_suite@notmuchmail.org\nDate: 01 Jan 2000 12:00:00 -0000\") -- 1.8.4.3 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: test/emacs: reduce use of smtp-dummy, notmuch-hello 2013-12-15 1:15 test/emacs: reduce use of smtp-dummy, notmuch-hello David Bremner 2013-12-15 1:15 ` [PATCH 1/2] test: add emacs_fcc_message that does not use smtp-dummy David Bremner 2013-12-15 1:15 ` [PATCH 2/2] test: remove call to notmuch-hello from emacs_deliver_message David Bremner @ 2013-12-15 10:35 ` Tomi Ollila 2 siblings, 0 replies; 6+ messages in thread From: Tomi Ollila @ 2013-12-15 10:35 UTC (permalink / raw) To: David Bremner, notmuch On Sun, Dec 15 2013, David Bremner <david@tethera.net> wrote: > In addition to the timestamp related failures in the test suite, > there were some intermittent problems [1] that turned out to be failure of > the emacs_deliver_message function. In this series we try to make > mail delivery from emacs more robust (mainly replace actual delivery > with fcc) > > [1]: https://buildd.debian.org/fetch.cgi?pkg=notmuch&arch=i386&ver=0.17%7Erc3-1&stamp=1386412394&file=log > > As I write this, the crypto tests have run successfully for 9155 > times. Of course that doesn't really prove anything, but it is twice > as many as the previously longest interval before failure. series LGTM. Tomi (sent using fingerterm on N9, review quality may vary ;/) ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-12-23 1:11 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-12-15 1:15 test/emacs: reduce use of smtp-dummy, notmuch-hello David Bremner 2013-12-15 1:15 ` [PATCH 1/2] test: add emacs_fcc_message that does not use smtp-dummy David Bremner 2013-12-22 11:39 ` Mark Walters 2013-12-23 1:11 ` David Bremner 2013-12-15 1:15 ` [PATCH 2/2] test: remove call to notmuch-hello from emacs_deliver_message David Bremner 2013-12-15 10:35 ` test/emacs: reduce use of smtp-dummy, notmuch-hello Tomi Ollila
Code repositories for project(s) associated with this public inbox https://yhetil.org/notmuch.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).