[PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Tomi Ollila]

If mail sending from emacs fails before it has chance to connect
to the smtp-dummy mail server, the opportunistic QUIT message
sending makes smtp-dummy to exit.
---
 test/test-lib.sh |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index af723ad..57cb088 100755
@@ -397,6 +397,8 @@ emacs_deliver_message ()
 	   (insert \"${body}\")
 	   $@
 	   (message-send-and-exit))" >/dev/null 2>&1
+    # opportunistically quit smtp-dummy in case above fails.
+    echo QUIT | nc 127.0.0.1 25025 >/dev/null 2>&1
     wait ${smtp_dummy_pid}
     notmuch new >/dev/null
 }
-- 
1.5.6.5

Re: [PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Austin Clements]

Good idea, but this introduces a dependency on netcat.  Would it work
to simply kill the SMTP dummy?

Quoth Tomi Ollila on Nov 11 at  1:38 am:
> If mail sending from emacs fails before it has chance to connect
> to the smtp-dummy mail server, the opportunistic QUIT message
> sending makes smtp-dummy to exit.
> ---
>  test/test-lib.sh |    8 +++++---
>  1 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index af723ad..57cb088 100755
> @@ -397,6 +397,8 @@ emacs_deliver_message ()
>  	   (insert \"${body}\")
>  	   $@
>  	   (message-send-and-exit))" >/dev/null 2>&1
> +    # opportunistically quit smtp-dummy in case above fails.
> +    echo QUIT | nc 127.0.0.1 25025 >/dev/null 2>&1
>      wait ${smtp_dummy_pid}
>      notmuch new >/dev/null
>  }

Re: [PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Tomi Ollila]

On Fri, 11 Nov 2011 21:49:41 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Good idea, but this introduces a dependency on netcat.  Would it work
> to simply kill the SMTP dummy?

I cannot guarantee 6 nines of probability (but can guarantee as many eights
as one migh wish ;)

Socket buffering and timing might cause smtp dummy to be killed before
it has read all content...

But... i recall bash has a way to do tcp stuff .../dev/tcp/host/port
is mentioned in manual page.
I'll investigate this later (or, anyone of you, provide a patch :)

Tomi

> Quoth Tomi Ollila on Nov 11 at  1:38 am:
> > If mail sending from emacs fails before it has chance to connect
> > to the smtp-dummy mail server, the opportunistic QUIT message
> > sending makes smtp-dummy to exit.
> > ---
> >  test/test-lib.sh |    8 +++++---
> >  1 files changed, 5 insertions(+), 3 deletions(-)
> > 
> > diff --git a/test/test-lib.sh b/test/test-lib.sh
> > index af723ad..57cb088 100755
> > @@ -397,6 +397,8 @@ emacs_deliver_message ()
> >  	   (insert \"${body}\")
> >  	   $@
> >  	   (message-send-and-exit))" >/dev/null 2>&1
> > +    # opportunistically quit smtp-dummy in case above fails.
> > +    echo QUIT | nc 127.0.0.1 25025 >/dev/null 2>&1
> >      wait ${smtp_dummy_pid}
> >      notmuch new >/dev/null
> >  }
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
> 

Re: [PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Austin Clements]

Quoth Tomi Ollila on Nov 12 at 12:02 pm:
> On Fri, 11 Nov 2011 21:49:41 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > Good idea, but this introduces a dependency on netcat.  Would it work
> > to simply kill the SMTP dummy?
> 
> I cannot guarantee 6 nines of probability (but can guarantee as many eights
> as one migh wish ;)
> 
> Socket buffering and timing might cause smtp dummy to be killed before
> it has read all content...
> 
> But... i recall bash has a way to do tcp stuff .../dev/tcp/host/port
> is mentioned in manual page.
> I'll investigate this later (or, anyone of you, provide a patch :)

Actually, since sending mail is synchronous, there shouldn't be any
issues with buffering or timing.  If Emacs successfully sends the
message, it will wait for the OK response (`smtpmail-via-smtp'), which
should indicate that the message is in stable storage on the SMTP
server (this is rather fundamental to SMTP's reliability as a
protocol, after all).  Thus, the smtp-dummy *should* be done by the
time Emacs exits.  If Emacs fails to send the message, then it doesn't
really matter and you just want the smtp-dummy to go away.

I say "should" because smtp-dummy needs a one line patch to add an
fflush at the end of receive_data_to_file.  Any real SMTP server would
do this (not to mention a full fsync) before acknowledging the
message.

Re: [PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Tomi Ollila]

On Sat, 12 Nov 2011 10:41:28 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> 
> Actually, since sending mail is synchronous, there shouldn't be any
> issues with buffering or timing.  If Emacs successfully sends the
> message, it will wait for the OK response (`smtpmail-via-smtp'), which
> should indicate that the message is in stable storage on the SMTP
> server (this is rather fundamental to SMTP's reliability as a
> protocol, after all).  Thus, the smtp-dummy *should* be done by the
> time Emacs exits.  If Emacs fails to send the message, then it doesn't
> really matter and you just want the smtp-dummy to go away.
> 
> I say "should" because smtp-dummy needs a one line patch to add an
> fflush at the end of receive_data_to_file.  Any real SMTP server would
> do this (not to mention a full fsync) before acknowledging the
> message.

SO, in our cases there are 2 options:


1) add line 
	{ echo QUIT > /dev/tcp/127.0.0.1/25025; } 2>/dev/null
  before 
	wait ${smtp_dummy_pid}

in test-lib.sh


2) add line
	kill ${smtp_dummy_pid}
  before 
	wait ${smtp_dummy_pid}

in test-lib.sh *and* add that fflush (output) before free (line) in 
smtp-dummy.c function receive_data_to_file.

folks! opinions!

Tomi

Re: [PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

On Sat, 12 Nov 2011 23:07:38 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> 1) add line 
> 	{ echo QUIT > /dev/tcp/127.0.0.1/25025; } 2>/dev/null

I have no directory called "/dev/tcp" on my system.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Pieter Praet]

On Sat, 12 Nov 2011 13:24:40 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Sat, 12 Nov 2011 23:07:38 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> > 1) add line 
> > 	{ echo QUIT > /dev/tcp/127.0.0.1/25025; } 2>/dev/null
> 
> I have no directory called "/dev/tcp" on my system.
> 

You shouldn't.  See (info "(bash)Redirections") :

  #+begin_quote
    Bash handles several filenames specially when they are used in
    redirections, as described in the following table:

    [...]
    `/dev/tcp/HOST/PORT'
         If HOST is a valid hostname or Internet address, and PORT is an
         integer port number or service name, Bash attempts to open a TCP
         connection to the corresponding socket.
    [...]
  #+end_quote

> jamie.
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


Peace

-- 
Pieter

Re: [PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Dmitry Kurochkin]

On Sat, 12 Nov 2011 23:07:38 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> On Sat, 12 Nov 2011 10:41:28 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > 
> > Actually, since sending mail is synchronous, there shouldn't be any
> > issues with buffering or timing.  If Emacs successfully sends the
> > message, it will wait for the OK response (`smtpmail-via-smtp'), which
> > should indicate that the message is in stable storage on the SMTP
> > server (this is rather fundamental to SMTP's reliability as a
> > protocol, after all).  Thus, the smtp-dummy *should* be done by the
> > time Emacs exits.  If Emacs fails to send the message, then it doesn't
> > really matter and you just want the smtp-dummy to go away.
> > 
> > I say "should" because smtp-dummy needs a one line patch to add an
> > fflush at the end of receive_data_to_file.  Any real SMTP server would
> > do this (not to mention a full fsync) before acknowledging the
> > message.
> 
> SO, in our cases there are 2 options:
> 
> 
> 1) add line 
> 	{ echo QUIT > /dev/tcp/127.0.0.1/25025; } 2>/dev/null
>   before 
> 	wait ${smtp_dummy_pid}
> 
> in test-lib.sh
> 
> 
> 2) add line
> 	kill ${smtp_dummy_pid}
>   before 
> 	wait ${smtp_dummy_pid}
> 
> in test-lib.sh *and* add that fflush (output) before free (line) in 
> smtp-dummy.c function receive_data_to_file.
> 
> folks! opinions!
> 

I think we should add fflush(3) call to smtp-dummy in any case.

As for sending QUIT over TCP vs kill, I prefer the former.  On one hand,
kill is more reliable.  But smtp-dummy must handle QUIT and it is a bug
if it does not.

An unrelated issue: currently we discard test_emacs output in
emacs_deliver_message and other cases.  Ideally we should not do that
because it may contain useful information.  We should capture the output
and print it in case of error, at least.

Regards,
  Dmitry

> Tomi
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Tomi Ollila]

On Sun, 13 Nov 2011 11:01:24 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> On Sat, 12 Nov 2011 23:07:38 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> > SO, in our cases there are 2 options:
> > 
> > 
> > 1) add line 
> > 	{ echo QUIT > /dev/tcp/127.0.0.1/25025; } 2>/dev/null
> >   before 
> > 	wait ${smtp_dummy_pid}
> > 
> > in test-lib.sh
> > 
> > 
> > 2) add line
> > 	kill ${smtp_dummy_pid}
> >   before 
> > 	wait ${smtp_dummy_pid}
> > 
> > in test-lib.sh *and* add that fflush (output) before free (line) in 
> > smtp-dummy.c function receive_data_to_file.
> > 
> > folks! opinions!
> > 
> 
> I think we should add fflush(3) call to smtp-dummy in any case.

True. In a separate patch. 

> As for sending QUIT over TCP vs kill, I prefer the former.  On one hand,
> kill is more reliable.  But smtp-dummy must handle QUIT and it is a bug
> if it does not.

Me too, basically for the same reasons. It is also aesthetically nicer
not to kill anything ;)

> An unrelated issue: currently we discard test_emacs output in
> emacs_deliver_message and other cases.  Ideally we should not do that
> because it may contain useful information.  We should capture the output
> and print it in case of error, at least.

I agree. And if id:"<1309487028-18786-1-git-send-email-dmitry.kurochkin@gmail.com>"
is applied before creating new patch based on this then these 2 apply cleanly.

> 
> Regards,
>   Dmitry
> 
> > Tomi

Tomi

[PATCH v2] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Tomi Ollila]

If mail sending from emacs fails before it has chance to connect
to the smtp-dummy mail server, the opportunistic QUIT message
sending makes smtp-dummy to exit.
---

Version 2 of the same feature. Now uses BASH's /dev/tcp/host/port 
feature instead of external 'nc' command.

 test/test-lib.sh |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 1ea7fa9..749d087 100755
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -395,6 +395,8 @@ emacs_deliver_message ()
 	   (insert \"${body}\")
 	   $@
 	   (message-send-and-exit))"
+    # opportunistically quit smtp-dummy in case above fails.
+    { echo QUIT > /dev/tcp/localhost/25025; } 2>/dev/null
     wait ${smtp_dummy_pid}
     notmuch new >/dev/null
 }
--
1.7.7.3

Re: [PATCH v2] test: attempt to send QUIT to smtp-dummy in case mail send failed

[Austin Clements]

LGTM.

Quoth Tomi Ollila on Nov 16 at 11:25 pm:
> If mail sending from emacs fails before it has chance to connect
> to the smtp-dummy mail server, the opportunistic QUIT message
> sending makes smtp-dummy to exit.

Re: [PATCH v2] test: attempt to send QUIT to smtp-dummy in case mail send failed

[David Bremner]

On Wed, 16 Nov 2011 23:25:19 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> If mail sending from emacs fails before it has chance to connect
> to the smtp-dummy mail server, the opportunistic QUIT message
> sending makes smtp-dummy to exit.

Pushed.

d