smtpmail/starttls starts connection but never sends

smtpmail/starttls starts connection but never sends

[Joost Kremers]

Hi all,

I'm running Emacs 24.2 on OS X Lion (more precisely, I'm using the
emacs-mac-port from <https://github.com/railwaycat/emacs-mac-port>) and
using it for email with mu4e. The other day, I ran into a problem with
sending mail, in that I can't anymore...

The emacs-mac-port is compiled without tls, so it uses the external
gnutls program for establishing tls connections, which worked fine,
until Homebrew upgraded gnutls. The problem is that gnutls-cli stopped
accepting certificates that cannot be verified. As a result, the
connection would be dropped and Emacs would tell me that no smtpmail
process was running. (Note: mu4e uses message-mode for composing and
smtpmail for sending email, so this issue shouldn't be mu4e-specific).

After some googling, I found that one way to get around this *should* be
to establish a connection manually once with the --tofu option, accept
the certificate when asked to, and then add the --tofu option to
starttls-extra-arguments, as explained in a post to gnu.emacs.gnus:
<mailman.471.1333501703.20052.info-gnus-english@gnu.org>.

In my case, this doesn't entirely work, however. Although the connection
appears to be established, nothing is actually sent. I get the following
message in the minibuffer:

Opening STARTTLS connection to `mailer.gwdg.de:25'...done

and that's where it stays. Emacs is locked, the message buffer remains
open and active and nothing happens. I have to hit C-g to get control
back.

Anyone know what might be going on, or how I could debug this further?

TIA

-- 
Joost Kremers                                      joostkremers@yahoo.com
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

Re: smtpmail/starttls starts connection but never sends

[William Gardella]

Hello Joost,

Joost Kremers <joostkremers@yahoo.com> writes:

> Hi all,
>
> I'm running Emacs 24.2 on OS X Lion (more precisely, I'm using the
> emacs-mac-port from <https://github.com/railwaycat/emacs-mac-port>) and
> using it for email with mu4e. The other day, I ran into a problem with
> sending mail, in that I can't anymore...
>
> The emacs-mac-port is compiled without tls, so it uses the external
> gnutls program for establishing tls connections, which worked fine,
> until Homebrew upgraded gnutls. The problem is that gnutls-cli stopped
> accepting certificates that cannot be verified. As a result, the
> connection would be dropped and Emacs would tell me that no smtpmail
> process was running. (Note: mu4e uses message-mode for composing and
> smtpmail for sending email, so this issue shouldn't be mu4e-specific).
>
> After some googling, I found that one way to get around this *should* be
> to establish a connection manually once with the --tofu option, accept
> the certificate when asked to, and then add the --tofu option to
> starttls-extra-arguments, as explained in a post to gnu.emacs.gnus:
> <mailman.471.1333501703.20052.info-gnus-english@gnu.org>.

I wonder if you've tried playing around with the `tls-checktrust'
variable.  If you (setq tls-checktrust 'ask), you should be able to
manually accept untrusted SSL/TLS keys by responding to a dialog in
Emacs.  Along with setting `tls-checktrust' to ask, you can set
`tls-program' to have a command line that will allow you to call
gnutls-cli with your own private list of certificates.  For me here in
Debian GNU/Linux, the following is a working TLS configuration that
checks certs but also lets me use invalid/self-signed ones when I wish:

(setq tls-checktrust 'ask
      tls-program
      '("gnutls-cli --x509cafile /etc/ssl/certs/ca-certificates.crt -p %p %h"
        "gnutls-cli --x509cafile /etc/ssl/certs/ca-certificates.crt -p %p %h --protocols ssl3"
        "openssl s_client -connect %h:%p -CAfile /etc/ssl/certs/ca-certificates.crt -no_ssl2 -ign_eof"))

Maybe this will help you if gnutls --insecure is no longer working.

> In my case, this doesn't entirely work, however. Although the connection
> appears to be established, nothing is actually sent. I get the following
> message in the minibuffer:
>
> Opening STARTTLS connection to `mailer.gwdg.de:25'...done
>
> and that's where it stays. Emacs is locked, the message buffer remains
> open and active and nothing happens. I have to hit C-g to get control
> back.
>
> Anyone know what might be going on, or how I could debug this further?
>
> TIA

If the above doesn't work, you can use strace or dtrace to see where in
the TLS-calling process Emacs hangs.  You can also try writing out an
`open-network-stream' function call of your Emacs trying to talk to your
mail server and stepping through it with (edebug-defun).

HTH,
WGG

-- 
I use grml (http://grml.org/)

Re: smtpmail/starttls starts connection but never sends

[Joost Kremers]

Hi William,

William Gardella wrote:
> I wonder if you've tried playing around with the `tls-checktrust'
> variable.  If you (setq tls-checktrust 'ask), you should be able to
> manually accept untrusted SSL/TLS keys by responding to a dialog in
> Emacs.  Along with setting `tls-checktrust' to ask, you can set
> `tls-program' to have a command line that will allow you to call
> gnutls-cli with your own private list of certificates.  For me here in
> Debian GNU/Linux, the following is a working TLS configuration that
> checks certs but also lets me use invalid/self-signed ones when I wish:
>
> (setq tls-checktrust 'ask
>       tls-program
>       '("gnutls-cli --x509cafile /etc/ssl/certs/ca-certificates.crt -p %p %h"
>         "gnutls-cli --x509cafile /etc/ssl/certs/ca-certificates.crt -p %p %h --protocols ssl3"
>         "openssl s_client -connect %h:%p -CAfile /etc/ssl/certs/ca-certificates.crt -no_ssl2 -ign_eof"))

I haven't been able to try this for the simple reason that I haven't
been able to find out where OS X stores its certificates...

> Maybe this will help you if gnutls --insecure is no longer working.

Well, right now, --insecure is the only thing that does work. What
puzzles me is that --tofu should also work, from what I read about it,
but it doesn't. The connection appears to be established, but the mail
is not sent and the connection remains open, it seems.

> If the above doesn't work, you can use strace or dtrace to see where in
> the TLS-calling process Emacs hangs.  You can also try writing out an
> `open-network-stream' function call of your Emacs trying to talk to your
> mail server and stepping through it with (edebug-defun).

and how would I go about doing that? :-)

-- 
Joost Kremers                                      joostkremers@yahoo.com
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

Re: smtpmail/starttls starts connection but never sends

[Peter Dyballa]

Am 14.11.2012 um 21:42 schrieb Joost Kremers:

> I haven't been able to try this for the simple reason that I haven't
> been able to find out where OS X stores its certificates...

They're either in Keychain Access.app or in files like (/opt/local/etc/openssl/cert.pem -> ) /opt/local/share/curl/curl-ca-bundle.crt, /sw/etc/ssl/certs/ca-bundle.crt, /etc/ssl/certs/ca-certificates.crt, /sw/share/gnupg/com-certs.pem, /usr/local/MacGPG2/share/gnupg/com-certs.pem, …, depending also on the application used.

You might also be able to check with lsof which files the utility keep open…

--
Mit friedvollen Grüßen

  Pete

Give a man a fish, and you've fed him for a day. Teach him to fish, and you've depleted the lake.

Re: smtpmail/starttls starts connection but never sends

[Joost Kremers]

Peter Dyballa wrote:
>
> Am 14.11.2012 um 21:42 schrieb Joost Kremers:
>
>> I haven't been able to try this for the simple reason that I haven't
>> been able to find out where OS X stores its certificates...
>
> They're either in Keychain Access.app or in files like
> (/opt/local/etc/openssl/cert.pem -> )
> /opt/local/share/curl/curl-ca-bundle.crt,
> /sw/etc/ssl/certs/ca-bundle.crt, /etc/ssl/certs/ca-certificates.crt,
> /sw/share/gnupg/com-certs.pem,

well, i use homebrew, so there's nothing in /opt/local/ and /sw/...

> /usr/local/MacGPG2/share/gnupg/com-certs.pem, …, depending also on the
> application used.

nothing there either...

> You might also be able to check with lsof which files the utility keep open…

good idea, will try that when i get to work.

thanks

-- 
Joost Kremers                                      joostkremers@yahoo.com
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

Re: smtpmail/starttls starts connection but never sends

[Peter Dyballa]

Am 16.11.2012 um 09:14 schrieb Joost Kremers:

> well, i use homebrew, so there's nothing in /opt/local/ and /sw/...

Then you can check the contents of the homebrew packages and grep for pem or crt file name extensions…

--
Mit friedvollen Grüßen

  Pete

Patriotismus ist die Überzeugung, dass unser Vaterland allen anderen Ländern überlegen ist, weil wir darin geboren wurden.
				(George Bernard Shaw)