gnutls tofu support? or even --insecure?

gnutls tofu support? or even --insecure?

[Nix]

So GnuTLS 3.2.21 has randomly (as in, I haven't updated it or touched
anything) started rejecting all connections to my work mailserver with
an apparently totally spurious certificate validation error:

- Status: The certificate is NOT trusted. The certificate issuer is unknown.
*** PKI verification of server certificate failed...
*** Fatal error: Error in the certificate.
*** Handshake has failed
GnuTLS error: Error in the certificate.

(when it's a perfectly normal Verisign cert in my certificate store, as
far as I can tell).

Life is *far* too short to figure out why this is (the whole thing is
happening over a VPN anyway, I trust this connection! I just can't tell
GnuTLS that!), so the thing that will save me is apparently --tofu,
though I'd be happy enough with --insecure. Unfortunately I can't get
Gnus to use either of these -- when (gnutls-available-p),
starttls-extra-arguments is ignored, as is tls-program, leaving me
forced to hack at gnutls.c if I want to read my work email any more. (I
find this somewhat unsatisfactory!)

Toke wrote a patch back in October of last year which implemented TOFU,
but now it doesn't remotely apply:
<https://lists.gnu.org/archive/html/emacs-devel/2014-10/msg00250.html>.

Does anyone know what happened to it? It doesn't seem to have been
applied, though it was applauded by several and adds a feature not
available in any other way (and a way of working *far* preferable to
trusting certificate authorities with anything ever).

I may have to reimplement it :)

-- 
NULL && (void)

Re: gnutls tofu support? or even --insecure?

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Emacs: ballast for RAM.
> Date: Tue, 11 Aug 2015 13:11:37 +0100
> Cc: Lars Magne Ingebrigtsen <larsi@gnus.org>, toke@toke.dk
> 
> So GnuTLS 3.2.21 has randomly (as in, I haven't updated it or touched
> anything) started rejecting all connections to my work mailserver with
> an apparently totally spurious certificate validation error:
> 
> - Status: The certificate is NOT trusted. The certificate issuer is unknown.
> *** PKI verification of server certificate failed...
> *** Fatal error: Error in the certificate.
> *** Handshake has failed
> GnuTLS error: Error in the certificate.
> 
> (when it's a perfectly normal Verisign cert in my certificate store, as
> far as I can tell).
> 
> Life is *far* too short to figure out why this is (the whole thing is
> happening over a VPN anyway, I trust this connection! I just can't tell
> GnuTLS that!), so the thing that will save me is apparently --tofu,

Before you press the panic button, wouldn't it be better to try to
find out why GnuTLS started failing?  My first suspect is the
certificate bundle.  Since this is Emacs 25.0 and GnuTLS > 3.0.20,
gnutls.c is supposed to use the system-stored certificates, but does
it?  The GnuTLS logs should show: if something goes wrong with that,
you should see a log message saying something like "setting system
trust failed with code NNN".

Or maybe you made some change to your system, unrelated to Emacs,
which somehow affected the system store of the certificates?

Or maybe one of your customizations forced it to use the external
bundle of certificate files, and that bundle is outdated or missing?
Again, there should be signs of that in the logs.

So I'd suggest to set gnutls-log-level to 10, and show all the logs
you get in *Messages* while initiating such a failing connection.

P.S. I have no opinions and no objections to applying the patch you
mentioned, but AFAICT the discussion back then raised a few issues
which the OP promised to work on.  Was there ever an updated patch?

Re: gnutls tofu support? or even --insecure?

[Toke Høiland-Jørgensen]

Eli Zaretskii <eliz@gnu.org> writes:

> P.S. I have no opinions and no objections to applying the patch you
> mentioned, but AFAICT the discussion back then raised a few issues
> which the OP promised to work on.  Was there ever an updated patch?

Well, the outcome was that the new network manager functionality would
replicate the tofu functionality in lisp and delegate only the
certificate checking to gnutls. But I lost track of what happened after
that; think the functionality was merged?

BTW, a quick fix can be to unset gnutls-available-p -- that will make
emacs fall back to the command line client. Did that for a while before
I wrote the original patch...

-Toke

Re: gnutls tofu support? or even --insecure?

[Nix]

On 11 Aug 2015, Toke Høiland-Jørgensen outgrape:

> Well, the outcome was that the new network manager functionality would
> replicate the tofu functionality in lisp and delegate only the
> certificate checking to gnutls. But I lost track of what happened after
> that; think the functionality was merged?

It was, but I'm still being asked about certs on every Emacs restart --
it doesn't seem to be remembering anything persistently.

It's clearly not the same as --tofu -- if gnutls doesn't like a
connection, it's not going to let you in, even if it connected perfectly
well to it in the past. This is true *no matter what* Emacs does -- I
don't see how you can possibly hope to replicate the 'connected fine
before, connect again without cert complaint or checking the cert chain'
without actually telling gnutls not to check the cert chain... which
this fault suggests is not happening.

-- 
NULL && (void)

Re: gnutls tofu support? or even --insecure?

[Ted Zlatanov]

On Wed, 12 Aug 2015 14:21:49 +0100 Nix <nix@esperi.org.uk> wrote: 

N> On 11 Aug 2015, Toke Høiland-Jørgensen outgrape:
>> Well, the outcome was that the new network manager functionality would
>> replicate the tofu functionality in lisp and delegate only the
>> certificate checking to gnutls. But I lost track of what happened after
>> that; think the functionality was merged?

N> It was, but I'm still being asked about certs on every Emacs restart --
N> it doesn't seem to be remembering anything persistently.

I'm not asked, and certs are stored for me. Can you check the cert
storage?

#+begin_src text
% cat ~/.emacs.d/network-security.data                                                                                  *[master] 
(
 (:id "sha1:f80ca63ad4bad106ae2a39e488272df8ddc4aaf8" :fingerprint "sha1:44:b4:29:38:9c:0c:68:f0:00:13:37:f3:90:42:94:17:08:e0:93:10" :host "letsencrypt.org:443")
...
)
#+end_src

N> It's clearly not the same as --tofu -- if gnutls doesn't like a
N> connection, it's not going to let you in, even if it connected perfectly
N> well to it in the past. This is true *no matter what* Emacs does -- I
N> don't see how you can possibly hope to replicate the 'connected fine
N> before, connect again without cert complaint or checking the cert chain'
N> without actually telling gnutls not to check the cert chain... which
N> this fault suggests is not happening.

I haven't experienced this issue myself, but maybe you can describe
the expected vs. actual behavior and how it can be improved in more
detail?  I can't follow your text, sorry.

Ted

Re: gnutls tofu support? or even --insecure?

[Nix]

[Back from holiday.]

On 19 Aug 2015, Ted Zlatanov stated:

> On Wed, 12 Aug 2015 14:21:49 +0100 Nix <nix@esperi.org.uk> wrote: 
>
> N> On 11 Aug 2015, Toke Høiland-Jørgensen outgrape:
>>> Well, the outcome was that the new network manager functionality would
>>> replicate the tofu functionality in lisp and delegate only the
>>> certificate checking to gnutls. But I lost track of what happened after
>>> that; think the functionality was merged?
>
> N> It was, but I'm still being asked about certs on every Emacs restart --
> N> it doesn't seem to be remembering anything persistently.
>
> I'm not asked, and certs are stored for me. Can you check the cert
> storage?

It's being stored. This was a stupidity in the site I was connecting to:
some sort of failover device whose nature is opaque to me was picking a
different cert (from an apparently infinite selection) or generating a
new cert or something like that about every five minutes. So Emacs was
right to warn: the effect was outwardly identical to a really-badly-
implemented man-in-the-middle attack.

-- 
NULL && (void)

Re: gnutls tofu support? or even --insecure?

[Lars Magne Ingebrigtsen]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Well, the outcome was that the new network manager functionality would
> replicate the tofu functionality in lisp and delegate only the
> certificate checking to gnutls. But I lost track of what happened after
> that; think the functionality was merged?

It was.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: gnutls tofu support? or even --insecure?

[Nix]

On 11 Aug 2015, Eli Zaretskii outgrape:

>> So GnuTLS 3.2.21 has randomly (as in, I haven't updated it or touched
>> anything) started rejecting all connections to my work mailserver with
>> an apparently totally spurious certificate validation error:
>> 
>> - Status: The certificate is NOT trusted. The certificate issuer is unknown.
>> *** PKI verification of server certificate failed...
>> *** Fatal error: Error in the certificate.
>> *** Handshake has failed
>> GnuTLS error: Error in the certificate.
>> 
>> (when it's a perfectly normal Verisign cert in my certificate store, as
>> far as I can tell).
>> 
>> Life is *far* too short to figure out why this is (the whole thing is
>> happening over a VPN anyway, I trust this connection! I just can't tell
>> GnuTLS that!), so the thing that will save me is apparently --tofu,
>
> Before you press the panic button, wouldn't it be better to try to
> find out why GnuTLS started failing?  My first suspect is the
> certificate bundle.

It was the certificate bundle. I dug out the relevant new root cert and
added it, and then things started working -- briefly. Now I've got into
a new state where it's failing in Emacs ("error in the pull function",
as usual), but working with gnutls-cli...

>                      Since this is Emacs 25.0 and GnuTLS > 3.0.20,
> gnutls.c is supposed to use the system-stored certificates, but does
> it?

Yes.

>      The GnuTLS logs should show: if something goes wrong with that,
> you should see a log message saying something like "setting system
> trust failed with code NNN".

I jacked up the gnutls-log-level to 2 and it failed:

Opening connection to xxxxx via tls...
gnutls.c: [1] (Emacs) connecting to host: xxxxx
gnutls.c: [1] (Emacs) allocating credentials
gnutls.c: [2] (Emacs) allocating x509 credentials
gnutls.c: [2] (Emacs) using default verification flags
gnutls.c: [1] (Emacs) setting the trustfile:  /etc/ssl/ca-bundle.pem
gnutls.c: [1] (Emacs) gnutls callbacks
gnutls.c: [1] (Emacs) gnutls_init
gnutls.c: [1] (Emacs) got non-default priority string: NORMAL
gnutls.c: [1] (Emacs) setting the priority string
gnutls.c: [audit] Note that the security level of the Diffie-Hellman key exchange has been lowered to 256 bits and this may allow decryption of the session data

gnutls.c: [2] ASSERT: gnutls_constate.c:583

gnutls.c: [2] ASSERT: gnutls_buffers.c:1078

gnutls.c: [2] ASSERT: gnutls_buffers.c:521

gnutls.c: [1] (Emacs) non-fatal error: Resource temporarily unavailable, try again.
gnutls.c: [2] ASSERT: gnutls_buffers.c:1078

gnutls.c: [2] ASSERT: gnutls_buffers.c:521

gnutls.c: [1] (Emacs) non-fatal error: Resource temporarily unavailable, try again.
gnutls.c: [2] ASSERT: gnutls_buffers.c:1078

gnutls.c: [2] ASSERT: gnutls_buffers.c:521

gnutls.c: [1] (Emacs) non-fatal error: Resource temporarily unavailable, try again.
gnutls.c: [2] ASSERT: gnutls_buffers.c:1078

gnutls.c: [2] ASSERT: gnutls_buffers.c:521

gnutls.c: [1] (Emacs) non-fatal error: Resource temporarily unavailable, try again.
gnutls.c: [2] ASSERT: gnutls_buffers.c:1078

gnutls.c: [2] ASSERT: gnutls_buffers.c:521

gnutls.c: [1] (Emacs) non-fatal error: Resource temporarily unavailable, try again.
gnutls.c: [2] ASSERT: gnutls_buffers.c:1078

gnutls.c: [2] ASSERT: gnutls_buffers.c:336

gnutls.c: [2] ASSERT: gnutls_buffers.c:521

gnutls.c: [2] ASSERT: gnutls_record.c:1026

gnutls.c: [2] ASSERT: gnutls_record.c:1146

gnutls.c: [2] ASSERT: gnutls_buffers.c:1329

gnutls.c: [2] ASSERT: gnutls_handshake.c:1414

gnutls.c: [2] ASSERT: gnutls_handshake.c:2701

gnutls.c: [1] (Emacs) fatal error: Error in the pull function.
gnutls.c: [2] errno: 32

gnutls.c: [2] ASSERT: gnutls_buffers.c:195

gnutls.c: [2] ASSERT: gnutls_buffers.c:653

gnutls.c: [2] ASSERT: gnutls_record.c:564

gnutls.el: (err=[-54] Error in the pull function.) boot: (:priority NORMAL :hostname xxxxx :loglevel 2 :min-prime-bits 256 :trustfiles (/etc/ssl/ca-bundle.pem) :crlfiles nil :keylist nil :verify-flags nil :verify-error nil :callbacks nil)
Unable to open server nnimap+xxxxx due to: GnuTLS error: #<process *nnimap*>, -54

(Errno 32 is -EPIPE.)

I jacked the log level up to 3, and everything worked! It looks like
control flow paths are quite different after gnutls_buffers.c:1078...

This is consistent behaviour, not a one-off.

This looks like a bug somewhere -- I'll report it as such shortly, after
spending a bit of time seeing if the bug is obvious enough that I can
fix it myself and just send a patch. I have a horrible workaround now,
which means I can at least get my work done today while sitting in a
nice pleasant Emacs :) -- always assuming that the other end doesn't
change again and make this problem go away for good.

> Or maybe you made some change to your system, unrelated to Emacs,
> which somehow affected the system store of the certificates?

No changes -- howeer, work's email is up the creek to a greater or
lesser degree right now, so who knows what sort of weird edge case
gnutls is being hit with -- but still it is working for gnutls-cli and
Thunderbird and the web client and everything but Emacs at this point :(

> Or maybe one of your customizations forced it to use the external
> bundle of certificate files, and that bundle is outdated or missing?
> Again, there should be signs of that in the logs.

That was definitely true at first: I found the bundle by stracing
gnutls-cli, and put one where it was expecting.

> So I'd suggest to set gnutls-log-level to 10, and show all the logs
> you get in *Messages* while initiating such a failing connection.

Tried that. This is, it seems, one of those bugs that goes away when you
look more closely at it.

> P.S. I have no opinions and no objections to applying the patch you
> mentioned, but AFAICT the discussion back then raised a few issues
> which the OP promised to work on.  Was there ever an updated patch?

Not that I've been able to find.

-- 
NULL && (void)

Re: gnutls tofu support? or even --insecure?

[Steinar Bang]

>>>>> Nix <nix@esperi.org.uk>:

> Tried that. This is, it seems, one of those bugs that goes away when you
> look more closely at it.

(aka. a "heisenbug"...)

Re: gnutls tofu support? or even --insecure?

[Nix]

On 12 Aug 2015, Steinar Bang told this:

>>>>>> Nix <nix@esperi.org.uk>:
>
>> Tried that. This is, it seems, one of those bugs that goes away when you
>> look more closely at it.
>
> (aka. a "heisenbug"...)

I've managed to reproduce it even in high verbosity: *sending* mail is
still broken. Time to debug...

-- 
NULL && (void)