[PATCH RFC] GnuTLS: Support TOFU certificate checking.

[PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

This implements rudimentary Trust On First Use certificate checking in
gnutls.c. This is useful for protecting against MITM attacks when
connecting to servers using self-signed certificates, as well as to
guard against rogue or compromised CAs issuing illegitimate
certificates that would otherwise be accepted.

To test:

(require 'gnutls)
(setq gnutls-verify-error '((".*" :tofu))
(open-gnutls-stream "test" nil "google.com" 443) ; this should fail

To add the certificate to the trust store, execute (in a shell)
`gnutls-cli --tofu -p 443 google.com` and answer yes when it asks
whether to trust the certificate. Doing so should cause the open to
success the next time around.

As noted above, this is quite rudimentary as it is right now, and will
probably need to be expanded. However, I wanted to solicit some feedback
first, and so here's a few questions:

1. Would this be viable to include at all? And if so, is this the right
   way to go about it?

2. There's currently no way to add a certificate to the store when it's
   first seen. What's the best way to go about this? I'm not sure how to
   actually communicate with the user from within gnutls-boot.

3. Currently this uses the global (well, per-user) GnuTLS certificate
   store. My thought was this makes the most sense (it's still per-port,
   so the same hostname can have different certificates for different
   services). But is it?

4. Setting gnutls-verify-error to t does not activate TOFU checking. The
   reasoning behind this is that TOFU works somewhat differently than
   the other trust models (CA and hostname matching), and so is a
   feature that is probably best left for people to explicitly ask for.
   Is this reasonable?

5. Any other comments. For instance, I have only tested this on Linux,
   so not sure if I fudged up all the library loading magic for W32...

Thanks in advance for looking this over.

As an aside, while testing this I found that using customize to set
gnutls-verify-error, a doubly-nested list of symbols would end up
getting passed to gnutls-boot, which would then subsequently fail all
the membership tests on it. This seems to stem from the fact that
customize produces list entries of the form '((".*" (:tofu))) while
gnutls-negotiate passes the cdr of each matched entry to cl-mapcan. Not
sure which should be fixed, so I just set it manually while testing, but
thought I'd point it out :)

Cheers,
-Toke
---
 lisp/net/gnutls.el | 10 +++++++---
 src/gnutls.c       | 43 ++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 49 insertions(+), 4 deletions(-)

diff --git a/lisp/net/gnutls.el b/lisp/net/gnutls.el
index 0c650f3..9809b0f 100644
--- a/lisp/net/gnutls.el
+++ b/lisp/net/gnutls.el
@@ -63,7 +63,8 @@ set this variable to \"normal:-dhe-rsa\"."
                     (const ".*" :tag "Any hostname")
                     regexp)
             (set (const :trustfiles)
-                 (const :hostname))))))
+                 (const :hostname)
+                 (const :tofu))))))
 
 (defcustom gnutls-trustfiles
   '(
@@ -123,7 +124,8 @@ GnuTLS connection, including specifying the credential type,
 trust and key files, and priority string."
   (gnutls-negotiate :process (open-network-stream name buffer host service)
                     :type 'gnutls-x509pki
-                    :hostname host))
+                    :hostname host
+                    :service service))
 
 (define-error 'gnutls-error "GnuTLS error")
 
@@ -133,7 +135,7 @@ trust and key files, and priority string."
 
 (cl-defun gnutls-negotiate
     (&rest spec
-           &key process type hostname priority-string
+           &key process type hostname service priority-string
            trustfiles crlfiles keylist min-prime-bits
            verify-flags verify-error verify-hostname-error
            &allow-other-keys)
@@ -144,6 +146,7 @@ Note arguments are passed CL style, :type TYPE instead of just TYPE.
 TYPE is `gnutls-x509pki' (default) or `gnutls-anon'.  Use nil for the default.
 PROCESS is a process returned by `open-network-stream'.
 HOSTNAME is the remote hostname.  It must be a valid string.
+SERVICE is the remote service.  It will be formatted as a string.
 PRIORITY-STRING is as per the GnuTLS docs, default is \"NORMAL\".
 TRUSTFILES is a list of CA bundles.  It defaults to `gnutls-trustfiles'.
 CRLFILES is a list of CRL files.
@@ -226,6 +229,7 @@ defaults to GNUTLS_VERIFY_ALLOW_X509_V1_CA_CRT."
 
     (setq params `(:priority ,priority-string
                              :hostname ,hostname
+                             :service ,(format "%s" service)
                              :loglevel ,gnutls-log-level
                              :min-prime-bits ,min-prime-bits
                              :trustfiles ,trustfiles
diff --git a/src/gnutls.c b/src/gnutls.c
index 5d48f78..a08ec8e 100644
--- a/src/gnutls.c
+++ b/src/gnutls.c
@@ -43,11 +43,13 @@ static bool gnutls_global_initialized;
 /* The following are for the property list of `gnutls-boot'.  */
 static Lisp_Object QCgnutls_bootprop_priority;
 static Lisp_Object QCgnutls_bootprop_trustfiles;
+static Lisp_Object QCgnutls_bootprop_tofu;
 static Lisp_Object QCgnutls_bootprop_keylist;
 static Lisp_Object QCgnutls_bootprop_crlfiles;
 static Lisp_Object QCgnutls_bootprop_callbacks;
 static Lisp_Object QCgnutls_bootprop_loglevel;
 static Lisp_Object QCgnutls_bootprop_hostname;
+static Lisp_Object QCgnutls_bootprop_service;
 static Lisp_Object QCgnutls_bootprop_min_prime_bits;
 static Lisp_Object QCgnutls_bootprop_verify_flags;
 static Lisp_Object QCgnutls_bootprop_verify_error;
@@ -141,6 +143,9 @@ DEF_GNUTLS_FN (void, gnutls_transport_set_push_function,
 	       (gnutls_session_t, gnutls_push_func));
 DEF_GNUTLS_FN (int, gnutls_x509_crt_check_hostname,
 	       (gnutls_x509_crt_t, const char *));
+DEF_GNUTLS_FN (int, gnutls_verify_stored_pubkey,
+	       (const char *, gnutls_tdb_t, const char *,const char *,
+		gnutls_cerificate_type_t,const gnutls_datum_t *,unsigned int));
 DEF_GNUTLS_FN (void, gnutls_x509_crt_deinit, (gnutls_x509_crt_t));
 DEF_GNUTLS_FN (int, gnutls_x509_crt_import,
 	       (gnutls_x509_crt_t, const gnutls_datum_t *,
@@ -202,6 +207,7 @@ init_gnutls_functions (void)
   LOAD_GNUTLS_FN (library, gnutls_transport_set_pull_function);
   LOAD_GNUTLS_FN (library, gnutls_transport_set_push_function);
   LOAD_GNUTLS_FN (library, gnutls_x509_crt_check_hostname);
+  LOAD_GNUTLS_FN (library, gnutls_verify_stored_pubkey);
   LOAD_GNUTLS_FN (library, gnutls_x509_crt_deinit);
   LOAD_GNUTLS_FN (library, gnutls_x509_crt_import);
   LOAD_GNUTLS_FN (library, gnutls_x509_crt_init);
@@ -257,6 +263,7 @@ init_gnutls_functions (void)
 #endif
 #define fn_gnutls_transport_set_ptr2		gnutls_transport_set_ptr2
 #define fn_gnutls_x509_crt_check_hostname	gnutls_x509_crt_check_hostname
+#define fn_gnutls_verify_stored_pubkey		gnutls_verify_stored_pubkey
 #define fn_gnutls_x509_crt_deinit		gnutls_x509_crt_deinit
 #define fn_gnutls_x509_crt_import		gnutls_x509_crt_import
 #define fn_gnutls_x509_crt_init			gnutls_x509_crt_init
@@ -738,6 +745,8 @@ PROPLIST is a property list with the following keys:
 
 :hostname is a string naming the remote host.
 
+:service is a port number naming the remote service.
+
 :priority is a GnuTLS priority string, defaults to "NORMAL".
 
 :trustfiles is a list of PEM-encoded trust files for `gnutls-x509pki'.
@@ -759,7 +768,8 @@ instead.
 
 :verify-error is a list of symbols to express verification checks or
 `t' to do all checks.  Currently it can contain `:trustfiles' and
-`:hostname' to verify the certificate or the hostname respectively.
+`:hostname' to verify the certificate or the hostname respectively, as
+well as `:tofu' to turn on Trust On First Use mode.
 
 :min-prime-bits is the minimum accepted number of bits the client will
 accept in Diffie-Hellman key exchange.
@@ -795,6 +805,7 @@ one trustfile (usually a CA bundle).  */)
   char const *priority_string_ptr = "NORMAL"; /* default priority string.  */
   unsigned int peer_verification;
   char *c_hostname;
+  char *c_service;
 
   /* Placeholders for the property list elements.  */
   Lisp_Object priority_string;
@@ -804,6 +815,7 @@ one trustfile (usually a CA bundle).  */)
   /* Lisp_Object callbacks; */
   Lisp_Object loglevel;
   Lisp_Object hostname;
+  Lisp_Object service;
   Lisp_Object verify_error;
   Lisp_Object prime_bits;
 
@@ -818,6 +830,7 @@ one trustfile (usually a CA bundle).  */)
     error ("Invalid GnuTLS credential type");
 
   hostname              = Fplist_get (proplist, QCgnutls_bootprop_hostname);
+  service               = Fplist_get (proplist, QCgnutls_bootprop_service);
   priority_string       = Fplist_get (proplist, QCgnutls_bootprop_priority);
   trustfiles            = Fplist_get (proplist, QCgnutls_bootprop_trustfiles);
   keylist               = Fplist_get (proplist, QCgnutls_bootprop_keylist);
@@ -839,6 +852,10 @@ one trustfile (usually a CA bundle).  */)
     error ("gnutls-boot: invalid :hostname parameter (not a string)");
   c_hostname = SSDATA (hostname);
 
+  if (!STRINGP (service))
+    error ("gnutls-boot: invalid :service parameter (not a string)");
+  c_service = SSDATA (service);
+
   state = XPROCESS (proc)->gnutls_state;
 
   if (TYPE_RANGED_INTEGERP (int, loglevel))
@@ -1141,6 +1158,28 @@ one trustfile (usually a CA bundle).  */)
                            c_hostname);
 	    }
 	}
+
+      if (!NILP (Fmember (QCgnutls_bootprop_tofu, verify_error)))
+	{
+	  ret = fn_gnutls_verify_stored_pubkey(NULL, NULL, c_hostname, c_service, GNUTLS_CRT_X509, gnutls_verify_cert_list, 0);
+	  if (ret == GNUTLS_E_NO_CERTIFICATE_FOUND)
+	    {
+	      fn_gnutls_x509_crt_deinit (gnutls_verify_cert);
+	      emacs_gnutls_deinit (proc);
+	      error ("No TOFU trust entry found for hostname \"%s\" and service \"%s\"", c_hostname, c_service);
+	    }
+	  else if (ret == GNUTLS_E_CERTIFICATE_KEY_MISMATCH)
+	    {
+	      fn_gnutls_x509_crt_deinit (gnutls_verify_cert);
+	      emacs_gnutls_deinit (proc);
+	      error ("TOFU trust MISMATCH for hostname \"%s\" and service \"%s\"", c_hostname, c_service);
+	    }
+	  else if (ret < GNUTLS_E_SUCCESS)
+	    {
+	      fn_gnutls_x509_crt_deinit (gnutls_verify_cert);
+	      return gnutls_make_error (ret);
+	    }
+	}
       fn_gnutls_x509_crt_deinit (gnutls_verify_cert);
     }
 
@@ -1189,8 +1228,10 @@ syms_of_gnutls (void)
   DEFSYM (Qgnutls_anon, "gnutls-anon");
   DEFSYM (Qgnutls_x509pki, "gnutls-x509pki");
   DEFSYM (QCgnutls_bootprop_hostname, ":hostname");
+  DEFSYM (QCgnutls_bootprop_service, ":service");
   DEFSYM (QCgnutls_bootprop_priority, ":priority");
   DEFSYM (QCgnutls_bootprop_trustfiles, ":trustfiles");
+  DEFSYM (QCgnutls_bootprop_tofu, ":tofu");
   DEFSYM (QCgnutls_bootprop_keylist, ":keylist");
   DEFSYM (QCgnutls_bootprop_crlfiles, ":crlfiles");
   DEFSYM (QCgnutls_bootprop_callbacks, ":callbacks");
-- 
2.1.2

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> (require 'gnutls)
> (setq gnutls-verify-error '((".*" :tofu))
> (open-gnutls-stream "test" nil "google.com" 443) ; this should fail
>
> To add the certificate to the trust store, execute (in a shell)
> `gnutls-cli --tofu -p 443 google.com` and answer yes when it asks
> whether to trust the certificate. Doing so should cause the open to
> success the next time around.

I think all the certificate checking should just work out of the box
without the user having to do any configuration or shell commands.
I.e., it should be done by `open-network-stream'.

See

http://permalink.gmane.org/gmane.emacs.devel/174908

for how I think this should work from the user's standpoint, if you want
to implement it.  >"?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I think all the certificate checking should just work out of the box
> without the user having to do any configuration or shell commands.
> I.e., it should be done by `open-network-stream'.

Right, I can definitely see the point of that, and ultimately this is
definitely desirable. The GnuTLS TOFU mode could be a way to do the
heavy lifting of certificate fingerprint storing and verification etc.

I don't think I'm sufficiently familiar with the innards of
open-network-stream to implement this, sorry. However, if you agree this
could be a reasonable building block for the user-facing functionality I
could rework the patch to (a) signal an appropriate error code when
verification fails and (b) add a parameter to add the certificate to the
trust chain. The lisp code could then use this functionality (by passing
the appropriate parameters to gnutls-boot) to implement the user-facing
y/no/maybe/whatever on top of it.

This would allow me to scratch my own itch (getting basic TOFU support
for me to put into gnutls-verify-error) while at the same time
contributing something to the overall problem.

Would you be interested in such a patch? :)


Also, I'll add that TOFU can also be used to ensure stronger trust than
just checking that the certificate validates; it can also be used for
certificate pinning to ensure that it doesn't change. This is what I use
it for personally, and I consider it a nice added security...

-Toke

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Right, I can definitely see the point of that, and ultimately this is
> definitely desirable. The GnuTLS TOFU mode could be a way to do the
> heavy lifting of certificate fingerprint storing and verification etc.
>
> I don't think I'm sufficiently familiar with the innards of
> open-network-stream to implement this, sorry. However, if you agree this
> could be a reasonable building block for the user-facing functionality I
> could rework the patch to (a) signal an appropriate error code when
> verification fails and (b) add a parameter to add the certificate to the
> trust chain. The lisp code could then use this functionality (by passing
> the appropriate parameters to gnutls-boot) to implement the user-facing
> y/no/maybe/whatever on top of it.

Yes, that's what the Emacs gnutls code needs: A way to access the
certificate, and the verification status of that certificate (i.e.,
whether it managed to validate it or not, and if not, why not).

Then the management of this could be done at a higher level, which would
be `open-network-stream'.

> Also, I'll add that TOFU can also be used to ensure stronger trust than
> just checking that the certificate validates; it can also be used for
> certificate pinning to ensure that it doesn't change. This is what I use
> it for personally, and I consider it a nice added security...

Yes, `open-network-stream' would implement certificate pinning.  That
is, it would store a fingerprint of the certificate and query the user
for what to do when that changes.  It would also use that to keep track
of whether a STARTTLS connection suddenly starts not offering STARTTLS,
which would be a typical symptom of a man-in-the-middle attack filtering
out the STARTTLS dialogue from the server.

If you implement the C gnutls bits, that would be great.  Then somebody
else (ahem, probably me) could do the `open-network-stream' bits...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Yes, `open-network-stream' would implement certificate pinning. 

(By default.  But there should probably be a way for the user to switch
it off, since the user may not want to store a list of servers (with
self-signed certificates) it has talked to -- for privacy reasons.  But
that's a pretty marginal issue.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Then the management of this could be done at a higher level, which
> would be `open-network-stream'.

Right, so (just to make sure I'm understanding you right), what you
propose is to get rid of all the current validation logic in C (i.e the
erroring out) and just return something like (<cert hash> <cert
hostname> <CA validity status>) -- and then make the lisp code work out
the rest?

Right now it seems the C code refuses to even return the opened network
stream object if validation fails; with this, that would have to change,
and the C code wouldn't make any policy decisions?

-Toke

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Right, so (just to make sure I'm understanding you right), what you
> propose is to get rid of all the current validation logic in C (i.e the
> erroring out) and just return something like (<cert hash> <cert
> hostname> <CA validity status>) -- and then make the lisp code work out
> the rest?

Yup, I think that would be more flexible.

I think it would also be nice if the entire cert was also returned (in a
convenient format), so that Emacs can display the pertinent parts while
querying the user about what action to take.

Like "signed by CA foo on date baz for host zot" and so on.

And perhaps display graphically the fingerprint like ssh does?  I have
no idea what's involved there, so I don't know whether that would be
possible (or easy)...

> Right now it seems the C code refuses to even return the opened network
> stream object if validation fails; with this, that would have to change,
> and the C code wouldn't make any policy decisions?

Yup.  But perhaps Ted (or other people) have opinions here.  >"?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I think it would also be nice if the entire cert was also returned (in
> a convenient format), so that Emacs can display the pertinent parts
> while querying the user about what action to take.

Okay, so basically a property list with everything that might be
interesting? Or did you mean a text string?

> And perhaps display graphically the fingerprint like ssh does? I have
> no idea what's involved there, so I don't know whether that would be
> possible (or easy)...

Well, gnutls-cli displays one of those as well. So I suppose it has a
way to do that; the question is whether it's exposed in the API
somewhere. I'll go poking around. Can't promise anything about when,
though; had expected this to be a small patch that would just be enough
for me to stop using gnutls-cli to get TOFU trust... ;)

-Toke

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Okay, so basically a property list with everything that might be
> interesting? Or did you mean a text string?

A property list would be most convenient.

>> And perhaps display graphically the fingerprint like ssh does? I have
>> no idea what's involved there, so I don't know whether that would be
>> possible (or easy)...
>
> Well, gnutls-cli displays one of those as well. So I suppose it has a
> way to do that; the question is whether it's exposed in the API
> somewhere. I'll go poking around.

Great!

> Can't promise anything about when, though; had expected this to be a
> small patch that would just be enough for me to stop using gnutls-cli
> to get TOFU trust... ;)

Hehe.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 08 Oct 2014 14:18:27 +0200
> MailScanner-NULL-Check: 1413375507.63653@xsOGipRhF1W7tfzR6ptQBA
> Cc: Ted Zlatanov <tzz@lifelogs.com>, emacs-devel@gnu.org
> 
> Toke Høiland-Jørgensen <toke@toke.dk> writes:
> 
> > Right, so (just to make sure I'm understanding you right), what you
> > propose is to get rid of all the current validation logic in C (i.e the
> > erroring out) and just return something like (<cert hash> <cert
> > hostname> <CA validity status>) -- and then make the lisp code work out
> > the rest?
> 
> Yup, I think that would be more flexible.

I don't see how this could be done: the initialization of TLS network
stream creates a descriptor and adds it to the descriptors we watch in
wait_reading_process_output.  If that descriptor is invalid, we will
likely crash.

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Right, so (just to make sure I'm understanding you right), what you
>> > propose is to get rid of all the current validation logic in C (i.e the
>> > erroring out) and just return something like (<cert hash> <cert
>> > hostname> <CA validity status>) -- and then make the lisp code work out
>> > the rest?
>> 
>> Yup, I think that would be more flexible.
>
> I don't see how this could be done: the initialization of TLS network
> stream creates a descriptor and adds it to the descriptors we watch in
> wait_reading_process_output.  If that descriptor is invalid, we will
> likely crash.

It would still need to return a file descriptor, but would have extra
accessors for accessing the certificate stuff.  Sorry if I was
misleading in my response to Toke...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Cc: toke@toke.dk,  tzz@lifelogs.com,  emacs-devel@gnu.org
> Date: Wed, 08 Oct 2014 14:56:42 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > Right, so (just to make sure I'm understanding you right), what you
> >> > propose is to get rid of all the current validation logic in C (i.e the
> >> > erroring out) and just return something like (<cert hash> <cert
> >> > hostname> <CA validity status>) -- and then make the lisp code work out
> >> > the rest?
> >> 
> >> Yup, I think that would be more flexible.
> >
> > I don't see how this could be done: the initialization of TLS network
> > stream creates a descriptor and adds it to the descriptors we watch in
> > wait_reading_process_output.  If that descriptor is invalid, we will
> > likely crash.
> 
> It would still need to return a file descriptor, but would have extra
> accessors for accessing the certificate stuff.

How can it return a valid descriptor without all the validations it
does before that, which (AFAIU) you want to delegate to Lisp now?

Apologies if I misunderstand your plan.

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> How can it return a valid descriptor without all the validations it
> does before that, which (AFAIU) you want to delegate to Lisp now?

It would validate the certificate, but never drop the connection even if
the certificate is invalid.  It would be up to `open-network-stream' to
close the connection if the user decides to not accept the invalid (or
self-signed) certificate.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Cc: toke@toke.dk,  tzz@lifelogs.com,  emacs-devel@gnu.org
> Date: Wed, 08 Oct 2014 15:06:52 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > How can it return a valid descriptor without all the validations it
> > does before that, which (AFAIU) you want to delegate to Lisp now?
> 
> It would validate the certificate, but never drop the connection even if
> the certificate is invalid.  It would be up to `open-network-stream' to
> close the connection if the user decides to not accept the invalid (or
> self-signed) certificate.

So you want to return a descriptor for a connection that failed
certificate validation, and let the application handle that?  That
could work, but I don't know what security-wary people here will tell
about keeping such connections.

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> So you want to return a descriptor for a connection that failed
> certificate validation, and let the application handle that?

The other option is to have the C layer close the connection, signal an
error, have `open-network-stream' query the user about the invalid
certificate, the user says "connect anyway", and then we'd reconnect
with other options.

That seems less ... convenient.

> That could work, but I don't know what security-wary people here will
> tell about keeping such connections.

I think I know.  >"?

But there should be no further security implications, really.  If you're
using `open-network-stream'.  If you're using the low-level C functions
yourself, you have to respond to the invalid certificate yourself, but
why would you?

We're just moving the certificate handling up to the Lisp level --
nothing more.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Cc: toke@toke.dk,  tzz@lifelogs.com,  emacs-devel@gnu.org
> Date: Wed, 08 Oct 2014 15:25:43 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > So you want to return a descriptor for a connection that failed
> > certificate validation, and let the application handle that?
> 
> The other option is to have the C layer close the connection, signal an
> error, have `open-network-stream' query the user about the invalid
> certificate, the user says "connect anyway", and then we'd reconnect
> with other options.
> 
> That seems less ... convenient.
> 
> > That could work, but I don't know what security-wary people here will
> > tell about keeping such connections.
> 
> I think I know.  >"?

What happens if some stuff comes out of the stream that failed to be
validated, while Emacs negotiates with the user about what to do?
Normally, we would pass this stuff to whatever sentinel was defined,
or insert it into a buffer.  Is that what you want?

> We're just moving the certificate handling up to the Lisp level --
> nothing more.

If what you want is to cause gnutls-boot call out to Lisp for
validation as part of its normal path, then that's fine, I think.  But
it does mean that we have no stream until the entire validation
completes.  My understanding of what was being suggested here was that
this is not what you have in mind:

> Right, so (just to make sure I'm understanding you right), what you
> propose is to get rid of all the current validation logic in C (i.e the
> erroring out) and just return something like (<cert hash> <cert
> hostname> <CA validity status>) -- and then make the lisp code work out
> the rest?
> 
> Right now it seems the C code refuses to even return the opened network
> stream object if validation fails; with this, that would have to change,
> and the C code wouldn't make any policy decisions?

That doesn't sound to me like "just moving the certificate handling up
to the Lisp level".

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> What happens if some stuff comes out of the stream that failed to be
> validated, while Emacs negotiates with the user about what to do?
> Normally, we would pass this stuff to whatever sentinel was defined,
> or insert it into a buffer.  Is that what you want?

Does that really happen that early in the connection process?  I thought
sentinels and buffers were attached at a point later, so that
`open-network-stream' would have a chance of inspecting the stream
first.

It's been a while since I looked at the code, so if that's wrong (and
can't be fixed), then we'd have to do it the way you suggest:

> If what you want is to cause gnutls-boot call out to Lisp for
> validation as part of its normal path, then that's fine, I think.  But
> it does mean that we have no stream until the entire validation
> completes. 

However, I was hoping to get the "bug out if the stream isn't encrypted
and you wanted that" into the same code, so it would be nice to have it
all in the same code path.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> However, I was hoping to get the "bug out if the stream isn't encrypted
> and you wanted that" into the same code, so it would be nice to have it
> all in the same code path.

So a generic callback from C to lisp that includes the encryption
status? However, doesn't open-network-stream use separate functions
depending on the encryption mode? I.e. the same C code is not run for
all of them?

-Toke

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> However, I was hoping to get the "bug out if the stream isn't encrypted
>> and you wanted that" into the same code, so it would be nice to have it
>> all in the same code path.
>
> So a generic callback from C to lisp that includes the encryption
> status? However, doesn't open-network-stream use separate functions
> depending on the encryption mode? I.e. the same C code is not run for
> all of them?

I've just had a fresh look at `open-network-stream'.  Looks to me like
it all happens way before sentinels are attached.  And if the buffer is
a problem (and I don't think it is), then we can attach the buffer at a
later point.

So I don't think this really is a problem.  The `open-gnutls-stream'
function can just return a descriptor, and `open-network-stream' can
then query the descriptor and then decide what to do with it (which
would be closing it and raising an error if the certificate is invalid
or self-signed, and the user confirms that no connection should happen).

I think.  Unless there's something subtle here I'm missing...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Cc: toke@toke.dk,  tzz@lifelogs.com,  emacs-devel@gnu.org
> Date: Wed, 08 Oct 2014 15:47:33 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > What happens if some stuff comes out of the stream that failed to be
> > validated, while Emacs negotiates with the user about what to do?
> > Normally, we would pass this stuff to whatever sentinel was defined,
> > or insert it into a buffer.  Is that what you want?
> 
> Does that really happen that early in the connection process?  I thought
> sentinels and buffers were attached at a point later, so that
> `open-network-stream' would have a chance of inspecting the stream
> first.

It won't happen before open-network-stream returns, or close to its
end, but then I don't quite understand where do you want to insert the
validation stuff.  Did you intend to do that inside
open-network-stream?  If so, it will suddenly start talking to the
user, something that applications might not be ready for.

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Did you intend to do that inside open-network-stream?

Yes.

> If so, it will suddenly start talking to the user, something that
> applications might not be ready for.

I don't see why the application would need to know much, if anything,
about it.  `open-network-stream' would say "Invalid certificate.  (Bla
bla.)  Connect anyway?" and the user would type `y' or `n'.

I can't really see how that would disturb smtpmail, Gnus, nnimap, pop3,
rmail, erc or any other application I can think of...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 08 Oct 2014 16:09:20 +0200
> Cc: tzz@lifelogs.com, toke@toke.dk, emacs-devel@gnu.org
> 
> I can't really see how that would disturb smtpmail, Gnus, nnimap, pop3,
> rmail, erc or any other application I can think of...

Then I'm sorry for making the noise.

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Ted Zlatanov]

On Wed, 08 Oct 2014 16:09:20 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> I don't see why the application would need to know much, if anything,
LMI> about it.  `open-network-stream' would say "Invalid certificate.  (Bla
LMI> bla.)  Connect anyway?" and the user would type `y' or `n'.

LMI> I can't really see how that would disturb smtpmail, Gnus, nnimap, pop3,
LMI> rmail, erc or any other application I can think of...

What about `emacs --batch'? That change would affect the URL retrieval
code and thus unattended package installs for instance. Do you just
reject certificates in batch mode? Or add a `--tofu-accept=SERVER_REGEX'
option to Emacs for batch mode?

Ted

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> What about `emacs --batch'? That change would affect the URL retrieval
> code and thus unattended package installs for instance. Do you just
> reject certificates in batch mode? Or add a `--tofu-accept=SERVER_REGEX'
> option to Emacs for batch mode?

If the user can't answer questions, the default would be to reject
invalid certificates.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Ted Zlatanov]

On Wed, 08 Oct 2014 17:31:33 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> What about `emacs --batch'? That change would affect the URL retrieval
>> code and thus unattended package installs for instance. Do you just
>> reject certificates in batch mode? Or add a `--tofu-accept=SERVER_REGEX'
>> option to Emacs for batch mode?

LMI> If the user can't answer questions, the default would be to reject
LMI> invalid certificates.

They are not necessarily invalid. But yeah, I agree that's reasonable
behavior as long as it's noted loudly (with a possible remedy in the
message).

Ted

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Stephen J. Turnbull]

Ted Zlatanov writes:
 > On Wed, 08 Oct 2014 17:31:33 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

 > LMI> If the user can't answer questions, the default would be to reject
 > LMI> invalid certificates.
 > 
 > They are not necessarily invalid.

Youngsters these days have trouble with precise use of English, and
the usage "invalid" for "unable to establish a chain of trust to a
trusted root"" is common.  Get used to it, old man. :-)

FWIW, I think I'm one of the more paranoid folks around Emacs
development, and I don't see a problem with completing whatever
operations are necessary to get safely back to Lisp to query the user,
as long as

(1) No data is transmitted from Emacs to the remote, except that
    needed by the protocol to establish the connection (transmitting
    credentials should be avoided if possible, but that may not be
    possible in some protocols).

(2) All data received is squirreled away in a buffer inaccessible to
    Emacs (except for the code that will eventually move it to Lisp,
    of course), and this buffer is "read-locked" until permission is
    received from the user.

(3) The amount of data accepted is effectively limited (to avoid DoS
    attacks -- this is probably gilding a lily, but I am paranoid).

Regards,

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Ted Zlatanov]

On Thu, 09 Oct 2014 11:43:49 +0900 "Stephen J. Turnbull" <stephen@xemacs.org> wrote: 

SJT> (2) All data received is squirreled away in a buffer inaccessible to
SJT>     Emacs (except for the code that will eventually move it to Lisp,
SJT>     of course), and this buffer is "read-locked" until permission is
SJT>     received from the user.

I don't think that kind of secure storage and authorization layering is
available in today's Emacs. It would be nice to have it for general
passwords as well. You can sort of hide data from casual dumping by
wrapping it in a closure, but unauthorized code can still extract it
easily.

Ted

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> It would validate the certificate, but never drop the connection even
> if the certificate is invalid. It would be up to `open-network-stream'
> to close the connection if the user decides to not accept the invalid
> (or self-signed) certificate.

FWIW this was what I understood from your earlier emails as well. It
does mean that the handshake will be completed before validation is
done, which goes counter to how these things are usually done. Not sure
if that is a problem; it surely will be if there's lisp code that
doesn't go through open-network-stream...

Another alternative would be to fail in the C code and report why, along
with the cert details; and then make it up to the lisp code to retry the
connection with appropriate flags set to ignore (some) errors based on
user response... That would be similar to how the gnutls lisp/C
interaction is now (I think), with the addition that certificate details
are returned along with any errors signalled...

-Toke

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Ted Zlatanov]

On Tue,  7 Oct 2014 23:16:05 +0200 Toke Høiland-Jørgensen <toke@toke.dk> wrote: 

TH> This implements rudimentary Trust On First Use certificate checking in
TH> gnutls.c. This is useful for protecting against MITM attacks when
TH> connecting to servers using self-signed certificates, as well as to
TH> guard against rogue or compromised CAs issuing illegitimate
TH> certificates that would otherwise be accepted.

Wonderful work, thank you!

You should check in the autoconf script whether
`gnutls_verify_stored_pubkey' is available.  It's a fairly new function
and we have to support older versions of GnuTLS without it.

TH> (require 'gnutls)
TH> (setq gnutls-verify-error '((".*" :tofu))
TH> (open-gnutls-stream "test" nil "google.com" 443) ; this should fail

TH> To add the certificate to the trust store, execute (in a shell)
TH> `gnutls-cli --tofu -p 443 google.com` and answer yes when it asks
TH> whether to trust the certificate. Doing so should cause the open to
TH> success the next time around.

I like the simplicity of it. It would be nice to do this inside Emacs
itself (it's OK if it requires some kind of `emacs --batch' call,
doesn't have to be immediate). Either way, the errors should tell us
specifically what to run from the command line in these two cases:

    error ("No TOFU trust entry found for hostname \"%s\" and service \"%s\"", c_hostname, c_service);
    error ("TOFU trust MISMATCH for hostname \"%s\" and service \"%s\"", c_hostname, c_service);

TH> 1. Would this be viable to include at all? And if so, is this the right
TH>    way to go about it?

I think it's good enough to include right now.  We can iterate the
design from the C side but the Lisp side seems simple enough.  I would
even make :tofu the default (we've had debates on whether it should be t
but IMHO :tofu is a better default).

TH> 2. There's currently no way to add a certificate to the store when it's
TH>    first seen. What's the best way to go about this? I'm not sure how to
TH>    actually communicate with the user from within gnutls-boot.

You can't, that's been the problem so far. That's a very picky part of
the Emacs code and breaking out for any kind of interaction caused
problems. That's why I like your pragmatic solution above.  But Lars'
proposal may be the nicer way if it can be hacked into place.

TH> 3. Currently this uses the global (well, per-user) GnuTLS certificate
TH>    store. My thought was this makes the most sense (it's still per-port,
TH>    so the same hostname can have different certificates for different
TH>    services). But is it?

I think so. But Emacs creates a Emacs-specific homedir for the GnuPG
keychain, for verification of the package archives, in
`~/.emacs.d/gnupg' which is an equally valid approach. So I don't have a
strong opinion.

What's the drawback of having a dedicated Emacs store? Do any other
programs besides `gnutls-cli` use the global GnuTLS store?

The planned certificate management UI can hide these choices from the
user and make the add/remove/list pinned cert operations simple.

TH> 4. Setting gnutls-verify-error to t does not activate TOFU checking. The
TH>    reasoning behind this is that TOFU works somewhat differently than
TH>    the other trust models (CA and hostname matching), and so is a
TH>    feature that is probably best left for people to explicitly ask for.
TH>    Is this reasonable?

I'm OK with that.

TH> 5. Any other comments. For instance, I have only tested this on Linux,
TH>    so not sure if I fudged up all the library loading magic for W32...

Don't worry about it much, it's fixable (modulo the autoconf note above).

TH> As an aside, while testing this I found that using customize to set
TH> gnutls-verify-error, a doubly-nested list of symbols would end up
TH> getting passed to gnutls-boot, which would then subsequently fail all
TH> the membership tests on it. This seems to stem from the fact that
TH> customize produces list entries of the form '((".*" (:tofu))) while
TH> gnutls-negotiate passes the cdr of each matched entry to cl-mapcan. Not
TH> sure which should be fixed, so I just set it manually while testing, but
TH> thought I'd point it out :)

If you can submit a bug with this, it would be wonderful.  I've been
meaning to get rid of the `cl-mapcan' call anyhow.

On Wed, 08 Oct 2014 14:18:27 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> I think it would also be nice if the entire cert was also returned (in a
LMI> convenient format), so that Emacs can display the pertinent parts while
LMI> querying the user about what action to take.

LMI> Like "signed by CA foo on date baz for host zot" and so on.

I am personally not too concerned about putting that at the ELisp level,
but it certainly changes the GnuTLS code fundamentally.  Right now it
stays at the C level to do everything, which is a bit more secure and
more standard.

On Wed, 08 Oct 2014 15:06:52 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Eli Zaretskii <eliz@gnu.org> writes:

>> How can it return a valid descriptor without all the validations it
>> does before that, which (AFAIU) you want to delegate to Lisp now?

LMI> It would validate the certificate, but never drop the connection even if
LMI> the certificate is invalid.  It would be up to `open-network-stream' to
LMI> close the connection if the user decides to not accept the invalid (or
LMI> self-signed) certificate.

I think that's pretty tricky with GnuTLS because it expects all the
validations to be C callbacks and just hands off the connection at the
end. You're not supposed to interact with the session during the
validation, IIUC. So it will probably require two attempts.

Do you want to ask in the GnuTLS mailing list?  Or I can?

On Wed, 08 Oct 2014 15:28:23 +0200 Toke Høiland-Jørgensen <toke@toke.dk> wrote: 

TH> Another alternative would be to fail in the C code and report why, along
TH> with the cert details; and then make it up to the lisp code to retry the
TH> connection with appropriate flags set to ignore (some) errors based on
TH> user response... That would be similar to how the gnutls lisp/C
TH> interaction is now (I think), with the addition that certificate details
TH> are returned along with any errors signalled...

Yes, that would be nice and clean. They can simply be attached as symbol
properties to the error.  Maybe you can adjust `gnutls_make_error'?

On Wed, 08 Oct 2014 15:25:43 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> The other option is to have the C layer close the connection, signal an
LMI> error, have `open-network-stream' query the user about the invalid
LMI> certificate, the user says "connect anyway", and then we'd reconnect
LMI> with other options.

LMI> That seems less ... convenient.

Unfortunately, I think it's how you're supposed to use GnuTLS.  But I'm
happy to be proven wrong!

On Wed, 08 Oct 2014 13:58:50 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>> Yes, `open-network-stream' would implement certificate pinning. 

LMI> (By default.  But there should probably be a way for the user to switch
LMI> it off, since the user may not want to store a list of servers (with
LMI> self-signed certificates) it has talked to -- for privacy reasons.  But
LMI> that's a pretty marginal issue.)

Yeah, I think it can be implemented later.

Ted

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Wonderful work, thank you!

Glad you think so :)

> You should check in the autoconf script whether
> `gnutls_verify_stored_pubkey' is available. It's a fairly new function
> and we have to support older versions of GnuTLS without it.

Right; will look into that.

> I like the simplicity of it. It would be nice to do this inside Emacs
> itself (it's OK if it requires some kind of `emacs --batch' call,
> doesn't have to be immediate). Either way, the errors should tell us
> specifically what to run from the command line in these two cases:

I have an updated version where I split out the parameters into
:tofu-strict and :tofu-auto, where the latter will automatically add a
certificate that hasn't been seen before (and fail on mismatch only).
This might be more suitable to have turned out by default.

Removing things from the key store has to be done manually, though; but
that goes for gnutls-cli as well (and even for ssh).

>     error ("No TOFU trust entry found for hostname \"%s\" and service \"%s\"", c_hostname, c_service);
>     error ("TOFU trust MISMATCH for hostname \"%s\" and service \"%s\"", c_hostname, c_service);

Will update the messages to be more helpful.

> I think so. But Emacs creates a Emacs-specific homedir for the GnuPG
> keychain, for verification of the package archives, in
> `~/.emacs.d/gnupg' which is an equally valid approach. So I don't have
> a strong opinion.
>
> What's the drawback of having a dedicated Emacs store? Do any other
> programs besides `gnutls-cli` use the global GnuTLS store?

Well, any programs that use gnutls and pass NULL as the trust store will
share the site default. Using this also has the nice side effect of not
having to come up with a portable way to find a suitable file name (I'm
sure this is solved elsewhere in the emacs code but it saved me from
going looking ;)).

> If you can submit a bug with this, it would be wonderful.  I've been
> meaning to get rid of the `cl-mapcan' call anyhow.

I've submitted a bug.

> I think that's pretty tricky with GnuTLS because it expects all the
> validations to be C callbacks and just hands off the connection at the
> end. You're not supposed to interact with the session during the
> validation, IIUC. So it will probably require two attempts.

Noted. I checked what gnutls-cli does (by way of packet dumps), and it
seems to keep the connection open, presumably in the middle of the
handshake, while waiting for the user to decide whether to trust it. So
presumably something similar could be done by Emacs, and I think it's
more a matter of whether or not it's possible to call back up into lisp
from this part of the code.

> Yes, that would be nice and clean. They can simply be attached as
> symbol properties to the error. Maybe you can adjust
> `gnutls_make_error'?

I will look into it. I'm out of time to hack on this for a while, so for
now I'll just resubmit the patch with the changes noted above, and then
return to this at a later date (heh, famous last words).

-Toke

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Ted Zlatanov]

On Wed, 08 Oct 2014 17:19:12 +0200 Toke Høiland-Jørgensen <toke@toke.dk> wrote: 

TH> Ted Zlatanov <tzz@lifelogs.com> writes:

>> I like the simplicity of it. It would be nice to do this inside Emacs
>> itself (it's OK if it requires some kind of `emacs --batch' call,
>> doesn't have to be immediate). Either way, the errors should tell us
>> specifically what to run from the command line in these two cases:

TH> I have an updated version where I split out the parameters into
TH> :tofu-strict and :tofu-auto, where the latter will automatically add a
TH> certificate that hasn't been seen before (and fail on mismatch only).
TH> This might be more suitable to have turned out by default.

That's very good.

TH> Removing things from the key store has to be done manually, though; but
TH> that goes for gnutls-cli as well (and even for ssh).

The certificate management UI will take care of that.

>> I think so. But Emacs creates a Emacs-specific homedir for the GnuPG
>> keychain, for verification of the package archives, in
>> `~/.emacs.d/gnupg' which is an equally valid approach. So I don't have
>> a strong opinion.
>> 
>> What's the drawback of having a dedicated Emacs store? Do any other
>> programs besides `gnutls-cli` use the global GnuTLS store?

TH> Well, any programs that use gnutls and pass NULL as the trust store will
TH> share the site default. Using this also has the nice side effect of not
TH> having to come up with a portable way to find a suitable file name (I'm
TH> sure this is solved elsewhere in the emacs code but it saved me from
TH> going looking ;)).

Emacs has this function already, e.g. `(locate-user-emacs-file "certs")'

I think it's better to make the store private than shared by default, so
I'd just give the user the choice to use nil (translated to NULL in C).

>> I think that's pretty tricky with GnuTLS because it expects all the
>> validations to be C callbacks and just hands off the connection at the
>> end. You're not supposed to interact with the session during the
>> validation, IIUC. So it will probably require two attempts.

TH> Noted. I checked what gnutls-cli does (by way of packet dumps), and it
TH> seems to keep the connection open, presumably in the middle of the
TH> handshake, while waiting for the user to decide whether to trust it. So
TH> presumably something similar could be done by Emacs, and I think it's
TH> more a matter of whether or not it's possible to call back up into lisp
TH> from this part of the code.

That would be great, please see how far you get with the exploration.
Your contribution is already very useful so I am excited to see it evolve.

>> Yes, that would be nice and clean. They can simply be attached as
>> symbol properties to the error. Maybe you can adjust
>> `gnutls_make_error'?

TH> I will look into it. I'm out of time to hack on this for a while, so for
TH> now I'll just resubmit the patch with the changes noted above, and then
TH> return to this at a later date (heh, famous last words).

OK; Lars and I will probably work on it as well as time allows in order
to get something into trunk.

Ted

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Emacs has this function already, e.g. `(locate-user-emacs-file "certs")'
>
> I think it's better to make the store private than shared by default, so
> I'd just give the user the choice to use nil (translated to NULL in
> C).

Ah, yes, this should come from the lisp side of course. Silly me, hadn't
even thought of that.

> That would be great, please see how far you get with the exploration.
> Your contribution is already very useful so I am excited to see it
> evolve.

Well, gnutls-cli asks the user in a callback (set with
gnutls_certificate_set_verify_function). The TOFU verification starts at
line 461 of
https://gitorious.org/gnutls/gnutls/source/ce47098eecba5fb3256b855f9674ee0ca458c60c:src/cli.c

so it seems it's just pausing in the middle of the handshake.

> OK; Lars and I will probably work on it as well as time allows in
> order to get something into trunk.

Oh, by all means. I didn't mean that as "don't touch it", more as "don't
expect anything more too soon" :)

Have updated the patch to use a configurable credentials file and put in
the autoconf stuff. Will resend it once I have tested it :)

-Toke

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Lars Magne Ingebrigtsen]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

>> I like the simplicity of it. It would be nice to do this inside Emacs
>> itself (it's OK if it requires some kind of `emacs --batch' call,
>> doesn't have to be immediate). Either way, the errors should tell us
>> specifically what to run from the command line in these two cases:
>
> I have an updated version where I split out the parameters into
> :tofu-strict and :tofu-auto, where the latter will automatically add a
> certificate that hasn't been seen before (and fail on mismatch only).
> This might be more suitable to have turned out by default.

Well, I kinda think the TOFU stuff is a fine band-aid, but we really
need a suture here, and the band-aid really sounds like it would more
get in the way of getting what we really need.  :-)

Anything less than what I described earlier in this thread doesn't
really give the Emacs users the security they need.  For instance, it
doesn't help with the man-in-the-middle-stripping-the-STARTTLS-announcement.

To make this workable (i.e., if we add lots of manual management, users
won't bother), we need a more complete solution.

I don't really see the TOFU thing in itself (which would arise naturally
as a side-effect from the Emacs-driven certificate handling we need) as
a step forward, I'm sorry to say.

But it all depends on how realistic the `open-network-stream' scenario I
proposed is -- I don't know the details of how the gnutls library works,
and whether it would allow us the proposed work flow.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Toke Høiland-Jørgensen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Well, I kinda think the TOFU stuff is a fine band-aid, but we really
> need a suture here, and the band-aid really sounds like it would more
> get in the way of getting what we really need. :-)

Yeah, well for right now I'm in the band-aid making business I guess :)

Resubmitted the updated patch and will return once I have some time for
making sutures...

-Toke

Re: [PATCH RFC] GnuTLS: Support TOFU certificate checking.

[Ted Zlatanov]

On Wed, 08 Oct 2014 19:07:48 +0200 Toke Høiland-Jørgensen <toke@toke.dk> wrote: 

TH> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>> Well, I kinda think the TOFU stuff is a fine band-aid, but we really
>> need a suture here, and the band-aid really sounds like it would more
>> get in the way of getting what we really need. :-)

TH> Yeah, well for right now I'm in the band-aid making business I guess :)

TH> Resubmitted the updated patch and will return once I have some time for
TH> making sutures...

Toke and Lars, I would really appreciate it if you could review this
thread, which was my preliminary research in 2010 on how we could store
and verify certificates, with comments from Nikos (the maintainer of
GnuTLS). It predates the TOFU features.

http://comments.gmane.org/gmane.comp.encryption.gpg.gnutls.devel/4580

Lars, I think it would be smart to resume that conversation and ask the
GnuTLS guys about Toke's approach vs. the oversight-from-ELisp approach
you suggested. I think Eli is on the GnuTLS mailing list and perhaps
others will join in.

Either way, I think the TOFU functions will at least have to be exposed
to ELisp when they are available so the certificate UI can use them. So
I can break Toke's patch in two pieces for that purpose, if that's OK
with everyone, and apply the part I know we'll need.

Thanks
Ted