bug#19284: 25.0.50; tls.el uses option --insecure

bug#19284: 25.0.50; tls.el uses option --insecure

[Jens Lechtenboerger]

This is a followup to bug#16978, where I reported multiple MITM
issues.

tls.el calls gnutls-cli with option --insecure.

As Emacs applies TOFU by default via nsm.el (great work, many
thanks!), the above is dangerous.  I continue to use the following:
(setq tls-program '("gnutls-cli --strict-tofu -p %p %h"))

I’m not sure under what conditions tls.el is necessary.  Is it?

Best wishes
Jens

bug#19284: 25.0.50; tls.el uses option --insecure

[Lars Ingebrigtsen]

Jens Lechtenboerger <jens.lechtenboerger@fsfe.org> writes:

> This is a followup to bug#16978, where I reported multiple MITM
> issues.
>
> tls.el calls gnutls-cli with option --insecure.
>
> As Emacs applies TOFU by default via nsm.el (great work, many
> thanks!), the above is dangerous.  I continue to use the following:
> (setq tls-program '("gnutls-cli --strict-tofu -p %p %h"))
>
> I’m not sure under what conditions tls.el is necessary.  Is it?

tls is not used if Emacs is build with GnuTLS (which all significant
distributions are, I think).  

As Stefan said in a different report -- perhaps we should just require
Emacs with built-in TLS support if you want to use TLS.  That would
essentially mean that we should just remove tls.el and starttls.el.

Alternatively we could, in Emacs 25.1, just remove the --insecure
settings and let people who try to connect to their IMAP server just
fail somewhat mysteriously (it's very common to have self-signed certs
for IMAP).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19284: 25.0.50; tls.el uses option --insecure

[Andreas Schwab]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> tls is not used if Emacs is build with GnuTLS

This is wrong.  Both tls and starttls are used by mail-source.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#19284: 25.0.50; tls.el uses option --insecure

[Lars Ingebrigtsen]

Andreas Schwab <schwab@linux-m68k.org> writes:

> This is wrong.  Both tls and starttls are used by mail-source.

Uhm...  I can't find any such usages (after I fixed imap.el to not do
that).  But I may be misreading.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19284: 25.0.50; tls.el uses option --insecure

[Andreas Schwab]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> (after I fixed imap.el to not do that)

Oh, I didn't notice that yet.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#19284: 25.0.50; tls.el uses option --insecure

[Ted Zlatanov]

On Sat, 26 Dec 2015 22:15:45 +0100 Lars Ingebrigtsen <larsi@gnus.org> wrote: 

LI> As Stefan said in a different report -- perhaps we should just require
LI> Emacs with built-in TLS support if you want to use TLS.  That would
LI> essentially mean that we should just remove tls.el and starttls.el.

LI> Alternatively we could, in Emacs 25.1, just remove the --insecure
LI> settings and let people who try to connect to their IMAP server just
LI> fail somewhat mysteriously (it's very common to have self-signed certs
LI> for IMAP).

I am in favor of either option and I think the first is cleaner.

There will be a small but vocal group that wants to use the external
tunnel utility. I think the benefit to the rest of the users will be
worth it, and that group can have a ELPA package to support them.

Ted

bug#19284: 25.0.50; tls.el uses option --insecure

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Sat, 26 Dec 2015 22:15:45 +0100 Lars Ingebrigtsen <larsi@gnus.org> wrote: 
>
> LI> As Stefan said in a different report -- perhaps we should just require
> LI> Emacs with built-in TLS support if you want to use TLS.  That would
> LI> essentially mean that we should just remove tls.el and starttls.el.
>
> LI> Alternatively we could, in Emacs 25.1, just remove the --insecure
> LI> settings and let people who try to connect to their IMAP server just
> LI> fail somewhat mysteriously (it's very common to have self-signed certs
> LI> for IMAP).
>
> I am in favor of either option and I think the first is cleaner.
>
> There will be a small but vocal group that wants to use the external
> tunnel utility. I think the benefit to the rest of the users will be
> worth it, and that group can have a ELPA package to support them.

I'd rather do the first, too, but perhaps we should wait a bit.

I'll just remove the --insecure for now.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19284: 25.0.50; tls.el uses option --insecure

[Ivan Shmakov]

>>>>> Ted Zlatanov <tzz@lifelogs.com> writes:
>>>>> On Sat, 26 Dec 2015 22:15:45 +0100 Lars Ingebrigtsen wrote:

 >> As Stefan said in a different report -- perhaps we should just
 >> require Emacs with built-in TLS support if you want to use TLS.
 >> That would essentially mean that we should just remove tls.el and
 >> starttls.el.
  
 >> Alternatively we could, in Emacs 25.1, just remove the --insecure
 >> settings

	FWIW, I tend to support this option.

 >> and let people who try to connect to their IMAP server just fail
 >> somewhat mysteriously (it's very common to have self-signed certs
 >> for IMAP).

	I see little value in self-signed certificates in general,
	especially given that there’s for a long-time a community-driven
	CA who offer X.509 certificates free of charge.

	Sure, for a small group, and assuming typical “desktop” TLS
	clients, self-signed certificates can be used to implement a
	public key dissemination model akin to that’s typical of SSH.
	However, I’ve seen them being used on MXes facing the world
	(say, the MX that serves bugs.debian.org), and I fail to see any
	point whatsoever in that.

 > I am in favor of either option and I think the first is cleaner.

 > There will be a small but vocal group that wants to use the external
 > tunnel utility.

	… Or there will be a group with a small number of its members
	being vocal; the difference may be not that easy to tell.

	To note is that Gnus’ nnimap method has its own “tunnel utility”
	support, which I use to interface the local IMAP server (below),
	and which (I suppose) could be used in place of tls.el.

   (nnimap-stream shell)
   (nnimap-shell-program "MAIL=maildir:\"$HOME\"/Maildir imapd")

	That said, the lack of possibility to use something similar for
	non-nnimap connections is not something I’d appreciate.

	I’ve sure seen external utility support in other software, too.
	Check the OpenSSH client’s ProxyCommand option, for instance.

 > I think the benefit to the rest of the users will be worth it, and
 > that group can have a ELPA package to support them.

	As long as the hooks are in place to route the requests via that
	package, I have no (strong) objections to the move.  But given
	that tls.el is about 300 LoC in total, and hardly incurs a high
	maintenance cost, I don’t see much value in the move, either.

-- 
FSF associate member #7257  http://am-1.org/~ivan/      … 3013 B6A0 230E 334A

bug#19284: 25.0.50; tls.el uses option --insecure

[Ted Zlatanov]

On Tue, 29 Dec 2015 19:25:48 +0000 Ivan Shmakov <ivan@siamics.net> wrote: 

IS> 	To note is that Gnus’ nnimap method has its own “tunnel utility”
IS> 	support, which I use to interface the local IMAP server (below),
IS> 	and which (I suppose) could be used in place of tls.el.

IS>    (nnimap-stream shell)
IS>    (nnimap-shell-program "MAIL=maildir:\"$HOME\"/Maildir imapd")

IS> 	That said, the lack of possibility to use something similar for
IS> 	non-nnimap connections is not something I’d appreciate.

IS> 	I’ve sure seen external utility support in other software, too.
IS> 	Check the OpenSSH client’s ProxyCommand option, for instance.

>> I think the benefit to the rest of the users will be worth it, and
>> that group can have a ELPA package to support them.

IS> 	As long as the hooks are in place to route the requests via that
IS> 	package, I have no (strong) objections to the move.

The package itself will install those hooks, I assume.

IS> 	But given that tls.el is about 300 LoC in total, and hardly
IS> 	incurs a high maintenance cost, I don’t see much value in the
IS> 	move, either.

There's a small but consistent amount of time spent checking "are you
using tls.el?" every time we debug a SSL/TLS issue (even if we don't ask
the user explicitly).

There is a user experience difference between relying on external tools
implicitly, which tls.el does, and explicitly, which ProxyCommand does.
Also, tls.el is not granular like ProxyCommand or the `nnimap-stream'
functionality, it applies to all connectivity. I hope that explains my
reasoning better.

Ted

bug#19284: 25.0.50; tls.el uses option --insecure

[Ivan Shmakov]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
>>>>> On Tue, 29 Dec 2015 19:25:48 +0000 Ivan Shmakov <ivan@siamics.net> wrote:

[…]

 TZ> I think the benefit to the rest of the users will be worth it, and
 TZ> that group can have a ELPA package to support them.

 IS> As long as the hooks are in place to route the requests via that
 IS> package, I have no (strong) objections to the move.

 TZ> The package itself will install those hooks, I assume.

	My point is that there’re no such hooks currently – the dispatch
	is instead hardcoded into network-stream-open-tls:

   357		   (stream
   358		    (funcall (if (gnutls-available-p)
   359				 'open-gnutls-stream
   360			       'open-tls-stream)
   361			     name buffer host service))

	For it to still be possible to use functions other than
	open-gnutls-stream, and assuming open-tls-stream is removed from
	the Emacs proper, this would’ve to be replaced with a
	(customizable) variable, like:

   (stream
    (funcall network-stream-open-tls-function
             name buffer host service))

 IS> But given that tls.el is about 300 LoC in total, and hardly incurs
 IS> a high maintenance cost, I don’t see much value in the move,
 IS> either.

 TZ> There's a small but consistent amount of time spent checking "are
 TZ> you using tls.el?" every time we debug a SSL/TLS issue (even if we
 TZ> don't ask the user explicitly).

 TZ> There is a user experience difference between relying on external
 TZ> tools implicitly, which tls.el does, and explicitly, which
 TZ> ProxyCommand does.

	But that’s trivial to solve; say:

(defcustom network-stream-open-tls-function 'open-gnutls-stream
  "The function to use to establish TLS/SSL connections."
  :type '(choice (function-item :tag "Native GnuTLS support"
                                open-gnutls-stream)
                 (function-item :tag "Use gnutls-cli external command"
                                open-tls-stream)))

	This way, tls.el would only be used if explicitly configured by
	the user.

 TZ> Also, tls.el is not granular like ProxyCommand or the
 TZ> `nnimap-stream' functionality, it applies to all connectivity.

	The user may set network-stream-open-tls-function to an entirely
	arbitrary function, which may take the target host and service
	names into account.  (Although I don’t have any sensible use
	case for that at hand.)

 TZ> I hope that explains my reasoning better.

	It does.

-- 
FSF associate member #7257  http://am-1.org/~ivan/      … 3013 B6A0 230E 334A

bug#19284: 25.0.50; tls.el uses option --insecure

[Ted Zlatanov]

On Wed, 30 Dec 2015 15:57:37 +0000 Ivan Shmakov <ivan@siamics.net> wrote: 

>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
>>>>>> On Tue, 29 Dec 2015 19:25:48 +0000 Ivan Shmakov <ivan@siamics.net> wrote:

IS> As long as the hooks are in place to route the requests via that
IS> package, I have no (strong) objections to the move.

TZ> The package itself will install those hooks, I assume.

IS> 	My point is that there’re no such hooks currently

You're right, I meant to say that the hooks will be provided and the
package will add itself to them.

IS> 	– the dispatch is instead hardcoded into
IS> 	network-stream-open-tls:

IS>    357		   (stream
IS>    358		    (funcall (if (gnutls-available-p)
IS>    359				 'open-gnutls-stream
IS>    360			       'open-tls-stream)
IS>    361			     name buffer host service))

Yes, this is exactly where the hook or function should go.

TZ> There is a user experience difference between relying on external
TZ> tools implicitly, which tls.el does, and explicitly, which
TZ> ProxyCommand does.

IS> 	But that’s trivial to solve; say:

IS> (defcustom network-stream-open-tls-function 'open-gnutls-stream
IS>   "The function to use to establish TLS/SSL connections."
IS>   :type '(choice (function-item :tag "Native GnuTLS support"
IS>                                 open-gnutls-stream)
IS>                  (function-item :tag "Use gnutls-cli external command"
IS>                                 open-tls-stream)))

IS> 	This way, tls.el would only be used if explicitly configured by
IS> 	the user.

Exactly, brilliant :)

But the user experience goes beyond configuration. External tools are
harder to debug and control, and the *user* ends up with the burden of
maintaining them (which can have security consequences too). I think if
the user *knows* he has chosen a proxy method, he's much more likely to
be aware of the burden he assumes.

It's also worth considering whether the GnuTLS integration itself can
support these use cases. Maybe `open-gnutls-stream-insecurely' would be
a good user-level function to provide.

TZ> Also, tls.el is not granular like ProxyCommand or the
TZ> `nnimap-stream' functionality, it applies to all connectivity.

IS> 	The user may set network-stream-open-tls-function to an entirely
IS> 	arbitrary function, which may take the target host and service
IS> 	names into account.  (Although I don’t have any sensible use
IS> 	case for that at hand.)

It makes sense in some very specifically constrained corporate
environments. It could be handled by making
`network-stream-open-tls-function' optionally specify the function by
host and port, not just a global choice. Gnus is full of this kind of
defcustom.

So that makes it fairly easy to configure, I think. The logging in the
network-stream code will probably have to be improved as well to support
the user experience.

I appreciate your thoughts, Ivan, but also anyone else that wants to
contribute is welcome... I think this is a very good discussion.

Ted

bug#19284: 25.0.50; tls.el uses option --insecure

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> There is a user experience difference between relying on external tools
> implicitly, which tls.el does, and explicitly, which ProxyCommand does.
> Also, tls.el is not granular like ProxyCommand or the `nnimap-stream'
> functionality, it applies to all connectivity. I hope that explains my
> reasoning better.

Yeah.  For the version after this, we should dump tls.el (and
starttls.el) completely.  If somebody wants a way to do TLS proxying, we
should add that as separate functionality, not something that plops out
as a side-effect of using gnutls-cli.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19284: 25.0.50; tls.el uses option --insecure

[Ted Zlatanov]

On Wed, 30 Dec 2015 19:22:49 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> There is a user experience difference between relying on external tools
>> implicitly, which tls.el does, and explicitly, which ProxyCommand does.
>> Also, tls.el is not granular like ProxyCommand or the `nnimap-stream'
>> functionality, it applies to all connectivity. I hope that explains my
>> reasoning better.

LMI> Yeah.  For the version after this, we should dump tls.el (and
LMI> starttls.el) completely.  If somebody wants a way to do TLS proxying, we
LMI> should add that as separate functionality, not something that plops out
LMI> as a side-effect of using gnutls-cli.

Ivan, do you want to summarize the three separate proposals to emacs-devel
or should I? I think it's time to move it out of this bug report since
Lars has committed the changes to fix it.

The proposals, I think, were:

1) provide a new function hook point for tls.el to provide
network-stream functionality, and make that a defcustom that can be
overridden by host and port

2) move tls.el out of Emacs into the GNU ELPA

3) support TLS proxying in gnutls.el or at the C level, if we can define
what that actually means

Thanks
Ted

bug#19284: 25.0.50; tls.el uses option --insecure

[Ivan Shmakov]

>>>>> Ted Zlatanov <tzz@lifelogs.com> writes:

[…]

 > Ivan, do you want to summarize the three separate proposals to
 > emacs-devel or should I?  I think it's time to move it out of this
 > bug report since Lars has committed the changes to fix it.

	I guess I’m going to be a bit busy over the next couple of days,
	so feel free to proceed.  TIA.

[…]

-- 
FSF associate member #7257  http://am-1.org/~ivan/      … 3013 B6A0 230E 334A