bug#18600: 24.3.94; EWW fails to check https certificates

bug#18600: 24.3.94; EWW fails to check https certificates

[Mark H Weaver]

I used EWW to visit an https website that uses a self-signed and
long-expired https certificate.  It failed to notify me of any problem.



In GNU Emacs 24.3.94.1 (i686-pc-linux-gnu, GTK+ Version 3.10.1)
 of 2014-10-02 on localhost
Windowing system distributor `The X.Org Foundation', version 11.0.11202000
Configured using:
 `configure
 CONFIG_SHELL=/gnu/store/wgvrj5q40prd4d1fb0j81n6gxdpqwz79-bash-4.3.27/bin/bash
 SHELL=/gnu/store/wgvrj5q40prd4d1fb0j81n6gxdpqwz79-bash-4.3.27/bin/bash
 --prefix=/gnu/store/6x3z5nwya75jgfs76qkpj25va9iwsqd4-emacs-24.3.94
 --enable-fast-install
 --with-crt-dir=/gnu/store/1zxdnj48g45pwram0s8nprvkkwxzp62b-glibc-2.20/lib'

Important settings:
  value of $LC_ALL: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Summary

Minor modes in effect:
  shell-dirtrack-mode: t
  diff-auto-refine-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t

Recent input:
  [removed; irrelevant]

Recent messages:
  [removed; irrelevant]

Load-path shadows:
None found.

Features:
(shadow term ehelp emacsbug sendmail sort gnus-cite mail-extr gnus-async
gnus-bcklg qp gnus-ml disp-table nndraft nnmh nnfolder netrc gnus-agent
gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art mm-uu
mml2015 epg-config mm-view mml-smime smime dig nntp gnus-cache gnus-sum
nnoo gnus-group gnus-undo nnmail mail-source gnus-start gnus-spec
gnus-int gnus-range gnus-win misearch multi-isearch gnutls shr-color
color timezone parse-time help-mode mule-util url-queue network-stream
starttls url-http tls url-gw url-cache url-auth eww mm-url gnus gnus-ems
nnheader wid-edit url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util url-parse auth-source eieio
byte-opt bytecomp byte-compile cconv eieio-core gnus-util password-cache
url-vars mailcap shr browse-url shell pcomplete comint ansi-color
paredit edmacro kmacro cl-loaddefs cl-lib server w3m-wget w3m-load
magit-bisect magit-key-mode magit diff-mode log-edit easy-mmode message
format-spec rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mm-util help-fns
mail-prsvr mailabbrev mail-utils gmm-utils mailheader ring pcvs-util
add-log geiser-install geiser scheme time-date tooltip electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 8 389222 31229)
 (symbols 24 32379 0)
 (miscs 20 195 588)
 (strings 16 51077 6316)
 (string-bytes 1 1780492)
 (vectors 8 25575)
 (vector-slots 4 590009 18268)
 (floats 8 413 443)
 (intervals 28 3719 319)
 (buffers 512 28)
 (heap 1024 44807 14554))

bug#18600: 24.3.94; EWW fails to check https certificates

[Glenn Morris]

Mark H Weaver wrote:

> I used EWW to visit an https website that uses a self-signed and
> long-expired https certificate.  It failed to notify me of any problem.

Setting gnutls-verify-error non-nil may help (I don't know what it does
with self-signed certificates).

bug#18600: 24.3.94; EWW fails to check https certificates

[Glenn Morris]

PS see previous discussion in http://debbugs.gnu.org/16978 .

bug#18600: 24.3.94; EWW fails to check https certificates

[Ted Zlatanov]

On Fri, 03 Oct 2014 19:01:42 -0400 Glenn Morris <rgm@gnu.org> wrote: 

GM> Mark H Weaver wrote:
>> I used EWW to visit an https website that uses a self-signed and
>> long-expired https certificate.  It failed to notify me of any problem.

GM> Setting gnutls-verify-error non-nil may help (I don't know what it does
GM> with self-signed certificates).

Emacs will reject such certificates then. I tested that as part of
http://debbugs.gnu.org/16978 and would appreciate Mark's verification.

After 24.4 (now 25.1) is released it will be t by default.  Mark, can we
close this bug since http://debbugs.gnu.org/16978 already has all the info?

Thanks
Ted

bug#18600: 24.3.94; EWW fails to check https certificates

[Mark H Weaver]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Fri, 03 Oct 2014 19:01:42 -0400 Glenn Morris <rgm@gnu.org> wrote: 
>
> GM> Mark H Weaver wrote:
>>> I used EWW to visit an https website that uses a self-signed and
>>> long-expired https certificate.  It failed to notify me of any problem.
>
> GM> Setting gnutls-verify-error non-nil may help (I don't know what it does
> GM> with self-signed certificates).
>
> Emacs will reject such certificates then. I tested that as part of
> http://debbugs.gnu.org/16978 and would appreciate Mark's verification.

Yes, that works, thanks.

> After 24.4 (now 25.1) is released it will be t by default.  Mark, can we
> close this bug since http://debbugs.gnu.org/16978 already has all the info?

I almost closed the bug myself, but on second thought I think this case
of eww https warrants special consideration, independent of the more
general question of how 'open-gnutls-stream' should behave by default.

There are a few reasons for this:

1. In the case of imaps, smtps, xmpp, etc, the most common use case is
   to connect to a single server only for each of these protocols, and
   very often that's one's own server with self-signed certs.

2. In the case of https, the typical use cases are very different, as
   are the expectations.  When browsing the web, one typically talks to
   a very large number of https servers.  More often than not, these
   servers have certificates signed by a well-known CA.  (Ideally it
   should be possible to disable checking based on URL).

3. Emacs 24.4 will be the first release that includes eww, so there are
   no preexisting users of eww that would be annoyed by suddenly having
   their existing functionality stop working.

With these in mind, I have two recommendations:

* I believe that eww https should check certificates by default in 24.4,
  even though other tls connections are tolerant by default.

* At minimum, it should be possible to enable certificate checking for
  eww https connections while still allowing self-signed certificates
  for other uses of 'open-gnutls-stream' such as imaps and smtps.  This
  is fairly common case.

IMO, anyway.  If you disagree, I'll defer to your judgment, but my
feeling is that the current behavior would not be well received.

    Thanks,
      Mark

bug#18600: 24.3.94; EWW fails to check https certificates

[Stefan Monnier]

> With these in mind, I have two recommendations:
> * I believe that eww https should check certificates by default in 24.4,
>   even though other tls connections are tolerant by default.
> * At minimum, it should be possible to enable certificate checking for
>   eww https connections while still allowing self-signed certificates
>   for other uses of 'open-gnutls-stream' such as imaps and smtps.  This
>   is fairly common case.

I think it's too late to do that for Emacs-24.4.  But we should apply
such a change to `emacs-24' after the 24.4 release, so that it will be
included in the next release regardless if the next release is 25.1 or
a 24.5 bugfix.


        Stefan

bug#18600: 24.3.94; EWW fails to check https certificates

[Glenn Morris]

Ted Zlatanov wrote:

> close this bug since http://debbugs.gnu.org/16978 already has all the info?

They are merged.

Marking changes to be backported

[Glenn Morris]

Stefan Monnier wrote:

> I think it's too late to do that for Emacs-24.4.  But we should apply
> such a change to `emacs-24' after the 24.4 release, so that it will be
> included in the next release regardless if the next release is 25.1 or
> a 24.5 bugfix.

There needs to be an agreed convention to mark trunk changes that should
be backported to the release branch at some later date. "Remember to do
it" doesn't scale.

A note in the commit log would do, but it should have some kind of
standard format. But if we only realize after the commit was made,
some other method is needed. Either a file in admin, or perhaps a git
notes file.

Re: Marking changes to be backported

[Glenn Morris]

A simple file in admin would do, one revision per line.
Someone could write a script to backport (labelling the commit with
"Backport...") each revision from trunk and remove it from the input
file.

bug#18600: 24.3.94; EWW fails to check https certificates

[Mark H Weaver]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> With these in mind, I have two recommendations:
>> * I believe that eww https should check certificates by default in 24.4,
>>   even though other tls connections are tolerant by default.
>> * At minimum, it should be possible to enable certificate checking for
>>   eww https connections while still allowing self-signed certificates
>>   for other uses of 'open-gnutls-stream' such as imaps and smtps.  This
>>   is fairly common case.
>
> I think it's too late to do that for Emacs-24.4.  But we should apply
> such a change to `emacs-24' after the 24.4 release, so that it will be
> included in the next release regardless if the next release is 25.1 or
> a 24.5 bugfix.

I continue to think this will be ill-received, and could result in more
bad PR for the GNU Project, but having said that, I'll let it go now.

     Thanks,
       Mark

Re: Marking changes to be backported

[Stefan Monnier]

> There needs to be an agreed convention to mark trunk changes that should
> be backported to the release branch at some later date. "Remember to do
> it" doesn't scale.

If it's really needed, we can have a "emacs-24-next" branch for that.
Then we merge emacs-24 into emacs-24-next, and then we merge
emacs-24-next into trunk.

But a few notes in an `admin' file sounds fine for now.


        Stefan

Re: Marking changes to be backported

[Glenn Morris]

Stefan Monnier wrote:

>> There needs to be an agreed convention to mark trunk changes that should
>> be backported to the release branch at some later date. "Remember to do
>> it" doesn't scale.
>
> If it's really needed, we can have a "emacs-24-next" branch for that.

I think it will be needed if: there are to be pure bug-fix releases, and
pretesting is going to continue to take months, and development is going
to continue at the same time. (There are probably tons of things fixed
in trunk today that should go into version 24.5, if there is one. Who's
going to dig them all out and backport them? No-one.)

> Then we merge emacs-24 into emacs-24-next, and then we merge
> emacs-24-next into trunk.

Yet more branches sounds more complicated to me (doesn't that mean you
need to know when you make a change where it should go?), but I don't
care about the system so long as there is one.

Re: Marking changes to be backported

[Stefan Monnier]

> Yet more branches sounds more complicated to me (doesn't that mean you
> need to know when you make a change where it should go?),

One way or another, someone will have to decide which change goes "in
the current pretest" (let's call it 24.4), "in the next bug-fix" (let's
call it 24.5), or "longer term" (let's call it 25.1).

If the committer knows before committing where that change should go,
then having 3 branches and committing to the proper branch is the
best solution.

If that's not the case, then having 3 branches doesn't help, indeed,
since the commit will have to be backported to the proper branch, if
applicable.  Note that this "backporting" will take place regardless of
whether we have 3 branches active at a time or not.  Having 3 branches
active at a time, does allow the backporting to be done any time you
want, whereas sticking to the "trunk vs emacs-24" scheme we have simply
means that some of the backporting will only take place once the
"emacs-24" branch is changed from "branch for 24.4" to "branch for
24.5".


        Stefan

Re: Marking changes to be backported

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Mon, 06 Oct 2014 02:37:30 -0400
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
> Stefan Monnier wrote:
> 
> >> There needs to be an agreed convention to mark trunk changes that should
> >> be backported to the release branch at some later date. "Remember to do
> >> it" doesn't scale.
> >
> > If it's really needed, we can have a "emacs-24-next" branch for that.
> 
> I think it will be needed if: there are to be pure bug-fix releases, and
> pretesting is going to continue to take months, and development is going
> to continue at the same time. (There are probably tons of things fixed
> in trunk today that should go into version 24.5, if there is one. Who's
> going to dig them all out and backport them? No-one.)
> 
> > Then we merge emacs-24 into emacs-24-next, and then we merge
> > emacs-24-next into trunk.
> 
> Yet more branches sounds more complicated to me (doesn't that mean you
> need to know when you make a change where it should go?), but I don't
> care about the system so long as there is one.

I think using another (3rd) branch is on balance the most reliable and
least error prone method.  It is also much less effort to merge
between branches than to cherry-pick.

Re: Marking changes to be backported

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 06 Oct 2014 09:25:49 -0400
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
> > Yet more branches sounds more complicated to me (doesn't that mean you
> > need to know when you make a change where it should go?),
> 
> One way or another, someone will have to decide which change goes "in
> the current pretest" (let's call it 24.4), "in the next bug-fix" (let's
> call it 24.5), or "longer term" (let's call it 25.1).

The dilemma is only between the 2 out of 3 branches, those that get
bugfixes.  One of them should only get very safe fixes for serious
problems, which should then be merged to the other.

As for trunk vs the other 2 branches, I think the decision is simple:
new features and changes that don't fix bugs go to trunk only.

Re: Marking changes to be backported

[Glenn Morris]

Stefan Monnier wrote:

> One way or another, someone will have to decide which change goes "in
> the current pretest" (let's call it 24.4), "in the next bug-fix" (let's
> call it 24.5), or "longer term" (let's call it 25.1).

I was hoping that this was something that everyone would think about,
not just a singular someone, and that's why I wanted there to be a
system in place.

Re: Marking changes to be backported

[Stefan Monnier]

>> One way or another, someone will have to decide which change goes "in
>> the current pretest" (let's call it 24.4), "in the next bug-fix" (let's
>> call it 24.5), or "longer term" (let's call it 25.1).
> I was hoping that this was something that everyone would think about,
> not just a singular someone, and that's why I wanted there to be a
> system in place.

I hope so too: I meant "someone" per patch.


        Stefan

bug#18600: 24.3.94; EWW fails to check https certificates

[Lars Magne Ingebrigtsen]

This has now been fixed on the trunk with the Network Security Manager
stuff.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no