Emacs 24.0.93 Pretest Windows Binaries published

Emacs 24.0.93 Pretest Windows Binaries published

[Christoph Scholtes]

The Emacs 24.0.93 Pretest Windows Binaries have been published in

http://alpha.gnu.org/gnu/emacs/pretest/windows/

The binaries were built using the following libraries:
giflib-4.1.4-1
gnutls-3.0.9
jpeg-6b-4
libXpm-3.5.8
libpng-1.4.3-1
tiff-3.8.2-1
zlib-1.2.5-2

Pre-built Windows binaries for GnuTLS are available at this location:
http://sourceforge.net/projects/ezwinports/files/

See the file included README.W32 file for more information on how to 
obtain other binaries necessary to make use of certain Emacs features.

Please report any bugs that you come across via M-x report-emacs-bugs,
or email bug-gnu-emacs@gnu.org.

For questions, email emacs-devel@gnu.org.

RE: Emacs 24.0.93 Pretest Windows Binaries published

[Drew Adams]

> The Emacs 24.0.93 Pretest Windows Binaries have been published in
> http://alpha.gnu.org/gnu/emacs/pretest/windows/
> 
> The binaries were built using the following libraries:
> giflib-4.1.4-1
> gnutls-3.0.9
> jpeg-6b-4
> libXpm-3.5.8
> libpng-1.4.3-1
> tiff-3.8.2-1
> zlib-1.2.5-2
> 
> Pre-built Windows binaries for GnuTLS are available at this location:
> http://sourceforge.net/projects/ezwinports/files/

Can you say what that means?  What is GnuTLS for, and why might an Emacs user
want to obtain a binary for it?  How does it relate to Emacs?

Re: Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Sun, 29 Jan 2012 20:55:30 -0800
> 
> > The Emacs 24.0.93 Pretest Windows Binaries have been published in
> > http://alpha.gnu.org/gnu/emacs/pretest/windows/
> > 
> > The binaries were built using the following libraries:
> > giflib-4.1.4-1
> > gnutls-3.0.9
> > jpeg-6b-4
> > libXpm-3.5.8
> > libpng-1.4.3-1
> > tiff-3.8.2-1
> > zlib-1.2.5-2
> > 
> > Pre-built Windows binaries for GnuTLS are available at this location:
> > http://sourceforge.net/projects/ezwinports/files/
> 
> Can you say what that means?  What is GnuTLS for, and why might an Emacs user
> want to obtain a binary for it?  How does it relate to Emacs?

It's an email authentication package.  See the node "Authentication"
in the smtpmail manual that comes with Emacs.

RE: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Drew Adams]

> > > Pre-built Windows binaries for GnuTLS are available at 
> > > this location: http://sourceforge.net/projects/ezwinports/files/
> > 
> > Can you say what that means?  What is GnuTLS for, and why 
> > might an Emacs user want to obtain a binary for it?
> > How does it relate to Emacs?
> 
> It's an email authentication package.  See the node "Authentication"
> in the smtpmail manual that comes with Emacs.

The point was for you to say that _in the announcement email_.

It doesn't make much sense, in an email about "Pretest Windows Binaries", to
provide a link to something that is separate and different from "Pretest Windows
Binaries", without saying anything about what that link is for.

Sure, users can always google "GnuTLS", but they should not have to.

Please add a one-line description to the link that is provided.  Preferably,
_link_ that description to a page with more info (e.g. GNU page or Wikipedia
page for GnuTLS).

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <cschol2112@googlemail.com>, <help-emacs-windows@gnu.org>,
>         <emacs-devel@gnu.org>
> Date: Mon, 30 Jan 2012 10:17:57 -0800
> 
> > > > Pre-built Windows binaries for GnuTLS are available at 
> > > > this location: http://sourceforge.net/projects/ezwinports/files/
> > > 
> > > Can you say what that means?  What is GnuTLS for, and why 
> > > might an Emacs user want to obtain a binary for it?
> > > How does it relate to Emacs?
> > 
> > It's an email authentication package.  See the node "Authentication"
> > in the smtpmail manual that comes with Emacs.
> 
> The point was for you to say that _in the announcement email_.

Then make your point explicitly.  For some weird reason I thought you
actually wanted to know the answer, and made an effort of looking for
an providing a reference for you to read about that.

RE: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Drew Adams]

> > > > Can you say what that means?  What is GnuTLS for, and why 
> > > > might an Emacs user want to obtain a binary for it?
> > > > How does it relate to Emacs?
> > > 
> > > It's an email authentication package.  See the node 
> > > "Authentication" in the smtpmail manual that comes with Emacs.
> > 
> > The point was for you to say that _in the announcement email_.
> 
> Then make your point explicitly.  For some weird reason I thought you
> actually wanted to know the answer, and made an effort of looking for
> an providing a reference for you to read about that.

I replied to the _announcement_ mail and its author, requesting that it describe
GnuTLS.

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <cschol2112@googlemail.com>, <help-emacs-windows@gnu.org>,
>         <emacs-devel@gnu.org>
> Date: Mon, 30 Jan 2012 11:08:42 -0800
> 
> > > > > Can you say what that means?  What is GnuTLS for, and why 
> > > > > might an Emacs user want to obtain a binary for it?
> > > > > How does it relate to Emacs?
> > > > 
> > > > It's an email authentication package.  See the node 
> > > > "Authentication" in the smtpmail manual that comes with Emacs.
> > > 
> > > The point was for you to say that _in the announcement email_.
> > 
> > Then make your point explicitly.  For some weird reason I thought you
> > actually wanted to know the answer, and made an effort of looking for
> > an providing a reference for you to read about that.
> 
> I replied to the _announcement_ mail and its author, requesting that it describe
> GnuTLS.

Your questions requested nothing about the announcement.  They were
questions about GnuTLS.

Re: Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> Date: Mon, 30 Jan 2012 19:47:24 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: cschol2112@googlemail.com, help-emacs-windows@gnu.org, emacs-devel@gnu.org
> 
> > From: "Drew Adams" <drew.adams@oracle.com>
> > Date: Sun, 29 Jan 2012 20:55:30 -0800
> > 
> > > The Emacs 24.0.93 Pretest Windows Binaries have been published in
> > > http://alpha.gnu.org/gnu/emacs/pretest/windows/
> > > 
> > > The binaries were built using the following libraries:
> > > giflib-4.1.4-1
> > > gnutls-3.0.9
> > > jpeg-6b-4
> > > libXpm-3.5.8
> > > libpng-1.4.3-1
> > > tiff-3.8.2-1
> > > zlib-1.2.5-2
> > > 
> > > Pre-built Windows binaries for GnuTLS are available at this location:
> > > http://sourceforge.net/projects/ezwinports/files/
> > 
> > Can you say what that means?  What is GnuTLS for, and why might an Emacs user
> > want to obtain a binary for it?  How does it relate to Emacs?
> 
> It's an email authentication package.  See the node "Authentication"
> in the smtpmail manual that comes with Emacs.

See also a short note in etc/NEWS (which is more Lisp programmer
oriented).

Re: Emacs 24.0.93 Pretest Windows Binaries published

[Christoph Scholtes]

On 1/29/2012 9:55 PM, Drew Adams wrote:

>> Pre-built Windows binaries for GnuTLS are available at this location:
>> http://sourceforge.net/projects/ezwinports/files/
>
> Can you say what that means?  What is GnuTLS for, and why might an Emacs user
> want to obtain a binary for it?  How does it relate to Emacs?

This is an announcement email nothing more. Do I also have to explain 
what a Pretest is?

What you omitted in your quote is the reference to README.W32. IF we 
need to include any clarifying information we should include it in 
README.W32 and I will only reference the it in any future announcement.

Christoph

RE: Emacs 24.0.93 Pretest Windows Binaries published

[Drew Adams]

> >> Pre-built Windows binaries for GnuTLS are available at 
> >> this location: http://sourceforge.net/projects/ezwinports/files/
> >
> > Can you say what that means?  What is GnuTLS for, and why 
> > might an Emacs user want to obtain a binary for it?
> > How does it relate to Emacs?
> 
> This is an announcement email nothing more. Do I also have to explain 
> what a Pretest is?

It is an _Emacs_ announcement.  If you also mention other stuff then some brief
description of its relation to Emacs is in order (i.e., helpful).

> IF we need to include any clarifying information we should
> include it in README.W32 and I will only reference the it
> in any future announcement.

Agreed.

The readme is the only place we mention other Windows binaries - e.g. image
binaries.  Why treat GnuTLS specially?  Either mention GnuTLS only in the readme
(preferred) or mention in the announcement each of the binaries that mentioned
in the readme (not preferred).

Wherever GnuTLS is mentioned, the mention should be accompanied by a short
description.  IMO, the readme is the proper place for this.

Re: Emacs 24.0.93 Pretest Windows Binaries published

[Ted Zlatanov]

On Tue, 31 Jan 2012 06:15:51 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> The readme is the only place we mention other Windows binaries - e.g. image
DA> binaries.  Why treat GnuTLS specially?  Either mention GnuTLS only in the readme
DA> (preferred) or mention in the announcement each of the binaries that mentioned
DA> in the readme (not preferred).

I requested that GnuTLS be treated specially.  I believe this is
necessary because it's important for secure networking on W32, unlike
any of the other libraries.  This is a temporary remedy; I will work on
a W32 installer and then it won't be necessary to mention GnuTLS
explicitly (a link to the installer in the announcement would be sufficient).

Ted

RE: Emacs 24.0.93 Pretest Windows Binaries published

[Drew Adams]

> DA> The readme is the only place we mention other Windows 
> DA> binaries - e.g. image binaries.  Why treat GnuTLS specially?  
> DA> Either mention GnuTLS only in the readme (preferred)
> DA> or mention in the announcement each of the binaries
> DA> that mentioned in the readme (not preferred).
> 
> I requested that GnuTLS be treated specially.  I believe this is
> necessary because it's important for secure networking on W32, unlike
> any of the other libraries.  This is a temporary remedy; I 
> will work on a W32 installer and then it won't be necessary
> to mention GnuTLS explicitly (a link to the installer in the
> announcement would be sufficient).

1. According to Eli, "It's an email authentication package."  Which would mean
that it is needed only by people who use Emacs for email.

At a minimum, that should be pointed out in the brief description that needs to
accompany this "special treatment".

And that's the case wherever we choose to describe GnuTLS.  (And it should be
described in the README, irregardless of whether it is described in the
announcement.)

(FWIW, I expect that most Windows users do not and will not use Emacs for
email.)


2. I'm not familiar with your proposed "installer", but I certainly hope that we
will continue to distribute a simple zip archive with a Windows binary.

A priori, I for one will not use an installer to "install" the binary.  I use
multiple Emacs Windows binaries, and I do not need an Emacs installer mucking
about with my registry etc.  This is one reason I do not use Lennart's
installer, for instance.

It is blindingly simple for a user to unzip an archive in a directory of choice,
and create a startup shortcut.  Nothing to it.

What's the motivation for this installer?  I can understand Lennart's
motivation, since he has apparently customized many things, including at the C
level.  But why do you think Emacs users on Windows need an installer for
vanilla Emacs?

I have nothing against the general idea of our providing an installer in
_addition_ to our providing zip archives, but I would not want to see the latter
practice dropped.

Re: Emacs 24.0.93 Pretest Windows Binaries published

[Ted Zlatanov]

On Thu, 2 Feb 2012 11:52:25 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> The readme is the only place we mention other Windows 
DA> binaries - e.g. image binaries.  Why treat GnuTLS specially?  
DA> Either mention GnuTLS only in the readme (preferred)
DA> or mention in the announcement each of the binaries
DA> that mentioned in the readme (not preferred).
>> 
>> I requested that GnuTLS be treated specially.  I believe this is
>> necessary because it's important for secure networking on W32, unlike
>> any of the other libraries.  This is a temporary remedy; I 
>> will work on a W32 installer and then it won't be necessary
>> to mention GnuTLS explicitly (a link to the installer in the
>> announcement would be sufficient).

DA> 1. According to Eli, "It's an email authentication package."  Which would mean
DA> that it is needed only by people who use Emacs for email.

DA> At a minimum, that should be pointed out in the brief description that needs to
DA> accompany this "special treatment".

DA> And that's the case wherever we choose to describe GnuTLS.  (And it should be
DA> described in the README, irregardless of whether it is described in the
DA> announcement.)

GnuTLS provides SSL and TLS encryption for any network connection, hence
"secure networking" in my earlier message.  It can encrypt e-mail
protocols like IMAP and SMTP but does not deal with e-mail messages.

DA> 2. I'm not familiar with your proposed "installer", but I certainly hope that we
DA> will continue to distribute a simple zip archive with a Windows binary.

DA> A priori, I for one will not use an installer to "install" the binary.  I use
DA> multiple Emacs Windows binaries, and I do not need an Emacs installer mucking
DA> about with my registry etc.  This is one reason I do not use Lennart's
DA> installer, for instance.

DA> It is blindingly simple for a user to unzip an archive in a directory of choice,
DA> and create a startup shortcut.  Nothing to it.

DA> What's the motivation for this installer?  I can understand Lennart's
DA> motivation, since he has apparently customized many things, including at the C
DA> level.  But why do you think Emacs users on Windows need an installer for
DA> vanilla Emacs?

DA> I have nothing against the general idea of our providing an installer in
DA> _addition_ to our providing zip archives, but I would not want to see the latter
DA> practice dropped.

Any installer I assemble will not replace the Emacs binaries and will
not be as "official" as Christoph's binaries.  It will not involve
emacs-devel resources, at least.

I've covered the reasons for the installer on this list, going back and
forth over several alternatives.  This was a very recent thread.  An
installer seems to be the best approach but you can read through the
discussion and decide for yourself.

Ted

RE: Emacs 24.0.93 Pretest Windows Binaries published

[Drew Adams]

> Any installer I assemble will not replace the Emacs binaries

Thanks for that.

> I've covered the reasons for the installer on this list,

There are actually two lists for this thread.  One of them is
help-emacs-windows, which has Windows users, not necessarily Emacs developers,
as participants and readers.

> going back and forth over several alternatives.  This was a very
> recent thread.  An installer seems to be the best approach but
> you can read through the discussion and decide for yourself.

You were not specific about which thread, so I searched the emacs-devel header
lines for "windows",  "installer" (no hits for that one), and "GnuTLS".

I came across this, as the last post for what is presumably the thread in
question (?):

> From: Chong Yidong
>
> Juanma Barranquero <lekktu@gmail.com> writes:
> 
> > The main argument is still that GnuTLS, etc. have their own projects
> > and maintainers, and they should be the ones taking care of building
> > and distributing it.
> 
> Agreed.  I think it is sufficient to have a URL to the GnuTLS binaries
> (and sources!).  If the worry is that Windows users won't see 
> that URL, we can put the notice somewhere on our webpage next to
> the download link, and/or in a "read me first" file on the FTP site.

That doesn't seem to quite correspond to what we're seeing now (GnuTLS mentioned
in the announcement, and with no description of it) or with what you are arguing
for (an Emacs installer for Windows), but perhaps you meant something different.

Anyway, I'm glad to hear that Windows binaries will continue to be published.
Thanks.

Re: Emacs 24.0.93 Pretest Windows Binaries published

[Ted Zlatanov]

On Thu, 2 Feb 2012 14:05:54 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 

>> I've covered the reasons for the installer on this list,

DA> There are actually two lists for this thread.  One of them is
DA> help-emacs-windows, which has Windows users, not necessarily Emacs developers,
DA> as participants and readers.

Yes, I apologize for the imprecision.  I meant emacs-devel.

>> going back and forth over several alternatives.  This was a very
>> recent thread.  An installer seems to be the best approach but
>> you can read through the discussion and decide for yourself.

DA> You were not specific about which thread, so I searched the emacs-devel header
DA> lines for "windows",  "installer" (no hits for that one), and "GnuTLS".

This is indexed under the GnuTLS mailing list by Gmane because that's
where the thread started, but it should give you enough reading for a
few days :)

http://thread.gmane.org/gmane.network.gnutls.general/2570/focus=147145

Ted

Re: Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Thu, 02 Feb 2012 16:46:17 -0500
> Cc: help-emacs-windows@gnu.org
> 
> DA> 1. According to Eli, "It's an email authentication package."  Which would mean
> DA> that it is needed only by people who use Emacs for email.
> 
> DA> At a minimum, that should be pointed out in the brief description that needs to
> DA> accompany this "special treatment".
> 
> DA> And that's the case wherever we choose to describe GnuTLS.  (And it should be
> DA> described in the README, irregardless of whether it is described in the
> DA> announcement.)
> 
> GnuTLS provides SSL and TLS encryption for any network connection, hence
> "secure networking" in my earlier message.  It can encrypt e-mail
> protocols like IMAP and SMTP but does not deal with e-mail messages.

What other features in Emacs use TLS as of this writing?  I thought
only email protocols do, which is why I described GnuTLS as I did.

If other protocols we have can be secured by GnuTLS, there seems to be
a gap in our documentation, because I couldn't find any place where
that is mentioned, except in relation to SMTP and the likes.

GnuTLS invasion of Emacs (was: Emacs 24.0.93 Pretest Windows Binaries published)

[Ted Zlatanov]

On Fri, 03 Feb 2012 09:48:39 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> 
>> GnuTLS provides SSL and TLS encryption for any network connection, hence
>> "secure networking" in my earlier message.  It can encrypt e-mail
>> protocols like IMAP and SMTP but does not deal with e-mail messages.

EZ> What other features in Emacs use TLS as of this writing?  I thought
EZ> only email protocols do, which is why I described GnuTLS as I did.

Any network connection can use it.  I think Lars introduced that option,
and at least HTTP/S connections can use it.

EZ> If other protocols we have can be secured by GnuTLS, there seems to be
EZ> a gap in our documentation, because I couldn't find any place where
EZ> that is mentioned, except in relation to SMTP and the likes.

(subject adjusted accordingly)

It's a replacement for the previous libraries that managed secure
connections, except it doesn't depend on external binaries.  So it
really doesn't change much in terms of Emacs functionality, only in the
underlying implementation.  There is one annoying detail with the cert
bundle on W32.  It defaults to /etc/ssl/certs/ca-certificates.crt which
is not valid on W32 and on many other platforms.  See
`open-gnutls-stream' and the rest of gnutls.el.  I was going to bring in
the Mozilla cert bundle with the binary installer I'm planning so I
didn't attack this problem sooner; if you have suggestions for the
default cert bundle on W32 let me know.

Thanks
Ted

Re: GnuTLS invasion of Emacs published)

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Fri, 03 Feb 2012 08:23:12 -0500
> Cc: help-emacs-windows@gnu.org
> 
> EZ> What other features in Emacs use TLS as of this writing?  I thought
> EZ> only email protocols do, which is why I described GnuTLS as I did.
> 
> Any network connection can use it.

I asked about actual use, not potential uses.

> I think Lars introduced that option, and at least HTTP/S connections
> can use it.

Then this needs to be documented somewhere.

> It's a replacement for the previous libraries that managed secure
> connections, except it doesn't depend on external binaries.  So it
> really doesn't change much in terms of Emacs functionality, only in the
> underlying implementation.

Lisp programmers should know they can use TLS when Emacs was compiled
with GnuTLS support.  Users should know that as well, because they
will need to set up their machines for that.  E.g., this:

> There is one annoying detail with the cert bundle on W32.  It
> defaults to /etc/ssl/certs/ca-certificates.crt which is not valid on
> W32 and on many other platforms.

Re: GnuTLS invasion of Emacs published)

[Ted Zlatanov]

On Fri, 03 Feb 2012 18:29:07 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Fri, 03 Feb 2012 08:23:12 -0500
>> Cc: help-emacs-windows@gnu.org
>> 
EZ> What other features in Emacs use TLS as of this writing?  I thought
EZ> only email protocols do, which is why I described GnuTLS as I did.
>> 
>> Any network connection can use it.

EZ> I asked about actual use, not potential uses.

I think the potential use is just as important, since much of Emacs's
utility is in 3rd party packages.  But Stefan answered about the actual
uses in the Emacs trunk; the URL package is most important (because of
package.el) to the Emacs users in general.

>> I think Lars introduced that option, and at least HTTP/S connections
>> can use it.

EZ> Then this needs to be documented somewhere.

I agree.

`open-network-stream' has some documentation, and is most useful as an
API.

User customization of gnutls.el is minimal right now, just
`gnutls-algorithm-priority' and `gnutls-min-prime-bits'.  But those are
tricky: the specific library that uses the API may need to override them
too.  And generally they should not be tweaked.  So I'm not sure those
two deserve more mention in the manual.

>> It's a replacement for the previous libraries that managed secure
>> connections, except it doesn't depend on external binaries.  So it
>> really doesn't change much in terms of Emacs functionality, only in the
>> underlying implementation.

EZ> Lisp programmers should know they can use TLS when Emacs was compiled
EZ> with GnuTLS support.  Users should know that as well, because they
EZ> will need to set up their machines for that.  E.g., this:

>> There is one annoying detail with the cert bundle on W32.  It
>> defaults to /etc/ssl/certs/ca-certificates.crt which is not valid on
>> W32 and on many other platforms.

I mentioned this because it's the only important GnuTLS-related
configuration bit on all platforms.  It should be in the manual, I
think, but consider that I proposed a while back that Emacs should ship
with its own version of the Mozilla cert bundle, so that this works on
all platforms, but that was not OK with the maintainers.  

So that leaves us with the options of 1) trusting the platform (which
doesn't work on W32, AFAIK it doesn't have a cert bundle we can use; and
many GNU/Linux distros don't have a cert bundle in a standard place or
at all), or 2) making the cert bundle a GNU ELPA package than any
installer or user can activate.

Because of these concerns, currently we don't verify the peer
certificate in SSL and TLS connections.  See `gnutls-negotiate' for how
that would work.  The connections are still encrypted, but you could be
talking to an impostor.

I prefer the GNU ELPA package approach instead of trusting the platform,
but I also think the user should be able to customize this (and an
installer should offer the choice).  Coming back to documentation, I'd
like to settle the greater question of how to distribute the cert bundle
before we document the configuration options for it.

WDYT?

Ted

need help with certificate bundles for ALL the platforms Emacs supports (was: GnuTLS invasion of Emacs published))

[Ted Zlatanov]

On Fri, 03 Feb 2012 11:51:01 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

>>> There is one annoying detail with the cert bundle on W32.  It
>>> defaults to /etc/ssl/certs/ca-certificates.crt which is not valid on
>>> W32 and on many other platforms.

TZ> I mentioned this because it's the only important GnuTLS-related
TZ> configuration bit on all platforms.  It should be in the manual, I
TZ> think, but consider that I proposed a while back that Emacs should ship
TZ> with its own version of the Mozilla cert bundle, so that this works on
TZ> all platforms, but that was not OK with the maintainers.  

After discussing this with Stefan Monnier, I've decided to proceed as
follows:

New variable `gnutls-trustfiles' will be a list of trustfiles for your
platform, filtered by file existence.  It can take functions in the
list, and the functions can return a list of files or a single file.
When the list is empty, you'll get a message to look in the GNU ELPA for
fallbacks.

A new GNU ELPA package "cert-bundle-mozilla" will provide a fallback
from Mozilla's certificate bundle.  It will be versioned same as that
bundle and updated periodically.  When you install that package, it will
add a function to `gnutls-trustfiles' to load the package's cert bundle
file.

I need a list of possible cert bundle locations on all the platforms
Emacs supports, or methods to retrieve them.  Please send to me directly
or follow up here.  The assembled list will help me greatly.

I'll start with the easiest ones (please correct me if any are wrong,
based on http://mercurial.selenic.com/wiki/CACertificates):

Debian, Ubuntu, Gentoo and Arch Linux: /etc/ssl/certs/ca-certificates.crt (maintained by `update-ca-certificates').

Fedora and RHEL: /etc/pki/tls/certs/ca-bundle.crt

Suse: /etc/ssl/ca-bundle.pem

Mac OS X has the certificate list in the system keychain.  If we had
keychain access functions in Emacs, or a shell call to dump the
contents, I could export it.  Any help is welcome.

W32 doesn't seem to have a system cert bundle and getting it from any
specific browser is unreliable, but any suggestions are welcome.

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports (was: GnuTLS invasion of Emacs published))

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Thu, 09 Feb 2012 09:16:16 -0500
> 
> I need a list of possible cert bundle locations on all the platforms
> Emacs supports, or methods to retrieve them.  Please send to me directly
> or follow up here.  The assembled list will help me greatly.
> [...]
> W32 doesn't seem to have a system cert bundle and getting it from any
> specific browser is unreliable, but any suggestions are welcome.

I think you are wrong about that.  Where did you get this information?

Can you show me an example of a "cert bundle", i.e. what kind of
directory hierarchy, if any, is there, and what files can one find
there?  Examples of how files are named and their contents will help.
I need this to compare with what I think is a cert bundle on my
Windows box (if I'm not mistaken).

TIA

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Thu, 09 Feb 2012 20:53:13 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Thu, 09 Feb 2012 09:16:16 -0500
>> 
>> I need a list of possible cert bundle locations on all the platforms
>> Emacs supports, or methods to retrieve them.  Please send to me directly
>> or follow up here.  The assembled list will help me greatly.
>> [...]
>> W32 doesn't seem to have a system cert bundle and getting it from any
>> specific browser is unreliable, but any suggestions are welcome.

EZ> I think you are wrong about that.  Where did you get this information?

Web searching, e.g. the URL I cited in the post you quoted.  I'd love to
be wrong!

EZ> Can you show me an example of a "cert bundle", i.e. what kind of
EZ> directory hierarchy, if any, is there, and what files can one find
EZ> there?  Examples of how files are named and their contents will help.
EZ> I need this to compare with what I think is a cert bundle on my
EZ> Windows box (if I'm not mistaken).

Certificate bundles are usually in a .pem format (I've also seen .crt,
and unfortunately there are at least 4 different formats).  

On W32, I know the MSysGit environment has a cert bundle (inherited from
curl/libcurl and placed under /usr/bin IIRC), but I don't think there's
a generally available bundle.  They consist of hundreds of text blocks
like this:

-----BEGIN CERTIFICATE-----
MIIDpDCCAoygAwIBAgIBATANBgkqhkiG9w0BAQUFADBjMQswCQYDVQQGEwJVUzEc
...
MMbHNYaz+ZZfRtsMRf3zUMNvxsNIrUam4SdHCh0Om7bCd39j8uB9Gr784N/Xx6ds
sPmuujz9dLQR6FgNgLzTqIA6me11zEZ7
-----END CERTIFICATE-----

which are simply individual .pem files, concatenated.  In Debian/Ubuntu
there is a directory structure under /etc/ssl, but Mozilla's bundle, for
instance, is offered as simply a monolithic download.

The question is how to obtain one reliably, and all my research leads me
to believe that W32 doesn't have it.

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Fri, 10 Feb 2012 08:06:31 -0500
> 
> >> W32 doesn't seem to have a system cert bundle and getting it from any
> >> specific browser is unreliable, but any suggestions are welcome.
> 
> EZ> I think you are wrong about that.  Where did you get this information?
> 
> Web searching, e.g. the URL I cited in the post you quoted.  I'd love to
> be wrong!

This URL:

   http://technet.microsoft.com/en-us/library/cc962104.aspx

and also a few others seem to indicate that each Windows user has
his/her certificates in this directory:

  C:\Documents and Settings\<username>\Application Data\Microsoft\SystemCertificates\My\Certificates

I do have such a directory on my XP box, but it is empty.  Meanwhile,
the application that is used on Windows to browse certificates does
show a long list of certificates I allegedly have on this box.

On another XP system I did see files in the above directory, but they
were binary files, unlike the contents you show:

> They consist of hundreds of text blocks like this:
> 
> -----BEGIN CERTIFICATE-----
> MIIDpDCCAoygAwIBAgIBATANBgkqhkiG9w0BAQUFADBjMQswCQYDVQQGEwJVUzEc
> ...
> MMbHNYaz+ZZfRtsMRf3zUMNvxsNIrUam4SdHCh0Om7bCd39j8uB9Gr784N/Xx6ds
> sPmuujz9dLQR6FgNgLzTqIA6me11zEZ7
> -----END CERTIFICATE-----
> 
> which are simply individual .pem files, concatenated.  In Debian/Ubuntu
> there is a directory structure under /etc/ssl, but Mozilla's bundle, for
> instance, is offered as simply a monolithic download.
> 
> The question is how to obtain one reliably, and all my research leads me
> to believe that W32 doesn't have it.

I know nothing about these issues, so I'm really not the right person
to look into this.  Perhaps someone else could chime in.

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Fri, 10 Feb 2012 17:51:45 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> The question is how to obtain one reliably, and all my research leads me
>> to believe that W32 doesn't have it.

EZ> This URL:

EZ>    http://technet.microsoft.com/en-us/library/cc962104.aspx

EZ> and also a few others seem to indicate that each Windows user has
EZ> his/her certificates in this directory:

EZ>   C:\Documents and Settings\<username>\Application Data\Microsoft\SystemCertificates\My\Certificates

EZ> I do have such a directory on my XP box, but it is empty.  Meanwhile,
EZ> the application that is used on Windows to browse certificates does
EZ> show a long list of certificates I allegedly have on this box.

EZ> On another XP system I did see files in the above directory, but they
EZ> were binary files, unlike the contents you show:

That's unfortunate.  I'll assume for now that on W32 we have to supply
our own certificate bundle through the GNU ELPA package, until someone
comes up with a better solution.  I think that's acceptable since we're
simply mimicking Mozilla's CA choices, and we can make incremental
improvements to gnutls.el as we find out more about each platform.

Thanks!
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Andy Moreton]

On Fri 10 Feb 2012, Ted Zlatanov wrote:

> On Fri, 10 Feb 2012 17:51:45 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>
>>> From: Ted Zlatanov <tzz@lifelogs.com>
>>> The question is how to obtain one reliably, and all my research leads me
>>> to believe that W32 doesn't have it.
>
> EZ> This URL:
>
> EZ>    http://technet.microsoft.com/en-us/library/cc962104.aspx
>
> EZ> and also a few others seem to indicate that each Windows user has
> EZ> his/her certificates in this directory:
>
> EZ>   C:\Documents and Settings\<username>\Application Data\Microsoft\SystemCertificates\My\Certificates
>
> EZ> I do have such a directory on my XP box, but it is empty.  Meanwhile,
> EZ> the application that is used on Windows to browse certificates does
> EZ> show a long list of certificates I allegedly have on this box.
>
> EZ> On another XP system I did see files in the above directory, but they
> EZ> were binary files, unlike the contents you show:
>
> That's unfortunate.  I'll assume for now that on W32 we have to supply
> our own certificate bundle through the GNU ELPA package, until someone
> comes up with a better solution.  I think that's acceptable since we're
> simply mimicking Mozilla's CA choices, and we can make incremental
> improvements to gnutls.el as we find out more about each platform.
>
> Thanks!
> Ted

It appears that Windows stores the certificates in the registry - see
"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates".

I expect that additonal locations are used under the control of group
policy for domain machines etc, and that this data should only be used
via the appropriate APIs.

Cygwin also has a cert bundle in the ca-certificates package - see 
http://cygwin.com/packages/ca-certificates/

    AndyM

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sat, 11 Feb 2012 17:22:40 +0000
> 
> It appears that Windows stores the certificates in the registry - see
> "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates".

Thanks.  FWIW, there's also

   HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates

for the user's certificates.  But what I see there, in both locations,
are binary blobs, not anything like what Ted showed.

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Sat, 11 Feb 2012 19:45:25 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Sat, 11 Feb 2012 17:22:40 +0000
>> 
>> It appears that Windows stores the certificates in the registry - see
>> "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates".

EZ> Thanks.  FWIW, there's also

EZ>    HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates

EZ> for the user's certificates.  But what I see there, in both locations,
EZ> are binary blobs, not anything like what Ted showed.

There are many certificate formats GnuTLS can speak; the .pem files I
showed are most common where legibility matters.  Can Emacs extract
everything under this registry path automatically?  I didn't see a way
in the C code.  If I can slurp them into a file, I may be able to use
that.

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Cc: Andy Moreton <andrewjmoreton@gmail.com>,  emacs-devel@gnu.org
> Date: Sat, 11 Feb 2012 21:43:27 -0500
> 
> Can Emacs extract everything under this registry path automatically?
> I didn't see a way in the C code.  If I can slurp them into a file,
> I may be able to use that.

Why do you need it to be on a file?  Emacs on Windows can access the
Registry as easily as it can access files.

The question is, can whatever you are using or writing read and use
the format of the certificates stored in the Windows Registry?

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Sun, 12 Feb 2012 06:05:22 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> The question is, can whatever you are using or writing read and use
EZ> the format of the certificates stored in the Windows Registry?

The GnuTLS API can take a file name or binary blobs in DER or PEM
format, according to the docs.  We only support file names right now.  I
would make the necessary changes if it was necessary to load the
registry blobs.

Unfortunately according to
http://citrixblogger.org/2010/09/13/public-key-certificate-locations-in-windows/
the story is much more complicated, with some certificates stored to
disk and so on.  It looks like a much better idea to use certreq.exe or
certutil.exe to dump all the trusted certificates, if those tools
support it.  Does anyone know?

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Andy Moreton]

On Sun 12 Feb 2012, Ted Zlatanov wrote:

> On Sat, 11 Feb 2012 19:45:25 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>
>>> From: Andy Moreton <andrewjmoreton@gmail.com>
>>> Date: Sat, 11 Feb 2012 17:22:40 +0000
>>> 
>>> It appears that Windows stores the certificates in the registry - see
>>> "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates".
>
> EZ> Thanks.  FWIW, there's also
>
> EZ>    HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates
>
> EZ> for the user's certificates.  But what I see there, in both locations,
> EZ> are binary blobs, not anything like what Ted showed.
>
> There are many certificate formats GnuTLS can speak; the .pem files I
> showed are most common where legibility matters.  Can Emacs extract
> everything under this registry path automatically?  I didn't see a way
> in the C code.  If I can slurp them into a file, I may be able to use
> that.

Please do not read these registry keys - you will almost certainly end
up using revoked certificates (e,.g. diginotar), and duplicating the
work of the existing system APIs but with added bugs.

Please read the following articles:

Certificate Status and Revocation Checking - TechNet Articles - Home - TechNet Wiki
<http://social.technet.microsoft.com/wiki/contents/articles/4954.certificate-status-and-revocation-checking.aspx>

How Certificate Revocation Works
<http://technet.microsoft.com/en-gb/library/ee619754(WS.10).aspx>

There is lots of information there about how this works for various
Windows versions.

    AndyM

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 10:29:36 +0000 Andy Moreton <andrewjmoreton@gmail.com> wrote: 

>> On Sat, 11 Feb 2012 19:45:25 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>> 
EZ> Thanks.  FWIW, there's also
>> 
EZ> HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates
>> 
EZ> for the user's certificates.  But what I see there, in both locations,
EZ> are binary blobs, not anything like what Ted showed.
...
AM> Please do not read these registry keys - you will almost certainly end
AM> up using revoked certificates (e,.g. diginotar), and duplicating the
AM> work of the existing system APIs but with added bugs.

AM> Please read the following articles:

AM> Certificate Status and Revocation Checking - TechNet Articles - Home - TechNet Wiki
AM> <http://social.technet.microsoft.com/wiki/contents/articles/4954.certificate-status-and-revocation-checking.aspx>

AM> How Certificate Revocation Works
AM> <http://technet.microsoft.com/en-gb/library/ee619754(WS.10).aspx>

AM> There is lots of information there about how this works for various
AM> Windows versions.

As I said later, the complexity of this task indicates we should use the
certutil.exe binary or something like it.  I am not excited to spend
hours reverse-engineering Microsoft's certificate storage strategy and
it would be a brittle solution in any case since it changes with W32
releases.

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Thu, 09 Feb 2012 09:16:16 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> I'll start with the easiest ones (please correct me if any are wrong,
TZ> based on http://mercurial.selenic.com/wiki/CACertificates):

TZ> Debian, Ubuntu, Gentoo and Arch Linux: /etc/ssl/certs/ca-certificates.crt (maintained by `update-ca-certificates').

TZ> Fedora and RHEL: /etc/pki/tls/certs/ca-bundle.crt

TZ> Suse: /etc/ssl/ca-bundle.pem

Maintainers: can I change gnutls.el to provide a customizable
`gnutls-trustfiles' and to probe these file locations or would you
consider that a new feature that has to wait?

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

> Maintainers: can I change gnutls.el to provide a customizable
> `gnutls-trustfiles' and to probe these file locations or would you
> consider that a new feature that has to wait?

I think it's OK to install now, but please show us the patch for
confirmation,


        Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

On Fri, 10 Feb 2012 13:57:18 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> Maintainers: can I change gnutls.el to provide a customizable
>> `gnutls-trustfiles' and to probe these file locations or would you
>> consider that a new feature that has to wait?

SM> I think it's OK to install now, but please show us the patch for
SM> confirmation,

No ChangeLog yet, just the code.  It's pretty simple.

`gnutls-flatten-list' seems like a nice general utility, maybe it
already exists?

Thanks
Ted


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnutls-trustfiles.patch --]
[-- Type: text/x-diff, Size: 2712 bytes --]

=== modified file 'lisp/net/gnutls.el'
--- lisp/net/gnutls.el	2012-02-12 21:40:25 +0000
+++ lisp/net/gnutls.el	2012-02-12 22:11:53 +0000
@@ -51,6 +51,22 @@
   :type '(choice (const nil)
 		 string))
 
+(defcustom gnutls-trustfiles '(
+                               ;; Debian, Ubuntu, Gentoo and Arch Linux
+                               "/etc/ssl/certs/ca-certificates.crt"
+                               ;; Fedora and RHEL
+                               "/etc/pki/tls/certs/ca-bundle.crt"
+                               ;; Suse
+                               "/etc/ssl/ca-bundle.pem"
+                               )
+  "List of functions or filenames yielding CA bundle locations.
+The files may be in PEM or DER format, as per the GnuTLS documentation.
+The files may not exist, in which case they will be ignored.
+Functions will be called and may return a filename or a list of filenames."
+  :group 'gnutls
+  :type '(repeat (choice (function :tag "Function")
+                         (file :tag "Bundle filename"))))
+
 ;;;###autoload
 (defcustom gnutls-min-prime-bits nil
   "The minimum number of bits to be used in Diffie-Hellman key exchange.
@@ -156,10 +172,14 @@
 It must be omitted, a number, or nil; if omitted or nil it
 defaults to GNUTLS_VERIFY_ALLOW_X509_V1_CA_CRT."
   (let* ((type (or type 'gnutls-x509pki))
-         (default-trustfile "/etc/ssl/certs/ca-certificates.crt")
          (trustfiles (or trustfiles
-                         (when (file-exists-p default-trustfile)
-                           (list default-trustfile))))
+                         (delq nil
+                               (mapcar (lambda (f) (and f (file-exists-p f) f))
+                                       (gnutls-flatten-list
+                                        (mapcar (lambda (tf) (if (functionp tf)
+                                                            (funcall tf)
+                                                          tf))
+                                                gnutls-trustfiles))))))
          (priority-string (or priority-string
                               (cond
                                ((eq type 'gnutls-anon)
@@ -203,6 +223,17 @@
              doit (gnutls-error-string doit)
              (apply 'format format (or params '(nil))))))
 
+;; copied from `eshell-flatten-list'
+(defun gnutls-flatten-list (args)
+  "Flatten any lists within ARGS, so that there are no sublists."
+  (let ((new-list (list t)))
+    (dolist (a args)
+      (if (and (listp a)
+               (listp (cdr a)))
+          (nconc new-list (eshell-flatten-list a))
+        (nconc new-list (list a))))
+    (cdr new-list)))
+
 (provide 'gnutls)
 
 ;;; gnutls.el ends here

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

> +(defcustom gnutls-trustfiles '(
> +                               ;; Debian, Ubuntu, Gentoo and Arch Linux
> +                               "/etc/ssl/certs/ca-certificates.crt"
> +                               ;; Fedora and RHEL
> +                               "/etc/pki/tls/certs/ca-bundle.crt"
> +                               ;; Suse
> +                               "/etc/ssl/ca-bundle.pem"
> +                               )
> +  "List of functions or filenames yielding CA bundle locations.
> +The files may be in PEM or DER format, as per the GnuTLS documentation.
> +The files may not exist, in which case they will be ignored.
> +Functions will be called and may return a filename or a list of filenames."
> +  :group 'gnutls
> +  :type '(repeat (choice (function :tag "Function")
> +                         (file :tag "Bundle filename"))))

How 'bout something like

   (defcustom gnutls-trustfile
       (let ((file (if (boundp 'cert-bundle-location)
                       cert-bundle-location))
             (candidates 
              '("/etc/ssl/certs/ca-certificates.crt" ; Debian, Gentoo, Arch.
                "/etc/pki/tls/certs/ca-bundle.crt"   ; Fedora and RHEL.
                "/etc/ssl/ca-bundle.pem"             ; Suse.
                )))
         (while candidates
           (if (file-readable-p (car candidates))
               (setq file (car candidate) candidates nil)
             (setq candidates (cdr candidates))))
         file)
     "Name of the CA bundle file.
   The file may be in PEM or DER format, as per the GnuTLS documentation."
     :group 'gnutls
     :type '(choice (const nil) (file :tag "Bundle filename")))


-- Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Sun, 12 Feb 2012 22:28:24 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> +(defcustom gnutls-trustfiles '(
>> +                               ;; Debian, Ubuntu, Gentoo and Arch Linux
>> +                               "/etc/ssl/certs/ca-certificates.crt"
>> +                               ;; Fedora and RHEL
>> +                               "/etc/pki/tls/certs/ca-bundle.crt"
>> +                               ;; Suse
>> +                               "/etc/ssl/ca-bundle.pem"
>> +                               )
>> +  "List of functions or filenames yielding CA bundle locations.
>> +The files may be in PEM or DER format, as per the GnuTLS documentation.
>> +The files may not exist, in which case they will be ignored.
>> +Functions will be called and may return a filename or a list of filenames."
>> +  :group 'gnutls
>> +  :type '(repeat (choice (function :tag "Function")
>> +                         (file :tag "Bundle filename"))))

SM> How 'bout something like

(defcustom gnutls-trustfile
    (let ((file (if (boundp 'cert-bundle-location)
                    cert-bundle-location))
          (candidates 
           '("/etc/ssl/certs/ca-certificates.crt" ; Debian, Gentoo, Arch.
             "/etc/pki/tls/certs/ca-bundle.crt"   ; Fedora and RHEL.
             "/etc/ssl/ca-bundle.pem"             ; Suse.
             )))
      (while candidates
        (if (file-readable-p (car candidates))
            (setq file (car candidate) candidates nil)
          (setq candidates (cdr candidates))))
      file)
  "Name of the CA bundle file.
The file may be in PEM or DER format, as per the GnuTLS documentation."
  :group 'gnutls
  :type '(choice (const nil) (file :tag "Bundle filename")))

The trustfiles parameter is a list of files, all the way through to
gnutls.c.  I don't think it should be demoted to a single file in the
customization interface, and it still needs a function choice.

Also I don't want to decide the default bundle file names at the time
the defcustom is evaluated.  Since `gnutls-trustfiles' can contain
function calls, I'd like it to be called when it's needed.  For
instance, it's very common to store certificates as PEM files in a
directory, and the user should be able to choose that approach instead
of managing a concatenated bundle.  If we built the file list only once,
the modular approach would fail.  Another situation is on W32, where the
cert bundle has to be dynamically built (which will require some caching
but should still be done as close to using the bundle as possible).

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

> Also I don't want to decide the default bundle file names at the time
> the defcustom is evaluated.  Since `gnutls-trustfiles' can contain
> function calls, I'd like it to be called when it's needed.  For
> instance, it's very common to store certificates as PEM files in a
> directory, and the user should be able to choose that approach instead
> of managing a concatenated bundle.  If we built the file list only once,
> the modular approach would fail.  Another situation is on W32, where the
> cert bundle has to be dynamically built (which will require some caching
> but should still be done as close to using the bundle as possible).

OK, but the variable should not be a "list of (function or filename)".
That's ugly.  Maybe we can have it be "a function or a list of files".


        Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]

On Mon, 13 Feb 2012 10:12:17 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> Also I don't want to decide the default bundle file names at the time
>> the defcustom is evaluated.  Since `gnutls-trustfiles' can contain
>> function calls, I'd like it to be called when it's needed.  For
>> instance, it's very common to store certificates as PEM files in a
>> directory, and the user should be able to choose that approach instead
>> of managing a concatenated bundle.  If we built the file list only once,
>> the modular approach would fail.  Another situation is on W32, where the
>> cert bundle has to be dynamically built (which will require some caching
>> but should still be done as close to using the bundle as possible).

SM> OK, but the variable should not be a "list of (function or filename)".
SM> That's ugly.

I see how it's confusing.

SM> Maybe we can have it be "a function or a list of files".

OK.  Patch attached for your review.  The code is simpler now and the
list flattening function is not needed.

If approved I think I should also write a manual entry for this new
variable.  Should I make a new manual subsection for GnuTLS-related
things?  Where?

Now we'll have three customizable variables in gnutls.el
(`gnutls-algorithm-priority', `gnutls-trustfiles', and
`gnutls-min-prime-bits') which is tipping the scales I think.  Plus it
will be good to explain what gnutls.el+gnutls.c do and how to debug
problems with them, since most users and developers don't know how
widely they are used in Emacs 24.

Thanks!
Ted


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnutls-trustfiles.patch --]
[-- Type: text/x-diff, Size: 2293 bytes --]

=== modified file 'lisp/net/gnutls.el'
--- lisp/net/gnutls.el	2012-02-12 21:40:25 +0000
+++ lisp/net/gnutls.el	2012-02-13 16:20:13 +0000
@@ -51,6 +51,19 @@
   :type '(choice (const nil)
 		 string))
 
+(defcustom gnutls-trustfiles
+  '(
+    "/etc/ssl/certs/ca-certificates.crt" ; Debian, Ubuntu, Gentoo and Arch Linux
+    "/etc/pki/tls/certs/ca-bundle.crt"   ; Fedora and RHEL
+    "/etc/ssl/ca-bundle.pem"             ; Suse
+    )
+  "List of CA bundle location filenames or a function returning said list.
+The files may be in PEM or DER format, as per the GnuTLS documentation.
+The files may not exist, in which case they will be ignored."
+  :group 'gnutls
+  :type '(choice (function :tag "Function to produce list of bundle filenames")
+                 (repeat (file :tag "Bundle filename"))))
+
 ;;;###autoload
 (defcustom gnutls-min-prime-bits nil
   "The minimum number of bits to be used in Diffie-Hellman key exchange.
@@ -118,7 +131,7 @@
 PROCESS is a process returned by `open-network-stream'.
 HOSTNAME is the remote hostname.  It must be a valid string.
 PRIORITY-STRING is as per the GnuTLS docs, default is \"NORMAL\".
-TRUSTFILES is a list of CA bundles.
+TRUSTFILES is a list of CA bundles.  It defaults to `gnutls-trustfiles'.
 CRLFILES is a list of CRL files.
 KEYLIST is an alist of (client key file, client cert file) pairs.
 MIN-PRIME-BITS is the minimum acceptable size of Diffie-Hellman keys
@@ -156,10 +169,12 @@
 It must be omitted, a number, or nil; if omitted or nil it
 defaults to GNUTLS_VERIFY_ALLOW_X509_V1_CA_CRT."
   (let* ((type (or type 'gnutls-x509pki))
-         (default-trustfile "/etc/ssl/certs/ca-certificates.crt")
          (trustfiles (or trustfiles
-                         (when (file-exists-p default-trustfile)
-                           (list default-trustfile))))
+                         (delq nil
+                               (mapcar (lambda (f) (and f (file-exists-p f) f))
+                                       (if (functionp gnutls-trustfiles)
+                                           (funcall gnutls-trustfiles)
+                                         gnutls-trustfiles)))))
          (priority-string (or priority-string
                               (cond
                                ((eq type 'gnutls-anon)

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

> OK.  Patch attached for your review.  The code is simpler now and the
> list flattening function is not needed.

Looks OK, please install.

> If approved I think I should also write a manual entry for this new
> variable.  Should I make a new manual subsection for GnuTLS-related
> things?  Where?

To the extent that the manual does not talk about TLS at all right now,
I don't think gnutls-trustfiles has a place yet.  But feel free to
update the documentation of open-network-stream.


        Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 16:04:46 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> OK.  Patch attached for your review.  The code is simpler now and the
>> list flattening function is not needed.

SM> Looks OK, please install.

Done, thank you.

>> If approved I think I should also write a manual entry for this new
>> variable.  Should I make a new manual subsection for GnuTLS-related
>> things?  Where?

SM> To the extent that the manual does not talk about TLS at all right now,
SM> I don't think gnutls-trustfiles has a place yet.  But feel free to
SM> update the documentation of open-network-stream.

I don't see how to update it appropriately.  I could add "Please see
`gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
random reference.  Maybe Lars has an opinion?

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I don't see how to update it appropriately.  I could add "Please see
> `gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
> random reference.  Maybe Lars has an opinion?

No opinion.  :-)  

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

SM> To the extent that the manual does not talk about TLS at all right now,
SM> I don't think gnutls-trustfiles has a place yet.  But feel free to
SM> update the documentation of open-network-stream.
> I don't see how to update it appropriately.  I could add "Please see
> `gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
> random reference.

I don't mean "update it with gnutls-trustfiles info", but "update it to
document the new "&rest PARAMS" keyword arguments.  At that point there
will be a place where you can document gnutls-trustfiles.


        Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 17:20:22 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

SM> To the extent that the manual does not talk about TLS at all right now,
SM> I don't think gnutls-trustfiles has a place yet.  But feel free to
SM> update the documentation of open-network-stream.
>> I don't see how to update it appropriately.  I could add "Please see
>> `gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
>> random reference.

SM> I don't mean "update it with gnutls-trustfiles info", but "update it to
SM> document the new "&rest PARAMS" keyword arguments.  At that point there
SM> will be a place where you can document gnutls-trustfiles.

I'm confused.  The keyword arguments of `open-network-stream' are
already documented.  Do you mean I should add a new :trustfiles argument
and pass that down to `network-stream-open-starttls', and in the
documentation for that argument mention `gnutls-trustfiles'?

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

SM> To the extent that the manual does not talk about TLS at all right now,
SM> I don't think gnutls-trustfiles has a place yet.  But feel free to
SM> update the documentation of open-network-stream.
>>> I don't see how to update it appropriately.  I could add "Please see
>>> `gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
>>> random reference.
SM> I don't mean "update it with gnutls-trustfiles info", but "update it to
SM> document the new "&rest PARAMS" keyword arguments.  At that point there
SM> will be a place where you can document gnutls-trustfiles.
> I'm confused.  The keyword arguments of `open-network-stream' are
> already documented.

Where?  In doc/lispref/processes.texi I only see

   @defun open-network-stream name buffer-or-name host service


-- Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Glenn Morris]

Ted Zlatanov wrote:

> +                               ;; Fedora and RHEL
> +                               "/etc/pki/tls/certs/ca-bundle.crt"

FWIW, on RHEL6 I have both /etc/pki/tls/certs/ca-bundle.crt:

# This is a bundle of X.509 certificates of public Certificate
# Authorities.  It was generated from the Mozilla root CA list.


and /etc/pki/tls/certs/ca-bundle.trust.crt:

# This is a bundle of X.509 certificates of public Certificate
# Authorities.  It was generated from the Mozilla root CA list.
# These certificates are in the OpenSSL "TRUSTED CERTIFICATE"
# format and have trust bits set accordingly.


I have no idea which of those you want. The latter is slightly larger.

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 21:32:14 -0500 Glenn Morris <rgm@gnu.org> wrote: 

GM> Ted Zlatanov wrote:
>> +                               ;; Fedora and RHEL
>> +                               "/etc/pki/tls/certs/ca-bundle.crt"

GM> FWIW, on RHEL6 I have both /etc/pki/tls/certs/ca-bundle.crt:

GM> # This is a bundle of X.509 certificates of public Certificate
GM> # Authorities.  It was generated from the Mozilla root CA list.

GM> and /etc/pki/tls/certs/ca-bundle.trust.crt:

GM> # This is a bundle of X.509 certificates of public Certificate
GM> # Authorities.  It was generated from the Mozilla root CA list.
GM> # These certificates are in the OpenSSL "TRUSTED CERTIFICATE"
GM> # format and have trust bits set accordingly.

GM> I have no idea which of those you want. The latter is slightly larger.

Me neither, and I have no RHEL systems.

According to
http://rpmfind.net/linux/RPM/fedora/15/i386/ca-certificates-2011.70-2.fc15.noarch.html
both of these are in the ca-certificates Fedora package.  So I would
guess the differences are cosmetic and the files are equivalent.  But if
anyone knows different, please let us know.

Thanks
Ted

Re: Emacs 24.0.93 Pretest Windows Binaries published

[Stefan Monnier]

> What other features in Emacs use TLS as of this writing?  I thought

At least IRC, NNTP and probably URL for `https',


        Stefan

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Richard M. Heiberger]

[-- Attachment #1: Type: text/plain, Size: 1416 bytes --]

I just downloaded and unzipped The Emacs 24.0.93 Pretest Windows Binaries

Symantec quarantined it with the message

Scan type: SONAR Scan
Event: Security Risk Found!
Security risk detected: Bloodhound.Sonar.9
File: c:\emacs\emacs-24.0.93\bin\runemacs.exe
Location: Quarantine
Computer: STAT-RHEIBERGER
User: rmh
Action taken: Quarantine succeeded
Date found: Thursday, February 02, 2012  7:40:31 PM


My inclination is to believe emacs rather than symantec, but nonetheless I
am checking with you.

Rich


On Sun, Jan 29, 2012 at 9:02 PM, Christoph Scholtes <
cschol2112@googlemail.com> wrote:

> The Emacs 24.0.93 Pretest Windows Binaries have been published in
>
> http://alpha.gnu.org/gnu/**emacs/pretest/windows/<http://alpha.gnu.org/gnu/emacs/pretest/windows/>
>
> The binaries were built using the following libraries:
> giflib-4.1.4-1
> gnutls-3.0.9
> jpeg-6b-4
> libXpm-3.5.8
> libpng-1.4.3-1
> tiff-3.8.2-1
> zlib-1.2.5-2
>
> Pre-built Windows binaries for GnuTLS are available at this location:
> http://sourceforge.net/**projects/ezwinports/files/<http://sourceforge.net/projects/ezwinports/files/>
>
> See the file included README.W32 file for more information on how to
> obtain other binaries necessary to make use of certain Emacs features.
>
> Please report any bugs that you come across via M-x report-emacs-bugs,
> or email bug-gnu-emacs@gnu.org.
>
> For questions, email emacs-devel@gnu.org.
>
>

[-- Attachment #2: Type: text/html, Size: 2050 bytes --]

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> Date: Thu, 2 Feb 2012 19:48:48 -0500
> From: "Richard M. Heiberger" <rmh@temple.edu>
> Cc: Emacs-Devel devel <emacs-devel@gnu.org>
> 
> I just downloaded and unzipped The Emacs 24.0.93 Pretest Windows Binaries
> 
> Symantec quarantined it with the message
> 
> Scan type: SONAR Scan
> Event: Security Risk Found!
> Security risk detected: Bloodhound.Sonar.9
> File: c:\emacs\emacs-24.0.93\bin\runemacs.exe
> Location: Quarantine
> Computer: STAT-RHEIBERGER
> User: rmh
> Action taken: Quarantine succeeded
> Date found: Thursday, February 02, 2012  7:40:31 PM
> 
> 
> My inclination is to believe emacs rather than symantec, but nonetheless I
> am checking with you.

You are right: this is a false alarm.  Let Symantec people know about
it, and ask them to get their act together.

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > You are right: this is a false alarm.  Let Symantec people know about
 > it, and ask them to get their act together.

That's hopeless, especially since we're now up to about 3 maybe 4 such
false alarms (ie, from different Wolf-Crying Peter companies).

(1) They'll say "better safe than sorry," and guess what? they're
    right! (as far as that goes, see (2)).

(2) It's impossible for anybody but Microsoft to truly get the act
    together, because the 3rd party virus checkers have to look for
    "signatures" in the content.  This is so that software whose whole
    selling point is "you don't need to know squat to use this because
    it's all automatic" can continue to oh-so-conveniently
    automatically run pretty much anything you download off the
    InterSewer.  False positives are pretty much inevitable with this
    technology.

And they're only going to only become more common, since viruses are
proliferating at the rate of what, about 1000 new variants a day?

I think you're just going to have to grin and bear this, because the
only alternative that's acceptable to the vast majority of Windows
customers is not safe 'nets, it's what Richard likes to call
"treacherous computing".  Let's pray that that does not become The
Final Solution.

But maybe Ted Z and GnuTLS can save the day.  GnuTLS is universally
applicable security for the network I hear! ;-)

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: "Richard M. Heiberger" <rmh@temple.edu>,
>     cschol2112@googlemail.com,
>     emacs-devel@gnu.org
> Date: Fri, 03 Feb 2012 19:03:29 +0900
> 
> Eli Zaretskii writes:
> 
>  > You are right: this is a false alarm.  Let Symantec people know about
>  > it, and ask them to get their act together.
> 
> That's hopeless, especially since we're now up to about 3 maybe 4 such
> false alarms (ie, from different Wolf-Crying Peter companies).
> 
> (1) They'll say "better safe than sorry," and guess what? they're
>     right! (as far as that goes, see (2)).
> 
> (2) It's impossible for anybody but Microsoft to truly get the act
>     together, because the 3rd party virus checkers have to look for
>     "signatures" in the content.  This is so that software whose whole
>     selling point is "you don't need to know squat to use this because
>     it's all automatic" can continue to oh-so-conveniently
>     automatically run pretty much anything you download off the
>     InterSewer.  False positives are pretty much inevitable with this
>     technology.

Is all this based on facts or on assumptions?  IOW, did you ever
report such problems to Symantec, and got the above as response?

I don't know about Symantec (don't use their products), but with AVG
it works as expected: you submit the offending file for their
analysis, via the GUI of the antivirus program, and get an email
notification, usually within hours, saying that it's a false alarm;
and the virus database is updated within a couple of days accordingly.

So if Symantec is really behaving like you describe, their users
should simply find a better product.

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > I don't know about Symantec (don't use their products), but with AVG
 > it works as expected: you submit the offending file for their
 > analysis, via the GUI of the antivirus program, and get an email
 > notification, usually within hours, saying that it's a false alarm;
 > and the virus database is updated within a couple of days accordingly.

That's what I would expect from Symantec, too.  But "submit the file
and get it fixed in a few days" is not what I understood from your
post.  "Get your act together" implies "you shouldn't be making
mistakes like this in the first place", not "I understand these things
happen, but you made a mistake here, please fix it."

I think they /should/ get their act together, but I'm not gonna hold my
breath.

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: rmh@temple.edu,
>     cschol2112@googlemail.com,
>     emacs-devel@gnu.org
> Date: Fri, 03 Feb 2012 22:49:59 +0900
> 
> But "submit the file and get it fixed in a few days" is not what I
> understood from your post.  "Get your act together" implies "you
> shouldn't be making mistakes like this in the first place", not "I
> understand these things happen, but you made a mistake here, please
> fix it."

In that case, I apologize for confusing wording, because I didn't mean
the latter at all.

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Lennart Borgman]

I just downloaded

  http://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-24.0.93-bin-i386.zip

I got a warning from Avast web shield:

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Lennart Borgman]

(Sorry for for double mail message.)
I just downloaded

   http://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-24.0.93-bin-i386.zip

I got a warning from Avast web shield when downloading:

  ...\addpm.exe Severity:High Threat: Win32:Malware-gen

Note: I have not unpacked anything. This warning came during downloading.

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> From: Lennart Borgman <lennart.borgman@gmail.com>
> Date: Sun, 12 Feb 2012 05:08:41 +0100
> Cc: "Stephen J. Turnbull" <stephen@xemacs.org>, rmh@temple.edu, cschol2112@googlemail.com, 
> 	emacs-devel@gnu.org
> 
> (Sorry for for double mail message.)
> I just downloaded
> 
>    http://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-24.0.93-bin-i386.zip
> 
> I got a warning from Avast web shield when downloading:
> 
>   ...\addpm.exe Severity:High Threat: Win32:Malware-gen

Report that to Avast as a false alarm.

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Lennart Borgman]

On Sun, Feb 12, 2012 at 17:23, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Lennart Borgman <lennart.borgman@gmail.com>
>> Date: Sun, 12 Feb 2012 05:08:41 +0100
>> Cc: "Stephen J. Turnbull" <stephen@xemacs.org>, rmh@temple.edu, cschol2112@googlemail.com,
>>       emacs-devel@gnu.org
>>
>> (Sorry for for double mail message.)
>> I just downloaded
>>
>>    http://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-24.0.93-bin-i386.zip
>>
>> I got a warning from Avast web shield when downloading:
>>
>>   ...\addpm.exe Severity:High Threat: Win32:Malware-gen
>
> Report that to Avast as a false alarm.

Ok, so no one else have seen this particular trouble?

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Stefan Monnier]

>> Report that to Avast as a false alarm.
> Ok, so no one else have seen this particular trouble?

Even if someone else has seen it, the right way to deal with it is to
bug the virus-s[cp]a[nm][nm]er.


        Stefan

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Eli Zaretskii]

> From: Lennart Borgman <lennart.borgman@gmail.com>
> Date: Sun, 12 Feb 2012 23:19:50 +0100
> Cc: rmh@temple.edu, cschol2112@googlemail.com, stephen@xemacs.org,
> 	emacs-devel@gnu.org
> 
> >>   ...\addpm.exe Severity:High Threat: Win32:Malware-gen
> >
> > Report that to Avast as a false alarm.
> 
> Ok, so no one else have seen this particular trouble?

Only the antivirus programs.

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Lennart Borgman]

On Mon, Feb 13, 2012 at 04:51, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Lennart Borgman <lennart.borgman@gmail.com>
>> Date: Sun, 12 Feb 2012 23:19:50 +0100
>> Cc: rmh@temple.edu, cschol2112@googlemail.com, stephen@xemacs.org,
>>       emacs-devel@gnu.org
>>
>> >>   ...\addpm.exe Severity:High Threat: Win32:Malware-gen
>> >
>> > Report that to Avast as a false alarm.
>>
>> Ok, so no one else have seen this particular trouble?
>
> Only the antivirus programs.

Ok, I have sent a bug report and pointed to this thread (since it was
here I mentioned it).

Re: [h-e-w] Emacs 24.0.93 Pretest Windows Binaries published

[Lennart Borgman]

On Sun, Feb 12, 2012 at 05:08, Lennart Borgman
<lennart.borgman@gmail.com> wrote:
> (Sorry for for double mail message.)
> I just downloaded
>
>   http://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-24.0.93-bin-i386.zip
>
> I got a warning from Avast web shield when downloading:
>
>  ...\addpm.exe Severity:High Threat: Win32:Malware-gen
>
> Note: I have not unpacked anything. This warning came during downloading.

I got a reply from Avast today. They said they were working on it and
when I just tested the problem seems to be fixed.