Re: need help with certificate bundles for ALL the platforms Emacs supports

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Thu, 09 Feb 2012 20:53:13 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Thu, 09 Feb 2012 09:16:16 -0500
>> 
>> I need a list of possible cert bundle locations on all the platforms
>> Emacs supports, or methods to retrieve them.  Please send to me directly
>> or follow up here.  The assembled list will help me greatly.
>> [...]
>> W32 doesn't seem to have a system cert bundle and getting it from any
>> specific browser is unreliable, but any suggestions are welcome.

EZ> I think you are wrong about that.  Where did you get this information?

Web searching, e.g. the URL I cited in the post you quoted.  I'd love to
be wrong!

EZ> Can you show me an example of a "cert bundle", i.e. what kind of
EZ> directory hierarchy, if any, is there, and what files can one find
EZ> there?  Examples of how files are named and their contents will help.
EZ> I need this to compare with what I think is a cert bundle on my
EZ> Windows box (if I'm not mistaken).

Certificate bundles are usually in a .pem format (I've also seen .crt,
and unfortunately there are at least 4 different formats).  

On W32, I know the MSysGit environment has a cert bundle (inherited from
curl/libcurl and placed under /usr/bin IIRC), but I don't think there's
a generally available bundle.  They consist of hundreds of text blocks
like this:

-----BEGIN CERTIFICATE-----
MIIDpDCCAoygAwIBAgIBATANBgkqhkiG9w0BAQUFADBjMQswCQYDVQQGEwJVUzEc
...
MMbHNYaz+ZZfRtsMRf3zUMNvxsNIrUam4SdHCh0Om7bCd39j8uB9Gr784N/Xx6ds
sPmuujz9dLQR6FgNgLzTqIA6me11zEZ7
-----END CERTIFICATE-----

which are simply individual .pem files, concatenated.  In Debian/Ubuntu
there is a directory structure under /etc/ssl, but Mozilla's bundle, for
instance, is offered as simply a monolithic download.

The question is how to obtain one reliably, and all my research leads me
to believe that W32 doesn't have it.

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Fri, 10 Feb 2012 08:06:31 -0500
> 
> >> W32 doesn't seem to have a system cert bundle and getting it from any
> >> specific browser is unreliable, but any suggestions are welcome.
> 
> EZ> I think you are wrong about that.  Where did you get this information?
> 
> Web searching, e.g. the URL I cited in the post you quoted.  I'd love to
> be wrong!

This URL:

   http://technet.microsoft.com/en-us/library/cc962104.aspx

and also a few others seem to indicate that each Windows user has
his/her certificates in this directory:

  C:\Documents and Settings\<username>\Application Data\Microsoft\SystemCertificates\My\Certificates

I do have such a directory on my XP box, but it is empty.  Meanwhile,
the application that is used on Windows to browse certificates does
show a long list of certificates I allegedly have on this box.

On another XP system I did see files in the above directory, but they
were binary files, unlike the contents you show:

> They consist of hundreds of text blocks like this:
> 
> -----BEGIN CERTIFICATE-----
> MIIDpDCCAoygAwIBAgIBATANBgkqhkiG9w0BAQUFADBjMQswCQYDVQQGEwJVUzEc
> ...
> MMbHNYaz+ZZfRtsMRf3zUMNvxsNIrUam4SdHCh0Om7bCd39j8uB9Gr784N/Xx6ds
> sPmuujz9dLQR6FgNgLzTqIA6me11zEZ7
> -----END CERTIFICATE-----
> 
> which are simply individual .pem files, concatenated.  In Debian/Ubuntu
> there is a directory structure under /etc/ssl, but Mozilla's bundle, for
> instance, is offered as simply a monolithic download.
> 
> The question is how to obtain one reliably, and all my research leads me
> to believe that W32 doesn't have it.

I know nothing about these issues, so I'm really not the right person
to look into this.  Perhaps someone else could chime in.

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Fri, 10 Feb 2012 17:51:45 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> The question is how to obtain one reliably, and all my research leads me
>> to believe that W32 doesn't have it.

EZ> This URL:

EZ>    http://technet.microsoft.com/en-us/library/cc962104.aspx

EZ> and also a few others seem to indicate that each Windows user has
EZ> his/her certificates in this directory:

EZ>   C:\Documents and Settings\<username>\Application Data\Microsoft\SystemCertificates\My\Certificates

EZ> I do have such a directory on my XP box, but it is empty.  Meanwhile,
EZ> the application that is used on Windows to browse certificates does
EZ> show a long list of certificates I allegedly have on this box.

EZ> On another XP system I did see files in the above directory, but they
EZ> were binary files, unlike the contents you show:

That's unfortunate.  I'll assume for now that on W32 we have to supply
our own certificate bundle through the GNU ELPA package, until someone
comes up with a better solution.  I think that's acceptable since we're
simply mimicking Mozilla's CA choices, and we can make incremental
improvements to gnutls.el as we find out more about each platform.

Thanks!
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Thu, 09 Feb 2012 09:16:16 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> I'll start with the easiest ones (please correct me if any are wrong,
TZ> based on http://mercurial.selenic.com/wiki/CACertificates):

TZ> Debian, Ubuntu, Gentoo and Arch Linux: /etc/ssl/certs/ca-certificates.crt (maintained by `update-ca-certificates').

TZ> Fedora and RHEL: /etc/pki/tls/certs/ca-bundle.crt

TZ> Suse: /etc/ssl/ca-bundle.pem

Maintainers: can I change gnutls.el to provide a customizable
`gnutls-trustfiles' and to probe these file locations or would you
consider that a new feature that has to wait?

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

> Maintainers: can I change gnutls.el to provide a customizable
> `gnutls-trustfiles' and to probe these file locations or would you
> consider that a new feature that has to wait?

I think it's OK to install now, but please show us the patch for
confirmation,


        Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Andy Moreton]

On Fri 10 Feb 2012, Ted Zlatanov wrote:

> On Fri, 10 Feb 2012 17:51:45 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>
>>> From: Ted Zlatanov <tzz@lifelogs.com>
>>> The question is how to obtain one reliably, and all my research leads me
>>> to believe that W32 doesn't have it.
>
> EZ> This URL:
>
> EZ>    http://technet.microsoft.com/en-us/library/cc962104.aspx
>
> EZ> and also a few others seem to indicate that each Windows user has
> EZ> his/her certificates in this directory:
>
> EZ>   C:\Documents and Settings\<username>\Application Data\Microsoft\SystemCertificates\My\Certificates
>
> EZ> I do have such a directory on my XP box, but it is empty.  Meanwhile,
> EZ> the application that is used on Windows to browse certificates does
> EZ> show a long list of certificates I allegedly have on this box.
>
> EZ> On another XP system I did see files in the above directory, but they
> EZ> were binary files, unlike the contents you show:
>
> That's unfortunate.  I'll assume for now that on W32 we have to supply
> our own certificate bundle through the GNU ELPA package, until someone
> comes up with a better solution.  I think that's acceptable since we're
> simply mimicking Mozilla's CA choices, and we can make incremental
> improvements to gnutls.el as we find out more about each platform.
>
> Thanks!
> Ted

It appears that Windows stores the certificates in the registry - see
"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates".

I expect that additonal locations are used under the control of group
policy for domain machines etc, and that this data should only be used
via the appropriate APIs.

Cygwin also has a cert bundle in the ca-certificates package - see 
http://cygwin.com/packages/ca-certificates/

    AndyM

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sat, 11 Feb 2012 17:22:40 +0000
> 
> It appears that Windows stores the certificates in the registry - see
> "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates".

Thanks.  FWIW, there's also

   HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates

for the user's certificates.  But what I see there, in both locations,
are binary blobs, not anything like what Ted showed.

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Sat, 11 Feb 2012 19:45:25 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Sat, 11 Feb 2012 17:22:40 +0000
>> 
>> It appears that Windows stores the certificates in the registry - see
>> "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates".

EZ> Thanks.  FWIW, there's also

EZ>    HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates

EZ> for the user's certificates.  But what I see there, in both locations,
EZ> are binary blobs, not anything like what Ted showed.

There are many certificate formats GnuTLS can speak; the .pem files I
showed are most common where legibility matters.  Can Emacs extract
everything under this registry path automatically?  I didn't see a way
in the C code.  If I can slurp them into a file, I may be able to use
that.

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Cc: Andy Moreton <andrewjmoreton@gmail.com>,  emacs-devel@gnu.org
> Date: Sat, 11 Feb 2012 21:43:27 -0500
> 
> Can Emacs extract everything under this registry path automatically?
> I didn't see a way in the C code.  If I can slurp them into a file,
> I may be able to use that.

Why do you need it to be on a file?  Emacs on Windows can access the
Registry as easily as it can access files.

The question is, can whatever you are using or writing read and use
the format of the certificates stored in the Windows Registry?

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Sun, 12 Feb 2012 06:05:22 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> The question is, can whatever you are using or writing read and use
EZ> the format of the certificates stored in the Windows Registry?

The GnuTLS API can take a file name or binary blobs in DER or PEM
format, according to the docs.  We only support file names right now.  I
would make the necessary changes if it was necessary to load the
registry blobs.

Unfortunately according to
http://citrixblogger.org/2010/09/13/public-key-certificate-locations-in-windows/
the story is much more complicated, with some certificates stored to
disk and so on.  It looks like a much better idea to use certreq.exe or
certutil.exe to dump all the trusted certificates, if those tools
support it.  Does anyone know?

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

On Fri, 10 Feb 2012 13:57:18 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> Maintainers: can I change gnutls.el to provide a customizable
>> `gnutls-trustfiles' and to probe these file locations or would you
>> consider that a new feature that has to wait?

SM> I think it's OK to install now, but please show us the patch for
SM> confirmation,

No ChangeLog yet, just the code.  It's pretty simple.

`gnutls-flatten-list' seems like a nice general utility, maybe it
already exists?

Thanks
Ted


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnutls-trustfiles.patch --]
[-- Type: text/x-diff, Size: 2712 bytes --]

=== modified file 'lisp/net/gnutls.el'
--- lisp/net/gnutls.el	2012-02-12 21:40:25 +0000
+++ lisp/net/gnutls.el	2012-02-12 22:11:53 +0000
@@ -51,6 +51,22 @@
   :type '(choice (const nil)
 		 string))
 
+(defcustom gnutls-trustfiles '(
+                               ;; Debian, Ubuntu, Gentoo and Arch Linux
+                               "/etc/ssl/certs/ca-certificates.crt"
+                               ;; Fedora and RHEL
+                               "/etc/pki/tls/certs/ca-bundle.crt"
+                               ;; Suse
+                               "/etc/ssl/ca-bundle.pem"
+                               )
+  "List of functions or filenames yielding CA bundle locations.
+The files may be in PEM or DER format, as per the GnuTLS documentation.
+The files may not exist, in which case they will be ignored.
+Functions will be called and may return a filename or a list of filenames."
+  :group 'gnutls
+  :type '(repeat (choice (function :tag "Function")
+                         (file :tag "Bundle filename"))))
+
 ;;;###autoload
 (defcustom gnutls-min-prime-bits nil
   "The minimum number of bits to be used in Diffie-Hellman key exchange.
@@ -156,10 +172,14 @@
 It must be omitted, a number, or nil; if omitted or nil it
 defaults to GNUTLS_VERIFY_ALLOW_X509_V1_CA_CRT."
   (let* ((type (or type 'gnutls-x509pki))
-         (default-trustfile "/etc/ssl/certs/ca-certificates.crt")
          (trustfiles (or trustfiles
-                         (when (file-exists-p default-trustfile)
-                           (list default-trustfile))))
+                         (delq nil
+                               (mapcar (lambda (f) (and f (file-exists-p f) f))
+                                       (gnutls-flatten-list
+                                        (mapcar (lambda (tf) (if (functionp tf)
+                                                            (funcall tf)
+                                                          tf))
+                                                gnutls-trustfiles))))))
          (priority-string (or priority-string
                               (cond
                                ((eq type 'gnutls-anon)
@@ -203,6 +223,17 @@
              doit (gnutls-error-string doit)
              (apply 'format format (or params '(nil))))))
 
+;; copied from `eshell-flatten-list'
+(defun gnutls-flatten-list (args)
+  "Flatten any lists within ARGS, so that there are no sublists."
+  (let ((new-list (list t)))
+    (dolist (a args)
+      (if (and (listp a)
+               (listp (cdr a)))
+          (nconc new-list (eshell-flatten-list a))
+        (nconc new-list (list a))))
+    (cdr new-list)))
+
 (provide 'gnutls)
 
 ;;; gnutls.el ends here

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

> +(defcustom gnutls-trustfiles '(
> +                               ;; Debian, Ubuntu, Gentoo and Arch Linux
> +                               "/etc/ssl/certs/ca-certificates.crt"
> +                               ;; Fedora and RHEL
> +                               "/etc/pki/tls/certs/ca-bundle.crt"
> +                               ;; Suse
> +                               "/etc/ssl/ca-bundle.pem"
> +                               )
> +  "List of functions or filenames yielding CA bundle locations.
> +The files may be in PEM or DER format, as per the GnuTLS documentation.
> +The files may not exist, in which case they will be ignored.
> +Functions will be called and may return a filename or a list of filenames."
> +  :group 'gnutls
> +  :type '(repeat (choice (function :tag "Function")
> +                         (file :tag "Bundle filename"))))

How 'bout something like

   (defcustom gnutls-trustfile
       (let ((file (if (boundp 'cert-bundle-location)
                       cert-bundle-location))
             (candidates 
              '("/etc/ssl/certs/ca-certificates.crt" ; Debian, Gentoo, Arch.
                "/etc/pki/tls/certs/ca-bundle.crt"   ; Fedora and RHEL.
                "/etc/ssl/ca-bundle.pem"             ; Suse.
                )))
         (while candidates
           (if (file-readable-p (car candidates))
               (setq file (car candidate) candidates nil)
             (setq candidates (cdr candidates))))
         file)
     "Name of the CA bundle file.
   The file may be in PEM or DER format, as per the GnuTLS documentation."
     :group 'gnutls
     :type '(choice (const nil) (file :tag "Bundle filename")))


-- Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Andy Moreton]

On Sun 12 Feb 2012, Ted Zlatanov wrote:

> On Sat, 11 Feb 2012 19:45:25 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>
>>> From: Andy Moreton <andrewjmoreton@gmail.com>
>>> Date: Sat, 11 Feb 2012 17:22:40 +0000
>>> 
>>> It appears that Windows stores the certificates in the registry - see
>>> "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates".
>
> EZ> Thanks.  FWIW, there's also
>
> EZ>    HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates
>
> EZ> for the user's certificates.  But what I see there, in both locations,
> EZ> are binary blobs, not anything like what Ted showed.
>
> There are many certificate formats GnuTLS can speak; the .pem files I
> showed are most common where legibility matters.  Can Emacs extract
> everything under this registry path automatically?  I didn't see a way
> in the C code.  If I can slurp them into a file, I may be able to use
> that.

Please do not read these registry keys - you will almost certainly end
up using revoked certificates (e,.g. diginotar), and duplicating the
work of the existing system APIs but with added bugs.

Please read the following articles:

Certificate Status and Revocation Checking - TechNet Articles - Home - TechNet Wiki
<http://social.technet.microsoft.com/wiki/contents/articles/4954.certificate-status-and-revocation-checking.aspx>

How Certificate Revocation Works
<http://technet.microsoft.com/en-gb/library/ee619754(WS.10).aspx>

There is lots of information there about how this works for various
Windows versions.

    AndyM

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 10:29:36 +0000 Andy Moreton <andrewjmoreton@gmail.com> wrote: 

>> On Sat, 11 Feb 2012 19:45:25 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>> 
EZ> Thanks.  FWIW, there's also
>> 
EZ> HKEY_CURRENT_USER\Software\Microsoft\SystemCertificates
>> 
EZ> for the user's certificates.  But what I see there, in both locations,
EZ> are binary blobs, not anything like what Ted showed.
...
AM> Please do not read these registry keys - you will almost certainly end
AM> up using revoked certificates (e,.g. diginotar), and duplicating the
AM> work of the existing system APIs but with added bugs.

AM> Please read the following articles:

AM> Certificate Status and Revocation Checking - TechNet Articles - Home - TechNet Wiki
AM> <http://social.technet.microsoft.com/wiki/contents/articles/4954.certificate-status-and-revocation-checking.aspx>

AM> How Certificate Revocation Works
AM> <http://technet.microsoft.com/en-gb/library/ee619754(WS.10).aspx>

AM> There is lots of information there about how this works for various
AM> Windows versions.

As I said later, the complexity of this task indicates we should use the
certutil.exe binary or something like it.  I am not excited to spend
hours reverse-engineering Microsoft's certificate storage strategy and
it would be a brittle solution in any case since it changes with W32
releases.

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Sun, 12 Feb 2012 22:28:24 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> +(defcustom gnutls-trustfiles '(
>> +                               ;; Debian, Ubuntu, Gentoo and Arch Linux
>> +                               "/etc/ssl/certs/ca-certificates.crt"
>> +                               ;; Fedora and RHEL
>> +                               "/etc/pki/tls/certs/ca-bundle.crt"
>> +                               ;; Suse
>> +                               "/etc/ssl/ca-bundle.pem"
>> +                               )
>> +  "List of functions or filenames yielding CA bundle locations.
>> +The files may be in PEM or DER format, as per the GnuTLS documentation.
>> +The files may not exist, in which case they will be ignored.
>> +Functions will be called and may return a filename or a list of filenames."
>> +  :group 'gnutls
>> +  :type '(repeat (choice (function :tag "Function")
>> +                         (file :tag "Bundle filename"))))

SM> How 'bout something like

(defcustom gnutls-trustfile
    (let ((file (if (boundp 'cert-bundle-location)
                    cert-bundle-location))
          (candidates 
           '("/etc/ssl/certs/ca-certificates.crt" ; Debian, Gentoo, Arch.
             "/etc/pki/tls/certs/ca-bundle.crt"   ; Fedora and RHEL.
             "/etc/ssl/ca-bundle.pem"             ; Suse.
             )))
      (while candidates
        (if (file-readable-p (car candidates))
            (setq file (car candidate) candidates nil)
          (setq candidates (cdr candidates))))
      file)
  "Name of the CA bundle file.
The file may be in PEM or DER format, as per the GnuTLS documentation."
  :group 'gnutls
  :type '(choice (const nil) (file :tag "Bundle filename")))

The trustfiles parameter is a list of files, all the way through to
gnutls.c.  I don't think it should be demoted to a single file in the
customization interface, and it still needs a function choice.

Also I don't want to decide the default bundle file names at the time
the defcustom is evaluated.  Since `gnutls-trustfiles' can contain
function calls, I'd like it to be called when it's needed.  For
instance, it's very common to store certificates as PEM files in a
directory, and the user should be able to choose that approach instead
of managing a concatenated bundle.  If we built the file list only once,
the modular approach would fail.  Another situation is on W32, where the
cert bundle has to be dynamically built (which will require some caching
but should still be done as close to using the bundle as possible).

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

> Also I don't want to decide the default bundle file names at the time
> the defcustom is evaluated.  Since `gnutls-trustfiles' can contain
> function calls, I'd like it to be called when it's needed.  For
> instance, it's very common to store certificates as PEM files in a
> directory, and the user should be able to choose that approach instead
> of managing a concatenated bundle.  If we built the file list only once,
> the modular approach would fail.  Another situation is on W32, where the
> cert bundle has to be dynamically built (which will require some caching
> but should still be done as close to using the bundle as possible).

OK, but the variable should not be a "list of (function or filename)".
That's ugly.  Maybe we can have it be "a function or a list of files".


        Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]

On Mon, 13 Feb 2012 10:12:17 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> Also I don't want to decide the default bundle file names at the time
>> the defcustom is evaluated.  Since `gnutls-trustfiles' can contain
>> function calls, I'd like it to be called when it's needed.  For
>> instance, it's very common to store certificates as PEM files in a
>> directory, and the user should be able to choose that approach instead
>> of managing a concatenated bundle.  If we built the file list only once,
>> the modular approach would fail.  Another situation is on W32, where the
>> cert bundle has to be dynamically built (which will require some caching
>> but should still be done as close to using the bundle as possible).

SM> OK, but the variable should not be a "list of (function or filename)".
SM> That's ugly.

I see how it's confusing.

SM> Maybe we can have it be "a function or a list of files".

OK.  Patch attached for your review.  The code is simpler now and the
list flattening function is not needed.

If approved I think I should also write a manual entry for this new
variable.  Should I make a new manual subsection for GnuTLS-related
things?  Where?

Now we'll have three customizable variables in gnutls.el
(`gnutls-algorithm-priority', `gnutls-trustfiles', and
`gnutls-min-prime-bits') which is tipping the scales I think.  Plus it
will be good to explain what gnutls.el+gnutls.c do and how to debug
problems with them, since most users and developers don't know how
widely they are used in Emacs 24.

Thanks!
Ted


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnutls-trustfiles.patch --]
[-- Type: text/x-diff, Size: 2293 bytes --]

=== modified file 'lisp/net/gnutls.el'
--- lisp/net/gnutls.el	2012-02-12 21:40:25 +0000
+++ lisp/net/gnutls.el	2012-02-13 16:20:13 +0000
@@ -51,6 +51,19 @@
   :type '(choice (const nil)
 		 string))
 
+(defcustom gnutls-trustfiles
+  '(
+    "/etc/ssl/certs/ca-certificates.crt" ; Debian, Ubuntu, Gentoo and Arch Linux
+    "/etc/pki/tls/certs/ca-bundle.crt"   ; Fedora and RHEL
+    "/etc/ssl/ca-bundle.pem"             ; Suse
+    )
+  "List of CA bundle location filenames or a function returning said list.
+The files may be in PEM or DER format, as per the GnuTLS documentation.
+The files may not exist, in which case they will be ignored."
+  :group 'gnutls
+  :type '(choice (function :tag "Function to produce list of bundle filenames")
+                 (repeat (file :tag "Bundle filename"))))
+
 ;;;###autoload
 (defcustom gnutls-min-prime-bits nil
   "The minimum number of bits to be used in Diffie-Hellman key exchange.
@@ -118,7 +131,7 @@
 PROCESS is a process returned by `open-network-stream'.
 HOSTNAME is the remote hostname.  It must be a valid string.
 PRIORITY-STRING is as per the GnuTLS docs, default is \"NORMAL\".
-TRUSTFILES is a list of CA bundles.
+TRUSTFILES is a list of CA bundles.  It defaults to `gnutls-trustfiles'.
 CRLFILES is a list of CRL files.
 KEYLIST is an alist of (client key file, client cert file) pairs.
 MIN-PRIME-BITS is the minimum acceptable size of Diffie-Hellman keys
@@ -156,10 +169,12 @@
 It must be omitted, a number, or nil; if omitted or nil it
 defaults to GNUTLS_VERIFY_ALLOW_X509_V1_CA_CRT."
   (let* ((type (or type 'gnutls-x509pki))
-         (default-trustfile "/etc/ssl/certs/ca-certificates.crt")
          (trustfiles (or trustfiles
-                         (when (file-exists-p default-trustfile)
-                           (list default-trustfile))))
+                         (delq nil
+                               (mapcar (lambda (f) (and f (file-exists-p f) f))
+                                       (if (functionp gnutls-trustfiles)
+                                           (funcall gnutls-trustfiles)
+                                         gnutls-trustfiles)))))
          (priority-string (or priority-string
                               (cond
                                ((eq type 'gnutls-anon)

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

> OK.  Patch attached for your review.  The code is simpler now and the
> list flattening function is not needed.

Looks OK, please install.

> If approved I think I should also write a manual entry for this new
> variable.  Should I make a new manual subsection for GnuTLS-related
> things?  Where?

To the extent that the manual does not talk about TLS at all right now,
I don't think gnutls-trustfiles has a place yet.  But feel free to
update the documentation of open-network-stream.


        Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 16:04:46 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> OK.  Patch attached for your review.  The code is simpler now and the
>> list flattening function is not needed.

SM> Looks OK, please install.

Done, thank you.

>> If approved I think I should also write a manual entry for this new
>> variable.  Should I make a new manual subsection for GnuTLS-related
>> things?  Where?

SM> To the extent that the manual does not talk about TLS at all right now,
SM> I don't think gnutls-trustfiles has a place yet.  But feel free to
SM> update the documentation of open-network-stream.

I don't see how to update it appropriately.  I could add "Please see
`gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
random reference.  Maybe Lars has an opinion?

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I don't see how to update it appropriately.  I could add "Please see
> `gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
> random reference.  Maybe Lars has an opinion?

No opinion.  :-)  

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

SM> To the extent that the manual does not talk about TLS at all right now,
SM> I don't think gnutls-trustfiles has a place yet.  But feel free to
SM> update the documentation of open-network-stream.
> I don't see how to update it appropriately.  I could add "Please see
> `gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
> random reference.

I don't mean "update it with gnutls-trustfiles info", but "update it to
document the new "&rest PARAMS" keyword arguments.  At that point there
will be a place where you can document gnutls-trustfiles.


        Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[nyc4bos]

On Thu, 09 Feb 2012 09:16:16 -0500 Ted Zlatanov <address@hidden> wrote: 

> TZ> I'll start with the easiest ones (please correct me if any are wrong,
> TZ> based on http://mercurial.selenic.com/wiki/CACertificates):
> 
> TZ> Debian, Ubuntu, Gentoo and Arch Linux: /etc/ssl/certs/ca-certificates.crt 
> (maintained by `update-ca-certificates').
> 
> TZ> Fedora and RHEL: /etc/pki/tls/certs/ca-bundle.crt
> 
> TZ> Suse: /etc/ssl/ca-bundle.pem
> 
> Maintainers: can I change gnutls.el to provide a customizable
> `gnutls-trustfiles' and to probe these file locations or would you
> consider that a new feature that has to wait?


Cygwin: /usr/ssl/cert/ca-bundle.crt

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 17:20:22 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

SM> To the extent that the manual does not talk about TLS at all right now,
SM> I don't think gnutls-trustfiles has a place yet.  But feel free to
SM> update the documentation of open-network-stream.
>> I don't see how to update it appropriately.  I could add "Please see
>> `gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
>> random reference.

SM> I don't mean "update it with gnutls-trustfiles info", but "update it to
SM> document the new "&rest PARAMS" keyword arguments.  At that point there
SM> will be a place where you can document gnutls-trustfiles.

I'm confused.  The keyword arguments of `open-network-stream' are
already documented.  Do you mean I should add a new :trustfiles argument
and pass that down to `network-stream-open-starttls', and in the
documentation for that argument mention `gnutls-trustfiles'?

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Stefan Monnier]

SM> To the extent that the manual does not talk about TLS at all right now,
SM> I don't think gnutls-trustfiles has a place yet.  But feel free to
SM> update the documentation of open-network-stream.
>>> I don't see how to update it appropriately.  I could add "Please see
>>> `gnutls-trustfiles'" somewhere in the docstring but it would be a pretty
>>> random reference.
SM> I don't mean "update it with gnutls-trustfiles info", but "update it to
SM> document the new "&rest PARAMS" keyword arguments.  At that point there
SM> will be a place where you can document gnutls-trustfiles.
> I'm confused.  The keyword arguments of `open-network-stream' are
> already documented.

Where?  In doc/lispref/processes.texi I only see

   @defun open-network-stream name buffer-or-name host service


-- Stefan

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Glenn Morris]

Ted Zlatanov wrote:

> +                               ;; Fedora and RHEL
> +                               "/etc/pki/tls/certs/ca-bundle.crt"

FWIW, on RHEL6 I have both /etc/pki/tls/certs/ca-bundle.crt:

# This is a bundle of X.509 certificates of public Certificate
# Authorities.  It was generated from the Mozilla root CA list.


and /etc/pki/tls/certs/ca-bundle.trust.crt:

# This is a bundle of X.509 certificates of public Certificate
# Authorities.  It was generated from the Mozilla root CA list.
# These certificates are in the OpenSSL "TRUSTED CERTIFICATE"
# format and have trust bits set accordingly.


I have no idea which of those you want. The latter is slightly larger.

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 21:32:14 -0500 Glenn Morris <rgm@gnu.org> wrote: 

GM> Ted Zlatanov wrote:
>> +                               ;; Fedora and RHEL
>> +                               "/etc/pki/tls/certs/ca-bundle.crt"

GM> FWIW, on RHEL6 I have both /etc/pki/tls/certs/ca-bundle.crt:

GM> # This is a bundle of X.509 certificates of public Certificate
GM> # Authorities.  It was generated from the Mozilla root CA list.

GM> and /etc/pki/tls/certs/ca-bundle.trust.crt:

GM> # This is a bundle of X.509 certificates of public Certificate
GM> # Authorities.  It was generated from the Mozilla root CA list.
GM> # These certificates are in the OpenSSL "TRUSTED CERTIFICATE"
GM> # format and have trust bits set accordingly.

GM> I have no idea which of those you want. The latter is slightly larger.

Me neither, and I have no RHEL systems.

According to
http://rpmfind.net/linux/RPM/fedora/15/i386/ca-certificates-2011.70-2.fc15.noarch.html
both of these are in the ca-certificates Fedora package.  So I would
guess the differences are cosmetic and the files are equivalent.  But if
anyone knows different, please let us know.

Thanks
Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Mon, 13 Feb 2012 17:30:00 -0500 nyc4bos@aol.com wrote: 

n> Cygwin: /usr/ssl/cert/ca-bundle.crt

Added to `gnutls-trustfiles', thank you.

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Andy Moreton]

On Tue 14 Feb 2012, Ted Zlatanov wrote:

> On Mon, 13 Feb 2012 17:30:00 -0500 nyc4bos@aol.com wrote: 
>
> n> Cygwin: /usr/ssl/cert/ca-bundle.crt
>
> Added to `gnutls-trustfiles', thank you.
>
> Ted

For Cygwin 1.7.x that should be as below:


1) /usr/ssl/certs/ca-bundle.crt

# This is a bundle of X.509 certificates of public Certificate
# Authorities.  It was generated from the Mozilla root CA list.


2) /usr/ssl/certs/ca-bundle.trust.crt

# This is a bundle of X.509 certificates of public Certificate
# Authorities.  It was generated from the Mozilla root CA list.
# These certificates are in the OpenSSL "TRUSTED CERTIFICATE"
# format and have trust bits set accordingly.


The comments suggest that this pair of files have similar meaning to the
Redhat certs mentioned upthread.

   AndyM

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Tue, 14 Feb 2012 14:04:31 +0000 Andy Moreton <andrewjmoreton@gmail.com> wrote: 

AM> On Tue 14 Feb 2012, Ted Zlatanov wrote:
>> On Mon, 13 Feb 2012 17:30:00 -0500 nyc4bos@aol.com wrote: 
>> 
n> Cygwin: /usr/ssl/cert/ca-bundle.crt
>> 
>> Added to `gnutls-trustfiles', thank you.

AM> For Cygwin 1.7.x that should be as below:

AM> 1) /usr/ssl/certs/ca-bundle.crt

AM> # This is a bundle of X.509 certificates of public Certificate
AM> # Authorities.  It was generated from the Mozilla root CA list.

AM> 2) /usr/ssl/certs/ca-bundle.trust.crt

AM> # This is a bundle of X.509 certificates of public Certificate
AM> # Authorities.  It was generated from the Mozilla root CA list.
AM> # These certificates are in the OpenSSL "TRUSTED CERTIFICATE"
AM> # format and have trust bits set accordingly.

AM> The comments suggest that this pair of files have similar meaning to the
AM> Redhat certs mentioned upthread.

I am reluctant to add them both blindly, as with RHEL.  What's the
difference in Cygwin and in RHEL?

Ted

Re: need help with certificate bundles for ALL the platforms Emacs supports

[nyc4bos]

nyc4bos@aol.com writes:


> Cygwin: /usr/ssl/cert/ca-bundle.crt

Whoops, I had typo.

It should be "certs" with an "s":


/usr/ssl/certs/ca-bundle.crt

Re: need help with certificate bundles for ALL the platforms Emacs supports

[Ted Zlatanov]

On Thu, 16 Feb 2012 13:47:49 -0500 nyc4bos@aol.com wrote: 

n> nyc4bos@aol.com writes:
>> Cygwin: /usr/ssl/cert/ca-bundle.crt

n> Whoops, I had typo.

n> It should be "certs" with an "s":

n> /usr/ssl/certs/ca-bundle.crt

Fixed, thank you.

Ted