[PATCH] gnu: curl: Update to 7.41.0. Fix #20121.

[PATCH] gnu: curl: Update to 7.41.0. Fix #20121.

[Tomáš Čech]

* gnu/packages/patches/curl-gss-api-fix.patch: Delete file.
* gnu/packages/patches/curl-enable_capath-conf.patch: New file.
* gnu/packages/patches/curl-enable_capath.patch: New file.
* gnu-system.am (dist_patch_DATA): Add new patches, remove old one.
* gnu/packages/curl.scm (curl): Update to 7.41.0. Remove old patch, add two
  new ones. Disable one unit test.
---
 gnu-system.am                                      |   3 +-
 gnu/packages/curl.scm                              |  29 +++++-
 gnu/packages/patches/curl-enable_capath-conf.patch |  16 ++++
 gnu/packages/patches/curl-enable_capath.patch      | 103 +++++++++++++++++++++
 gnu/packages/patches/curl-gss-api-fix.patch        |  38 --------
 5 files changed, 145 insertions(+), 44 deletions(-)
 create mode 100644 gnu/packages/patches/curl-enable_capath-conf.patch
 create mode 100644 gnu/packages/patches/curl-enable_capath.patch
 delete mode 100644 gnu/packages/patches/curl-gss-api-fix.patch

diff --git a/gnu-system.am b/gnu-system.am
index 1cb0b11..b86ec09 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -391,8 +391,9 @@ dist_patch_DATA =						\
   gnu/packages/patches/cssc-gets-undeclared.patch               \
   gnu/packages/patches/cssc-missing-include.patch               \
   gnu/packages/patches/clucene-contribs-lib.patch               \
-  gnu/packages/patches/curl-gss-api-fix.patch			\
   gnu/packages/patches/cursynth-wave-rand.patch			\
+  gnu/packages/patches/curl-enable_capath.patch			\
+  gnu/packages/patches/curl-enable_capath-conf.patch		\
   gnu/packages/patches/dbus-localstatedir.patch			\
   gnu/packages/patches/diffutils-gets-undeclared.patch		\
   gnu/packages/patches/dfu-programmer-fix-libusb.patch		\
diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index 821a957..41b2675 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -37,15 +37,19 @@
 (define-public curl
   (package
    (name "curl")
-   (version "7.40.0")
+   (version "7.41.0")
    (source (origin
             (method url-fetch)
             (uri (string-append "http://curl.haxx.se/download/curl-"
                                 version ".tar.lzma"))
             (sha256
              (base32
-              "1a15fdc26b3vwwmchzzpd3l1hfyhx06dn7b6lkikqd7kgwvg5ps7"))
-            (patches (list (search-patch "curl-gss-api-fix.patch")))))
+              "08n7vrhdfzziy3a7n93r7qjhzk8p26q464hxg8w9irdk3v60pi62"))
+            ;; This is backport of patch allowing --with-ca-path for curl built
+            ;; against GnuTLS. First part is identical to upstream, second
+            ;; patches also configure without need of reconfigure. Fixes #20121.
+            (patches (list (search-patch "curl-enable_capath.patch")
+                           (search-patch "curl-enable_capath-conf.patch")))))
    (build-system gnu-build-system)
    (inputs `(("gnutls" ,gnutls)
              ("gss" ,gss)
@@ -60,7 +64,18 @@
        ("pkg-config" ,pkg-config)
        ("python" ,python-2)))
    (arguments
-    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
+    `(#:configure-flags
+      '("--with-gnutls" "--with-gssapi" "--enable-http"
+        ;; This option requires parameter so let's use this one
+        ;; - when it exists, it makes sense
+        ;; - when it doesn't exist, we can still use SSL_CERT_DIR environment
+        ;;   variable to set it per user or for whole system
+        ;; Fixes #20121.
+        "--with-ca-path=/etc/ssl/certs"
+        "--enable-ftp" "--enable-proxy" "--with-libidn"
+        "--enable-cookies" "CFLAGS=-ggdb")
+      #:make-flags '("CFLAGS=-ggdb")
+      #:strip-binaries? #f
       ;; Add a phase to patch '/bin/sh' occurances in tests/runtests.pl
       #:phases
       (alist-replace
@@ -68,12 +83,16 @@
        (lambda _
          (substitute* "tests/runtests.pl"
            (("/bin/sh") (which "sh")))
-
+         ;; Test #1135 requires extern-scan.pl, which is not part of the
+         ;; tarball due to mistake. It was fixed already in upstream. We can
+         ;; simply ignore the test as it aims VMS and OS/400.
+         (delete-file "tests/data/test1135")
          ;; The top-level "make check" does "make -C tests quiet-test", which
          ;; is too quiet.  Use the "test" target instead, which is more
          ;; verbose.
          (zero? (system* "make" "-C" "tests" "test")))
        %standard-phases)))
+   (outputs '("out" "debug"))   
    (synopsis "Command line tool for transferring data with URL syntax")
    (description
     "curl is a command line tool for transferring data with URL syntax,
diff --git a/gnu/packages/patches/curl-enable_capath-conf.patch b/gnu/packages/patches/curl-enable_capath-conf.patch
new file mode 100644
index 0000000..6d4ba8e
--- /dev/null
+++ b/gnu/packages/patches/curl-enable_capath-conf.patch
@@ -0,0 +1,16 @@
+Following patch allows --with-ca-path for curl built against GnuTLS even
+without need of reconfigure.
+
+--- a/configure       2015-03-22 01:11:23.178743705 +0100
++++ b/configure       2015-02-25 00:05:37.000000000 +0100
+@@ -23952,8 +24432,8 @@
+         ca="$want_ca"
+     capath="no"
+   elif test "x$want_capath" != "xno" -a "x$want_capath" != "xunset"; then
+-        if test "x$OPENSSL_ENABLED" != "x1" -a "x$POLARSSL_ENABLED" != "x1"; then
+-      as_fn_error $? "--with-ca-path only works with openSSL or PolarSSL" "$LINENO" 5
++        if test "x$OPENSSL_ENABLED" != "x1" -a "x$GNUTLS_ENABLED" != "x1" -a "x$POLARSSL_ENABLED" != "x1"; then
++      as_fn_error $? "--with-ca-path only works with OpenSSL, GnuTLS or PolarSSL" "$LINENO" 5
+     fi
+     capath="$want_capath"
+     ca="no"
diff --git a/gnu/packages/patches/curl-enable_capath.patch b/gnu/packages/patches/curl-enable_capath.patch
new file mode 100644
index 0000000..0094a1b
--- /dev/null
+++ b/gnu/packages/patches/curl-enable_capath.patch
@@ -0,0 +1,103 @@
+Following patch allows to use --with-ca-path for curl built against GnuTLS.
+
+
+From 5a1614cecdd57cab8b4ae3e9bc19dfff5ba77e80 Mon Sep 17 00:00:00 2001
+From: Alessandro Ghedini <alessandro@ghedini.me>
+Date: Sun, 8 Mar 2015 20:11:06 +0100
+Subject: [PATCH] gtls: add support for CURLOPT_CAPATH
+
+---
+ acinclude.m4                       |  4 ++--
+ docs/libcurl/opts/CURLOPT_CAPATH.3 |  5 ++---
+ lib/vtls/gtls.c                    | 22 ++++++++++++++++++++++
+ lib/vtls/gtls.h                    |  3 +++
+ 4 files changed, 29 insertions(+), 5 deletions(-)
+
+diff --git a/acinclude.m4 b/acinclude.m4
+index 6ed7ffb..ca01869 100644
+--- a/acinclude.m4
++++ b/acinclude.m4
+@@ -2615,8 +2615,8 @@ AC_HELP_STRING([--without-ca-path], [Don't use a default CA path]),
+     capath="no"
+   elif test "x$want_capath" != "xno" -a "x$want_capath" != "xunset"; then
+     dnl --with-ca-path given
+-    if test "x$OPENSSL_ENABLED" != "x1" -a "x$POLARSSL_ENABLED" != "x1"; then
+-      AC_MSG_ERROR([--with-ca-path only works with openSSL or PolarSSL])
++    if test "x$OPENSSL_ENABLED" != "x1" -a "x$GNUTLS_ENABLED" != "x1" -a "x$POLARSSL_ENABLED" != "x1"; then
++      AC_MSG_ERROR([--with-ca-path only works with OpenSSL, GnuTLS or PolarSSL])
+     fi
+     capath="$want_capath"
+     ca="no"
+diff --git a/docs/libcurl/opts/CURLOPT_CAPATH.3 b/docs/libcurl/opts/CURLOPT_CAPATH.3
+index 642953d..6695f9f 100644
+--- a/docs/libcurl/opts/CURLOPT_CAPATH.3
++++ b/docs/libcurl/opts/CURLOPT_CAPATH.3
+@@ -43,9 +43,8 @@ All TLS based protocols: HTTPS, FTPS, IMAPS, POP3, SMTPS etc.
+ .SH EXAMPLE
+ TODO
+ .SH AVAILABILITY
+-This option is OpenSSL-specific and does nothing if libcurl is built to use
+-GnuTLS. NSS-powered libcurl provides the option only for backward
+-compatibility.
++This option is supported by the OpenSSL, GnuTLS and PolarSSL backends. The NSS
++backend provides the option only for backward compatibility.
+ .SH RETURN VALUE
+ Returns CURLE_OK if TLS enabled, and CURLE_UNKNOWN_OPTION if not, or
+ CURLE_OUT_OF_MEMORY if there was insufficient heap space.
+diff --git a/lib/vtls/gtls.c b/lib/vtls/gtls.c
+index 05aef19..c792540 100644
+--- a/lib/vtls/gtls.c
++++ b/lib/vtls/gtls.c
+@@ -97,6 +97,10 @@ static bool gtls_inited = FALSE;
+ #  if (GNUTLS_VERSION_NUMBER >= 0x03020d)
+ #    define HAS_OCSP
+ #  endif
++
++#  if (GNUTLS_VERSION_NUMBER >= 0x030306)
++#    define HAS_CAPATH
++#  endif
+ #endif
+ 
+ #ifdef HAS_OCSP
+@@ -462,6 +466,24 @@ gtls_connect_step1(struct connectdata *conn,
+             rc, data->set.ssl.CAfile);
+   }
+ 
++#ifdef HAS_CAPATH
++  if(data->set.ssl.CApath) {
++    /* set the trusted CA cert directory */
++    rc = gnutls_certificate_set_x509_trust_dir(conn->ssl[sockindex].cred,
++                                                data->set.ssl.CApath,
++                                                GNUTLS_X509_FMT_PEM);
++    if(rc < 0) {
++      infof(data, "error reading ca cert file %s (%s)\n",
++            data->set.ssl.CAfile, gnutls_strerror(rc));
++      if(data->set.ssl.verifypeer)
++        return CURLE_SSL_CACERT_BADFILE;
++    }
++    else
++      infof(data, "found %d certificates in %s\n",
++            rc, data->set.ssl.CApath);
++  }
++#endif
++
+   if(data->set.ssl.CRLfile) {
+     /* set the CRL list file */
+     rc = gnutls_certificate_set_x509_crl_file(conn->ssl[sockindex].cred,
+diff --git a/lib/vtls/gtls.h b/lib/vtls/gtls.h
+index c3867e5..af1cb5b 100644
+--- a/lib/vtls/gtls.h
++++ b/lib/vtls/gtls.h
+@@ -54,6 +54,9 @@ bool Curl_gtls_cert_status_request(void);
+ /* Set the API backend definition to GnuTLS */
+ #define CURL_SSL_BACKEND CURLSSLBACKEND_GNUTLS
+ 
++/* this backend supports the CAPATH option */
++#define have_curlssl_ca_path 1
++
+ /* API setup for GnuTLS */
+ #define curlssl_init Curl_gtls_init
+ #define curlssl_cleanup Curl_gtls_cleanup
+-- 
+2.2.1
+
diff --git a/gnu/packages/patches/curl-gss-api-fix.patch b/gnu/packages/patches/curl-gss-api-fix.patch
deleted file mode 100644
index ea838ae..0000000
--- a/gnu/packages/patches/curl-gss-api-fix.patch
+++ /dev/null
@@ -1,38 +0,0 @@
-Copied from upstream:
-https://github.com/bagder/curl/commit/5c0e66d63214e0306197c5a3f162441e074f3401.patch
-
-From 5c0e66d63214e0306197c5a3f162441e074f3401 Mon Sep 17 00:00:00 2001
-From: Steve Holme <steve_holme@hotmail.com>
-Date: Thu, 8 Jan 2015 19:23:53 +0000
-Subject: [PATCH] sasl_gssapi: Fixed build on NetBSD with built-in GSS-API
-
-Bug: http://curl.haxx.se/bug/view.cgi?id=1469
-Reported-by: Thomas Klausner
----
- lib/curl_sasl_gssapi.c | 3 ++-
- 1 file changed, 2 insertions(+), 1 deletion(-)
-
-diff --git a/lib/curl_sasl_gssapi.c b/lib/curl_sasl_gssapi.c
-index 6dda0e9..a50646a 100644
---- a/lib/curl_sasl_gssapi.c
-+++ b/lib/curl_sasl_gssapi.c
-@@ -6,6 +6,7 @@
-  *                             \___|\___/|_| \_\_____|
-  *
-  * Copyright (C) 2014, Steve Holme, <steve_holme@hotmail.com>.
-+ * Copyright (C) 2015, Daniel Stenberg, <daniel@haxx.se>, et al.
-  *
-  * This software is licensed as described in the file COPYING, which
-  * you should have received as part of this distribution. The terms
-@@ -126,7 +127,7 @@ CURLcode Curl_sasl_create_gssapi_user_message(struct SessionHandle *data,
- 
-     /* Import the SPN */
-     gss_major_status = gss_import_name(&gss_minor_status, &spn_token,
--                                       gss_nt_service_name, &krb5->spn);
-+                                       GSS_C_NT_HOSTBASED_SERVICE, &krb5->spn);
-     if(GSS_ERROR(gss_major_status)) {
-       Curl_gss_log_error(data, gss_minor_status, "gss_import_name() failed: ");
- 
--- 
-2.2.1
-
-- 
2.2.1

Re: [PATCH] gnu: curl: Update to 7.41.0. Fix #20121.

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 1677 bytes --]

On Sun, Mar 22, 2015 at 11:20:36AM +0100, Tomáš Čech wrote:
> ...
>    (arguments
>-    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
>+    `(#:configure-flags
>+      '("--with-gnutls" "--with-gssapi" "--enable-http"
>+        ;; This option requires parameter so let's use this one
>+        ;; - when it exists, it makes sense
>+        ;; - when it doesn't exist, we can still use SSL_CERT_DIR environment
>+        ;;   variable to set it per user or for whole system
>+        ;; Fixes #20121.
>+        "--with-ca-path=/etc/ssl/certs"
>+        "--enable-ftp" "--enable-proxy" "--with-libidn"
>+        "--enable-cookies" "CFLAGS=-ggdb")
>+      #:make-flags '("CFLAGS=-ggdb")

Damn, I forgot my debugging code pieces here...
>+      #:strip-binaries? #f
>       ;; Add a phase to patch '/bin/sh' occurances in tests/runtests.pl
>       #:phases
>       (alist-replace
>@@ -68,12 +83,16 @@
>        (lambda _
>          (substitute* "tests/runtests.pl"
>            (("/bin/sh") (which "sh")))
>-
>+         ;; Test #1135 requires extern-scan.pl, which is not part of the
>+         ;; tarball due to mistake. It was fixed already in upstream. We can
>+         ;; simply ignore the test as it aims VMS and OS/400.
>+         (delete-file "tests/data/test1135")
>          ;; The top-level "make check" does "make -C tests quiet-test", which
>          ;; is too quiet.  Use the "test" target instead, which is more
>          ;; verbose.
>          (zero? (system* "make" "-C" "tests" "test")))
>        %standard-phases)))
>+   (outputs '("out" "debug"))

...and here.


I removed it from the patch already.

S_W

[-- Attachment #2: Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH] gnu: curl: Update to 7.41.0. Fix #20121.

[Mark H Weaver]

Tomáš Čech <sleep_walker@gnu.org> writes:

> @@ -60,7 +64,18 @@
>         ("pkg-config" ,pkg-config)
>         ("python" ,python-2)))
>     (arguments
> -    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
> +    `(#:configure-flags
> +      '("--with-gnutls" "--with-gssapi" "--enable-http"
> +        ;; This option requires parameter so let's use this one
> +        ;; - when it exists, it makes sense
> +        ;; - when it doesn't exist, we can still use SSL_CERT_DIR environment
> +        ;;   variable to set it per user or for whole system
> +        ;; Fixes #20121.
> +        "--with-ca-path=/etc/ssl/certs"

It would be better to avoid passing the --with-ca-path= option.  We are
attempting to move away from having any compiled-in system-wide location
for the CA trust store.  Each user should be able to specify their
desired trust store using environment variables or other settings.

We currently pass a similar configure-time option to GnuTLS for now, but
only because GnuTLS provides no other way to specify the location of an
old-style system trust store.  We should be able to remove this when we
have the more modern p11-kit system working.

Can you find a way to make 'curl' work without compiling-in a hard-coded
system trust store location?

      Mark

Re: [PATCH] gnu: curl: Update to 7.41.0. Fix #20121.

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

Thanks for review!

On Sun, Mar 22, 2015 at 12:56:43PM -0400, Mark H Weaver wrote:
>Tomáš Čech <sleep_walker@gnu.org> writes:
>
>> @@ -60,7 +64,18 @@
>>         ("pkg-config" ,pkg-config)
>>         ("python" ,python-2)))
>>     (arguments
>> -    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
>> +    `(#:configure-flags
>> +      '("--with-gnutls" "--with-gssapi" "--enable-http"
>> +        ;; This option requires parameter so let's use this one
>> +        ;; - when it exists, it makes sense
>> +        ;; - when it doesn't exist, we can still use SSL_CERT_DIR environment
>> +        ;;   variable to set it per user or for whole system
>> +        ;; Fixes #20121.
>> +        "--with-ca-path=/etc/ssl/certs"
>
>It would be better to avoid passing the --with-ca-path= option.  We are
>attempting to move away from having any compiled-in system-wide location
>for the CA trust store.  Each user should be able to specify their
>desired trust store using environment variables or other settings.

I verified that patch is now sufficient as solution and I verified
that it respects SSL_CERT_DIR with that so it is ideal solution.

Thanks for kicking me step further. (And yes, that patch is really needed :)

I'll update the patch accordingly.

S_W

[-- Attachment #2: Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH] gnu: curl: Update to 7.41.0. Fix #20121.

[Mark H Weaver]

Tomáš Čech <sleep_walker@gnu.org> writes:

> On Sun, Mar 22, 2015 at 12:56:43PM -0400, Mark H Weaver wrote:
>>
>>It would be better to avoid passing the --with-ca-path= option.  We are
>>attempting to move away from having any compiled-in system-wide location
>>for the CA trust store.  Each user should be able to specify their
>>desired trust store using environment variables or other settings.
>
> I verified that patch is now sufficient as solution and I verified
> that it respects SSL_CERT_DIR with that so it is ideal solution.

Sounds perfect!

> Thanks for kicking me step further. (And yes, that patch is really
> needed :)

Thanks very much for your role in getting that patch produced and
upstreamed.  It sounds like this will allow our 'git' to consult
SSL_CERT_DIR when checking https certificates, which was not possible
before when libcurl was linked with GnuTLS.  Instead, we had to create a
legacy single-file trust store and set SSL_CERT_FILE.  There may still
be other programs that require the single-file trust store, but I'm glad
that libcurl-based programs can now be crossed off that list :)

      Mark

Re: [PATCH] gnu: curl: Update to 7.41.0. Fix #20121.

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

On Sun, Mar 22, 2015 at 02:16:00PM -0400, Mark H Weaver wrote:
>Tomáš Čech <sleep_walker@gnu.org> writes:
>
>> On Sun, Mar 22, 2015 at 12:56:43PM -0400, Mark H Weaver wrote:
>>>
>>>It would be better to avoid passing the --with-ca-path= option.  We are
>>>attempting to move away from having any compiled-in system-wide location
>>>for the CA trust store.  Each user should be able to specify their
>>>desired trust store using environment variables or other settings.
>>
>> I verified that patch is now sufficient as solution and I verified
>> that it respects SSL_CERT_DIR with that so it is ideal solution.
>
>Sounds perfect!
>
>> Thanks for kicking me step further. (And yes, that patch is really
>> needed :)
>
>Thanks very much for your role in getting that patch produced and
>upstreamed.

FTR, I just did some debugging and in the end bisected the right patch
from upstream GIT, author of the patch is someone else.

S_W


[-- Attachment #2: Type: application/pgp-signature, Size: 181 bytes --]