PATCH as first attempt to fix the sad curl situation

PATCH as first attempt to fix the sad curl situation

[ng0]

[PATCH] gnu: curl: Add ca-bundle to config.

I had to disable test 324, but the rest passed. I hope this fixes the cURL problem, though I haven't tested reconfiguring a system with it as it would require rebasing some of my older, still waiting final aproval, patches.

[PATCH] gnu: curl: Add ca-bundle to config.

[ng0]

* gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
[arguments]: Disable failing test number 324.
---
 gnu/packages/curl.scm | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index 7329d870d..3473055b8 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2015 Tomáš Čech <sleep_walker@suse.cz>
 ;;; Copyright © 2015 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
+;;; Copyright © 2017 ng0 <ng0@libertad.pw>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -65,7 +66,8 @@
        ("pkg-config" ,pkg-config)
        ("python" ,python-2)))
    (arguments
-    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
+    `(#:configure-flags '("--with-gnutls" "--with-gssapi"
+                          "--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt")
       ;; Add a phase to patch '/bin/sh' occurances in tests/runtests.pl
       #:phases
       (modify-phases %standard-phases
@@ -104,6 +106,15 @@
            ;;  * Connection #3 to host server4.example.com left intact[LF]
            (delete-file "tests/data/test1510")
 
+           ;; XXX: FIXME: Test #0324 requires access to the
+           ;; /etc/ssl/certs/ca-certificates.crt, which can't happen for the builder.
+           ;; It fails with the following error:
+           ;; test 0324...[TLS-SRP with server cert checking]
+           ;; curl returned 77, when expecting 51
+           ;; exit FAILED
+           ;; - abort tests
+           (delete-file "tests/data/test324")
+
            ;; The top-level "make check" does "make -C tests quiet-test", which
            ;; is too quiet.  Use the "test" target instead, which is more
            ;; verbose.
-- 
2.11.0

Re: PATCH as first attempt to fix the sad curl situation

[ng0]

ng0 <ng0@libertad.pw> writes:

> [PATCH] gnu: curl: Add ca-bundle to config.
>
> I had to disable test 324, but the rest passed. I hope this fixes the cURL problem, though I haven't tested reconfiguring a system with it as it would require rebasing some of my older, still waiting final aproval, patches.
>
>

ng0@wasp /gnu/store/09ncbabac6vniawypgwqxshx6dyjmph1-curl-7.50.3/bin$ ./curl-config --configure
 'CONFIG_SHELL=/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/bash' 'SHELL=/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/bash' '--prefix=/gnu/store/09ncbabac6vniawypgwqxshx6dyjmph1-curl-7.50.3' '--enable-fast-install' '--docdir=/gnu/store/fq85pwcn5kvy1f5a82n259pj9wv00ix7-curl-7.50.3-doc/share/doc/curl' '--build=x86_64-unknown-linux-gnu' '--with-gnutls' '--with-gssapi' '--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt' 'build_alias=x86_64-unknown-linux-gnu'


-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: [PATCH] gnu: curl: Add ca-bundle to config.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1311 bytes --]

ng0 <ng0@libertad.pw> writes:

> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
> [arguments]: Disable failing test number 324.
> ---
>  gnu/packages/curl.scm | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
> index 7329d870d..3473055b8 100644
> --- a/gnu/packages/curl.scm
> +++ b/gnu/packages/curl.scm
> @@ -4,6 +4,7 @@
>  ;;; Copyright © 2015 Tomáš Čech <sleep_walker@suse.cz>
>  ;;; Copyright © 2015 Ludovic Courtès <ludo@gnu.org>
>  ;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
> +;;; Copyright © 2017 ng0 <ng0@libertad.pw>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -65,7 +66,8 @@
>         ("pkg-config" ,pkg-config)
>         ("python" ,python-2)))
>     (arguments
> -    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
> +    `(#:configure-flags '("--with-gnutls" "--with-gssapi"
> +                          "--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt")

This may not work on all distros, and is "impure" since this path is not
managed by Guix. If we are doing this, it should be referring to
(string-append (assoc-ref %build-inputs "nss-certs") "/etc/ssl/...").
That will likely fix the test as well.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH] gnu: curl: Add ca-bundle to config.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1784 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> ng0 <ng0@libertad.pw> writes:
>
>> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
>> [arguments]: Disable failing test number 324.
>> ---
>>  gnu/packages/curl.scm | 13 ++++++++++++-
>>  1 file changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
>> index 7329d870d..3473055b8 100644
>> --- a/gnu/packages/curl.scm
>> +++ b/gnu/packages/curl.scm
>> @@ -4,6 +4,7 @@
>>  ;;; Copyright © 2015 Tomáš Čech <sleep_walker@suse.cz>
>>  ;;; Copyright © 2015 Ludovic Courtès <ludo@gnu.org>
>>  ;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
>> +;;; Copyright © 2017 ng0 <ng0@libertad.pw>
>>  ;;;
>>  ;;; This file is part of GNU Guix.
>>  ;;;
>> @@ -65,7 +66,8 @@
>>         ("pkg-config" ,pkg-config)
>>         ("python" ,python-2)))
>>     (arguments
>> -    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
>> +    `(#:configure-flags '("--with-gnutls" "--with-gssapi"
>> +                          "--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt")
>
> This may not work on all distros, and is "impure" since this path is not
> managed by Guix. If we are doing this, it should be referring to
> (string-append (assoc-ref %build-inputs "nss-certs") "/etc/ssl/...").
> That will likely fix the test as well.

I realized shortly after posting why this wasn't done already. Curl has
1403 dependent packages, which would apply for "nss-certs" as well if
that is added as input. Obviously we want to be able to update TLS
certificates quickly without rebuilding ~1/4 of the tree.

Perhaps it could be added as a separate package, or by e.g. renaming the
current curl package to "curl-minimal".

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH] gnu: curl: Add ca-bundle to config.

[ng0]

Marius Bakke <mbakke@fastmail.com> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> ng0 <ng0@libertad.pw> writes:
>>
>>> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
>>> [arguments]: Disable failing test number 324.
>>> ---
>>>  gnu/packages/curl.scm | 13 ++++++++++++-
>>>  1 file changed, 12 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
>>> index 7329d870d..3473055b8 100644
>>> --- a/gnu/packages/curl.scm
>>> +++ b/gnu/packages/curl.scm
>>> @@ -4,6 +4,7 @@
>>>  ;;; Copyright © 2015 Tomáš Čech <sleep_walker@suse.cz>
>>>  ;;; Copyright © 2015 Ludovic Courtès <ludo@gnu.org>
>>>  ;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
>>> +;;; Copyright © 2017 ng0 <ng0@libertad.pw>
>>>  ;;;
>>>  ;;; This file is part of GNU Guix.
>>>  ;;;
>>> @@ -65,7 +66,8 @@
>>>         ("pkg-config" ,pkg-config)
>>>         ("python" ,python-2)))
>>>     (arguments
>>> -    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
>>> +    `(#:configure-flags '("--with-gnutls" "--with-gssapi"
>>> +                          "--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt")
>>
>> This may not work on all distros, and is "impure" since this path is not
>> managed by Guix. If we are doing this, it should be referring to
>> (string-append (assoc-ref %build-inputs "nss-certs") "/etc/ssl/...").
>> That will likely fix the test as well.
>
> I realized shortly after posting why this wasn't done already. Curl has
> 1403 dependent packages, which would apply for "nss-certs" as well if
> that is added as input. Obviously we want to be able to update TLS
> certificates quickly without rebuilding ~1/4 of the tree.
>
> Perhaps it could be added as a separate package, or by e.g. renaming the
> current curl package to "curl-minimal".

Okay, I will wait for more people to comment on the best route to
follow here for curl.

For gnurl:
As right now gnurl is only required by gnunet and related
packages, I will send in a patch for this in advance for gnurl.
-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: [PATCH] gnu: curl: Add ca-bundle to config.

[ng0]

Marius Bakke <mbakke@fastmail.com> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> ng0 <ng0@libertad.pw> writes:
>>
>>> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
>>> [arguments]: Disable failing test number 324.
>>> ---
>>>  gnu/packages/curl.scm | 13 ++++++++++++-
>>>  1 file changed, 12 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
>>> index 7329d870d..3473055b8 100644
>>> --- a/gnu/packages/curl.scm
>>> +++ b/gnu/packages/curl.scm
>>> @@ -4,6 +4,7 @@
>>>  ;;; Copyright © 2015 Tomáš Čech <sleep_walker@suse.cz>
>>>  ;;; Copyright © 2015 Ludovic Courtès <ludo@gnu.org>
>>>  ;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
>>> +;;; Copyright © 2017 ng0 <ng0@libertad.pw>
>>>  ;;;
>>>  ;;; This file is part of GNU Guix.
>>>  ;;;
>>> @@ -65,7 +66,8 @@
>>>         ("pkg-config" ,pkg-config)
>>>         ("python" ,python-2)))
>>>     (arguments
>>> -    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
>>> +    `(#:configure-flags '("--with-gnutls" "--with-gssapi"
>>> +                          "--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt")
>>
>> This may not work on all distros, and is "impure" since this path is not
>> managed by Guix. If we are doing this, it should be referring to
>> (string-append (assoc-ref %build-inputs "nss-certs") "/etc/ssl/...").
>> That will likely fix the test as well.
>
> I realized shortly after posting why this wasn't done already. Curl has
> 1403 dependent packages, which would apply for "nss-certs" as well if
> that is added as input. Obviously we want to be able to update TLS
> certificates quickly without rebuilding ~1/4 of the tree.
>
> Perhaps it could be added as a separate package, or by e.g. renaming the
> current curl package to "curl-minimal".

Appending to my last message:
how can I make this valid:

   (arguments
    `(#:configure-flags '("--enable-ipv6" "--with-gnutls" "--without-libssh2"
                          "--without-libmetalink" "--without-winidn"
                          "--without-librtmp" "--without-nghttp2"
                          "--without-nss" "--without-cyassl"
                          "--without-polarssl" "--without-ssl"
                          "--without-winssl" "--without-darwinssl"
                          "--disable-sspi" "--disable-ntlm-wb"
                          "--disable-ldap" "--disable-rtsp" "--disable-dict"
                          "--disable-telnet" "--disable-tftp" "--disable-pop3"
                          "--disable-imap" "--disable-smtp" "--disable-gopher"
                          "--disable-file" "--disable-ftp" "--disable-smb"
                          (string-append
                           "--with-ca-bundle="
                           (string-append (assoc-ref %build-inputs "nss-certs")
                                          "/etc/ssl/certs/ca-certificates.crt")))

The string-append is not valid here.
-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: [PATCH] gnu: curl: Add ca-bundle to config.

[ng0]

ng0 <ng0@libertad.pw> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> Marius Bakke <mbakke@fastmail.com> writes:
>>
>>> ng0 <ng0@libertad.pw> writes:
>>>
>>>> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
>>>> [arguments]: Disable failing test number 324.
>>>> ---
>>>>  gnu/packages/curl.scm | 13 ++++++++++++-
>>>>  1 file changed, 12 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
>>>> index 7329d870d..3473055b8 100644
>>>> --- a/gnu/packages/curl.scm
>>>> +++ b/gnu/packages/curl.scm
>>>> @@ -4,6 +4,7 @@
>>>>  ;;; Copyright © 2015 Tomáš Čech <sleep_walker@suse.cz>
>>>>  ;;; Copyright © 2015 Ludovic Courtès <ludo@gnu.org>
>>>>  ;;; Copyright © 2016 Leo Famulari <leo@famulari.name>
>>>> +;;; Copyright © 2017 ng0 <ng0@libertad.pw>
>>>>  ;;;
>>>>  ;;; This file is part of GNU Guix.
>>>>  ;;;
>>>> @@ -65,7 +66,8 @@
>>>>         ("pkg-config" ,pkg-config)
>>>>         ("python" ,python-2)))
>>>>     (arguments
>>>> -    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
>>>> +    `(#:configure-flags '("--with-gnutls" "--with-gssapi"
>>>> +                          "--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt")
>>>
>>> This may not work on all distros, and is "impure" since this path is not
>>> managed by Guix. If we are doing this, it should be referring to
>>> (string-append (assoc-ref %build-inputs "nss-certs") "/etc/ssl/...").
>>> That will likely fix the test as well.
>>
>> I realized shortly after posting why this wasn't done already. Curl has
>> 1403 dependent packages, which would apply for "nss-certs" as well if
>> that is added as input. Obviously we want to be able to update TLS
>> certificates quickly without rebuilding ~1/4 of the tree.
>>
>> Perhaps it could be added as a separate package, or by e.g. renaming the
>> current curl package to "curl-minimal".
>
> Appending to my last message:
> how can I make this valid:
>
>    (arguments
>     `(#:configure-flags '("--enable-ipv6" "--with-gnutls" "--without-libssh2"
>                           "--without-libmetalink" "--without-winidn"
>                           "--without-librtmp" "--without-nghttp2"
>                           "--without-nss" "--without-cyassl"
>                           "--without-polarssl" "--without-ssl"
>                           "--without-winssl" "--without-darwinssl"
>                           "--disable-sspi" "--disable-ntlm-wb"
>                           "--disable-ldap" "--disable-rtsp" "--disable-dict"
>                           "--disable-telnet" "--disable-tftp" "--disable-pop3"
>                           "--disable-imap" "--disable-smtp" "--disable-gopher"
>                           "--disable-file" "--disable-ftp" "--disable-smb"
>                           (string-append
>                            "--with-ca-bundle="
>                            (string-append (assoc-ref %build-inputs "nss-certs")
>                                           "/etc/ssl/certs/ca-certificates.crt")))
>
> The string-append is not valid here.

Solved, by using "(list" here.

-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: PATCH as first attempt to fix the sad curl situation

[Ricardo Wurmus]

ng0 <ng0@libertad.pw> writes:

> ng0 <ng0@libertad.pw> writes:
>
>> [PATCH] gnu: curl: Add ca-bundle to config.
>>
>> I had to disable test 324, but the rest passed. I hope this fixes the cURL problem, though I haven't tested reconfiguring a system with it as it would require rebasing some of my older, still waiting final aproval, patches.
>>
>>
>
> ng0@wasp /gnu/store/09ncbabac6vniawypgwqxshx6dyjmph1-curl-7.50.3/bin$ ./curl-config --configure
>  'CONFIG_SHELL=/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/bash' 'SHELL=/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/bash' '--prefix=/gnu/store/09ncbabac6vniawypgwqxshx6dyjmph1-curl-7.50.3' '--enable-fast-install' '--docdir=/gnu/store/fq85pwcn5kvy1f5a82n259pj9wv00ix7-curl-7.50.3-doc/share/doc/curl' '--build=x86_64-unknown-linux-gnu' '--with-gnutls' '--with-gssapi' '--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt' 'build_alias=x86_64-unknown-linux-gnu'

What does this mean?

(Please consider the amount of emails we have to go through before
sending mail to the list.)

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
http://elephly.net

GnuTLS and the “trust store”

[Ludovic Courtès]

Hello!

Marius Bakke <mbakke@fastmail.com> skribis:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> ng0 <ng0@libertad.pw> writes:
>>
>>> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.

[...]

> I realized shortly after posting why this wasn't done already. Curl has
> 1403 dependent packages, which would apply for "nss-certs" as well if
> that is added as input. Obviously we want to be able to update TLS
> certificates quickly without rebuilding ~1/4 of the tree.

Indeed.  It’s a situation where we do not want to have a static binding
between cURL and nss-certs; instead, they should be composed
dynamically, along the lines of what we already recommend at:

  https://www.gnu.org/software/guix/manual/html_node/X_002e509-Certificates.html

cURL depends on GnuTLS, and GnuTLS doesn’t honor an environment variable
like ‘SSL_CERT_DIR’.  Its recipe has this comment:

         ;; GnuTLS doesn't consult any environment variables to specify
         ;; the location of the system-wide trust store.  Instead it has a
         ;; configure-time option.  Unless specified, its configure script
         ;; attempts to auto-detect the location by looking for common
         ;; places in the file system, none of which are present in our
         ;; chroot build environment.  If not found, then no default trust
         ;; store is used, so each program has to provide its own
         ;; fallback, and users have to configure each program
         ;; independently.  This seems suboptimal.
         "--with-default-trust-store-dir=/etc/ssl/certs"

Original discussion:

  https://lists.gnu.org/archive/html/guix-devel/2014-02/msg00245.html

Ludo’.

Re: GnuTLS and the “trust store”

[ng0]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Marius Bakke <mbakke@fastmail.com> skribis:
>
>> Marius Bakke <mbakke@fastmail.com> writes:
>>
>>> ng0 <ng0@libertad.pw> writes:
>>>
>>>> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
>
> [...]
>
>> I realized shortly after posting why this wasn't done already. Curl has
>> 1403 dependent packages, which would apply for "nss-certs" as well if
>> that is added as input. Obviously we want to be able to update TLS
>> certificates quickly without rebuilding ~1/4 of the tree.
>
> Indeed.  It’s a situation where we do not want to have a static binding
> between cURL and nss-certs; instead, they should be composed
> dynamically, along the lines of what we already recommend at:

Okay, so my proposed gnURL patch should not be applied at
all. Reading the old threads I'm starting to understand the
situation, but not completely.

>   https://www.gnu.org/software/guix/manual/html_node/X_002e509-Certificates.html
>
> cURL depends on GnuTLS, and GnuTLS doesn’t honor an environment variable
> like ‘SSL_CERT_DIR’.  Its recipe has this comment:

The 3rd option in 2015, subject: [PATCH] gnu: gnutls: Configure
location of system-wide trust store, was to use openssl. Now
we have libressl, so why not try and give that a try in the
future when we (that is, the people with commit access) have
rebuild everything with libressl and it turns out alright?

I'm trying to understand the problem here, the problem why
packages like darcs, pbpst, and others are just sitting, waiting
for months because of issues with cURL. There's a problem, and
I'd like to fix (and understand) it.
Do I have to fix the curl dependent applications? Doesn't sound
like a solution for me which would scale.

>          ;; GnuTLS doesn't consult any environment variables to specify
>          ;; the location of the system-wide trust store.  Instead it has a
>          ;; configure-time option.  Unless specified, its configure script
>          ;; attempts to auto-detect the location by looking for common
>          ;; places in the file system, none of which are present in our
>          ;; chroot build environment.  If not found, then no default trust
>          ;; store is used, so each program has to provide its own
>          ;; fallback, and users have to configure each program
>          ;; independently.  This seems suboptimal.
>          "--with-default-trust-store-dir=/etc/ssl/certs"
>
> Original discussion:
>
>   https://lists.gnu.org/archive/html/guix-devel/2014-02/msg00245.html

I've read some of the threads connected to this one after I
learned about the subject. It usually helps when the subject is
added so I can search locally.
What happened to the p11-kit Andreas mentioned back in 2014 or
2015?

> Ludo’.
>

-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: GnuTLS and the “trust store”

[Ludovic Courtès]

ng0 <ng0@libertad.pw> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Marius Bakke <mbakke@fastmail.com> skribis:
>>
>>> Marius Bakke <mbakke@fastmail.com> writes:
>>>
>>>> ng0 <ng0@libertad.pw> writes:
>>>>
>>>>> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
>>
>> [...]
>>
>>> I realized shortly after posting why this wasn't done already. Curl has
>>> 1403 dependent packages, which would apply for "nss-certs" as well if
>>> that is added as input. Obviously we want to be able to update TLS
>>> certificates quickly without rebuilding ~1/4 of the tree.
>>
>> Indeed.  It’s a situation where we do not want to have a static binding
>> between cURL and nss-certs; instead, they should be composed
>> dynamically, along the lines of what we already recommend at:
>
> Okay, so my proposed gnURL patch should not be applied at
> all. Reading the old threads I'm starting to understand the
> situation, but not completely.
>
>>   https://www.gnu.org/software/guix/manual/html_node/X_002e509-Certificates.html
>>
>> cURL depends on GnuTLS, and GnuTLS doesn’t honor an environment variable
>> like ‘SSL_CERT_DIR’.  Its recipe has this comment:
>
> The 3rd option in 2015, subject: [PATCH] gnu: gnutls: Configure
> location of system-wide trust store, was to use openssl.

Not an option: we use GnuTLS anytime there’s a choice (which also avoids
licensing issues with the OpenSSL license).

> I'm trying to understand the problem here, the problem why
> packages like darcs, pbpst, and others are just sitting, waiting
> for months because of issues with cURL.

What is “these issues with cURL”?  It builds fine, and it’s perfectly
usable as long as /etc/ssl/certs is populated.

>>          ;; GnuTLS doesn't consult any environment variables to specify
>>          ;; the location of the system-wide trust store.  Instead it has a
>>          ;; configure-time option.  Unless specified, its configure script
>>          ;; attempts to auto-detect the location by looking for common
>>          ;; places in the file system, none of which are present in our
>>          ;; chroot build environment.  If not found, then no default trust
>>          ;; store is used, so each program has to provide its own
>>          ;; fallback, and users have to configure each program
>>          ;; independently.  This seems suboptimal.
>>          "--with-default-trust-store-dir=/etc/ssl/certs"
>>
>> Original discussion:
>>
>>   https://lists.gnu.org/archive/html/guix-devel/2014-02/msg00245.html
>
> I've read some of the threads connected to this one after I
> learned about the subject. It usually helps when the subject is
> added so I can search locally.
> What happened to the p11-kit Andreas mentioned back in 2014 or
> 2015?

Good question, I don’t know!  Perhaps we can revisit this.

Thanks,
Ludo’.

Re: GnuTLS and the “trust store”

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Marius Bakke <mbakke@fastmail.com> skribis:
>
>> Marius Bakke <mbakke@fastmail.com> writes:
>>
>>> ng0 <ng0@libertad.pw> writes:
>>>
>>>> * gnu/packages/curl.scm (curl)[arguments]: Add "--with-ca-bundle" configure flag.
>
> [...]
>
>> I realized shortly after posting why this wasn't done already. Curl has
>> 1403 dependent packages, which would apply for "nss-certs" as well if
>> that is added as input. Obviously we want to be able to update TLS
>> certificates quickly without rebuilding ~1/4 of the tree.
>
> Indeed.  It’s a situation where we do not want to have a static binding
> between cURL and nss-certs; instead, they should be composed
> dynamically, along the lines of what we already recommend at:
>
>   https://www.gnu.org/software/guix/manual/html_node/X_002e509-Certificates.html

Curl respects the variable "CURL_CA_BUNDLE". I think we could add a
"native-search-path" for that, similar to how it's done for "git".

ng0, can you try that?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: GnuTLS and the “trust store”

[Ricardo Wurmus]

Marius Bakke <mbakke@fastmail.com> writes:

> Curl respects the variable "CURL_CA_BUNDLE". I think we could add a
> "native-search-path" for that, similar to how it's done for "git".

“curl” does but libcurl does not.

~~ Ricardo

Re: GnuTLS and the “trust store”

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

>> I'm trying to understand the problem here, the problem why
>> packages like darcs, pbpst, and others are just sitting, waiting
>> for months because of issues with cURL.
>
> What is “these issues with cURL”?  It builds fine, and it’s perfectly
> usable as long as /etc/ssl/certs is populated.

I still think that we ought to patch libcurl such that it checks the
environment variable CURL_CA_BUNDLE.  NixOS used to do this in the past,
IIRC.

It’s an easy fix and allows us to solve the problem at the root instead
of trying to come up with a fix for every user of libcurl.

~~ Ricardo

Re: [PATCH] gnu: curl: Add ca-bundle to config.

[Ricardo Wurmus]

ng0 <ng0@libertad.pw> writes:

>> how can I make this valid:
>>
>>    (arguments
>>     `(#:configure-flags '("--enable-ipv6" "--with-gnutls" "--without-libssh2"
>>                           "--without-libmetalink" "--without-winidn"
>>                           "--without-librtmp" "--without-nghttp2"
>>                           "--without-nss" "--without-cyassl"
>>                           "--without-polarssl" "--without-ssl"
>>                           "--without-winssl" "--without-darwinssl"
>>                           "--disable-sspi" "--disable-ntlm-wb"
>>                           "--disable-ldap" "--disable-rtsp" "--disable-dict"
>>                           "--disable-telnet" "--disable-tftp" "--disable-pop3"
>>                           "--disable-imap" "--disable-smtp" "--disable-gopher"
>>                           "--disable-file" "--disable-ftp" "--disable-smb"
>>                           (string-append
>>                            "--with-ca-bundle="
>>                            (string-append (assoc-ref %build-inputs "nss-certs")
>>                                           "/etc/ssl/certs/ca-certificates.crt")))
>>
>> The string-append is not valid here.
>
> Solved, by using "(list" here.

The reason why this didn’t work is because you’re expecting code to be
evaluated inside of an “inert” expression.

(+ 1 2) is evaluated right away and the result is 3

'(+ 1 2) is a quoted expression, so it’s just a list of '+, 1, and 2.
Think of the ' as “DATA MODE”

`(+ 1 2) is a quasiquoted expression.  Think of the backtick as a toggle
switch.  When it’s up it means “DATA MODE”, when it is down (,) it means
“CODE MODE”.

Example:

    `(+ 1 2 ,(string->number "4"))

this means: DATA MODE + 1 2 CODE MODE (string->number "4")

so you get a list with the following contents: '+, 1, 2, and the number 4.

Your configure flags above are a quoted list, so everything that follows
is just data.  “string-append” is not special, it’s just another symbol
in the list.  You can try this in the REPL to convince yourself that
this is how quoting works.

Note the difference between:

    `(#:configure-flags '("foo" "bar"
                          (string-append "baz" "lightyear")))

and

    `(#:configure-flags '("foo" "bar"
                          ,(string-append "baz" "lightyear")))

The comma (“unquote”) flips the toggle switch and the expression is
evaluated.

Using “list” just means that you are not using quotation at all.


One final note:

>>                           (string-append
>>                            "--with-ca-bundle="
>>                            (string-append (assoc-ref %build-inputs "nss-certs")
>>                                           "/etc/ssl/certs/ca-certificates.crt")))

That’s really not pretty.  You don’t need to nest string-append
expressions.

    (string-append "this" "and" "that")

returns the same value as

    (string-append "this" (string-append "and" "that"))

~~ Ricardo

Re: GnuTLS and the “trust store”

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> Curl respects the variable "CURL_CA_BUNDLE". I think we could add a
>> "native-search-path" for that, similar to how it's done for "git".
>
> “curl” does but libcurl does not.

But that’s probably on purpose.  What do the cURL developers recommend
for their users?

If they recommend that users roll their own mechanism to designate the
trust store, then they probably do (?), and I think we should avoid
interfering with that.

Thoughts?

Ludo’.

Re: GnuTLS and the “trust store”

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>
>> Marius Bakke <mbakke@fastmail.com> writes:
>>
>>> Curl respects the variable "CURL_CA_BUNDLE". I think we could add a
>>> "native-search-path" for that, similar to how it's done for "git".
>>
>> “curl” does but libcurl does not.
>
> But that’s probably on purpose.  What do the cURL developers recommend
> for their users?
>
> If they recommend that users roll their own mechanism to designate the
> trust store, then they probably do (?), and I think we should avoid
> interfering with that.

I don’t know what they recommend but on an FHS-compliant system libcurl
would be configured to default to a well-known path for the default CA
bundle.  This allows users of libcurl to just not care about
implementing a mechanism to override the default CA bundle, because it
would fall back to the well-known system-wide path.

One of these packages is “r-curl”, which just assumes that the libcurl
defaults are fine.  We patch it to enable CURL_CA_BUNDLE lookup (a
feature that was intended only for Windows).

Since GuixSD does not offer this path and Guix can be used on different
systems I think we need to provide an alternative.  One alternative is
to replace the well-known path with a call to getenv("CURL_CA_BUNDLE").

~~ Ricardo

Re: GnuTLS and the “trust store”

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>>
>>> Marius Bakke <mbakke@fastmail.com> writes:
>>>
>>>> Curl respects the variable "CURL_CA_BUNDLE". I think we could add a
>>>> "native-search-path" for that, similar to how it's done for "git".
>>>
>>> “curl” does but libcurl does not.
>>
>> But that’s probably on purpose.  What do the cURL developers recommend
>> for their users?
>>
>> If they recommend that users roll their own mechanism to designate the
>> trust store, then they probably do (?), and I think we should avoid
>> interfering with that.
>
> I don’t know what they recommend but on an FHS-compliant system libcurl
> would be configured to default to a well-known path for the default CA
> bundle.  This allows users of libcurl to just not care about
> implementing a mechanism to override the default CA bundle, because it
> would fall back to the well-known system-wide path.

That’s also the case with Guix: GnuTLS looks for things in
/etc/ssl/certs by default, doesn’t it?

> One of these packages is “r-curl”, which just assumes that the libcurl
> defaults are fine.  We patch it to enable CURL_CA_BUNDLE lookup (a
> feature that was intended only for Windows).

So r-curl doesn’t try /etc/ssl/certs?  That makes me wonder if the
--with-default-trust-store-dir option of GnuTLS works as expected.

> Since GuixSD does not offer this path and Guix can be used on different
> systems I think we need to provide an alternative.  One alternative is
> to replace the well-known path with a call to getenv("CURL_CA_BUNDLE").

OK.

Ludo’.