curl server certificate verification failed for a few sites

curl server certificate verification failed for a few sites

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 3396 bytes --]

Hello Guix,

this is my current Guix version:

--8<---------------cut here---------------start------------->8---

Generation 71	giu 03 2020 17:44:58	(current)
  guix 2f49007
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 2f49007dd076b14feb40d7c3331dee3e737265c844

--8<---------------cut here---------------end--------------->8---

I'm having a strange error with curl from Guix (on a foreign distro):

--8<---------------cut here---------------start------------->8---
giovanni@roquette: curl -I https://voices.transparency.org
curl: (60) server certificate verification failed. CAfile: /home/giovanni/.guix-extra-profiles/emacs/emacs/etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
--8<---------------cut here---------------end--------------->8---

I'm working in a profile with "curl" and "nss-certs" installed, I my
profile activate with:

--8<---------------cut here---------------start------------->8---
GUIX_PROFILE="/home/giovanni/.guix-extra-profiles/emacs/emacs" ; . "$GUIX_PROFILE"/etc/profile
--8<---------------cut here---------------end--------------->8---

and it seems to me that the env is set up correctly, especially I have:

--8<---------------cut here---------------start------------->8---
CURL_CA_BUNDLE=/home/giovanni/.guix-extra-profiles/emacs/emacs/etc/ssl/certs/ca-certificates.crt
--8<---------------cut here---------------end--------------->8---

With other sites I have no problems, e.g. with "curl https://google.com"
or "curl https://fsf.org"

I also have no problem using curl from my foreign distro, i.e.:

--8<---------------cut here---------------start------------->8---
giovanni@roquette: /usr/bin/curl -I https://voices.transparency.org
HTTP/2 200 
server: nginx
date: Thu, 04 Jun 2020 12:31:30 GMT
content-type: text/html; charset=utf-8
content-length: 331031
set-cookie: uid=lo_qSsMbQ4z5dnK; Expires=Fri, 04-Jun-21 12:31:30 GMT; Path=/; Secure; HttpOnly
sepia-upstream: medium
x-opentracing: {"ot-tracer-spanid":"390949417d4930c0","ot-tracer-traceid":"7f0fd7d0262c0c9b","ot-tracer-sampled":"true"}

[...]

x-frame-options: sameorigin
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
x-ua-compatible: IE=edge, Chrome=1
x-powered-by: Medium
x-obvious-tid: 1591273890249:f076faaf3e63
x-obvious-info: 41228-18f1264,18f1264f828
link: <https://medium.com/humans.txt>; rel="humans"
cache-control: no-cache, no-store, max-age=0, must-revalidate
expires: Thu, 09 Sep 1999 09:09:09 GMT
pragma: no-cache
set-cookie: sid=1:h/sJWfMuvxt6xsp9CxiSVZhE/m0Gkrnt1Aj+VWcPGV1cpLwK14Dg7Rfr4fWgzCz9ru4qnVfGEhX+mljUMOriPA==; path=/; expires=Fri, 04 Jun 2021 12:31:30 GMT; samesite=none; secure; httponly
set-cookie: optimizelyEndUserId=lo_qSsMbQ4z5dnK; path=/; expires=Fri, 04 Jun 2021 12:31:30 GMT; samesite=none; secure
--8<---------------cut here---------------end--------------->8---

Am I missing something in my env, there is a problem in
https://voices.transparency.org setting or is something else?

Thanks! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: curl server certificate verification failed for a few sites

[Jack Hill]

On Thu, 4 Jun 2020, Giovanni Biscuolo wrote:

> Hello Guix,
>
> --8<---------------cut here---------------end--------------->8---
>
> I'm having a strange error with curl from Guix (on a foreign distro):
>
> --8<---------------cut here---------------start------------->8---
> giovanni@roquette: curl -I https://voices.transparency.org
> curl: (60) server certificate verification failed. CAfile: /home/giovanni/.guix-extra-profiles/emacs/emacs/etc/ssl/certs/ca-certificates.crt CRLfile: none
> More details here: https://curl.haxx.se/docs/sslcerts.html
>
> curl failed to verify the legitimacy of the server and therefore could not
> establish a secure connection to it. To learn more about this situation and
> how to fix it, please visit the web page mentioned above.
> --8<---------------cut here---------------end--------------->8---

Giovanni,

I think that this is due to the recent AdTrust Root CA cert expiration 
[0]. The error wget gives is a little bit better, but you know about the 
situation to interpret it correctly:

"""
$ wget "https://voices.transparency.org" -O /dev/null
--2020-06-04 10:37:29--  https://voices.transparency.org/
Resolving voices.transparency.org (voices.transparency.org)... 
52.4.225.124, 52.4.240.221, 52.1.119.170, ...
Connecting to voices.transparency.org 
(voices.transparency.org)|52.4.225.124|:443... connected.
ERROR: The certificate of ‘voices.transparency.org’ is not trusted.
ERROR: The certificate of ‘voices.transparency.org’ has expired.
"""

In my experience, sometimes this cert expiration is easy to miss by site 
administrators or others connecting to the site if they have one of the 
intermediate certificates in their trust store. Our nss-certs package 
tends not to have such intermediates.

Therefore, I think the fix is for voices.transparency.org to update the 
certificate chain/bundle that they are sending.

[0] https://support.sectigo.com/Com_KnowledgeDetailPage?Id=kA03l00000117LT

Best,
Jack

Re: curl server certificate verification failed for a few sites

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 2439 bytes --]

Hi Jack,

thanks for your help!
...and sorry if this is not Guix specific

Jack Hill <jackhill@jackhill.us> writes:

[...]

> I think that this is due to the recent AdTrust Root CA cert expiration 
> [0]. The error wget gives is a little bit better, but you know about the 
> situation to interpret it correctly:
>
> """
> $ wget "https://voices.transparency.org" -O /dev/null
> --2020-06-04 10:37:29--  https://voices.transparency.org/
> Resolving voices.transparency.org (voices.transparency.org)... 
> 52.4.225.124, 52.4.240.221, 52.1.119.170, ...
> Connecting to voices.transparency.org 
> (voices.transparency.org)|52.4.225.124|:443... connected.
> ERROR: The certificate of ‘voices.transparency.org’ is not trusted.
> ERROR: The certificate of ‘voices.transparency.org’ has expired.
> """

oh I see, I get this error also... but I do not understand the different
behaviour with what I see in Firefox (from Debian) or ungoogled-chromium
(from Guix): using one of those browsers the certificate is valid, the
certificate viewer shows that the root in cert hierarchy is "USERTrust
RSA Certification Authority"

The section in [0] titled "Certificate Chain Diagram" states:

--8<---------------cut here---------------start------------->8---

A legacy browser or older device that does not have the modern
“USERTRust” root would not trust it and so would look further up the
chain to a root it does trust, the AddTrust External CA Root. A more
modern browser would have the USERTrust root already installed and trust
itwithout needing to rely on the older AddTrust root.

--8<---------------cut here---------------end--------------->8---

I do not fully understand why curl and wget return error while Firefox
and ungoogled-chromium not

[...]

> Therefore, I think the fix is for voices.transparency.org to update the 
> certificate chain/bundle that they are sending.
>
> [0]
> https://support.sectigo.com/Com_KnowledgeDetailPage?Id=kA03l00000117LT

this page states:

--8<---------------cut here---------------start------------->8---

Will my certificate still be trusted after May 30, 2020?

Yes. All modern clients and operating systems have the newer, modern
COMODO and USERTrust roots which don’t expire until 2038.

--8<---------------cut here---------------end--------------->8---

Thanks! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: curl server certificate verification failed for a few sites

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

Giovanni Biscuolo 写道：
> Jack Hill <jackhill@jackhill.us> writes:
>> The error wget gives is a little bit better,

FWIW, I use this (extremely verbose) command to debug/check my own 
servers:

  $ openssl s_client -showcerts -servername 
  voices.transparency.org \
    -connect voices.transparency.org:443

>> Therefore, I think the fix is for voices.transparency.org to 
>> update the 
>> certificate chain/bundle that they are sending.

They're also sending intermediate certificates that they shouldn't 
be sending in the first place[0] which doesn't help matters.  I 
agree that this looks like an outdated server (mis)configuration.

> Yes. All modern clients and operating systems have the newer, 
> modern
> COMODO and USERTrust roots which don’t expire until 2038.

Right, but ‘modern’ there means ~2015.

Kind regards,

T G-R

[0]: 
https://www.ssllabs.com/ssltest/analyze.html?d=voices.transparency.org&s=52.4.38.70&hideResults=on

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: curl server certificate verification failed for a few sites

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 3609 bytes --]

Hi Tobias,

thank you for your clear explanation and patience

...and sorry again to all other Guix users for the "noise": this is not
strictly related to Guix but just to the most recent version of
curl/wget

I still I don't understand the differences between curl (and wget)
behaviour and the last Guix available ungoogled-chromium (see below).

Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Giovanni Biscuolo 写道：
>> Jack Hill <jackhill@jackhill.us> writes:
>>> The error wget gives is a little bit better,
>
> FWIW, I use this (extremely verbose) command to debug/check my own 
> servers:
>
>   $ openssl s_client -showcerts -servername 
>   voices.transparency.org \
>     -connect voices.transparency.org:443

With this output I'm able to understand what's going on with this
certificate, thanks!

This command clearly shows the depth of this certificate is 3 and that
the top level cert is expired:

--8<---------------cut here---------------start------------->8---

depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify error:num=10:certificate has expired
notAfter=May 30 10:48:38 2020 GMT

--8<---------------cut here---------------end--------------->8---

I guess that this information, client side, is the same for all browsers
and CLI interfaces (like curl) since long ago: right?

[...]

> They're also sending intermediate certificates that they shouldn't 
> be sending in the first place[0] which doesn't help matters.  I 
> agree that this looks like an outdated server (mis)configuration.

OK but I really don't understand why with a recent browser from Guix -
ungoogled-chromium 81.0.4044.138 - the certificate is detected as valid:
the top root certificate shown in it's graphical "Certificate viewer"
interface is "USERTrust".

It seems that ungoogled-chromium stops the verification at the level=1 certificate:

--8<---------------cut here---------------start------------->8---

 1 s:/C=GB/ST=Greater Manchester/L=Salford/O=Sectigo Limited/CN=Sectigo RSA Domain Validation Secure Server CA
   i:/C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority

--8<---------------cut here---------------end--------------->8---

>> Yes. All modern clients and operating systems have the newer, 
>> modern
>> COMODO and USERTrust roots which don’t expire until 2038.
>
> Right, but ‘modern’ there means ~2015.

I don't fully understand what this means, sorry... but it's not
important :-)

> [0]: 
> https://www.ssllabs.com/ssltest/analyze.html?d=voices.transparency.org&s=52.4.38.70&hideResults=on

I had a look at three random IP addresses from the list of checked ones
(all grade B): they give three certification paths and path #3 is
expired.

Nonetheless, I still do not understand why ungoogled-chromium is
behaving diffrerently than the most recent curl/wget

A similar thing is happening when trying to fetch content (for elfeed)
using curl from:

1. www.skepticalscience.com (server's certificate chain is incomplete)
2. firstmonday.org (uses the expired AddTrust External TTP Network root
certificate)

Both are detected as valid in ungoogle-chromium.

I can ask each of them to update their certificates but I fear it will
be difficult to explain why, given that all "modern browsers" have
absolutely no problem with them :-S

...and yes, I agree they **have** a problem with their certificate
chains :-(


Thanks! Giovanni.

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: curl server certificate verification failed for a few sites

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

Giovanni Biscuolo <g@xelera.eu> writes:

> Hi Tobias,
>
> thank you for your clear explanation and patience
>
> ...and sorry again to all other Guix users for the "noise": this is not
> strictly related to Guix but just to the most recent version of
> curl/wget
>
> I still I don't understand the differences between curl (and wget)
> behaviour and the last Guix available ungoogled-chromium (see below).

The problem is with GnuTLS, which failed if one of the trust roots were
using an expired certificate:

  https://gitlab.com/gnutls/gnutls/-/issues/1008

It has been fixed in the latest GnuTLS, which is in Guix as of commit
8951b9496b5c390adb3b3292d234bb8ab9936c40.  Thanks for reporting it!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: curl server certificate verification failed for a few sites

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 499 bytes --]

Marius Bakke <marius@gnu.org> writes:

[...]

> The problem is with GnuTLS, which failed if one of the trust roots were
> using an expired certificate:
>
>   https://gitlab.com/gnutls/gnutls/-/issues/1008

Oh I see, fine!

> It has been fixed in the latest GnuTLS, which is in Guix as of commit
> 8951b9496b5c390adb3b3292d234bb8ab9936c40.  Thanks for reporting it!

Thank you for the explanation and the patch!

Best, Giovanni

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: curl server certificate verification failed for a few sites

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 1513 bytes --]

Giovanni,

Giovanni Biscuolo 写道：
> ...and sorry again to all other Guix users for the "noise": this 
> is not
> strictly related to Guix but just to the most recent version of
> curl/wget

Don't be.  It was a legitimate bug in a Guix package.  Thanks to 
Marius for the quick fix, by the way!

> I still I don't understand the differences between curl (and 
> wget)
> behaviour and the last Guix available ungoogled-chromium (see 
> below).

The expiration of the Sectigo root triggered a dormant bug in 
GnuTLS.  Users of other crypto libraries were unaffected.

> I guess that this information, client side, is the same for all 
> browsers
> and CLI interfaces (like curl) since long ago: right?

Yes.  Including GnuTLS.  It had the right data but drew the wrong 
conclusion from it.

> It seems that ungoogled-chromium stops the verification at the 
> level=1 certificate:

As your browser and SSLLabs knew, there *was* a valid chain (two, 
even) and GnuTLS should have returned success.  Instead it 
reported failure because there was *also* an invalid expired one.

At the risk of being flamed for oversimplifying: paranoid GnuTLS 
was using AND where it should have used OR.

Here's the actual bug report: 
<https://gitlab.com/gnutls/gnutls/-/issues/1008>.

(I think the server's still sending too many intermediates, but at 
least now all clients will correctly ignore them.  They'll just 
waste some bandwidth on every handshake.)

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]