bug#32942: nss-certs not deterministic

bug#32942: nss-certs not deterministic

[Julien Lepiller]

While updating a profile, I found that nss-certs was not deterministic. 
 From ludo:

$ wget -O - -q 
https://mirror.hydra.gnu.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo | 
grep Hash
NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla
$ wget -O - -q 
https://berlin.guixsd.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo | 
grep Hash
NarHash: sha256:08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s

bug#32942: nss-certs not deterministic

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 2702 bytes --]

Hello,

Julien Lepiller <julien@lepiller.eu> skribis:

> While updating a profile, I found that nss-certs was not
> deterministic. From ludo:
>
> $ wget -O - -q 
> https://mirror.hydra.gnu.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo| grep Hash
> NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla
> $ wget -O - -q 
> https://berlin.guixsd.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo |
> grep Hash
> NarHash: sha256:08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s

As shown above, berlin and hydra disagree on nss-certs.

The difference is an encoding bug:

--8<---------------cut here---------------start------------->8---
$ wget -O - https://berlin.guixsd.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.berlin
$ wget -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.hydra
$ diff -ru /tmp/nss-certs.{hydra,berlin}
Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
Only in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
Only in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pem
Only in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem
--8<---------------cut here---------------end--------------->8---

The problem was already reported as <https://bugs.gnu.org/26948> and
since commit 412701b0e5e073e6767eed162c14698db99df69c (July 2017) ‘guix
publish’ on GuixSD runs in a UTF-8 locale to avoid that problem.

The faulty narinfo/nar on berlin were generated on Oct. 17, 2018, so
clearly the above commit was in effect.  Indeed, after removing them and
regenerating them, I’m still getting
08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s (aka. the wrong
hash).

On closer inspection the problem is elsewhere: the
/gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 directory on
berlin has question marks in file names, so ‘guix publish’ is not to
blame; instead the problem likely comes from ‘guix offload’.

Indeed ‘guix-daemon’ and its child processes such as ‘guix offload’ run
with an empty environment, and thus in the C locale.  Specifically,
‘restore-file-set’ on the build farm front-end must be the one
substituting question marks to the non-ASCII characters.

If this analysis is correct, the patch below should fix it.  I’ll try it
later.

Thanks,
Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 972 bytes --]

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index cee9898d79..9fe64e8087 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -1603,7 +1603,15 @@ failed to register public key '~a': ~a~%" key status))))))))
                             '())
                      #$@(if tmpdir
                             (list (string-append "TMPDIR=" tmpdir))
-                            '()))
+                            '())
+
+                     ;; Make sure we run in a UTF-8 locale so that 'guix
+                     ;; offload' correctly restores nars that contain UTF-8
+                     ;; file names such as 'nss-certs'.  See
+                     ;; <https://bugs.gnu.org/32942>.
+                     (string-append "GUIX_LOCPATH="
+                                    #$glibc-utf8-locales "/lib/locale")
+                     "LC_ALL=en_US.utf8")
 
                #:log-file #$log-file))
            (stop #~(make-kill-destructor))))))

bug#32942: nss-certs not deterministic

[Ludovic Courtès]

Hello,

ludo@gnu.org (Ludovic Courtès) skribis:

> The difference is an encoding bug:
>
> $ wget -O - https://berlin.guixsd.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.berlin
> $ wget -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.hydra
> $ diff -ru /tmp/nss-certs.{hydra,berlin}
> Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
> Only in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
> Only in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pem
> Only in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem

[...]

> On closer inspection the problem is elsewhere: the
> /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 directory on
> berlin has question marks in file names, so ‘guix publish’ is not to
> blame; instead the problem likely comes from ‘guix offload’.
>
> Indeed ‘guix-daemon’ and its child processes such as ‘guix offload’ run
> with an empty environment, and thus in the C locale.  Specifically,
> ‘restore-file-set’ on the build farm front-end must be the one
> substituting question marks to the non-ASCII characters.
>
> If this analysis is correct, the patch below should fix it.  I’ll try it
> later.

Pushed as 7e4bc215098f334bc2a11737f2665dd4992fc2da.

Thanks,
Ludo'.

bug#32942: nss-certs not deterministic

[swedebugia]

On 2018-12-05 15:01, Ludovic Courtès wrote:
> Hello,
> 
> Julien Lepiller <julien@lepiller.eu> skribis:
> 
>> While updating a profile, I found that nss-certs was not
>> deterministic. From ludo:
>>
>> $ wget -O - -q
>> https://mirror.hydra.gnu.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo| grep Hash
>> NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla
>> $ wget -O - -q
>> https://berlin.guixsd.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo |
>> grep Hash
>> NarHash: sha256:08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s
> 
> As shown above, berlin and hydra disagree on nss-certs.
> 
> The difference is an encoding bug:
> 
> --8<---------------cut here---------------start------------->8---
> $ wget -O - https://berlin.guixsd.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.berlin
> $ wget -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.hydra
> $ diff -ru /tmp/nss-certs.{hydra,berlin}
> Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
> Only in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
> Only in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pem
> Only in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem
> --8<---------------cut here---------------end--------------->8---
> 
> The problem was already reported as <https://bugs.gnu.org/26948> and
> since commit 412701b0e5e073e6767eed162c14698db99df69c (July 2017) ‘guix
> publish’ on GuixSD runs in a UTF-8 locale to avoid that problem.
> 
> The faulty narinfo/nar on berlin were generated on Oct. 17, 2018, so
> clearly the above commit was in effect.  Indeed, after removing them and
> regenerating them, I’m still getting
> 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s (aka. the wrong
> hash).
> 
> On closer inspection the problem is elsewhere: the
> /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 directory on
> berlin has question marks in file names, so ‘guix publish’ is not to
> blame; instead the problem likely comes from ‘guix offload’.
> 
> Indeed ‘guix-daemon’ and its child processes such as ‘guix offload’ run
> with an empty environment, and thus in the C locale.  Specifically,
> ‘restore-file-set’ on the build farm front-end must be the one
> substituting question marks to the non-ASCII characters.
> 
> If this analysis is correct, the patch below should fix it.  I’ll try it
> later.
> 
> Thanks,
> Ludo’.
> 
> 
> 
> diff --git a/gnu/services/base.scm b/gnu/services/base.scm
> index cee9898d79..9fe64e8087 100644
> --- a/gnu/services/base.scm
> +++ b/gnu/services/base.scm
> @@ -1603,7 +1603,15 @@ failed to register public key '~a': ~a~%" key status))))))))
>                               '())
>                        #$@(if tmpdir
>                               (list (string-append "TMPDIR=" tmpdir))
> -                            '()))
> +                            '())
> +
> +                     ;; Make sure we run in a UTF-8 locale so that 'guix
> +                     ;; offload' correctly restores nars that contain UTF-8
> +                     ;; file names such as 'nss-certs'.  See
> +                     ;; <https://bugs.gnu.org/32942>.
> +                     (string-append "GUIX_LOCPATH="
> +                                    #$glibc-utf8-locales "/lib/locale")
> +                     "LC_ALL=en_US.utf8")
>   
>                  #:log-file #$log-file))
>              (stop #~(make-kill-destructor))))))
> 

Congratulations with the succeded hunt and thanks a lot for showing all 
the steps you took so I can improve my hunting skills and eventually 
begin helping by hunting on my own :D

-- 
Cheers Swedebugia