bug#32942: nss-certs not deterministic

[swedebugia <swedebugia@riseup.net>]

On 2018-12-05 15:01, Ludovic Courtès wrote:
> Hello,
> 
> Julien Lepiller <julien@lepiller.eu> skribis:
> 
>> While updating a profile, I found that nss-certs was not
>> deterministic. From ludo:
>>
>> $ wget -O - -q
>> https://mirror.hydra.gnu.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo| grep Hash
>> NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla
>> $ wget -O - -q
>> https://berlin.guixsd.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo |
>> grep Hash
>> NarHash: sha256:08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s
> 
> As shown above, berlin and hydra disagree on nss-certs.
> 
> The difference is an encoding bug:
> 
> --8<---------------cut here---------------start------------->8---
> $ wget -O - https://berlin.guixsd.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.berlin
> $ wget -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.hydra
> $ diff -ru /tmp/nss-certs.{hydra,berlin}
> Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
> Only in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
> Only in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pem
> Only in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem
> --8<---------------cut here---------------end--------------->8---
> 
> The problem was already reported as <https://bugs.gnu.org/26948> and
> since commit 412701b0e5e073e6767eed162c14698db99df69c (July 2017) ‘guix
> publish’ on GuixSD runs in a UTF-8 locale to avoid that problem.
> 
> The faulty narinfo/nar on berlin were generated on Oct. 17, 2018, so
> clearly the above commit was in effect.  Indeed, after removing them and
> regenerating them, I’m still getting
> 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s (aka. the wrong
> hash).
> 
> On closer inspection the problem is elsewhere: the
> /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 directory on
> berlin has question marks in file names, so ‘guix publish’ is not to
> blame; instead the problem likely comes from ‘guix offload’.
> 
> Indeed ‘guix-daemon’ and its child processes such as ‘guix offload’ run
> with an empty environment, and thus in the C locale.  Specifically,
> ‘restore-file-set’ on the build farm front-end must be the one
> substituting question marks to the non-ASCII characters.
> 
> If this analysis is correct, the patch below should fix it.  I’ll try it
> later.
> 
> Thanks,
> Ludo’.
> 
> 
> 
> diff --git a/gnu/services/base.scm b/gnu/services/base.scm
> index cee9898d79..9fe64e8087 100644
> --- a/gnu/services/base.scm
> +++ b/gnu/services/base.scm
> @@ -1603,7 +1603,15 @@ failed to register public key '~a': ~a~%" key status))))))))
>                               '())
>                        #$@(if tmpdir
>                               (list (string-append "TMPDIR=" tmpdir))
> -                            '()))
> +                            '())
> +
> +                     ;; Make sure we run in a UTF-8 locale so that 'guix
> +                     ;; offload' correctly restores nars that contain UTF-8
> +                     ;; file names such as 'nss-certs'.  See
> +                     ;; <https://bugs.gnu.org/32942>.
> +                     (string-append "GUIX_LOCPATH="
> +                                    #$glibc-utf8-locales "/lib/locale")
> +                     "LC_ALL=en_US.utf8")
>   
>                  #:log-file #$log-file))
>              (stop #~(make-kill-destructor))))))
> 

Congratulations with the succeded hunt and thanks a lot for showing all 
the steps you took so I can improve my hunting skills and eventually 
begin helping by hunting on my own :D

-- 
Cheers Swedebugia