From mboxrd@z Thu Jan 1 00:00:00 1970 From: swedebugia Subject: bug#32942: nss-certs not deterministic Date: Wed, 19 Dec 2018 18:42:19 +0100 Message-ID: <0fc840ed-c560-79ea-c765-7ab66af17d97@riseup.net> References: <3974e5005881951012bb5e55a5bfabe2@lepiller.eu> <87woooxebu.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:48065) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZflN-000818-CE for bug-guix@gnu.org; Wed, 19 Dec 2018 12:36:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZflK-0007gJ-4R for bug-guix@gnu.org; Wed, 19 Dec 2018 12:36:05 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:50794) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gZflJ-0007g7-Vw for bug-guix@gnu.org; Wed, 19 Dec 2018 12:36:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gZflJ-0006cT-PB for bug-guix@gnu.org; Wed, 19 Dec 2018 12:36:01 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87woooxebu.fsf@gnu.org> Content-Language: en-US List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Julien Lepiller Cc: 32942@debbugs.gnu.org On 2018-12-05 15:01, Ludovic Courtès wrote: > Hello, > > Julien Lepiller skribis: > >> While updating a profile, I found that nss-certs was not >> deterministic. From ludo: >> >> $ wget -O - -q >> https://mirror.hydra.gnu.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo| grep Hash >> NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla >> $ wget -O - -q >> https://berlin.guixsd.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo | >> grep Hash >> NarHash: sha256:08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s > > As shown above, berlin and hydra disagree on nss-certs. > > The difference is an encoding bug: > > --8<---------------cut here---------------start------------->8--- > $ wget -O - https://berlin.guixsd.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.berlin > $ wget -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.hydra > $ diff -ru /tmp/nss-certs.{hydra,berlin} > Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem > Only in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem > Only in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pem > Only in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem > --8<---------------cut here---------------end--------------->8--- > > The problem was already reported as and > since commit 412701b0e5e073e6767eed162c14698db99df69c (July 2017) ‘guix > publish’ on GuixSD runs in a UTF-8 locale to avoid that problem. > > The faulty narinfo/nar on berlin were generated on Oct. 17, 2018, so > clearly the above commit was in effect. Indeed, after removing them and > regenerating them, I’m still getting > 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s (aka. the wrong > hash). > > On closer inspection the problem is elsewhere: the > /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 directory on > berlin has question marks in file names, so ‘guix publish’ is not to > blame; instead the problem likely comes from ‘guix offload’. > > Indeed ‘guix-daemon’ and its child processes such as ‘guix offload’ run > with an empty environment, and thus in the C locale. Specifically, > ‘restore-file-set’ on the build farm front-end must be the one > substituting question marks to the non-ASCII characters. > > If this analysis is correct, the patch below should fix it. I’ll try it > later. > > Thanks, > Ludo’. > > > > diff --git a/gnu/services/base.scm b/gnu/services/base.scm > index cee9898d79..9fe64e8087 100644 > --- a/gnu/services/base.scm > +++ b/gnu/services/base.scm > @@ -1603,7 +1603,15 @@ failed to register public key '~a': ~a~%" key status)))))))) > '()) > #$@(if tmpdir > (list (string-append "TMPDIR=" tmpdir)) > - '())) > + '()) > + > + ;; Make sure we run in a UTF-8 locale so that 'guix > + ;; offload' correctly restores nars that contain UTF-8 > + ;; file names such as 'nss-certs'. See > + ;; . > + (string-append "GUIX_LOCPATH=" > + #$glibc-utf8-locales "/lib/locale") > + "LC_ALL=en_US.utf8") > > #:log-file #$log-file)) > (stop #~(make-kill-destructor)))))) > Congratulations with the succeded hunt and thanks a lot for showing all the steps you took so I can improve my hunting skills and eventually begin helping by hunting on my own :D -- Cheers Swedebugia