From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Hash mismatch for Python 2.7.10 (x86_64-linux) Date: Mon, 21 Mar 2016 18:22:24 +0100 Message-ID: <87lh5b690v.fsf_-_@gnu.org> References: <20160319152540.GA29638@thebird.nl> <87d1qqf9i8.fsf@gnu.org> <20160321154731.GB26140@jasmine> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:48926) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ai3XE-0000Hf-AS for guix-devel@gnu.org; Mon, 21 Mar 2016 13:22:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ai3XB-0007iu-0C for guix-devel@gnu.org; Mon, 21 Mar 2016 13:22:32 -0400 In-Reply-To: <20160321154731.GB26140@jasmine> (Leo Famulari's message of "Mon, 21 Mar 2016 11:47:31 -0400") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Leo Famulari Cc: guix-devel@gnu.org Leo Famulari skribis: > Can you take a look at python-2.7.10? > > Specifically, /gnu/store/4bs4mx6xyx0jx9827vmmbxyhjln7cmcv-python-2.7.10 > > This is from "ifur" on #guix: > http://paste.lisp.org/display/311051 This is an =E2=80=9Cinteresting=E2=80=9D case. There=E2=80=99s indeed a ha= sh mismatch between that narinfo and the nar (hydra.gnu.org and mirror.guixsd.org provide the exact same narinfo and nar though): --8<---------------cut here---------------start------------->8--- $ wget -q -O - https://hydra.gnu.org/4bs4mx6xyx0jx9827vmmbxyhjln7cmcv.narin= fo |grep Hash NarHash: sha256:1ifhxz7ng64ckrsfp0cy6rcybwrdf2mgq7p9w73ynbwy1w2frhp5 $ wget -q -O - https://hydra.gnu.org/nar/4bs4mx6xyx0jx9827vmmbxyhjln7cmcv-p= ython-2.7.10 | bunzip2 | guix hash /dev/stdin 1rb34mfv32zj6ir9nkrmw2mrc00n2gf9p2chihh1qpl9fmsxhhif $ guix hash -r /gnu/store/4bs4mx6xyx0jx9827vmmbxyhjln7cmcv-python-2.7.10 1ifhxz7ng64ckrsfp0cy6rcybwrdf2mgq7p9w73ynbwy1w2frhp5 --8<---------------cut here---------------end--------------->8--- To spot the differences, I did: --8<---------------cut here---------------start------------->8--- $ wget -O - https://mirror.guixsd.org/nar/4bs4mx6xyx0jx9827vmmbxyhjln7cmcv-= python-2.7.10 | bunzip2 | guix archive -x /tmp/python $ diff -ur --no-dereference /tmp/python /gnu/store/4bs4mx6xyx0jx9827vmmbxyh= jln7cmcv-python-2.7.10 Binary files /tmp/python/lib/python2.7/_abcoll.pyc and /gnu/store/4bs4mx6xy= x0jx9827vmmbxyhjln7cmcv-python-2.7.10/lib/python2.7/_abcoll.pyc differ Binary files /tmp/python/lib/python2.7/abc.pyc and /gnu/store/4bs4mx6xyx0jx= 9827vmmbxyhjln7cmcv-python-2.7.10/lib/python2.7/abc.pyc differ Binary files /tmp/python/lib/python2.7/codecs.pyc and /gnu/store/4bs4mx6xyx= 0jx9827vmmbxyhjln7cmcv-python-2.7.10/lib/python2.7/codecs.pyc differ Binary files /tmp/python/lib/python2.7/copy_reg.pyc and /gnu/store/4bs4mx6x= yx0jx9827vmmbxyhjln7cmcv-python-2.7.10/lib/python2.7/copy_reg.pyc differ Binary files /tmp/python/lib/python2.7/encodings/aliases.pyc and /gnu/store= /4bs4mx6xyx0jx9827vmmbxyhjln7cmcv-python-2.7.10/lib/python2.7/encodings/ali= ases.pyc differ Binary files /tmp/python/lib/python2.7/encodings/__init__.pyc and /gnu/stor= e/4bs4mx6xyx0jx9827vmmbxyhjln7cmcv-python-2.7.10/lib/python2.7/encodings/__= init__.pyc differ Binary files /tmp/python/lib/python2.7/encodings/utf_8.pyc and /gnu/store/4= bs4mx6xyx0jx9827vmmbxyhjln7cmcv-python-2.7.10/lib/python2.7/encodings/utf_8= .pyc differ [...] $ diffoscope --html t.html /tmp/python/lib/python2.7 /gnu/store/4bs4mx6xyx0= jx9827vmmbxyhjln7cmcv-python-2.7.10/lib/python2.7/ --8<---------------cut here---------------end--------------->8--- Diffoscope shows several hundred of bytes of differences in several .pyc files (so it=E2=80=99s not just 32-byte timestamps as discussed in , IIRC.) That the differences are so important is fishy. Could you check what you have on your machine? Is anyone able to make an independent build of this thing? For non-deterministic packages, something that could in theory happen (although it seems unlikely) is this: 1. nginx caches the narinfo of a first build with hash X; 2. nobody downloads the corresponding nar, and the result is eventually GC=E2=80=99d from hydra; 3. Hydra rebuilds the thing, obtains hash Y; the nar with hash Y is downloaded and cached by nginx, but, no luck, it doesn=E2=80=99t match= the hash in the narinfo that was cached in step #1. That could happen, but it seems pretty far-fetched, especially for a package like Python. On my machine I looked for the corresponding =E2=80=9Cderivers=E2=80=9D: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> ,use(guix) scheme@(guile-user)> (define s (open-connection)) scheme@(guile-user)> (valid-derivers s "/gnu/store/4bs4mx6xyx0jx9827vmmbxyh= jln7cmcv-python-2.7.10") $4 =3D ("/gnu/store/a5s3mkl387wwkklj3ss5hw34ib9rb2p6-python-2.7.10.drv" "/g= nu/store/pvsdsjn15rj1ciwr37vl2vqprvl57szg-python-2.7.10.drv") --8<---------------cut here---------------end--------------->8--- And then run: --8<---------------cut here---------------start------------->8--- $ guix build "/gnu/store/pvsdsjn15rj1ciwr37vl2vqprvl57szg-python-2.7.10.drv= " --check=20=20 --8<---------------cut here---------------end--------------->8--- However that=E2=80=99s not very helpful since it unsurprisingly fails with a =E2=80=9Cmay not be deterministic=E2=80=9D error. Ludo=E2=80=99.