From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id ABOlH3xCwV5eFwAA0tVLHw (envelope-from ) for ; Sun, 17 May 2020 13:56:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id GNCIG3xCwV7bZgAAbx9fmQ (envelope-from ) for ; Sun, 17 May 2020 13:56:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 006F7940510 for ; Sun, 17 May 2020 13:56:12 +0000 (UTC) Received: from localhost ([::1]:39066 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jaJly-0004ka-TX for larch@yhetil.org; Sun, 17 May 2020 09:56:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jaJlq-0004k1-BM for bug-guix@gnu.org; Sun, 17 May 2020 09:56:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:60870) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jaJlq-0002ro-2Z for bug-guix@gnu.org; Sun, 17 May 2020 09:56:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jaJlq-0000Mf-0k for bug-guix@gnu.org; Sun, 17 May 2020 09:56:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#22304: =?UTF-8?Q?[Nicol=C3=B2?= Balzarotti] Re: Julia v1.4.1 still not reproducible In-Reply-To: Resent-From: anothersms@gmail.com (=?UTF-8?Q?Nicol=C3=B2?= Balzarotti) Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 17 May 2020 13:56:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22304 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 22304@debbugs.gnu.org Received: via spool by 22304-submit@debbugs.gnu.org id=B22304.15897237431372 (code B ref 22304); Sun, 17 May 2020 13:56:01 +0000 Received: (at 22304) by debbugs.gnu.org; 17 May 2020 13:55:43 +0000 Received: from localhost ([127.0.0.1]:44178 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jaJlM-0000M1-Ni for submit@debbugs.gnu.org; Sun, 17 May 2020 09:55:42 -0400 Received: from mail-wm1-f46.google.com ([209.85.128.46]:39902) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jaJlK-0000Ea-Sh for 22304@debbugs.gnu.org; Sun, 17 May 2020 09:55:31 -0400 Received: by mail-wm1-f46.google.com with SMTP id w64so7395639wmg.4 for <22304@debbugs.gnu.org>; Sun, 17 May 2020 06:55:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:references:date:message-id:mime-version; bh=q5Gy5B0o70k1xjeMRCnsaGDYHaWRo4Q8hpzTMVK5Rkg=; b=g1CfxFfJxN0P0t7lx1hCb2ZiS1NR1ATMcFSNmmqYztCw40tX52JqJI4TDgTp1aEMbD rB2v5ugTCjyF+3VKVcAvUqbLdYJeRcCaD6Q/uC8SsOFzzSZ98fBXNEwbFUPVWSN2LYLY vcBCtBv8+RAb7vbFwaHVi4uq0FgrhTvmsF5ymWPzJomuz2P9gsxqK07J1irNnOfCmwg0 X06uhNnyy5XnVqibiPcPptM0e/t3qGNbHkrJETHZDpPR4puDJzmjOLT8uFI2+jULuGxK k/M43lrTMnDhjKrr9dyQZDbRuA1ZUt+30BkYAvlSu7LL5SaH662DTo/Kuitcb4nNQf2W bhUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:references:date:message-id :mime-version; bh=q5Gy5B0o70k1xjeMRCnsaGDYHaWRo4Q8hpzTMVK5Rkg=; b=qije+YlvJktF0sWsgD7eFpZCdc0BTW9ylOwak/yPi6w0/bCrbDJj4XQvM+WQ0FDJa6 SsBixIO8xgdtp3lKqThR/DIE6yA8qdYUwIa2HMZqp2+AIQKGdraTNeHKqMk0IriRzW9j X+fG1yZmVcEQnK+9cA1ZFvcNwXHJE97H9uwDMTDS/6XFdnuwRXC0TOyGRimMubZMxf4z 3nzrZrxi37eA8b5OTyHljKKCwdLAPO2REDyisJKwW1i1+mc4ckRmSf27VwRr6owJX7Zs wepE9oaWJeSOg7Gi9HNtIsNkMZIIufPhqC1TxDmH8CJcmFh6KQFxLrVD0gcK7MwdfkES wokQ== X-Gm-Message-State: AOAM533sqorGwXMgUf+olyrSFO/trTWeva4nBzJavEziIcTfmTQwcOy/ zOoKBhYCFh6iJryzGzQihWe1WrB6 X-Google-Smtp-Source: ABdhPJxEYM8PXnIZnEypLI2Oazqon7VpdPIgGFn2HgpWK55e1+BCfNOzf2kXqu3hLc5rKVTnXqdbxQ== X-Received: by 2002:a7b:cc84:: with SMTP id p4mr15091257wma.159.1589723724500; Sun, 17 May 2020 06:55:24 -0700 (PDT) Received: from guixSD (host77-80-dynamic.52-82-r.retail.telecomitalia.it. [82.52.80.77]) by smtp.gmail.com with ESMTPSA id t14sm8146566wrs.1.2020.05.17.06.55.23 for <22304@debbugs.gnu.org> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 May 2020 06:55:23 -0700 (PDT) From: anothersms@gmail.com (=?UTF-8?Q?Nicol=C3=B2?= Balzarotti) References: <87y2pqwtyz.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> Date: Sun, 17 May 2020 15:55:22 +0200 Message-ID: <87v9kuwtrp.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=g1CfxFfJ; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: NvbLS2SpOJUi --=-=-= Content-Type: text/plain Ops, I sent it to 22304@debuugs.gnu.org instead of 22304@debbugs.gnu.org --=-=-= Content-Type: message/rfc822 Content-Disposition: inline From: =?utf-8?Q?Nicol=C3=B2?= Balzarotti To: Efraim Flashner , zimoun Cc: 22304@debuugs.gnu.org Subject: Re: Julia v1.4.1 still not reproducible In-Reply-To: <20200517134039.GF31833@E5400> References: <873688i57c.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> <20200513110416.GE918@E5400> <20200517134039.GF31833@E5400> Date: Sun, 17 May 2020 15:51:00 +0200 Message-ID: <87y2pqwtyz.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="====-=-=" --====-=-= Content-Type: text/plain Content-Disposition: inline Hi Efraim, Simon, The attached patches provide Simon's suggested julia-deterministic build. With this, I get deterministic julia build, *but* only on a single machine. Building on a different machine I get a different output. I still need to investigate why. Also, some of the patches are probably not needed at all. In the near future, as discussed previously, I aim at submitting required patches upstream. I disabled tests in julia-deterministic, so it get built quite fast (~10minutes). If you want to try to see if this is reproducible//know how to figure out what might be happening on different machines, feedback is wellcome! My ouptut is (patches applied on 75741af9b2): guix hash -r /gnu/store/mp8hl7j09nfnsxbb3a6ka23ww44gq0z2-julia-deterministic-1.4.1 1k6mak38nkiaj0b9l6rcmxfcjwvbijmczxs703hgawwi6z0jikxl --====-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-gnu-julia-Specify-correct-llvm-version.patch >From baf3f3853079ae375342f626516945998c15f1ca Mon Sep 17 00:00:00 2001 From: nixo Date: Thu, 14 May 2020 17:17:08 +0200 Subject: [PATCH 1/2] gnu: julia: Specify correct llvm version. --- gnu/packages/julia.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm index c636fa756b..8db5e66131 100644 --- a/gnu/packages/julia.scm +++ b/gnu/packages/julia.scm @@ -416,7 +416,7 @@ libraries. It is also a bit like @code{ldd} and @code{otool -L}.") (assoc-ref %build-inputs "utf8proc") "/include") "USE_SYSTEM_LLVM=1" - "LLVM_VER=6.0.1" + "LLVM_VER=8.0.0" "USE_LLVM_SHLIB=1" "USE_SYSTEM_LIBUNWIND=1" -- 2.26.2 --====-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-gnu-Add-julia-deterministic.patch >From fc0bd3f4e8d83898e64fe8cae8c4c61defa7ba4d Mon Sep 17 00:00:00 2001 From: nixo Date: Sat, 16 May 2020 11:56:26 +0200 Subject: [PATCH 2/2] gnu: Add julia-deterministic. * gnu/packages/julia.scm (julia-deterministic): New variable. --- gnu/packages/julia.scm | 64 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm index 8db5e66131..0b4cb82966 100644 --- a/gnu/packages/julia.scm +++ b/gnu/packages/julia.scm @@ -493,3 +493,67 @@ computing environments. It provides a sophisticated compiler, distributed parallel execution, numerical accuracy, and an extensive mathematical function library.") (license license:expat))) + +;; Deterministic build of the julia executable +(define-public julia-deterministic + (package + (inherit julia) + (name "julia-deterministic") + (source + (origin + (inherit (package-source julia)) + ;; I wrote this patches to get a deterministic build. + ;; We need to discuss the best way to add them upstream. + (patches + (list + (origin ; timefuncs: support SOURCE_DATE_EPOCH for reproducible builds + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/1c56fb0975c7701596b5bdc9ad4e716a3e2fcfc9.patch") + (sha256 (base32 "0zw5554s0k38kgq8bgyxcjflrbjk6xqfrh8kidwsh6zvmpkmgnf5"))) + (origin ; don't use jl_hrtime to store module build_id + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/c13691df6747849dec8f654180dabc55e8044a64.patch") + (sha256 (base32 "0n7mnbwzr411j9z557jrcib5vr8v8mgqbj5xdrxv7b2n0mrfcp5d"))) + (origin ; contrib: generate_precompile: do not use random temporary files + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/ae2929a5d3d1f724b421a7f04b6c7b84c9987583.patch") + (sha256 (base32 "00vpj4m9f4wzfkvd4flarki58jbfhi0zngw35zrhaph53mfi6y27"))) + ;; (origin ; base: Base: deterministic random + ;; (method url-fetch) + ;; (uri "https://github.com/nico202/julia/commit/49a4edc26e9db110586c590c17df276301c344a7.patch") + ;; (sha256 (base32 "16p4axwr3shf2jp6hq8dk19c9s1l667ki5a8bv3dbv5b0ji9w7ra"))) + (origin ; remove some use of time() + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/5561587256b29761fe3a7d8fac169b4305afa240.patch") + (sha256 (base32 "0s9b90x61jjrc057myqdck52yj4lb4xlm7ypjn5qvpypnxrbkz6b"))) + (origin ; contrib: generate_precompile: bare minimum + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/5cd454695e7f6a7e448c19d8f9b2d41fcdeb812e.patch") + (sha256 (base32 "0c9pz9ligd6wfhvra5wn7k2k2qr1zw3x67b2y9dhf108xf4z79sm"))) + (origin ; src: codegen: don't dump time to precompile stream + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/5445b5dc2ea30f2f39acb0fb45ba552122a7e6f5.patch") + (sha256 (base32 "0sa1a73ajk8cm3a5l25hyc2bg8kq6l16r2vvdn2qbcnpnjff9skx"))) + (origin ; do not import repl + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/9a45d44ba387d7386b14abfe76e18f1f000eecc2.patch") + (sha256 (base32 "0k6yyxr8rd3dcj8rh32pjzlyc0izvi34xglmxsdhqq6q3j0rf0wb"))) + (origin ; no more interactive + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/e78f791e048ed5192d6cb3166fe089207e2e994d.patch") + (sha256 (base32 "1bvgiishhc0csypymknff46cik5dnanyx4q5kgipjk0a47wi80x2"))) + (origin ; generate precompile: sort modules + (method url-fetch) + (uri "https://github.com/nico202/julia/commit/e5359e1ce383d549db87659ecd72dcb615eb649d.patch") + (sha256 (base32 "0slw4jczba88hpqyd0gq9x7jnw75yvq8wmbrl7dd8f83jmci8lzz"))))))) + (arguments + (substitute-keyword-arguments (package-arguments julia) + ((#:phases phases '%standard-phases) + `(modify-phases ,phases + (delete 'check) + (add-after 'unpack 'generic-target + (lambda _ + ;; Not sure is needed/is the best target + (substitute* "contrib/generate_precompile.jl" + (("native") "generic")) + #t)))))))) -- 2.26.2 --====-=-= Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Thanks, Nicol=C3=B2 Efraim Flashner writes: > On Wed, May 13, 2020 at 08:43:08PM +0200, zimoun wrote: >> Hi Efraim, >>=20 >> On Wed, 13 May 2020 at 13:04, Efraim Flashner wr= ote: >>=20 >> > guix build: error: derivation `/gnu/store/z21c12ijspavaclcins5pja8z1s8= wbm0-julia-1.4.1.drv' may not be deterministic: output `/gnu/store/ri931fk8= hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1' differs from ?/gnu/store/ri931fk8hf5w= vagk76zhmwbphq2pwgnp-julia-1.4.1-check? >> > >> > I built it on bayfront twice. I'll get the diffoscope output once it >> > finally builds. > > I wasn't able to provide the output of diffoscope, it turns out that the > output was too large to attach. > > --=20 > Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7= =9D =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 > GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 > Confidentiality cannot be guaranteed on emails sent or received unencrypt= ed --====-=-=-- --=-=-=--