From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ricardo Wurmus Subject: bug#22533: Python bytecode reproducibility Date: Sun, 04 Mar 2018 20:18:23 +0100 Message-ID: <871sgz1wg0.fsf@elephly.net> References: <20160202051544.GA11744@jasmine> <87bmqfu44s.fsf@fastmail.com> <87606c23bq.fsf@elephly.net> <874llw101c.fsf@elephly.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:42520) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1esZA2-0006vw-4E for bug-guix@gnu.org; Sun, 04 Mar 2018 14:19:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1esZ9y-0002Et-2h for bug-guix@gnu.org; Sun, 04 Mar 2018 14:19:06 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:36813) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1esZ9x-0002Ep-T0 for bug-guix@gnu.org; Sun, 04 Mar 2018 14:19:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1esZ9x-0004Tt-N3 for bug-guix@gnu.org; Sun, 04 Mar 2018 14:19:01 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-reply-to: <874llw101c.fsf@elephly.net> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: =?UTF-8?Q?G=C3=A1bor?= Boskovits Cc: 22533@debbugs.gnu.org --=-=-= Content-Type: text/plain I have applied this patch locally: --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=1.diff diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 5f701701a..0d1ecc3c6 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -359,8 +359,42 @@ data types.") "Lib/ctypes/test/test_win32.py" ; fails on aarch64 "Lib/test/test_fcntl.py")) ; fails on aarch64 #t)))) - (arguments (substitute-keyword-arguments (package-arguments python-2) - ((#:tests? _) #t))) + (arguments + (substitute-keyword-arguments (package-arguments python-2) + ((#:tests? _) #t) + ((#:phases phases) + `(modify-phases ,phases + (add-after 'unpack 'patch-timestamp-for-pyc-files + (lambda _ + ;; We set DETERMINISTIC_BUILD to only override the mtime when + ;; building with Guix, lest we break auto-compilation in + ;; environments. + (setenv "DETERMINISTIC_BUILD" "1") + (substitute* "Lib/py_compile.py" + (("source_stats\\['mtime'\\]") + "(1 if 'DETERMINISTIC_BUILD' in os.environ else source_stats['mtime'])")) + + ;; Use deterministic hashes for strings, bytes, and datetime + ;; objects. + (setenv "PYTHONHASHSEED" "0") + + ;; Reset mtime when validating bytecode header. + (substitute* "Lib/importlib/_bootstrap_external.py" + (("source_mtime = int\\(source_stats\\['mtime'\\]\\)") + "source_mtime = 1")) + #t)) + (add-after 'unpack 'disable-timestamp-tests + (lambda _ + (substitute* "Lib/test/test_importlib/source/test_file_loader.py" + (("test_bad_marshal") + "disable_test_bad_marshal") + (("test_no_marshal") + "disable_test_no_marshal") + (("test_non_code_marshal") + "disable_test_non_code_marshal")) + #t)) + (add-before 'check 'allow-non-deterministic-compilation + (lambda _ (unsetenv "DETERMINISTIC_BUILD") #t)))))) (native-search-paths (list (search-path-specification (variable "PYTHONPATH") --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable It allows me to build python-six and python-sip reproducibly. It does not fix problems with Python 2, and I haven=E2=80=99t yet tested if it caus= es any new problems. It=E2=80=99s a little worrying that I had to disable three more tests that = I think shouldn=E2=80=99t have failed. What do you think? -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net --=-=-=--