* bug#57005: PETSc is not reproducible
@ 2022-08-05 14:51 Ludovic Courtès
2024-02-29 9:46 ` bug#57005: [PATCH v1] gnu: petsc: scrub build machine specifics Lars Bilke
` (4 more replies)
0 siblings, 5 replies; 9+ messages in thread
From: Ludovic Courtès @ 2022-08-05 14:51 UTC (permalink / raw)
To: 57005
PETSc 3.16.1 is not reproducible as it captures the kernel version and
number of processors of the machine it’s built on:
--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 223 Aug 01 2022 00:30:24 (current)
guix ab59155
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: ab59155c5a38dda7efaceb47c7528578fcf0def4
$ guix challenge petsc --diff="$(type -P diff) -ru"
/gnu/store/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1 contents differ:
no local build for '/gnu/store/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1'
https://ci.guix.gnu.org/nar/lzip/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1: 1fqg24x0qq0vww8a21a4ifyxlwy0l7bg8zzrzvg024lk76n54kxh
https://bordeaux.guix.gnu.org/nar/lzip/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1: 162fp2wsxxjjl1pd42522jb3i37hpq8lnfs12qaipzhkzqda5h34
bordeaux.guix.gnu.org 10.3MiB 9.7MiB/s 00:01 [##################] 100.0%diff -ru /tmp/guix-directory.4kiUDI/include/petscmachineinfo.h /tmp/guix-directory.0i2wFi/include/petscmachineinfo.h
--- /tmp/guix-directory.4kiUDI/include/petscmachineinfo.h 1970-01-01 01:00:01.000000000 +0100
+++ /tmp/guix-directory.0i2wFi/include/petscmachineinfo.h 1970-01-01 01:00:01.000000000 +0100
@@ -1,6 +1,6 @@
static const char *petscmachineinfo = "\n"
"-----------------------------------------\n"
-"Machine characteristics: Linux-5.15.10-gnu-x86_64-with-glibc2.33\n"
+"Machine characteristics: Linux-5.15.11-gnu-x86_64-with-glibc2.33\n"
"Using PETSc directory: /gnu/store/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1\n"
"Using PETSc arch: \n"
"-----------------------------------------\n";
Binary files /tmp/guix-directory.4kiUDI/lib/libpetsc.so and /tmp/guix-directory.0i2wFi/lib/libpetsc.so differ
Binary files /tmp/guix-directory.4kiUDI/lib/libpetsc.so.3.16 and /tmp/guix-directory.0i2wFi/lib/libpetsc.so.3.16 differ
Binary files /tmp/guix-directory.4kiUDI/lib/libpetsc.so.3.16.1 and /tmp/guix-directory.0i2wFi/lib/libpetsc.so.3.16.1 differ
diff -ru /tmp/guix-directory.4kiUDI/lib/petsc/conf/petscvariables /tmp/guix-directory.0i2wFi/lib/petsc/conf/petscvariables
--- /tmp/guix-directory.4kiUDI/lib/petsc/conf/petscvariables 1970-01-01 01:00:01.000000000 +0100
+++ /tmp/guix-directory.0i2wFi/lib/petsc/conf/petscvariables 1970-01-01 01:00:01.000000000 +0100
@@ -107,10 +107,10 @@
OMAKE_PRINTDIR = /gnu/store/55cbpsi18mahg131nmiya6km5b4mscfa-make-4.3/bin/make --print-directory
OMAKE = /gnu/store/55cbpsi18mahg131nmiya6km5b4mscfa-make-4.3/bin/make --no-print-directory
MAKE_PAR_OUT_FLG = --output-sync=recurse
-MAKE_NP = 40
-MAKE_TEST_NP = 40
-MAKE_LOAD = 96.0
-NPMAX = 64
+MAKE_NP = 24
+MAKE_TEST_NP = 24
+MAKE_LOAD = 48.0
+NPMAX = 32
SONAME_FUNCTION = $(1).$(SL_LINKER_SUFFIX).$(2)
SL_LINKER_FUNCTION = -shared -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g3 -O0 -Wl,-soname,$(call SONAME_FUNCTION,$(notdir $(1)),$(2))
BUILDSHAREDLIB = yes
1 store items were analyzed:
- 0 (0.0%) were identical
- 1 (100.0%) differed
- 0 (0.0%) were inconclusive
--8<---------------cut here---------------end--------------->8---
Ludo’.
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#57005: [PATCH v1] gnu: petsc: scrub build machine specifics
2022-08-05 14:51 bug#57005: PETSc is not reproducible Ludovic Courtès
@ 2024-02-29 9:46 ` Lars Bilke
2024-02-29 10:00 ` Lars Bilke
2024-02-29 17:48 ` bug#57005: Suggestion Andreas Enge
` (3 subsequent siblings)
4 siblings, 1 reply; 9+ messages in thread
From: Lars Bilke @ 2024-02-29 9:46 UTC (permalink / raw)
To: 57005; +Cc: Lars Bilke, Andreas Enge, Eric Bavier, Sharlatan Hellseher
'Libraries compiled on' does not have to be removed since PETSc 3.9:
https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
---
gnu/packages/maths.scm | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 1f61ee0583..f09cbee48b 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3455,18 +3455,21 @@ (define-public petsc
(substitute* "config/example_template.py"
(("#!/usr/bin/env bash")
(string-append "#!" (which "bash")))))))
- (add-after 'configure 'clean-local-references
+ (add-after 'install 'clean-local-references
(lambda* (#:key outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
(substitute* (find-files "." "^petsc(conf|machineinfo).h$")
;; Prevent build directory from leaking into compiled code
(((getcwd)) out)
- ;; Scrub timestamp for reproducibility
- ((".*Libraries compiled on.*") ""))
+ ;; Scrub build machine characteristic
+ (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
+ "Machine characteristics: Linux-x.x.x"))
(substitute* (find-files "." "petscvariables")
;; Do not expose build machine characteristics, set to defaults.
- (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
- (("NPMAX = [:digit:]+") "NPMAX = 2")))))
+ (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
+ (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
+ (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
+ (("NPMAX = [[:digit:]]+") "NPMAX = 2")))))
(add-after 'install 'clean-install
;; Try to keep installed files from leaking build directory names.
(lambda* (#:key inputs outputs #:allow-other-keys)
base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* bug#57005: [PATCH v1] gnu: petsc: scrub build machine specifics
2024-02-29 9:46 ` bug#57005: [PATCH v1] gnu: petsc: scrub build machine specifics Lars Bilke
@ 2024-02-29 10:00 ` Lars Bilke
0 siblings, 0 replies; 9+ messages in thread
From: Lars Bilke @ 2024-02-29 10:00 UTC (permalink / raw)
To: 57005
[-- Attachment #1.1: Type: text/plain, Size: 2691 bytes --]
This patch makes these files deterministic:
- petscmachineinfo.h
- petscvariables
Unfortunately libpetsc.so.3.16.1 still differs between builds, see e.g. this diffoscope output:
https://jsfiddle.net/dLx15obp/show (click on run)
Maybe someone has an idea how to also fix the library?
Sincerely,
Lars
On 29 Feb 2024, at 10:46, Lars Bilke wrote:
> 'Libraries compiled on' does not have to be removed since PETSc 3.9:
>
> https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
>
> Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
>
> ---
> gnu/packages/maths.scm | 13 ++++++++-----
> 1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
> index 1f61ee0583..f09cbee48b 100644
> --- a/gnu/packages/maths.scm
> +++ b/gnu/packages/maths.scm
> @@ -3455,18 +3455,21 @@ (define-public petsc
> (substitute* "config/example_template.py"
> (("#!/usr/bin/env bash")
> (string-append "#!" (which "bash")))))))
> - (add-after 'configure 'clean-local-references
> + (add-after 'install 'clean-local-references
> (lambda* (#:key outputs #:allow-other-keys)
> (let ((out (assoc-ref outputs "out")))
> (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
> ;; Prevent build directory from leaking into compiled code
> (((getcwd)) out)
> - ;; Scrub timestamp for reproducibility
> - ((".*Libraries compiled on.*") ""))
> + ;; Scrub build machine characteristic
> + (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
> + "Machine characteristics: Linux-x.x.x"))
> (substitute* (find-files "." "petscvariables")
> ;; Do not expose build machine characteristics, set to defaults.
> - (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
> - (("NPMAX = [:digit:]+") "NPMAX = 2")))))
> + (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
> + (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
> + (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
> + (("NPMAX = [[:digit:]]+") "NPMAX = 2")))))
> (add-after 'install 'clean-install
> ;; Try to keep installed files from leaking build directory names.
> (lambda* (#:key inputs outputs #:allow-other-keys)
>
> base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
> --
> 2.43.0
[-- Attachment #1.2: Type: text/html, Size: 3724 bytes --]
[-- Attachment #2: S/MIME digital signature --]
[-- Type: application/pkcs7-signature, Size: 5435 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#57005: Suggestion
2022-08-05 14:51 bug#57005: PETSc is not reproducible Ludovic Courtès
2024-02-29 9:46 ` bug#57005: [PATCH v1] gnu: petsc: scrub build machine specifics Lars Bilke
@ 2024-02-29 17:48 ` Andreas Enge
2024-03-01 8:27 ` bug#57005: [PATCH v2] gnu: petsc: scrub build machine specifics Lars Bilke
` (2 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Andreas Enge @ 2024-02-29 17:48 UTC (permalink / raw)
To: 57005
Hello Lars,
thanks for the patch! Concerning "Machine characteristics", I see it only
in petscmachineinfo.h, so we can probably drop looking for petscconf.h
in find-files.
There is already the cleaning phase 'clean-install after 'install.
So I would either suggest to keep the phase 'clean-local-references
where it is, or to merge its content with 'clean-install (I do not see
why one should be preferred over the other).
Andreas
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#57005: [PATCH v2] gnu: petsc: scrub build machine specifics
2022-08-05 14:51 bug#57005: PETSc is not reproducible Ludovic Courtès
2024-02-29 9:46 ` bug#57005: [PATCH v1] gnu: petsc: scrub build machine specifics Lars Bilke
2024-02-29 17:48 ` bug#57005: Suggestion Andreas Enge
@ 2024-03-01 8:27 ` Lars Bilke
2024-03-05 14:58 ` bug#57005: [PATCH v3] " Lars Bilke
2024-03-06 11:51 ` bug#57005: [PATCH v4] " Lars Bilke
4 siblings, 0 replies; 9+ messages in thread
From: Lars Bilke @ 2024-03-01 8:27 UTC (permalink / raw)
To: 57005; +Cc: Lars Bilke, Andreas Enge, Eric Bavier, Sharlatan Hellseher
'Libraries compiled on' does not have to be removed since PETSc 3.9:
https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
Also merged all add-after 'install phases into `clean-install.
Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
---
gnu/packages/maths.scm | 43 ++++++++++++++++++------------------------
1 file changed, 18 insertions(+), 25 deletions(-)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 1f61ee0583..4b4d033506 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3455,30 +3455,34 @@ (define-public petsc
(substitute* "config/example_template.py"
(("#!/usr/bin/env bash")
(string-append "#!" (which "bash")))))))
- (add-after 'configure 'clean-local-references
- (lambda* (#:key outputs #:allow-other-keys)
+ (add-after 'install 'clean-install
+ ;; Try to keep installed files from leaking build directory names.
+ (lambda* (#:key inputs outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
- (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
+ (substitute* (find-files "." "petscmachineinfo.h")
;; Prevent build directory from leaking into compiled code
(((getcwd)) out)
- ;; Scrub timestamp for reproducibility
- ((".*Libraries compiled on.*") ""))
+ ;; Scrub build machine characteristic
+ (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
+ "Machine characteristics: Linux-x.x.x"))
(substitute* (find-files "." "petscvariables")
;; Do not expose build machine characteristics, set to defaults.
- (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
- (("NPMAX = [:digit:]+") "NPMAX = 2")))))
- (add-after 'install 'clean-install
- ;; Try to keep installed files from leaking build directory names.
- (lambda* (#:key inputs outputs #:allow-other-keys)
- (let ((out (assoc-ref outputs "out")))
+ (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
+ (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
+ (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
+ (("NPMAX = [[:digit:]]+") "NPMAX = 2"))
(substitute* (map (lambda (file)
(string-append out "/lib/petsc/conf/" file))
'("petscvariables"))
(((getcwd)) out))
- ;; Make compiler references point to the store
(substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("= (gcc|g\\+\\+|gfortran)" _ compiler)
- (string-append "= " (which compiler))))
+ ;; Do not retain a reference to GCC and other build only inputs.
+ (("([[:graph:]]+)/bin/gcc") "gcc")
+ (("([[:graph:]]+)/bin/g\\+\\+") "g++")
+ (("([[:graph:]]+)/bin/make") "make")
+ (("([[:graph:]]+)/bin/diff") "diff")
+ (("([[:graph:]]+)/bin/sed") "sed")
+ (("([[:graph:]]+)/bin/gfortran") "gfortran"))
;; PETSc installs some build logs, which aren't necessary.
(for-each (lambda (file)
(let ((f (string-append out "/lib/petsc/conf/" file)))
@@ -3490,17 +3494,6 @@ (define-public petsc
"configure-hash"
;; Once installed, should uninstall with Guix
"uninstall.py")))))
- (add-after 'clean-install 'clear-reference-to-compiler
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Do not retain a reference to GCC and other build only inputs.
- (let ((out (assoc-ref outputs "out")))
- (substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("([[:graph:]]+)/bin/gcc") "gcc")
- (("([[:graph:]]+)/bin/g\\+\\+") "g++")
- (("([[:graph:]]+)/bin/make") "make")
- (("([[:graph:]]+)/bin/diff") "diff")
- (("([[:graph:]]+)/bin/sed") "sed")
- (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
(add-after 'install 'move-examples
(lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))
base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* bug#57005: [PATCH v3] gnu: petsc: scrub build machine specifics
2022-08-05 14:51 bug#57005: PETSc is not reproducible Ludovic Courtès
` (2 preceding siblings ...)
2024-03-01 8:27 ` bug#57005: [PATCH v2] gnu: petsc: scrub build machine specifics Lars Bilke
@ 2024-03-05 14:58 ` Lars Bilke
2024-03-05 15:01 ` Lars Bilke
2024-03-06 11:51 ` bug#57005: [PATCH v4] " Lars Bilke
4 siblings, 1 reply; 9+ messages in thread
From: Lars Bilke @ 2024-03-05 14:58 UTC (permalink / raw)
To: 57005; +Cc: Lars Bilke, Andreas Enge, Eric Bavier, Sharlatan Hellseher
'Libraries compiled on' does not have to be removed since PETSc 3.9:
https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
Also merged all add-after 'install phases into `clean-install.
Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
sort source files in generated makefiles for reproducibility
Change-Id: I86e5b9cd0d7184019617bd8ae9a23eaf459891d7
---
gnu/packages/maths.scm | 46 +++++++++++++++++++-----------------------
1 file changed, 21 insertions(+), 25 deletions(-)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 1f61ee0583..b828d5d5cb 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3447,6 +3447,9 @@ (define-public petsc
(let* ((prefix (assoc-ref outputs "out"))
(flags `(,(string-append "--prefix=" prefix)
,@configure-flags)))
+ ;; sort source files in configure (for reproducibility)
+ (substitute* "config/gmakegen.py"
+ (("join\\(srcs\\[lang\\]\\)") "join(sorted(srcs[lang]))"))
(format #t "build directory: ~s~%" (getcwd))
(format #t "configure flags: ~s~%" flags)
(apply invoke "./configure" flags)
@@ -3455,30 +3458,34 @@ (define-public petsc
(substitute* "config/example_template.py"
(("#!/usr/bin/env bash")
(string-append "#!" (which "bash")))))))
- (add-after 'configure 'clean-local-references
- (lambda* (#:key outputs #:allow-other-keys)
+ (add-after 'install 'clean-install
+ ;; Try to keep installed files from leaking build directory names.
+ (lambda* (#:key inputs outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
- (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
+ (substitute* (find-files "." "petscmachineinfo.h")
;; Prevent build directory from leaking into compiled code
(((getcwd)) out)
- ;; Scrub timestamp for reproducibility
- ((".*Libraries compiled on.*") ""))
+ ;; Scrub build machine characteristic
+ (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
+ "Machine characteristics: Linux-x.x.x"))
(substitute* (find-files "." "petscvariables")
;; Do not expose build machine characteristics, set to defaults.
- (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
- (("NPMAX = [:digit:]+") "NPMAX = 2")))))
- (add-after 'install 'clean-install
- ;; Try to keep installed files from leaking build directory names.
- (lambda* (#:key inputs outputs #:allow-other-keys)
- (let ((out (assoc-ref outputs "out")))
+ (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
+ (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
+ (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
+ (("NPMAX = [[:digit:]]+") "NPMAX = 2"))
(substitute* (map (lambda (file)
(string-append out "/lib/petsc/conf/" file))
'("petscvariables"))
(((getcwd)) out))
- ;; Make compiler references point to the store
(substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("= (gcc|g\\+\\+|gfortran)" _ compiler)
- (string-append "= " (which compiler))))
+ ;; Do not retain a reference to GCC and other build only inputs.
+ (("([[:graph:]]+)/bin/gcc") "gcc")
+ (("([[:graph:]]+)/bin/g\\+\\+") "g++")
+ (("([[:graph:]]+)/bin/make") "make")
+ (("([[:graph:]]+)/bin/diff") "diff")
+ (("([[:graph:]]+)/bin/sed") "sed")
+ (("([[:graph:]]+)/bin/gfortran") "gfortran"))
;; PETSc installs some build logs, which aren't necessary.
(for-each (lambda (file)
(let ((f (string-append out "/lib/petsc/conf/" file)))
@@ -3490,17 +3497,6 @@ (define-public petsc
"configure-hash"
;; Once installed, should uninstall with Guix
"uninstall.py")))))
- (add-after 'clean-install 'clear-reference-to-compiler
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Do not retain a reference to GCC and other build only inputs.
- (let ((out (assoc-ref outputs "out")))
- (substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("([[:graph:]]+)/bin/gcc") "gcc")
- (("([[:graph:]]+)/bin/g\\+\\+") "g++")
- (("([[:graph:]]+)/bin/make") "make")
- (("([[:graph:]]+)/bin/diff") "diff")
- (("([[:graph:]]+)/bin/sed") "sed")
- (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
(add-after 'install 'move-examples
(lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))
base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* bug#57005: [PATCH v3] gnu: petsc: scrub build machine specifics
2024-03-05 14:58 ` bug#57005: [PATCH v3] " Lars Bilke
@ 2024-03-05 15:01 ` Lars Bilke
0 siblings, 0 replies; 9+ messages in thread
From: Lars Bilke @ 2024-03-05 15:01 UTC (permalink / raw)
To: 57005; +Cc: andreas, Ludovic Courtès
[-- Attachment #1: Type: text/plain, Size: 5824 bytes --]
v3 of this patch finally makes PETSc fully reproducible!
On 5 Mar 2024, at 15:58, Lars Bilke wrote:
> 'Libraries compiled on' does not have to be removed since PETSc 3.9:
>
> https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
>
> Also merged all add-after 'install phases into `clean-install.
>
> Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
>
> sort source files in generated makefiles for reproducibility
>
> Change-Id: I86e5b9cd0d7184019617bd8ae9a23eaf459891d7
> ---
> gnu/packages/maths.scm | 46 +++++++++++++++++++-----------------------
> 1 file changed, 21 insertions(+), 25 deletions(-)
>
> diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
> index 1f61ee0583..b828d5d5cb 100644
> --- a/gnu/packages/maths.scm
> +++ b/gnu/packages/maths.scm
> @@ -3447,6 +3447,9 @@ (define-public petsc
> (let* ((prefix (assoc-ref outputs "out"))
> (flags `(,(string-append "--prefix=" prefix)
> ,@configure-flags)))
> + ;; sort source files in configure (for reproducibility)
> + (substitute* "config/gmakegen.py"
> + (("join\\(srcs\\[lang\\]\\)") "join(sorted(srcs[lang]))"))
> (format #t "build directory: ~s~%" (getcwd))
> (format #t "configure flags: ~s~%" flags)
> (apply invoke "./configure" flags)
> @@ -3455,30 +3458,34 @@ (define-public petsc
> (substitute* "config/example_template.py"
> (("#!/usr/bin/env bash")
> (string-append "#!" (which "bash")))))))
> - (add-after 'configure 'clean-local-references
> - (lambda* (#:key outputs #:allow-other-keys)
> + (add-after 'install 'clean-install
> + ;; Try to keep installed files from leaking build directory names.
> + (lambda* (#:key inputs outputs #:allow-other-keys)
> (let ((out (assoc-ref outputs "out")))
> - (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
> + (substitute* (find-files "." "petscmachineinfo.h")
> ;; Prevent build directory from leaking into compiled code
> (((getcwd)) out)
> - ;; Scrub timestamp for reproducibility
> - ((".*Libraries compiled on.*") ""))
> + ;; Scrub build machine characteristic
> + (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
> + "Machine characteristics: Linux-x.x.x"))
> (substitute* (find-files "." "petscvariables")
> ;; Do not expose build machine characteristics, set to defaults.
> - (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
> - (("NPMAX = [:digit:]+") "NPMAX = 2")))))
> - (add-after 'install 'clean-install
> - ;; Try to keep installed files from leaking build directory names.
> - (lambda* (#:key inputs outputs #:allow-other-keys)
> - (let ((out (assoc-ref outputs "out")))
> + (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
> + (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
> + (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
> + (("NPMAX = [[:digit:]]+") "NPMAX = 2"))
> (substitute* (map (lambda (file)
> (string-append out "/lib/petsc/conf/" file))
> '("petscvariables"))
> (((getcwd)) out))
> - ;; Make compiler references point to the store
> (substitute* (string-append out "/lib/petsc/conf/petscvariables")
> - (("= (gcc|g\\+\\+|gfortran)" _ compiler)
> - (string-append "= " (which compiler))))
> + ;; Do not retain a reference to GCC and other build only inputs.
> + (("([[:graph:]]+)/bin/gcc") "gcc")
> + (("([[:graph:]]+)/bin/g\\+\\+") "g++")
> + (("([[:graph:]]+)/bin/make") "make")
> + (("([[:graph:]]+)/bin/diff") "diff")
> + (("([[:graph:]]+)/bin/sed") "sed")
> + (("([[:graph:]]+)/bin/gfortran") "gfortran"))
> ;; PETSc installs some build logs, which aren't necessary.
> (for-each (lambda (file)
> (let ((f (string-append out "/lib/petsc/conf/" file)))
> @@ -3490,17 +3497,6 @@ (define-public petsc
> "configure-hash"
> ;; Once installed, should uninstall with Guix
> "uninstall.py")))))
> - (add-after 'clean-install 'clear-reference-to-compiler
> - (lambda* (#:key inputs outputs #:allow-other-keys)
> - ;; Do not retain a reference to GCC and other build only inputs.
> - (let ((out (assoc-ref outputs "out")))
> - (substitute* (string-append out "/lib/petsc/conf/petscvariables")
> - (("([[:graph:]]+)/bin/gcc") "gcc")
> - (("([[:graph:]]+)/bin/g\\+\\+") "g++")
> - (("([[:graph:]]+)/bin/make") "make")
> - (("([[:graph:]]+)/bin/diff") "diff")
> - (("([[:graph:]]+)/bin/sed") "sed")
> - (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
> (add-after 'install 'move-examples
> (lambda* (#:key outputs #:allow-other-keys)
> (let* ((out (assoc-ref outputs "out"))
>
> base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
> --
> 2.43.0
[-- Attachment #2: S/MIME digital signature --]
[-- Type: application/pkcs7-signature, Size: 5435 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#57005: [PATCH v4] gnu: petsc: scrub build machine specifics
2022-08-05 14:51 bug#57005: PETSc is not reproducible Ludovic Courtès
` (3 preceding siblings ...)
2024-03-05 14:58 ` bug#57005: [PATCH v3] " Lars Bilke
@ 2024-03-06 11:51 ` Lars Bilke
2024-03-06 11:54 ` Lars Bilke
4 siblings, 1 reply; 9+ messages in thread
From: Lars Bilke @ 2024-03-06 11:51 UTC (permalink / raw)
To: 57005; +Cc: Lars Bilke, Andreas Enge, Eric Bavier, Sharlatan Hellseher
'Libraries compiled on' does not have to be removed since PETSc 3.9:
https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
Also merged all add-after 'install phases into `clean-install.
Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
sort source files in generated makefiles for reproducibility
Change-Id: I86e5b9cd0d7184019617bd8ae9a23eaf459891d7
---
gnu/packages/maths.scm | 56 ++++++++++++++++++------------------------
1 file changed, 24 insertions(+), 32 deletions(-)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 1f61ee0583..1ea5ccee74 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3447,38 +3447,41 @@ (define-public petsc
(let* ((prefix (assoc-ref outputs "out"))
(flags `(,(string-append "--prefix=" prefix)
,@configure-flags)))
+ ;; sort source files in configure (for reproducibility)
+ (substitute* "config/gmakegen.py"
+ (("join\\(srcs\\[lang\\]\\)") "join(sorted(srcs[lang]))"))
(format #t "build directory: ~s~%" (getcwd))
(format #t "configure flags: ~s~%" flags)
(apply invoke "./configure" flags)
-
+ ;; Prevent build directory Linux version from leaking into
+ ;; compiled code
+ (substitute* (find-files "." "petscmachineinfo.h")
+ (((getcwd)) out)
+ ;; Scrub build machine characteristic
+ (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
+ "Machine characteristics: Linux-x.x.x"))
;; Generate test scripts with the right shebang.
(substitute* "config/example_template.py"
(("#!/usr/bin/env bash")
(string-append "#!" (which "bash")))))))
- (add-after 'configure 'clean-local-references
- (lambda* (#:key outputs #:allow-other-keys)
- (let ((out (assoc-ref outputs "out")))
- (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
- ;; Prevent build directory from leaking into compiled code
- (((getcwd)) out)
- ;; Scrub timestamp for reproducibility
- ((".*Libraries compiled on.*") ""))
- (substitute* (find-files "." "petscvariables")
- ;; Do not expose build machine characteristics, set to defaults.
- (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
- (("NPMAX = [:digit:]+") "NPMAX = 2")))))
(add-after 'install 'clean-install
- ;; Try to keep installed files from leaking build directory names.
(lambda* (#:key inputs outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
- (substitute* (map (lambda (file)
- (string-append out "/lib/petsc/conf/" file))
- '("petscvariables"))
- (((getcwd)) out))
- ;; Make compiler references point to the store
(substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("= (gcc|g\\+\\+|gfortran)" _ compiler)
- (string-append "= " (which compiler))))
+ ;; Try to keep installed files from leaking build directory names.
+ (((getcwd)) out)
+ ;; Do not expose build machine characteristics, set to defaults.
+ (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
+ (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
+ (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
+ (("NPMAX = [[:digit:]]+") "NPMAX = 2")
+ ;; Do not retain a reference to GCC and other build only inputs.
+ (("([[:graph:]]+)/bin/gcc") "gcc")
+ (("([[:graph:]]+)/bin/g\\+\\+") "g++")
+ (("([[:graph:]]+)/bin/make") "make")
+ (("([[:graph:]]+)/bin/diff") "diff")
+ (("([[:graph:]]+)/bin/sed") "sed")
+ (("([[:graph:]]+)/bin/gfortran") "gfortran"))
;; PETSc installs some build logs, which aren't necessary.
(for-each (lambda (file)
(let ((f (string-append out "/lib/petsc/conf/" file)))
@@ -3490,17 +3493,6 @@ (define-public petsc
"configure-hash"
;; Once installed, should uninstall with Guix
"uninstall.py")))))
- (add-after 'clean-install 'clear-reference-to-compiler
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Do not retain a reference to GCC and other build only inputs.
- (let ((out (assoc-ref outputs "out")))
- (substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("([[:graph:]]+)/bin/gcc") "gcc")
- (("([[:graph:]]+)/bin/g\\+\\+") "g++")
- (("([[:graph:]]+)/bin/make") "make")
- (("([[:graph:]]+)/bin/diff") "diff")
- (("([[:graph:]]+)/bin/sed") "sed")
- (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
(add-after 'install 'move-examples
(lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))
base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* bug#57005: [PATCH v4] gnu: petsc: scrub build machine specifics
2024-03-06 11:51 ` bug#57005: [PATCH v4] " Lars Bilke
@ 2024-03-06 11:54 ` Lars Bilke
0 siblings, 0 replies; 9+ messages in thread
From: Lars Bilke @ 2024-03-06 11:54 UTC (permalink / raw)
To: 57005
[-- Attachment #1.1: Type: text/plain, Size: 6208 bytes --]
v4 should (hopefully) fix all issues regarding reprocuibility of petsc:
I have tested v4 on multiple machines with exact identical output (ran `guix hash -S nar ...` on the packages) and also tested on one machine with `guix build --check`
On 6 Mar 2024, at 12:51, Lars Bilke wrote:
> 'Libraries compiled on' does not have to be removed since PETSc 3.9:
>
> https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
>
> Also merged all add-after 'install phases into `clean-install.
>
> Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
>
> sort source files in generated makefiles for reproducibility
>
> Change-Id: I86e5b9cd0d7184019617bd8ae9a23eaf459891d7
> ---
> gnu/packages/maths.scm | 56 ++++++++++++++++++------------------------
> 1 file changed, 24 insertions(+), 32 deletions(-)
>
> diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
> index 1f61ee0583..1ea5ccee74 100644
> --- a/gnu/packages/maths.scm
> +++ b/gnu/packages/maths.scm
> @@ -3447,38 +3447,41 @@ (define-public petsc
> (let* ((prefix (assoc-ref outputs "out"))
> (flags `(,(string-append "--prefix=" prefix)
> ,@configure-flags)))
> + ;; sort source files in configure (for reproducibility)
> + (substitute* "config/gmakegen.py"
> + (("join\\(srcs\\[lang\\]\\)") "join(sorted(srcs[lang]))"))
> (format #t "build directory: ~s~%" (getcwd))
> (format #t "configure flags: ~s~%" flags)
> (apply invoke "./configure" flags)
> -
> + ;; Prevent build directory Linux version from leaking into
> + ;; compiled code
> + (substitute* (find-files "." "petscmachineinfo.h")
> + (((getcwd)) out)
> + ;; Scrub build machine characteristic
> + (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
> + "Machine characteristics: Linux-x.x.x"))
> ;; Generate test scripts with the right shebang.
> (substitute* "config/example_template.py"
> (("#!/usr/bin/env bash")
> (string-append "#!" (which "bash")))))))
> - (add-after 'configure 'clean-local-references
> - (lambda* (#:key outputs #:allow-other-keys)
> - (let ((out (assoc-ref outputs "out")))
> - (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
> - ;; Prevent build directory from leaking into compiled code
> - (((getcwd)) out)
> - ;; Scrub timestamp for reproducibility
> - ((".*Libraries compiled on.*") ""))
> - (substitute* (find-files "." "petscvariables")
> - ;; Do not expose build machine characteristics, set to defaults.
> - (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
> - (("NPMAX = [:digit:]+") "NPMAX = 2")))))
> (add-after 'install 'clean-install
> - ;; Try to keep installed files from leaking build directory names.
> (lambda* (#:key inputs outputs #:allow-other-keys)
> (let ((out (assoc-ref outputs "out")))
> - (substitute* (map (lambda (file)
> - (string-append out "/lib/petsc/conf/" file))
> - '("petscvariables"))
> - (((getcwd)) out))
> - ;; Make compiler references point to the store
> (substitute* (string-append out "/lib/petsc/conf/petscvariables")
> - (("= (gcc|g\\+\\+|gfortran)" _ compiler)
> - (string-append "= " (which compiler))))
> + ;; Try to keep installed files from leaking build directory names.
> + (((getcwd)) out)
> + ;; Do not expose build machine characteristics, set to defaults.
> + (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
> + (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
> + (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
> + (("NPMAX = [[:digit:]]+") "NPMAX = 2")
> + ;; Do not retain a reference to GCC and other build only inputs.
> + (("([[:graph:]]+)/bin/gcc") "gcc")
> + (("([[:graph:]]+)/bin/g\\+\\+") "g++")
> + (("([[:graph:]]+)/bin/make") "make")
> + (("([[:graph:]]+)/bin/diff") "diff")
> + (("([[:graph:]]+)/bin/sed") "sed")
> + (("([[:graph:]]+)/bin/gfortran") "gfortran"))
> ;; PETSc installs some build logs, which aren't necessary.
> (for-each (lambda (file)
> (let ((f (string-append out "/lib/petsc/conf/" file)))
> @@ -3490,17 +3493,6 @@ (define-public petsc
> "configure-hash"
> ;; Once installed, should uninstall with Guix
> "uninstall.py")))))
> - (add-after 'clean-install 'clear-reference-to-compiler
> - (lambda* (#:key inputs outputs #:allow-other-keys)
> - ;; Do not retain a reference to GCC and other build only inputs.
> - (let ((out (assoc-ref outputs "out")))
> - (substitute* (string-append out "/lib/petsc/conf/petscvariables")
> - (("([[:graph:]]+)/bin/gcc") "gcc")
> - (("([[:graph:]]+)/bin/g\\+\\+") "g++")
> - (("([[:graph:]]+)/bin/make") "make")
> - (("([[:graph:]]+)/bin/diff") "diff")
> - (("([[:graph:]]+)/bin/sed") "sed")
> - (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
> (add-after 'install 'move-examples
> (lambda* (#:key outputs #:allow-other-keys)
> (let* ((out (assoc-ref outputs "out"))
>
> base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
> --
> 2.43.0
[-- Attachment #1.2: Type: text/html, Size: 7533 bytes --]
[-- Attachment #2: S/MIME digital signature --]
[-- Type: application/pkcs7-signature, Size: 5435 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2024-03-06 11:55 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-05 14:51 bug#57005: PETSc is not reproducible Ludovic Courtès
2024-02-29 9:46 ` bug#57005: [PATCH v1] gnu: petsc: scrub build machine specifics Lars Bilke
2024-02-29 10:00 ` Lars Bilke
2024-02-29 17:48 ` bug#57005: Suggestion Andreas Enge
2024-03-01 8:27 ` bug#57005: [PATCH v2] gnu: petsc: scrub build machine specifics Lars Bilke
2024-03-05 14:58 ` bug#57005: [PATCH v3] " Lars Bilke
2024-03-05 15:01 ` Lars Bilke
2024-03-06 11:51 ` bug#57005: [PATCH v4] " Lars Bilke
2024-03-06 11:54 ` Lars Bilke
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).