* bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible
@ 2021-11-20 15:36 Ricardo Wurmus
2021-11-20 16:26 ` Maxime Devos
2021-12-10 22:42 ` Ludovic Courtès
0 siblings, 2 replies; 3+ messages in thread
From: Ricardo Wurmus @ 2021-11-20 15:36 UTC (permalink / raw)
To: 52001
Numpy accelerates operations with native code. Early in the build
it probes the CPU / compiler for supported instruction extensions
and then proceeds to build extensions for different features. I’m
guessing that this won’t work when building on older CPUs, so the
results of two builds on different CPUs may differ.
It would be helpful if someone could confirm this by building
numpy on an older CPU and then compare the build result with that
provided by ci.guix.gnu.org.
--
Ricardo
^ permalink raw reply [flat|nested] 3+ messages in thread
* bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible
2021-11-20 15:36 bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible Ricardo Wurmus
@ 2021-11-20 16:26 ` Maxime Devos
2021-12-10 22:42 ` Ludovic Courtès
1 sibling, 0 replies; 3+ messages in thread
From: Maxime Devos @ 2021-11-20 16:26 UTC (permalink / raw)
To: Ricardo Wurmus, 52001
Ricardo Wurmus schreef op za 20-11-2021 om 15:36 [+0000]:
> Numpy accelerates operations with native code. Early in the build
> it probes the CPU / compiler for supported instruction extensions
> and then proceeds to build extensions for different features. I’m
> guessing that this won’t work when building on older CPUs, so the
> results of two builds on different CPUs may differ.
>
> It would be helpful if someone could confirm this by building
> numpy on an older CPU and then compare the build result with that
> provided by ci.guix.gnu.org.
>
I built it locally, and have the same result as ci.guix.gnu.org.
However, the bordeaux substitute is different:
$ LANGUAGE=en guix challenge python-numpy
/gnu/store/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3
contents differ:
local hash: 07var0wn8fywxchldz5pjqpnlavrlbc8s81aqwsqyi0i7qlh6ka7
https://ci.guix.gnu.org/nar/lzip/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3
: 07var0wn8fywxchldz5pjqpnlavrlbc8s81aqwsqyi0i7qlh6ka7
https://bordeaux.guix.gnu.org/nar/lzip/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3
: 0cbl3q19bshb6ddz8xkcrjzkcmillsqii4z852ybzixyp7rg40qa
bordeaux.guix.gnu.org 3.9MiB 6.7MiB/s
00:01 [##################] 100.0%
1 store items were analyzed:
- 0 (0.0%) were identical
- 1 (100.0%) differed
- 0 (0.0%) were inconclusive
Diffoscope output:
bordeaux.guix.gnu.org 3.9MiB 4.2MiB/s 00:01 [##################] 100.0%
[...]
│ │ │ │ │ │ │ │ --- /tmp/guix-directory.6ivbVt/lib/python3.8/site-packages/numpy/distutils/fcompiler/__pycache__/vast.cpython-38.pyc
│ │ │ │ │ │ │ ├── +++ /gnu/store/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/distutils/fcompiler/__pycache__/vast.cpython-38.pyc
│ │ │ │ │ │ │ │ @@ -106,15 +106,15 @@
│ │ │ │ │ │ │ │ 00000690: 0104 0106 0104 0104 0106 0104 f906 0904 ................
│ │ │ │ │ │ │ │ 000006a0: 0104 0208 0308 06da 085f 5f6d 6169 6e5f .........__main_
│ │ │ │ │ │ │ │ 000006b0: 5f29 01da 036c 6f67 e902 0000 0029 01da _)...log.....)..
│ │ │ │ │ │ │ │ 000006c0: 1463 7573 746f 6d69 7a65 645f 6663 6f6d .customized_fcom
│ │ │ │ │ │ │ │ 000006d0: 7069 6c65 7272 0700 0000 2901 da08 636f pilerr....)...co
│ │ │ │ │ │ │ │ 000006e0: 6d70 696c 6572 2911 da0a 5f5f 6675 7475 mpiler)...__futu
│ │ │ │ │ │ │ │ 000006f0: 7265 5f5f 7202 0000 0072 0300 0000 7204 re__r....r....r.
│ │ │ │ │ │ │ │ -00000700: 0000 0072 1900 0000 da1d 6e75 6d70 792e ...r......numpy.
│ │ │ │ │ │ │ │ +00000700: 0000 0072 1900 0000 5a1d 6e75 6d70 792e ...r....Z.numpy.
│ │ │ │ │ │ │ │ 00000710: 6469 7374 7574 696c 732e 6663 6f6d 7069 distutils.fcompi
│ │ │ │ │ │ │ │ 00000720: 6c65 722e 676e 7572 0500 0000 da09 636f ler.gnur......co
│ │ │ │ │ │ │ │ 00000730: 6d70 696c 6572 7372 0600 0000 7227 0000 mpilersr....r'..
│ │ │ │ │ │ │ │ 00000740: 00da 0964 6973 7475 7469 6c73 7233 0000 ...distutilsr3..
│ │ │ │ │ │ │ │ 00000750: 00da 0d73 6574 5f76 6572 626f 7369 7479 ...set_verbosity
│ │ │ │ │ │ │ │ 00000760: da0f 6e75 6d70 792e 6469 7374 7574 696c ..numpy.distutil
│ │ │ │ │ │ │ │ 00000770: 7372 3500 0000 da05 7072 696e 7472 2000 sr5.....printr .
1 bit is different.
Greetings,
Maxime.
^ permalink raw reply [flat|nested] 3+ messages in thread
* bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible
2021-11-20 15:36 bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible Ricardo Wurmus
2021-11-20 16:26 ` Maxime Devos
@ 2021-12-10 22:42 ` Ludovic Courtès
1 sibling, 0 replies; 3+ messages in thread
From: Ludovic Courtès @ 2021-12-10 22:42 UTC (permalink / raw)
To: Ricardo Wurmus; +Cc: 52001
Hi,
Ricardo Wurmus <rekado@elephly.net> skribis:
> Numpy accelerates operations with native code. Early in the build it
> probes the CPU / compiler for supported instruction extensions and
> then proceeds to build extensions for different features. I’m
> guessing that this won’t work when building on older CPUs, so the
> results of two builds on different CPUs may differ.
I don’t see anything in the build log that suggests it might be building
for a specific ISA extension (on x86_64).
On the contrary, it seems to do the right thing by dispatching to the
right optimized implementation at run time, in numpy/core/src/umath.
Apparently this directory contains templates for home-made function
multi-versioning, with variants and dispatch code generated by
numpy/core/code_generators/generate_umath.py.
Besides, I see the same .pyc difference as Maxime, but I don’t think it
relates to ISA extensions.
Ludo’.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-12-10 22:43 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-20 15:36 bug#52001: numpy CPU dispatch probably prevents builds of python-numpy from being reproducible Ricardo Wurmus
2021-11-20 16:26 ` Maxime Devos
2021-12-10 22:42 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).