* Re: Tensorflow fixes on core-updates-frozen
2021-12-14 12:39 ` Tensorflow fixes on core-updates-frozen Guillaume Le Vaillant
@ 2021-12-14 13:17 ` Ricardo Wurmus
2021-12-15 7:49 ` Ricardo Wurmus
0 siblings, 1 reply; 4+ messages in thread
From: Ricardo Wurmus @ 2021-12-14 13:17 UTC (permalink / raw)
To: Guillaume Le Vaillant; +Cc: guix-devel
Hi,
> [[PGP Signed Part:Undecided]]
> guix-commits@gnu.org skribis:
>
>> rekado pushed a change to branch core-updates-frozen
>> in repository guix.
>>
>> new d503194 gnu: python2-entrypoints: Add missing input.
>> new 672c7a2 gnu: tensorflow: Do not unpack directory.
>> new 6d3439b gnu: eigen-for-tensorflow: Build with GCC 7.
>
> Hi Ricardo,
>
> Given that the core-update-frozen branch has been merged in master,
> can't these fixes be pushed on master directly?
I accidentally pushed those to core-updates-frozen. I pushed almost the
same commits to the “master” branch afterwards.
The only difference is in the list of inputs in python2-entrypoints. It
uses the new style on “master”.
Unfortunately, this is not enough to build tensorflow. At the very end
we have this problem:
--8<---------------cut here---------------start------------->8---
[ 99%] Building CXX object CMakeFiles/pywrap_tensorflow_internal.dir/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc.o
/gnu/store/vakvgvrb839igv16jkif4lmx11d25jqb-gcc-10.3.0/bin/c++ -DEIGEN_AVOID_STL_ARRAY -DSQLITE_OMIT_LOAD_EXTENSION -DTF_USE_SNAPPY -Dpywrap_tensorflow_internal_EXPORTS -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/external/eigen_archive -I/gnu/store/54dclv4bsbg5894pyh0q5g768113368a-eigen-for-tensorflow-3.3.5-1.fd6845384b86/include/eigen3 -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/gemmlowp/src/gemmlowp -I/gnu/store/jh8w9a3gnkisp8zgkklydhc372vgcsk6-jsoncpp-1.7.3 -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/external/farmhash_archive -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/external/farmhash_archive/util -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/external/highwayhash -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/cub/src/cub -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/external/nsync/public -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/re2/install/include -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/double_conversion/src/double_conversion -I/gnu/store/g88hslvix5j2rkdm13ay7biqyyqfja0f-snappy-1.1.9 -I/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/include/python3.9 -I/gnu/store/wgsmkn68q8h178sqc7ywjcdr330z9rb6-python-numpy-1.20.3/lib/python3.9/site-packages/numpy/core/include -fPIC -fno-exceptions -std=c++11 -fopenmp -O3 -DNDEBUG -fPIC -std=gnu++14 -MD -MT CMakeFiles/pywrap_tensorflow_internal.dir/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc.o -MF CMakeFiles/pywrap_tensorflow_internal.dir/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc.o.d -o CMakeFiles/pywrap_tensorflow_internal.dir/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc.o -c /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:362:1: warning: converting to non-pointer type ‘long int’ from NULL [-Wconversion-null]
362 | };
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc: In function ‘bool tensorflow::{anonymous}::Initialize()’:
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:623:36: error: no match for call to ‘(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [6], <unresolved overloaded function type>, const std::array<int, 3>&)’
623 | compare_types)) {
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: candidate: ‘tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>’
596 | auto register_ufunc = [&](const char* name, PyUFuncGenericFunction fn,
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: no known conversion for argument 2 from ‘<unresolved overloaded function type>’ to ‘PyUFuncGenericFunction’ {aka ‘void (*)(char**, const long int*, const long int*, void*)’}
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:627:36: error: no match for call to ‘(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [10], <unresolved overloaded function type>, const std::array<int, 3>&)’
627 | compare_types)) {
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: candidate: ‘tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>’
596 | auto register_ufunc = [&](const char* name, PyUFuncGenericFunction fn,
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: no known conversion for argument 2 from ‘<unresolved overloaded function type>’ to ‘PyUFuncGenericFunction’ {aka ‘void (*)(char**, const long int*, const long int*, void*)’}
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:630:77: error: no match for call to ‘(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [5], <unresolved overloaded function type>, const std::array<int, 3>&)’
630 | if (!register_ufunc("less", CompareUFunc<Bfloat16LtFunctor>, compare_types)) {
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: candidate: ‘tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>’
596 | auto register_ufunc = [&](const char* name, PyUFuncGenericFunction fn,
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: no known conversion for argument 2 from ‘<unresolved overloaded function type>’ to ‘PyUFuncGenericFunction’ {aka ‘void (*)(char**, const long int*, const long int*, void*)’}
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:634:36: error: no match for call to ‘(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [8], <unresolved overloaded function type>, const std::array<int, 3>&)’
634 | compare_types)) {
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: candidate: ‘tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>’
596 | auto register_ufunc = [&](const char* name, PyUFuncGenericFunction fn,
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: no known conversion for argument 2 from ‘<unresolved overloaded function type>’ to ‘PyUFuncGenericFunction’ {aka ‘void (*)(char**, const long int*, const long int*, void*)’}
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:638:36: error: no match for call to ‘(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [11], <unresolved overloaded function type>, const std::array<int, 3>&)’
638 | compare_types)) {
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: candidate: ‘tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>’
596 | auto register_ufunc = [&](const char* name, PyUFuncGenericFunction fn,
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: no known conversion for argument 2 from ‘<unresolved overloaded function type>’ to ‘PyUFuncGenericFunction’ {aka ‘void (*)(char**, const long int*, const long int*, void*)’}
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:642:36: error: no match for call to ‘(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [14], <unresolved overloaded function type>, const std::array<int, 3>&)’
642 | compare_types)) {
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: candidate: ‘tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>’
596 | auto register_ufunc = [&](const char* name, PyUFuncGenericFunction fn,
| ^
/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc:596:25: note: no known conversion for argument 2 from ‘<unresolved overloaded function type>’ to ‘PyUFuncGenericFunction’ {aka ‘void (*)(char**, const long int*, const long int*, void*)’}
make[2]: *** [CMakeFiles/pywrap_tensorflow_internal.dir/build.make:188: CMakeFiles/pywrap_tensorflow_internal.dir/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build'
make[1]: *** [CMakeFiles/Makefile2:3807: CMakeFiles/pywrap_tensorflow_internal.dir/all] Error 2
make[1]: Leaving directory '/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build'
make: *** [Makefile:139: all] Error 2
--8<---------------cut here---------------end--------------->8---
That’s because Tensorflow 1.9.0 is not meant to be built with Python
3.9. It didn’t even like Python 3.6, which is why we already carry a
few 3.7 compatibility patches. We should figure out how the calling
convention changed and then patch the C++ files.
My changes compared to the “master” branch:
--8<---------------cut here---------------start------------->8---
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 58f7c7c013..9d8bed09f7 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -1901,6 +1901,9 @@ (define-public tensorflow
(mkdir-p "eigen-src")
(copy-recursively (assoc-ref inputs "eigen:src") "eigen-src")
+ (substitute* "tensorflow/tools/ci_build/install/install_pip_packages.sh"
+ (("setuptools==39.1.0") "setuptools"))
+
(substitute* "tensorflow/contrib/cmake/tf_python.cmake"
;; Take protobuf source files from our source package.
(("\\$\\{CMAKE_CURRENT_BINARY_DIR\\}/protobuf/src/protobuf/src/google")
@@ -1957,8 +1960,7 @@ (define-public tensorflow
("protobuf:native" ,protobuf-3.6) ; protoc
("protobuf:src" ,(package-source protobuf-3.6))
("eigen:src" ,(package-source eigen-for-tensorflow))
- ;; install_pip_packages.sh wants setuptools 39.1.0 specifically.
- ("python-setuptools" ,python-setuptools-for-tensorflow)
+ ("python-setuptools" ,python-setuptools)
;; The commit hashes and URLs for third-party source code are taken
;; from "tensorflow/workspace.bzl".
--8<---------------cut here---------------end--------------->8---
--
Ricardo
^ permalink raw reply related [flat|nested] 4+ messages in thread