From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id KKGDOtubuGEz7gAAgWs5BA (envelope-from ) for ; Tue, 14 Dec 2021 14:27:55 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id GK4/NtubuGFGWQAAB5/wlQ (envelope-from ) for ; Tue, 14 Dec 2021 13:27:55 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4EE5425279 for ; Tue, 14 Dec 2021 14:27:55 +0100 (CET) Received: from localhost ([::1]:50780 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mx7qU-0004tU-FE for larch@yhetil.org; Tue, 14 Dec 2021 08:27:54 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51426) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mx7jw-0006VN-JM for guix-devel@gnu.org; Tue, 14 Dec 2021 08:21:08 -0500 Received: from sender3-of-o51.zoho.com ([136.143.184.51]:21711) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mx7jp-000279-7W for guix-devel@gnu.org; Tue, 14 Dec 2021 08:21:07 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1639488056; cv=none; d=zohomail.com; s=zohoarc; b=g+9TDefBGXPcEmH14SgHosNuYFTp2QKvroRKwiAp4s+EiVfvCh2B+GmoKMUig43DFKflWF9iJNOqH7ocwlTO3PNhaoBzvVPVG+rm/F873VoncqvntGcOgVMjpe9x1YUCISTaIiEKbztF1MfAyf5ptukNYg5X7GCnGWiYExEnqb4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1639488056; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=usxSaHbAWfuwOe5iEJdkdsi6PVEUXyFFeM1lzWYu8G4=; b=MEmmjeeSAswcqQYVW6ZdSOxKlA/MCEsosBQEMSnxIYgRVi+k10vnFgTFYX9286bRJIBFOHjP0y6z3o0LHFTcRkFDDYAdkcWwKnOCmXDaWR7fYIhoHsNNAnmxrYcq5XixlSClxBsmuIjZD3K1WBgZnJHnok8NGux3eT3J+FQwG4M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1639488056; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:Date:In-reply-to:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=usxSaHbAWfuwOe5iEJdkdsi6PVEUXyFFeM1lzWYu8G4=; b=P3owInBDuk2exXsALT1O0lSB+sncVCIw/EMygYTtiseJ3r4xFJco0hsZPjF9v3Qh MVXmLpOOr48zcXxtgnctun1VtR+8y4PxNhJUenVxaLx5tKnw1vGn2qqnO3NoWyKoGMb Y4s13SWW2K5hN5TogQYgsPXhOjn68jHVfm/jSu18= Received: from localhost (p54ad4ca9.dip0.t-ipconnect.de [84.173.76.169]) by mx.zohomail.com with SMTPS id 16394880542951017.5052471807275; Tue, 14 Dec 2021 05:20:54 -0800 (PST) References: <20211213231602.12276.49991@vcs0.savannah.gnu.org> <874k7bbber.fsf@kitej> User-agent: mu4e 1.6.6; emacs 28.0.50 From: Ricardo Wurmus To: Guillaume Le Vaillant Subject: Re: Tensorflow fixes on core-updates-frozen Date: Tue, 14 Dec 2021 13:17:10 +0000 In-reply-to: <874k7bbber.fsf@kitej> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Message-ID: <87ee6fmifh.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.184.51; envelope-from=rekado@elephly.net; helo=sender3-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1639488475; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=usxSaHbAWfuwOe5iEJdkdsi6PVEUXyFFeM1lzWYu8G4=; b=JxRtchnkUMsAJM4OySnZ5hWbJ+M6I5MeH8fDKUtqjC/3SlFwvjZowxrzC882yqS7VjOaY9 R3aEn/17G5o0fLSvZUV+fxlxdbHH2KIVZ177i9JGR2uh1w7lQJxoDjOnQ0dxwcjWiDQa/l 8BbNivv09ZqdriGdw2qfkaBHt+BduKcNn1uyG/dwhXeVaijYZVUEJvie2eSTJHMhqxEWFj RULDUJRZaBgSz00JnrLt9mDF9oDbCYLYK3zG5VaY9hIN6CP1PiMYV0P6sRWSK1mjUHokJg 9lmZb8qKo1Xyrj3SKvhep6Z6umQJuCrxKs6SyxSLvFQj1sPFgXsG3tkL8KxgCg== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1639488475; a=rsa-sha256; cv=pass; b=lMU81dobTkhezytfZ7bBc0IxQxgsMOmJzbMSqv4FVuD1yv2whfNoZ+jk+uSd13lfu0Ve/F nYk6VDa0coI6tb8a23daEvZTkC8BZT4h23dD9jeJaKpdhQyUCFvKlaV89IHzvgRuOgXUls 9B1ObBdub86HcaUR6xYOsw5q3MQZR7qodPI02b0v1O/qbK+x3SxuBWKTKs3ros7VI6CtSd MU4m1/JMXn/AZyGl7fzBz7eWy1siZSAvCpbx+nPMqHJZOYNL+bfTCcsQJhxeml8hNbROry ZMQDP/YOTPRvcxncv+60bsILZ1wA8ssGXzWSJG4IN3hnjwRdVa/wk8r6kNg8xw== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=P3owInBD; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.67 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=P3owInBD; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 4EE5425279 X-Spam-Score: -4.67 X-Migadu-Scanner: scn1.migadu.com X-TUID: qPXk5DFZusZ7 Hi, > [[PGP Signed Part:Undecided]] > guix-commits@gnu.org skribis: > >> rekado pushed a change to branch core-updates-frozen >> in repository guix. >> >> new d503194 gnu: python2-entrypoints: Add missing input. >> new 672c7a2 gnu: tensorflow: Do not unpack directory. >> new 6d3439b gnu: eigen-for-tensorflow: Build with GCC 7. > > Hi Ricardo, > > Given that the core-update-frozen branch has been merged in master, > can't these fixes be pushed on master directly? I accidentally pushed those to core-updates-frozen. I pushed almost the same commits to the =E2=80=9Cmaster=E2=80=9D branch afterwards. The only difference is in the list of inputs in python2-entrypoints. It uses the new style on =E2=80=9Cmaster=E2=80=9D. Unfortunately, this is not enough to build tensorflow. At the very end we have this problem: --8<---------------cut here---------------start------------->8--- [ 99%] Building CXX object CMakeFiles/pywrap_tensorflow_internal.dir/tmp/gu= ix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.= cc.o /gnu/store/vakvgvrb839igv16jkif4lmx11d25jqb-gcc-10.3.0/bin/c++ -DEIGEN_AVOI= D_STL_ARRAY -DSQLITE_OMIT_LOAD_EXTENSION -DTF_USE_SNAPPY -Dpywrap_tensorflo= w_internal_EXPORTS -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source -I/tmp/g= uix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build -I/tmp/gui= x-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/external/eig= en_archive -I/gnu/store/54dclv4bsbg5894pyh0q5g768113368a-eigen-for-tensorfl= ow-3.3.5-1.fd6845384b86/include/eigen3 -I/tmp/guix-build-tensorflow-1.9.0.d= rv-0/source/tensorflow/contrib/build/gemmlowp/src/gemmlowp -I/gnu/store/jh8= w9a3gnkisp8zgkklydhc372vgcsk6-jsoncpp-1.7.3 -I/tmp/guix-build-tensorflow-1.= 9.0.drv-0/source/tensorflow/contrib/build/external/farmhash_archive -I/tmp/= guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/external/= farmhash_archive/util -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tenso= rflow/contrib/build/external/highwayhash -I/tmp/guix-build-tensorflow-1.9.0= .drv-0/source/tensorflow/contrib/build/cub/src/cub -I/tmp/guix-build-tensor= flow-1.9.0.drv-0/source/tensorflow/contrib/build/external/nsync/public -I/t= mp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/contrib/build/re2/in= stall/include -I/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/co= ntrib/build/double_conversion/src/double_conversion -I/gnu/store/g88hslvix5= j2rkdm13ay7biqyyqfja0f-snappy-1.1.9 -I/gnu/store/p5fgysbcnnp8b1d91mrvjvabab= mczga0-python-3.9.6/include/python3.9 -I/gnu/store/wgsmkn68q8h178sqc7ywjcdr= 330z9rb6-python-numpy-1.20.3/lib/python3.9/site-packages/numpy/core/include= -fPIC -fno-exceptions -std=3Dc++11 -fopenmp -O3 -DNDEBUG -fPIC -std=3Dgnu+= +14 -MD -MT CMakeFiles/pywrap_tensorflow_internal.dir/tmp/guix-build-tensor= flow-1.9.0.drv-0/source/tensorflow/python/lib/core/bfloat16.cc.o -MF CMakeF= iles/pywrap_tensorflow_internal.dir/tmp/guix-build-tensorflow-1.9.0.drv-0/s= ource/tensorflow/python/lib/core/bfloat16.cc.o.d -o CMakeFiles/pywrap_tenso= rflow_internal.dir/tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/= python/lib/core/bfloat16.cc.o -c /tmp/guix-build-tensorflow-1.9.0.drv-0/sou= rce/tensorflow/python/lib/core/bfloat16.cc /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:362:1: warning: converting to non-pointer type =E2=80=98long int= =E2=80=99 from NULL [-Wconversion-null] 362 | }; | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc: In function =E2=80=98bool tensorflow::{anonymous}::Initialize()= =E2=80=99: /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:623:36: error: no match for call to =E2=80=98(tensorflow::{anonym= ous}::Initialize()::&)>) (const char [6], , = const std::array&)=E2=80=99 623 | compare_types)) { | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: candidate: =E2=80=98tensorflow::{anonymous}::Initia= lize()::&)>=E2=80=99 596 | auto register_ufunc =3D [&](const char* name, PyUFuncGenericFunct= ion fn, | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: no known conversion for argument 2 from =E2=80=98= =E2=80=99 to =E2=80=98PyUFuncGenericFu= nction=E2=80=99 {aka =E2=80=98void (*)(char**, const long int*, const long = int*, void*)=E2=80=99} /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:627:36: error: no match for call to =E2=80=98(tensorflow::{anonym= ous}::Initialize()::&)>) (const char [10], ,= const std::array&)=E2=80=99 627 | compare_types)) { | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: candidate: =E2=80=98tensorflow::{anonymous}::Initia= lize()::&)>=E2=80=99 596 | auto register_ufunc =3D [&](const char* name, PyUFuncGenericFunct= ion fn, | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: no known conversion for argument 2 from =E2=80=98= =E2=80=99 to =E2=80=98PyUFuncGenericFu= nction=E2=80=99 {aka =E2=80=98void (*)(char**, const long int*, const long = int*, void*)=E2=80=99} /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:630:77: error: no match for call to =E2=80=98(tensorflow::{anonym= ous}::Initialize()::&)>) (const char [5], , = const std::array&)=E2=80=99 630 | if (!register_ufunc("less", CompareUFunc, comp= are_types)) { | = ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: candidate: =E2=80=98tensorflow::{anonymous}::Initia= lize()::&)>=E2=80=99 596 | auto register_ufunc =3D [&](const char* name, PyUFuncGenericFunct= ion fn, | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: no known conversion for argument 2 from =E2=80=98= =E2=80=99 to =E2=80=98PyUFuncGenericFu= nction=E2=80=99 {aka =E2=80=98void (*)(char**, const long int*, const long = int*, void*)=E2=80=99} /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:634:36: error: no match for call to =E2=80=98(tensorflow::{anonym= ous}::Initialize()::&)>) (const char [8], , = const std::array&)=E2=80=99 634 | compare_types)) { | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: candidate: =E2=80=98tensorflow::{anonymous}::Initia= lize()::&)>=E2=80=99 596 | auto register_ufunc =3D [&](const char* name, PyUFuncGenericFunct= ion fn, | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: no known conversion for argument 2 from =E2=80=98= =E2=80=99 to =E2=80=98PyUFuncGenericFu= nction=E2=80=99 {aka =E2=80=98void (*)(char**, const long int*, const long = int*, void*)=E2=80=99} /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:638:36: error: no match for call to =E2=80=98(tensorflow::{anonym= ous}::Initialize()::&)>) (const char [11], ,= const std::array&)=E2=80=99 638 | compare_types)) { | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: candidate: =E2=80=98tensorflow::{anonymous}::Initia= lize()::&)>=E2=80=99 596 | auto register_ufunc =3D [&](const char* name, PyUFuncGenericFunct= ion fn, | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: no known conversion for argument 2 from =E2=80=98= =E2=80=99 to =E2=80=98PyUFuncGenericFu= nction=E2=80=99 {aka =E2=80=98void (*)(char**, const long int*, const long = int*, void*)=E2=80=99} /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:642:36: error: no match for call to =E2=80=98(tensorflow::{anonym= ous}::Initialize()::&)>) (const char [14], ,= const std::array&)=E2=80=99 642 | compare_types)) { | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: candidate: =E2=80=98tensorflow::{anonymous}::Initia= lize()::&)>=E2=80=99 596 | auto register_ufunc =3D [&](const char* name, PyUFuncGenericFunct= ion fn, | ^ /tmp/guix-build-tensorflow-1.9.0.drv-0/source/tensorflow/python/lib/core/bf= loat16.cc:596:25: note: no known conversion for argument 2 from =E2=80=98= =E2=80=99 to =E2=80=98PyUFuncGenericFu= nction=E2=80=99 {aka =E2=80=98void (*)(char**, const long int*, const long = int*, void*)=E2=80=99} make[2]: *** [CMakeFiles/pywrap_tensorflow_internal.dir/build.make:188: CMa= keFiles/pywrap_tensorflow_internal.dir/tmp/guix-build-tensorflow-1.9.0.drv-= 0/source/tensorflow/python/lib/core/bfloat16.cc.o] Error 1 make[2]: *** Waiting for unfinished jobs.... make[2]: Leaving directory '/tmp/guix-build-tensorflow-1.9.0.drv-0/source/t= ensorflow/contrib/build' make[1]: *** [CMakeFiles/Makefile2:3807: CMakeFiles/pywrap_tensorflow_inter= nal.dir/all] Error 2 make[1]: Leaving directory '/tmp/guix-build-tensorflow-1.9.0.drv-0/source/t= ensorflow/contrib/build' make: *** [Makefile:139: all] Error 2 --8<---------------cut here---------------end--------------->8--- That=E2=80=99s because Tensorflow 1.9.0 is not meant to be built with Python 3.9. It didn=E2=80=99t even like Python 3.6, which is why we already carry= a few 3.7 compatibility patches. We should figure out how the calling convention changed and then patch the C++ files. My changes compared to the =E2=80=9Cmaster=E2=80=9D branch: --8<---------------cut here---------------start------------->8--- diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learn= ing.scm index 58f7c7c013..9d8bed09f7 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1901,6 +1901,9 @@ (define-public tensorflow (mkdir-p "eigen-src") (copy-recursively (assoc-ref inputs "eigen:src") "eigen-src") =20 + (substitute* "tensorflow/tools/ci_build/install/install_pip_p= ackages.sh" + (("setuptools=3D=3D39.1.0") "setuptools")) + (substitute* "tensorflow/contrib/cmake/tf_python.cmake" ;; Take protobuf source files from our source package. (("\\$\\{CMAKE_CURRENT_BINARY_DIR\\}/protobuf/src/protobuf/= src/google") @@ -1957,8 +1960,7 @@ (define-public tensorflow ("protobuf:native" ,protobuf-3.6) ; protoc ("protobuf:src" ,(package-source protobuf-3.6)) ("eigen:src" ,(package-source eigen-for-tensorflow)) - ;; install_pip_packages.sh wants setuptools 39.1.0 specifically. - ("python-setuptools" ,python-setuptools-for-tensorflow) + ("python-setuptools" ,python-setuptools) =20 ;; The commit hashes and URLs for third-party source code are taken ;; from "tensorflow/workspace.bzl". --8<---------------cut here---------------end--------------->8--- --=20 Ricardo