* [bug#48785] [PATCH 0/3] Update LightGBM and split frontends.
@ 2021-06-01 22:05 Vinicius Monego
2021-06-01 22:06 ` [bug#48785] [PATCH 1/3] gnu: Add fast-double-parser Vinicius Monego
` (5 more replies)
0 siblings, 6 replies; 19+ messages in thread
From: Vinicius Monego @ 2021-06-01 22:05 UTC (permalink / raw)
To: 48785; +Cc: Vinicius Monego
Same considerations from https://issues.guix.gnu.org/48388
Vinicius Monego (3):
gnu: Add fast-double-parser.
gnu: lightgbm: Update to 3.2.1.
gnu: Add python-lightgbm.
gnu/local.mk | 2 +
gnu/packages/cpp.scm | 38 ++++++-
gnu/packages/machine-learning.scm | 100 +++++++++++++-----
...htgbm-python-use-system-lib-lightgbm.patch | 74 +++++++++++++
.../lightgbm-use-system-libraries.patch | 58 ++++++++++
5 files changed, 247 insertions(+), 25 deletions(-)
create mode 100644 gnu/packages/patches/lightgbm-python-use-system-lib-lightgbm.patch
create mode 100644 gnu/packages/patches/lightgbm-use-system-libraries.patch
--
2.31.1
^ permalink raw reply [flat|nested] 19+ messages in thread* [bug#48785] [PATCH 1/3] gnu: Add fast-double-parser. 2021-06-01 22:05 [bug#48785] [PATCH 0/3] Update LightGBM and split frontends Vinicius Monego @ 2021-06-01 22:06 ` Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 3/3] gnu: Add python-lightgbm Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 1/3] gnu: Add fast-double-parser Vinicius Monego ` (4 subsequent siblings) 5 siblings, 2 replies; 19+ messages in thread From: Vinicius Monego @ 2021-06-01 22:06 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/cpp.scm (fast-double-parser): New variable. --- gnu/packages/cpp.scm | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm index 3089094eb8..1cddf7209b 100644 --- a/gnu/packages/cpp.scm +++ b/gnu/packages/cpp.scm @@ -10,7 +10,7 @@ ;;; Copyright © 2020 Roel Janssen <roel@gnu.org> ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re> -;;; Copyright © 2020 Vinicius Monego <monego@posteo.net> +;;; Copyright © 2020, 2021 Vinicius Monego <monego@posteo.net> ;;; Copyright © 2020 Marius Bakke <marius@gnu.org> ;;; Copyright © 2020 Michael Rohleder <mike@rohleder.de> ;;; Copyright © 2020 Alexandros Theodotou <alex@zrythm.org> @@ -330,6 +330,42 @@ functions, class methods, and stl containers. @end enumerate\n") (license license:bsd-3))) +(define-public fast-double-parser + (package + (name "fast-double-parser") + (version "0.5.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/lemire/fast_double_parser") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "10mfpy4i7lpavghb15s4d0920nlwsac1ri2sxkcqbrj9iq94qj71")))) + (build-system cmake-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'install ;no install target, have to do it manually + (lambda* (#:key outputs #:allow-other-keys) + (let ((out-include (string-append (assoc-ref outputs "out") + "/include"))) + (mkdir-p out-include) + (copy-file "../source/include/fast_double_parser.h" + (string-append out-include + "/fast_double_parser.h")))))))) + (home-page "https://github.com/lemire/fast_double_parser") + (synopsis "Parse strings into double (binary64) floating-point values") + (description "Fast function to parse strings containing decimal numbers +into double-precision (binary64) floating-point values. That is, given the +string \"1.0e10\", it should return a 64-bit floating-point value equal to +10000000000. The function will match exactly (down the smallest bit) the +result of a standard function like @code{strtod}.") + ;; Dual-licensed, either license applies. + (license (list license:asl2.0 + license:boost1.0)))) + (define-public fifo-map (let* ((commit "0dfbf5dacbb15a32c43f912a7e66a54aae39d0f9") (revision "0") -- 2.31.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH 2/3] gnu: lightgbm: Update to 3.2.1. 2021-06-01 22:06 ` [bug#48785] [PATCH 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2021-06-01 22:06 ` Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 3/3] gnu: Add python-lightgbm Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2021-06-01 22:06 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (lightgbm): Update to 3.2.1. [source]: Add patches. [native-inputs]: Add googletest, opencl-headers. [inputs]: Add boost, eigen, fast-double-parser, fmt, ocl-icd. [propagated-inputs]: Remove. [arguments]: Add #:tests?, #:test-target. Pass "-DBUILD_CPP_TEST=OFF" and "-DUSE_GPU=ON" to #:configure-flags. Do not replace check phase. * gnu/packages/patches/lightgbm-use-system-libraries.patch, gnu/packages/patches/lightgbm-python-use-system-lib-lightgbm.patch: New files. * gnu/local.mk (dist_PATCH_DATA): Add them. --- gnu/local.mk | 2 + gnu/packages/machine-learning.scm | 57 ++++++++------ ...htgbm-python-use-system-lib-lightgbm.patch | 74 +++++++++++++++++++ .../lightgbm-use-system-libraries.patch | 58 +++++++++++++++ 4 files changed, 167 insertions(+), 24 deletions(-) create mode 100644 gnu/packages/patches/lightgbm-python-use-system-lib-lightgbm.patch create mode 100644 gnu/packages/patches/lightgbm-use-system-libraries.patch diff --git a/gnu/local.mk b/gnu/local.mk index 6a1e459e4e..0016ac12af 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1375,6 +1375,8 @@ dist_patch_DATA = \ %D%/packages/patches/lierolibre-newer-libconfig.patch \ %D%/packages/patches/lierolibre-remove-arch-warning.patch \ %D%/packages/patches/lierolibre-try-building-other-arch.patch \ + %D%/packages/patches/lightgbm-python-use-system-lib-lightgbm.patch \ + %D%/packages/patches/lightgbm-use-system-libraries.patch \ %D%/packages/patches/linbox-fix-pkgconfig.patch \ %D%/packages/patches/linkchecker-tests-require-network.patch \ %D%/packages/patches/linphone-desktop-without-sdk.patch \ diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index ead47a534a..1a3f647dde 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -56,6 +56,7 @@ #:use-module (gnu packages check) #:use-module (gnu packages compression) #:use-module (gnu packages cmake) + #:use-module (gnu packages cpp) #:use-module (gnu packages cran) #:use-module (gnu packages databases) #:use-module (gnu packages dejagnu) @@ -70,8 +71,10 @@ #:use-module (gnu packages mpi) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages pretty-print) #:use-module (gnu packages protobuf) #:use-module (gnu packages python) #:use-module (gnu packages python-build) @@ -1058,33 +1061,39 @@ main intended application of Autograd is gradient-based optimization.") (define-public lightgbm (package (name "lightgbm") - (version "2.0.12") - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/Microsoft/LightGBM") - (commit (string-append "v" version)))) - (sha256 - (base32 - "0jlvyn7k81dzrh9ij3zw576wbgiwmmr26rzpdxjn1dbpc3njpvzi")) - (file-name (git-file-name name version)))) + (version "3.2.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/Microsoft/LightGBM") + (commit (string-append "v" version)))) + (patches + (search-patches "lightgbm-use-system-libraries.patch" + "lightgbm-python-use-system-lib-lightgbm.patch")) + (sha256 + (base32 "1fchiiwl3pz1kpi0kbfanif6c6b688wmmjjbqz4ff4djc6g2sg7z")) + (file-name (git-file-name name version)))) (native-inputs - `(("python-pytest" ,python-pytest) - ("python-nose" ,python-nose))) + `(("googletest" ,googletest) + ("opencl-headers" ,opencl-headers))) (inputs - `(("openmpi" ,openmpi))) - (propagated-inputs - `(("python-numpy" ,python-numpy) - ("python-scipy" ,python-scipy))) + `(("boost" ,boost) + ("eigen" ,eigen) + ("fast-double-parser" ,fast-double-parser) + ("fmt" ,fmt) + ("ocl-icd" ,ocl-icd) + ("openmpi" ,openmpi))) (arguments - `(#:configure-flags - '("-DUSE_MPI=ON") - #:phases - (modify-phases %standard-phases - (replace 'check - (lambda _ - (with-directory-excursion "../source" - (invoke "pytest" "tests/c_api_test/test_.py"))))))) + `(;; FIXME: With 'find_package(GTest CONFIG)' (default), googletest + ;; is not found by CMake. If we replace CONFIG with REQUIRED, the + ;; test files fail to link with googletest in the build phase. + #:tests? #f + #:test-target "testlightgbm" + #:configure-flags + (list "-DBUILD_CPP_TEST=OFF" + "-DUSE_GPU=ON" ;using OpenCL + "-DUSE_MPI=ON"))) (build-system cmake-build-system) (home-page "https://github.com/Microsoft/LightGBM") (synopsis "Gradient boosting framework based on decision tree algorithms") diff --git a/gnu/packages/patches/lightgbm-python-use-system-lib-lightgbm.patch b/gnu/packages/patches/lightgbm-python-use-system-lib-lightgbm.patch new file mode 100644 index 0000000000..69cb721daa --- /dev/null +++ b/gnu/packages/patches/lightgbm-python-use-system-lib-lightgbm.patch @@ -0,0 +1,74 @@ +From c73be84d8ec6ac71dfdaa71b7943ae344f5d7cde Mon Sep 17 00:00:00 2001 +From: Vinicius Monego <monego@posteo.net> +Date: Tue, 1 Jun 2021 00:24:38 -0300 +Subject: [PATCH] Use system lib_lightgbm. + +--- + python-package/lightgbm/libpath.py | 4 ++++ + python-package/setup.py | 12 ++++++++---- + 2 files changed, 12 insertions(+), 4 deletions(-) + +diff --git a/python-package/lightgbm/libpath.py b/python-package/lightgbm/libpath.py +index 6533792..27af2b9 100644 +--- a/python-package/lightgbm/libpath.py ++++ b/python-package/lightgbm/libpath.py +@@ -17,8 +17,12 @@ def find_lib_path() -> List[str]: + # we don't need lib_lightgbm while building docs + return [] + ++ # This variable will be set during substitution. ++ lib_lightgbm_path = '' ++ + curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__))) + dll_path = [curr_path, ++ lib_lightgbm_path, + os.path.join(curr_path, '../../'), + os.path.join(curr_path, 'compile'), + os.path.join(curr_path, '../compile'), +diff --git a/python-package/setup.py b/python-package/setup.py +index ee98b1a..2f7ce38 100644 +--- a/python-package/setup.py ++++ b/python-package/setup.py +@@ -231,7 +231,7 @@ class CustomInstall(install): + self.opencl_library = None + self.mpi = 0 + self.hdfs = 0 +- self.precompile = 0 ++ self.precompile = 1 + self.nomp = 0 + self.bit32 = 0 + +@@ -274,7 +274,7 @@ class CustomBdistWheel(bdist_wheel): + self.opencl_library = None + self.mpi = 0 + self.hdfs = 0 +- self.precompile = 0 ++ self.precompile = 1 + self.nomp = 0 + self.bit32 = 0 + +@@ -324,7 +324,10 @@ if __name__ == "__main__": + copy_file(os.path.join(CURRENT_DIR, os.path.pardir, 'VERSION.txt'), + os.path.join(CURRENT_DIR, 'lightgbm', 'VERSION.txt'), + verbose=0) # type:ignore +- version = open(os.path.join(CURRENT_DIR, 'lightgbm', 'VERSION.txt'), encoding='utf-8').read().strip() ++ ++ # This variable will be set during substitution. ++ version = '' ++ + readme = open(os.path.join(CURRENT_DIR, 'README.rst'), encoding='utf-8').read() + + sys.path.insert(0, CURRENT_DIR) +@@ -355,7 +358,8 @@ if __name__ == "__main__": + zip_safe=False, + cmdclass={ + 'install': CustomInstall, +- 'install_lib': CustomInstallLib, ++ # Lib will be loaded from the lightgbm package. ++ # 'install_lib': CustomInstallLib, + 'bdist_wheel': CustomBdistWheel, + 'sdist': CustomSdist, + }, +-- +2.31.1 + diff --git a/gnu/packages/patches/lightgbm-use-system-libraries.patch b/gnu/packages/patches/lightgbm-use-system-libraries.patch new file mode 100644 index 0000000000..115ca02775 --- /dev/null +++ b/gnu/packages/patches/lightgbm-use-system-libraries.patch @@ -0,0 +1,58 @@ +From 894648291b9629d51785e4fd1cae820d2858e039 Mon Sep 17 00:00:00 2001 +From: Vinicius Monego <monego@posteo.net> +Date: Mon, 31 May 2021 21:23:52 -0300 +Subject: [PATCH] Use system libraries. + +--- + CMakeLists.txt | 11 ++++++++--- + include/LightGBM/utils/common.h | 4 ++-- + 2 files changed, 10 insertions(+), 5 deletions(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index b845f36..560e477 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -93,9 +93,6 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + +-SET(EIGEN_DIR "${PROJECT_SOURCE_DIR}/external_libs/eigen") +-include_directories(${EIGEN_DIR}) +- + # See https://gitlab.com/libeigen/eigen/-/blob/master/COPYING.README + ADD_DEFINITIONS(-DEIGEN_MPL2_ONLY) + +@@ -399,6 +396,14 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + ++find_package (Eigen3 3.3 REQUIRED NO_MODULE) ++TARGET_LINK_LIBRARIES (lightgbm Eigen3::Eigen) ++TARGET_LINK_LIBRARIES (_lightgbm Eigen3::Eigen) ++ ++find_package(fmt) ++TARGET_LINK_LIBRARIES(lightgbm fmt::fmt) ++TARGET_LINK_LIBRARIES(_lightgbm fmt::fmt) ++ + if(USE_MPI) + TARGET_LINK_LIBRARIES(lightgbm ${MPI_CXX_LIBRARIES}) + TARGET_LINK_LIBRARIES(_lightgbm ${MPI_CXX_LIBRARIES}) +diff --git a/include/LightGBM/utils/common.h b/include/LightGBM/utils/common.h +index 4357357..00101a5 100644 +--- a/include/LightGBM/utils/common.h ++++ b/include/LightGBM/utils/common.h +@@ -32,9 +32,9 @@ + + #if (!((defined(sun) || defined(__sun)) && (defined(__SVR4) || defined(__svr4__)))) + #define FMT_HEADER_ONLY +-#include "../../../external_libs/fmt/include/fmt/format.h" ++#include "fmt/format.h" + #endif +-#include "../../../external_libs/fast_double_parser/include/fast_double_parser.h" ++#include "fast_double_parser.h" + + #ifdef _MSC_VER + #include <intrin.h> +-- +2.31.1 + -- 2.31.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH 3/3] gnu: Add python-lightgbm. 2021-06-01 22:06 ` [bug#48785] [PATCH 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego @ 2021-06-01 22:06 ` Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2021-06-01 22:06 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (python-lightgbm): New variable. --- gnu/packages/machine-learning.scm | 43 +++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 1a3f647dde..900ca15b28 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1110,6 +1110,49 @@ the following advantages: @end itemize\n") (license license:expat))) +(define-public python-lightgbm + (package + (inherit lightgbm) + (name "python-lightgbm") + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'preparations + (lambda* (#:key inputs #:allow-other-keys) + (setenv "HOME" "/tmp") ;to write a log file + ;; Move Python files to source root to silence some warnings. + (rename-file "python-package/setup.py" "setup.py") + (rename-file "python-package/README.rst" "README.rst") + (rename-file "python-package/lightgbm" "lightgbm") + (substitute* "setup.py" + (("version = ''") + (string-append "version = " "'" + ,(package-version lightgbm) "'"))) + (substitute* "lightgbm/libpath.py" + (("lib_lightgbm_path = ''") + (string-append "lib_lightgbm_path = " "'" + (assoc-ref inputs "lightgbm") + "/lib" "'"))))) + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest" + "tests/python_package_test"))))))) + (native-inputs + `(("python-psutil" ,python-psutil) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) + (inputs + `(("lightgbm" ,lightgbm))) + (propagated-inputs + `(("python-numpy" ,python-numpy) + ("python-pandas" ,python-pandas) + ("python-scipy" ,python-scipy) + ("python-scikit-learn" ,python-scikit-learn))) + (synopsis "Python frontend for LightGBM"))) + (define-public vowpal-wabbit ;; Language bindings not included. (package -- 2.31.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v2 1/3] gnu: Add fast-double-parser. 2021-06-01 22:05 [bug#48785] [PATCH 0/3] Update LightGBM and split frontends Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2021-06-30 20:09 ` Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 3/3] gnu: Add python-lightgbm Vinicius Monego 2021-10-08 2:54 ` [bug#48785] [PATCH v3 1/3] gnu: Add fast-double-parser Vinicius Monego ` (3 subsequent siblings) 5 siblings, 2 replies; 19+ messages in thread From: Vinicius Monego @ 2021-06-30 20:09 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/cpp.scm (fast-double-parser): New variable. --- gnu/packages/cpp.scm | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm index 54f9da4b27..3ca184b4b0 100644 --- a/gnu/packages/cpp.scm +++ b/gnu/packages/cpp.scm @@ -10,7 +10,7 @@ ;;; Copyright © 2020 Roel Janssen <roel@gnu.org> ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re> -;;; Copyright © 2020 Vinicius Monego <monego@posteo.net> +;;; Copyright © 2020, 2021 Vinicius Monego <monego@posteo.net> ;;; Copyright © 2020 Marius Bakke <marius@gnu.org> ;;; Copyright © 2020 Michael Rohleder <mike@rohleder.de> ;;; Copyright © 2020 Alexandros Theodotou <alex@zrythm.org> @@ -330,6 +330,42 @@ functions, class methods, and stl containers. @end enumerate\n") (license license:bsd-3))) +(define-public fast-double-parser + (package + (name "fast-double-parser") + (version "0.5.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/lemire/fast_double_parser") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "10mfpy4i7lpavghb15s4d0920nlwsac1ri2sxkcqbrj9iq94qj71")))) + (build-system cmake-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'install ;no install target, have to do it manually + (lambda* (#:key outputs #:allow-other-keys) + (let ((out-include (string-append (assoc-ref outputs "out") + "/include"))) + (mkdir-p out-include) + (copy-file "../source/include/fast_double_parser.h" + (string-append out-include + "/fast_double_parser.h")))))))) + (home-page "https://github.com/lemire/fast_double_parser") + (synopsis "Parse strings into double (binary64) floating-point values") + (description "Fast function to parse strings containing decimal numbers +into double-precision (binary64) floating-point values. That is, given the +string \"1.0e10\", it should return a 64-bit floating-point value equal to +10000000000. The function will match exactly (down the smallest bit) the +result of a standard function like @code{strtod}.") + ;; Dual-licensed, either license applies. + (license (list license:asl2.0 + license:boost1.0)))) + (define-public fifo-map (let* ((commit "0dfbf5dacbb15a32c43f912a7e66a54aae39d0f9") (revision "0") -- 2.32.0 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v2 2/3] gnu: lightgbm: Update to 3.2.1. 2021-06-30 20:09 ` [bug#48785] [PATCH v2 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2021-06-30 20:09 ` Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 3/3] gnu: Add python-lightgbm Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2021-06-30 20:09 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (lightgbm): Update to 3.2.1. [source]: Add patch. Make cosmetic changes. [native-inputs]: Add googletest, opencl-headers. [inputs]: Add boost, eigen, fast-double-parser, fmt, ocl-icd. [propagated-inputs]: Remove. [arguments]: Add #:tests?, #:test-target. Pass "-DBUILD_CPP_TEST=OFF" and "-DUSE_GPU=ON" to #:configure-flags. Do not replace check phase. * gnu/packages/patches/lightgbm-use-system-libraries.patch: New file. * gnu/local.mk (dist_patch_DATA): Add it. --- Removed the python patch. gnu/local.mk | 1 + gnu/packages/machine-learning.scm | 56 ++++++++++-------- .../lightgbm-use-system-libraries.patch | 58 +++++++++++++++++++ 3 files changed, 91 insertions(+), 24 deletions(-) create mode 100644 gnu/packages/patches/lightgbm-use-system-libraries.patch diff --git a/gnu/local.mk b/gnu/local.mk index 19db9787dc..fa30cc7e9b 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1386,6 +1386,7 @@ dist_patch_DATA = \ %D%/packages/patches/lierolibre-newer-libconfig.patch \ %D%/packages/patches/lierolibre-remove-arch-warning.patch \ %D%/packages/patches/lierolibre-try-building-other-arch.patch \ + %D%/packages/patches/lightgbm-use-system-libraries.patch \ %D%/packages/patches/linbox-fix-pkgconfig.patch \ %D%/packages/patches/linphone-desktop-without-sdk.patch \ %D%/packages/patches/linux-libre-support-for-Pinebook-Pro.patch \ diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index b8d79821e0..04d1fa0c4d 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -56,6 +56,7 @@ #:use-module (gnu packages check) #:use-module (gnu packages compression) #:use-module (gnu packages cmake) + #:use-module (gnu packages cpp) #:use-module (gnu packages cran) #:use-module (gnu packages databases) #:use-module (gnu packages dejagnu) @@ -70,8 +71,10 @@ #:use-module (gnu packages mpi) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages pretty-print) #:use-module (gnu packages protobuf) #:use-module (gnu packages python) #:use-module (gnu packages python-build) @@ -1058,33 +1061,38 @@ main intended application of Autograd is gradient-based optimization.") (define-public lightgbm (package (name "lightgbm") - (version "2.0.12") - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/Microsoft/LightGBM") - (commit (string-append "v" version)))) - (sha256 - (base32 - "0jlvyn7k81dzrh9ij3zw576wbgiwmmr26rzpdxjn1dbpc3njpvzi")) - (file-name (git-file-name name version)))) + (version "3.2.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/Microsoft/LightGBM") + (commit (string-append "v" version)))) + (patches + (search-patches "lightgbm-use-system-libraries.patch")) + (sha256 + (base32 "1fchiiwl3pz1kpi0kbfanif6c6b688wmmjjbqz4ff4djc6g2sg7z")) + (file-name (git-file-name name version)))) (native-inputs - `(("python-pytest" ,python-pytest) - ("python-nose" ,python-nose))) + `(("googletest" ,googletest) + ("opencl-headers" ,opencl-headers))) (inputs - `(("openmpi" ,openmpi))) - (propagated-inputs - `(("python-numpy" ,python-numpy) - ("python-scipy" ,python-scipy))) + `(("boost" ,boost) + ("eigen" ,eigen) + ("fast-double-parser" ,fast-double-parser) + ("fmt" ,fmt) + ("ocl-icd" ,ocl-icd) + ("openmpi" ,openmpi))) (arguments - `(#:configure-flags - '("-DUSE_MPI=ON") - #:phases - (modify-phases %standard-phases - (replace 'check - (lambda _ - (with-directory-excursion "../source" - (invoke "pytest" "tests/c_api_test/test_.py"))))))) + `(;; FIXME: With 'find_package(GTest CONFIG)' (default), googletest + ;; is not found by CMake. If we replace CONFIG with REQUIRED, the + ;; test files fail to link with googletest in the build phase. + #:tests? #f + #:test-target "testlightgbm" + #:configure-flags + (list "-DBUILD_CPP_TEST=OFF" + "-DUSE_GPU=ON" ;using OpenCL + "-DUSE_MPI=ON"))) (build-system cmake-build-system) (home-page "https://github.com/Microsoft/LightGBM") (synopsis "Gradient boosting framework based on decision tree algorithms") diff --git a/gnu/packages/patches/lightgbm-use-system-libraries.patch b/gnu/packages/patches/lightgbm-use-system-libraries.patch new file mode 100644 index 0000000000..115ca02775 --- /dev/null +++ b/gnu/packages/patches/lightgbm-use-system-libraries.patch @@ -0,0 +1,58 @@ +From 894648291b9629d51785e4fd1cae820d2858e039 Mon Sep 17 00:00:00 2001 +From: Vinicius Monego <monego@posteo.net> +Date: Mon, 31 May 2021 21:23:52 -0300 +Subject: [PATCH] Use system libraries. + +--- + CMakeLists.txt | 11 ++++++++--- + include/LightGBM/utils/common.h | 4 ++-- + 2 files changed, 10 insertions(+), 5 deletions(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index b845f36..560e477 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -93,9 +93,6 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + +-SET(EIGEN_DIR "${PROJECT_SOURCE_DIR}/external_libs/eigen") +-include_directories(${EIGEN_DIR}) +- + # See https://gitlab.com/libeigen/eigen/-/blob/master/COPYING.README + ADD_DEFINITIONS(-DEIGEN_MPL2_ONLY) + +@@ -399,6 +396,14 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + ++find_package (Eigen3 3.3 REQUIRED NO_MODULE) ++TARGET_LINK_LIBRARIES (lightgbm Eigen3::Eigen) ++TARGET_LINK_LIBRARIES (_lightgbm Eigen3::Eigen) ++ ++find_package(fmt) ++TARGET_LINK_LIBRARIES(lightgbm fmt::fmt) ++TARGET_LINK_LIBRARIES(_lightgbm fmt::fmt) ++ + if(USE_MPI) + TARGET_LINK_LIBRARIES(lightgbm ${MPI_CXX_LIBRARIES}) + TARGET_LINK_LIBRARIES(_lightgbm ${MPI_CXX_LIBRARIES}) +diff --git a/include/LightGBM/utils/common.h b/include/LightGBM/utils/common.h +index 4357357..00101a5 100644 +--- a/include/LightGBM/utils/common.h ++++ b/include/LightGBM/utils/common.h +@@ -32,9 +32,9 @@ + + #if (!((defined(sun) || defined(__sun)) && (defined(__SVR4) || defined(__svr4__)))) + #define FMT_HEADER_ONLY +-#include "../../../external_libs/fmt/include/fmt/format.h" ++#include "fmt/format.h" + #endif +-#include "../../../external_libs/fast_double_parser/include/fast_double_parser.h" ++#include "fast_double_parser.h" + + #ifdef _MSC_VER + #include <intrin.h> +-- +2.31.1 + -- 2.32.0 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v2 3/3] gnu: Add python-lightgbm. 2021-06-30 20:09 ` [bug#48785] [PATCH v2 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego @ 2021-06-30 20:09 ` Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2021-06-30 20:09 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (python-lightgbm): New variable. --- Replace patch changes with code in 'preparations phase and added phase 'install-liblightgbm gnu/packages/machine-learning.scm | 59 +++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 04d1fa0c4d..79bf0bbbfd 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1109,6 +1109,65 @@ the following advantages: @end itemize\n") (license license:expat))) +(define-public python-lightgbm + (package + (inherit lightgbm) + (name "python-lightgbm") + (source (package-source lightgbm)) + (build-system python-build-system) + (arguments + `(#:configure-flags (list "--precompile") + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'preparations + (lambda _ + (setenv "HOME" "/tmp") ;to write a log file + ;; Move Python files to source root to silence some warnings. + (rename-file "python-package/setup.py" "setup.py") + (rename-file "python-package/README.rst" "README.rst") + (rename-file "python-package/lightgbm" "lightgbm") + ;; Install version file to read the library's version. + (with-output-to-file "lightgbm/VERSION.txt" + (lambda () + (display ,(package-version lightgbm)))) + ;; Skip building the shared library. + (substitute* "setup.py" + (("'install_lib': CustomInstallLib,") "")))) + (add-after 'install 'install-liblightgbm + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (pylib (string-append out "/lib/python" + ,(version-major+minor + (package-version python)) + "/site-packages")) + (lgbdir (string-append pylib "/lightgbm")) + (lib-lightgbm (string-append (assoc-ref inputs "lightgbm") + "/lib/lib_lightgbm.so"))) + ;; This symlink is necessary for tests. + (symlink lib-lightgbm "lightgbm/lib_lightgbm.so") + ;; And this one for runtime. + (mkdir-p (string-append lgbdir "/lib")) + (symlink lib-lightgbm (string-append lgbdir + "/lib_lightgbm.so"))))) + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest" + "tests/python_package_test"))))))) + (native-inputs + `(("python-psutil" ,python-psutil) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) + (inputs + `(("lightgbm" ,lightgbm))) + (propagated-inputs + `(("python-numpy" ,python-numpy) + ("python-pandas" ,python-pandas) + ("python-scipy" ,python-scipy) + ("python-scikit-learn" ,python-scikit-learn))) + (synopsis "Python frontend for LightGBM"))) + (define-public vowpal-wabbit ;; Language bindings not included. (package -- 2.32.0 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v3 1/3] gnu: Add fast-double-parser. 2021-06-01 22:05 [bug#48785] [PATCH 0/3] Update LightGBM and split frontends Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2021-10-08 2:54 ` Vinicius Monego 2021-10-08 2:54 ` [bug#48785] [PATCH v3 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego 2021-10-08 2:54 ` [bug#48785] [PATCH v3 3/3] gnu: Add python-lightgbm Vinicius Monego 2021-10-31 1:40 ` [bug#48785] [PATCH v4 1/3] gnu: Add fast-double-parser Vinicius Monego ` (2 subsequent siblings) 5 siblings, 2 replies; 19+ messages in thread From: Vinicius Monego @ 2021-10-08 2:54 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/cpp.scm (fast-double-parser): New variable. --- Changed string to @code{} in description. gnu/packages/cpp.scm | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm index 4af71ba93a..4a8cd3a120 100644 --- a/gnu/packages/cpp.scm +++ b/gnu/packages/cpp.scm @@ -10,7 +10,7 @@ ;;; Copyright © 2020 Roel Janssen <roel@gnu.org> ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re> -;;; Copyright © 2020 Vinicius Monego <monego@posteo.net> +;;; Copyright © 2020, 2021 Vinicius Monego <monego@posteo.net> ;;; Copyright © 2020 Marius Bakke <marius@gnu.org> ;;; Copyright © 2020 Michael Rohleder <mike@rohleder.de> ;;; Copyright © 2020 Alexandros Theodotou <alex@zrythm.org> @@ -331,6 +331,42 @@ functions, class methods, and stl containers. @end enumerate\n") (license license:bsd-3))) +(define-public fast-double-parser + (package + (name "fast-double-parser") + (version "0.5.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/lemire/fast_double_parser") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "10mfpy4i7lpavghb15s4d0920nlwsac1ri2sxkcqbrj9iq94qj71")))) + (build-system cmake-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'install ;no install target, have to do it manually + (lambda* (#:key outputs #:allow-other-keys) + (let ((out-include (string-append (assoc-ref outputs "out") + "/include"))) + (mkdir-p out-include) + (copy-file "../source/include/fast_double_parser.h" + (string-append out-include + "/fast_double_parser.h")))))))) + (home-page "https://github.com/lemire/fast_double_parser") + (synopsis "Parse strings into double (binary64) floating-point values") + (description "Fast function to parse strings containing decimal numbers +into double-precision (binary64) floating-point values. That is, given the +string @code{1.0e10}, it should return a 64-bit floating-point value equal to +10000000000. The function will match exactly (down the smallest bit) the +result of a standard function like @code{strtod}.") + ;; Dual-licensed, either license applies. + (license (list license:asl2.0 + license:boost1.0)))) + (define-public fifo-map (let* ((commit "0dfbf5dacbb15a32c43f912a7e66a54aae39d0f9") (revision "0") -- 2.30.2 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v3 2/3] gnu: lightgbm: Update to 3.2.1. 2021-10-08 2:54 ` [bug#48785] [PATCH v3 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2021-10-08 2:54 ` Vinicius Monego 2021-10-08 2:54 ` [bug#48785] [PATCH v3 3/3] gnu: Add python-lightgbm Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2021-10-08 2:54 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (lightgbm): Update to 3.2.1. [source]: Add patch. Make some cosmetic changes. [native-inputs]: Add googletest, opencl-headers. [inputs]: Add boost, eigen, fast-double-parser, fmt, ocl-icd. [propagated-inputs]: Remove. [arguments]: Add #:tests?, #:test-target. Pass "-DBUILD_CPP_TEST=OFF" and "-DUSE_GPU=ON" to #:configure-flags. Remove custom 'check phase and skip running tests. * gnu/packages/patches/lightgbm-use-system-libraries.patch: New patch. * gnu/local.mk (dist_patch_DATA): Add it. --- Changed commit message slightly. gnu/local.mk | 1 + gnu/packages/machine-learning.scm | 55 ++++++++++-------- .../lightgbm-use-system-libraries.patch | 58 +++++++++++++++++++ 3 files changed, 90 insertions(+), 24 deletions(-) create mode 100644 gnu/packages/patches/lightgbm-use-system-libraries.patch diff --git a/gnu/local.mk b/gnu/local.mk index d415b892e9..35942e9188 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1420,6 +1420,7 @@ dist_patch_DATA = \ %D%/packages/patches/lierolibre-newer-libconfig.patch \ %D%/packages/patches/lierolibre-remove-arch-warning.patch \ %D%/packages/patches/lierolibre-try-building-other-arch.patch \ + %D%/packages/patches/lightgbm-use-system-libraries.patch \ %D%/packages/patches/linbox-fix-pkgconfig.patch \ %D%/packages/patches/linphone-desktop-without-sdk.patch \ %D%/packages/patches/linux-libre-support-for-Pinebook-Pro.patch \ diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 029422677a..99d864a25c 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -74,8 +74,10 @@ #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) #:use-module (gnu packages parallel) + #:use-module (gnu packages opencl) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages pretty-print) #:use-module (gnu packages protobuf) #:use-module (gnu packages python) #:use-module (gnu packages python-build) @@ -1221,33 +1223,38 @@ main intended application of Autograd is gradient-based optimization.") (define-public lightgbm (package (name "lightgbm") - (version "2.0.12") - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/Microsoft/LightGBM") - (commit (string-append "v" version)))) - (sha256 - (base32 - "0jlvyn7k81dzrh9ij3zw576wbgiwmmr26rzpdxjn1dbpc3njpvzi")) - (file-name (git-file-name name version)))) + (version "3.2.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/Microsoft/LightGBM") + (commit (string-append "v" version)))) + (patches + (search-patches "lightgbm-use-system-libraries.patch")) + (sha256 + (base32 "1fchiiwl3pz1kpi0kbfanif6c6b688wmmjjbqz4ff4djc6g2sg7z")) + (file-name (git-file-name name version)))) (native-inputs - `(("python-pytest" ,python-pytest) - ("python-nose" ,python-nose))) + `(("googletest" ,googletest) + ("opencl-headers" ,opencl-headers))) (inputs - `(("openmpi" ,openmpi))) - (propagated-inputs - `(("python-numpy" ,python-numpy) - ("python-scipy" ,python-scipy))) + `(("boost" ,boost) + ("eigen" ,eigen) + ("fast-double-parser" ,fast-double-parser) + ("fmt" ,fmt) + ("ocl-icd" ,ocl-icd) + ("openmpi" ,openmpi))) (arguments - `(#:configure-flags - '("-DUSE_MPI=ON") - #:phases - (modify-phases %standard-phases - (replace 'check - (lambda _ - (with-directory-excursion "../source" - (invoke "pytest" "tests/c_api_test/test_.py"))))))) + `(;; FIXME: With 'find_package(GTest CONFIG)' (default), googletest + ;; is not found by CMake. If we replace CONFIG with REQUIRED, the + ;; test files fail to link with googletest in the build phase. + #:tests? #f + #:test-target "testlightgbm" + #:configure-flags + (list "-DBUILD_CPP_TEST=OFF" + "-DUSE_GPU=ON" ;using OpenCL + "-DUSE_MPI=ON"))) (build-system cmake-build-system) (home-page "https://github.com/Microsoft/LightGBM") (synopsis "Gradient boosting framework based on decision tree algorithms") diff --git a/gnu/packages/patches/lightgbm-use-system-libraries.patch b/gnu/packages/patches/lightgbm-use-system-libraries.patch new file mode 100644 index 0000000000..115ca02775 --- /dev/null +++ b/gnu/packages/patches/lightgbm-use-system-libraries.patch @@ -0,0 +1,58 @@ +From 894648291b9629d51785e4fd1cae820d2858e039 Mon Sep 17 00:00:00 2001 +From: Vinicius Monego <monego@posteo.net> +Date: Mon, 31 May 2021 21:23:52 -0300 +Subject: [PATCH] Use system libraries. + +--- + CMakeLists.txt | 11 ++++++++--- + include/LightGBM/utils/common.h | 4 ++-- + 2 files changed, 10 insertions(+), 5 deletions(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index b845f36..560e477 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -93,9 +93,6 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + +-SET(EIGEN_DIR "${PROJECT_SOURCE_DIR}/external_libs/eigen") +-include_directories(${EIGEN_DIR}) +- + # See https://gitlab.com/libeigen/eigen/-/blob/master/COPYING.README + ADD_DEFINITIONS(-DEIGEN_MPL2_ONLY) + +@@ -399,6 +396,14 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + ++find_package (Eigen3 3.3 REQUIRED NO_MODULE) ++TARGET_LINK_LIBRARIES (lightgbm Eigen3::Eigen) ++TARGET_LINK_LIBRARIES (_lightgbm Eigen3::Eigen) ++ ++find_package(fmt) ++TARGET_LINK_LIBRARIES(lightgbm fmt::fmt) ++TARGET_LINK_LIBRARIES(_lightgbm fmt::fmt) ++ + if(USE_MPI) + TARGET_LINK_LIBRARIES(lightgbm ${MPI_CXX_LIBRARIES}) + TARGET_LINK_LIBRARIES(_lightgbm ${MPI_CXX_LIBRARIES}) +diff --git a/include/LightGBM/utils/common.h b/include/LightGBM/utils/common.h +index 4357357..00101a5 100644 +--- a/include/LightGBM/utils/common.h ++++ b/include/LightGBM/utils/common.h +@@ -32,9 +32,9 @@ + + #if (!((defined(sun) || defined(__sun)) && (defined(__SVR4) || defined(__svr4__)))) + #define FMT_HEADER_ONLY +-#include "../../../external_libs/fmt/include/fmt/format.h" ++#include "fmt/format.h" + #endif +-#include "../../../external_libs/fast_double_parser/include/fast_double_parser.h" ++#include "fast_double_parser.h" + + #ifdef _MSC_VER + #include <intrin.h> +-- +2.31.1 + -- 2.30.2 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v3 3/3] gnu: Add python-lightgbm. 2021-10-08 2:54 ` [bug#48785] [PATCH v3 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-10-08 2:54 ` [bug#48785] [PATCH v3 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego @ 2021-10-08 2:54 ` Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2021-10-08 2:54 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (python-lightgbm): New variable. --- Rebase only. gnu/packages/machine-learning.scm | 59 +++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 99d864a25c..e7ce107637 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1271,6 +1271,65 @@ the following advantages: @end itemize\n") (license license:expat))) +(define-public python-lightgbm + (package + (inherit lightgbm) + (name "python-lightgbm") + (source (package-source lightgbm)) + (build-system python-build-system) + (arguments + `(#:configure-flags (list "--precompile") + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'preparations + (lambda _ + (setenv "HOME" "/tmp") ;to write a log file + ;; Move Python files to source root to silence some warnings. + (rename-file "python-package/setup.py" "setup.py") + (rename-file "python-package/README.rst" "README.rst") + (rename-file "python-package/lightgbm" "lightgbm") + ;; Install version file to read the library's version. + (with-output-to-file "lightgbm/VERSION.txt" + (lambda () + (display ,(package-version lightgbm)))) + ;; Skip building the shared library. + (substitute* "setup.py" + (("'install_lib': CustomInstallLib,") "")))) + (add-after 'install 'install-liblightgbm + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (pylib (string-append out "/lib/python" + ,(version-major+minor + (package-version python)) + "/site-packages")) + (lgbdir (string-append pylib "/lightgbm")) + (lib-lightgbm (string-append (assoc-ref inputs "lightgbm") + "/lib/lib_lightgbm.so"))) + ;; This symlink is necessary for tests. + (symlink lib-lightgbm "lightgbm/lib_lightgbm.so") + ;; And this one for runtime. + (mkdir-p (string-append lgbdir "/lib")) + (symlink lib-lightgbm (string-append lgbdir + "/lib_lightgbm.so"))))) + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest" + "tests/python_package_test"))))))) + (native-inputs + `(("python-psutil" ,python-psutil) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) + (inputs + `(("lightgbm" ,lightgbm))) + (propagated-inputs + `(("python-numpy" ,python-numpy) + ("python-pandas" ,python-pandas) + ("python-scikit-learn" ,python-scikit-learn) + ("python-scipy" ,python-scipy))) + (synopsis "Python frontend for LightGBM"))) + (define-public vowpal-wabbit ;; Language bindings not included. (package -- 2.30.2 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v4 1/3] gnu: Add fast-double-parser. 2021-06-01 22:05 [bug#48785] [PATCH 0/3] Update LightGBM and split frontends Vinicius Monego ` (2 preceding siblings ...) 2021-10-08 2:54 ` [bug#48785] [PATCH v3 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2021-10-31 1:40 ` Vinicius Monego 2021-10-31 1:40 ` [bug#48785] [PATCH v4 2/3] gnu: lightgbm: Update to 3.3.1 Vinicius Monego 2021-10-31 1:40 ` [bug#48785] [PATCH v4 3/3] gnu: Add python-lightgbm Vinicius Monego 2023-04-07 15:17 ` [bug#48785] [PATCH v5 1/3] gnu: Add fast-double-parser Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 1/3] gnu: Add fast-double-parser Vinicius Monego 5 siblings, 2 replies; 19+ messages in thread From: Vinicius Monego @ 2021-10-31 1:40 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/cpp.scm (fast-double-parser): New variable. --- gnu/packages/cpp.scm | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm index 643b85a8db..3bf51e6e78 100644 --- a/gnu/packages/cpp.scm +++ b/gnu/packages/cpp.scm @@ -10,7 +10,7 @@ ;;; Copyright © 2020 Roel Janssen <roel@gnu.org> ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re> -;;; Copyright © 2020 Vinicius Monego <monego@posteo.net> +;;; Copyright © 2020, 2021 Vinicius Monego <monego@posteo.net> ;;; Copyright © 2020 Marius Bakke <marius@gnu.org> ;;; Copyright © 2020 Michael Rohleder <mike@rohleder.de> ;;; Copyright © 2020 Alexandros Theodotou <alex@zrythm.org> @@ -330,6 +330,42 @@ functions, class methods, and stl containers. @end enumerate\n") (license license:bsd-3))) +(define-public fast-double-parser + (package + (name "fast-double-parser") + (version "0.5.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/lemire/fast_double_parser") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "10mfpy4i7lpavghb15s4d0920nlwsac1ri2sxkcqbrj9iq94qj71")))) + (build-system cmake-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'install ;no install target, have to do it manually + (lambda* (#:key outputs #:allow-other-keys) + (let ((out-include (string-append (assoc-ref outputs "out") + "/include"))) + (mkdir-p out-include) + (copy-file "../source/include/fast_double_parser.h" + (string-append out-include + "/fast_double_parser.h")))))))) + (home-page "https://github.com/lemire/fast_double_parser") + (synopsis "Parse strings into double (binary64) floating-point values") + (description "Fast function to parse strings containing decimal numbers +into double-precision (binary64) floating-point values. That is, given the +string @code{1.0e10}, it should return a 64-bit floating-point value equal to +10000000000. The function will match exactly (down the smallest bit) the +result of a standard function like @code{strtod}.") + ;; Dual-licensed, either license applies. + (license (list license:asl2.0 + license:boost1.0)))) + (define-public fifo-map (let* ((commit "0dfbf5dacbb15a32c43f912a7e66a54aae39d0f9") (revision "0") base-commit: 142344b6efa42e1a4d57a4bdf14dfb3ca1cba25a -- 2.30.2 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v4 2/3] gnu: lightgbm: Update to 3.3.1. 2021-10-31 1:40 ` [bug#48785] [PATCH v4 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2021-10-31 1:40 ` Vinicius Monego 2021-10-31 1:40 ` [bug#48785] [PATCH v4 3/3] gnu: Add python-lightgbm Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2021-10-31 1:40 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (lightgbm): Update to 3.3.1. [source]: Add patch. [native-inputs]: Add opencl-headers. [inputs]: Add boost, eigen, fast-double-parser, fmt, ocl-icd. [propagated-inputs]: Remove. [arguments]: Add #:tests?, #:test-target. Pass "-DBUILD_CPP_TEST=OFF" and "-DUSE_GPU=ON" to #:configure-flags. Don't override 'check phase. * gnu/packages/patches/lightgbm-use-system-libraries.patch: New patch. * gnu/local.mk (dist_patch_DATA): Add it. --- Updated to 3.3.1. gnu/local.mk | 1 + gnu/packages/machine-learning.scm | 52 +++++++++-------- .../lightgbm-use-system-libraries.patch | 58 +++++++++++++++++++ 3 files changed, 87 insertions(+), 24 deletions(-) create mode 100644 gnu/packages/patches/lightgbm-use-system-libraries.patch diff --git a/gnu/local.mk b/gnu/local.mk index baba6be058..4dcfe9996c 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1437,6 +1437,7 @@ dist_patch_DATA = \ %D%/packages/patches/lierolibre-newer-libconfig.patch \ %D%/packages/patches/lierolibre-remove-arch-warning.patch \ %D%/packages/patches/lierolibre-try-building-other-arch.patch \ + %D%/packages/patches/lightgbm-use-system-libraries.patch \ %D%/packages/patches/linbox-fix-pkgconfig.patch \ %D%/packages/patches/linphone-desktop-without-sdk.patch \ %D%/packages/patches/linux-libre-support-for-Pinebook-Pro.patch \ diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 029422677a..c7613ebe4e 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -74,8 +74,10 @@ #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) #:use-module (gnu packages parallel) + #:use-module (gnu packages opencl) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages pretty-print) #:use-module (gnu packages protobuf) #:use-module (gnu packages python) #:use-module (gnu packages python-build) @@ -1221,33 +1223,35 @@ main intended application of Autograd is gradient-based optimization.") (define-public lightgbm (package (name "lightgbm") - (version "2.0.12") - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/Microsoft/LightGBM") - (commit (string-append "v" version)))) - (sha256 - (base32 - "0jlvyn7k81dzrh9ij3zw576wbgiwmmr26rzpdxjn1dbpc3njpvzi")) - (file-name (git-file-name name version)))) + (version "3.3.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/Microsoft/LightGBM") + (commit (string-append "v" version)))) + (patches + (search-patches "lightgbm-use-system-libraries.patch")) + (sha256 + (base32 "0cwyb371xra5qnwkhpn223z3yqrppisgvx173an8i3f6m83zwb38")) + (file-name (git-file-name name version)))) (native-inputs - `(("python-pytest" ,python-pytest) - ("python-nose" ,python-nose))) + `(;; ("googletest" ,googletest) + ("opencl-headers" ,opencl-headers))) (inputs - `(("openmpi" ,openmpi))) - (propagated-inputs - `(("python-numpy" ,python-numpy) - ("python-scipy" ,python-scipy))) + `(("boost" ,boost) + ("eigen" ,eigen) + ("fast-double-parser" ,fast-double-parser) + ("fmt" ,fmt) + ("ocl-icd" ,ocl-icd) + ("openmpi" ,openmpi))) (arguments - `(#:configure-flags - '("-DUSE_MPI=ON") - #:phases - (modify-phases %standard-phases - (replace 'check - (lambda _ - (with-directory-excursion "../source" - (invoke "pytest" "tests/c_api_test/test_.py"))))))) + `(#:tests? #f ;; FIXME: googletest is not found by CMake + #:test-target "testlightgbm" + #:configure-flags + (list "-DBUILD_CPP_TEST=OFF" + "-DUSE_GPU=ON" ;using OpenCL + "-DUSE_MPI=ON"))) (build-system cmake-build-system) (home-page "https://github.com/Microsoft/LightGBM") (synopsis "Gradient boosting framework based on decision tree algorithms") diff --git a/gnu/packages/patches/lightgbm-use-system-libraries.patch b/gnu/packages/patches/lightgbm-use-system-libraries.patch new file mode 100644 index 0000000000..115ca02775 --- /dev/null +++ b/gnu/packages/patches/lightgbm-use-system-libraries.patch @@ -0,0 +1,58 @@ +From 894648291b9629d51785e4fd1cae820d2858e039 Mon Sep 17 00:00:00 2001 +From: Vinicius Monego <monego@posteo.net> +Date: Mon, 31 May 2021 21:23:52 -0300 +Subject: [PATCH] Use system libraries. + +--- + CMakeLists.txt | 11 ++++++++--- + include/LightGBM/utils/common.h | 4 ++-- + 2 files changed, 10 insertions(+), 5 deletions(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index b845f36..560e477 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -93,9 +93,6 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + +-SET(EIGEN_DIR "${PROJECT_SOURCE_DIR}/external_libs/eigen") +-include_directories(${EIGEN_DIR}) +- + # See https://gitlab.com/libeigen/eigen/-/blob/master/COPYING.README + ADD_DEFINITIONS(-DEIGEN_MPL2_ONLY) + +@@ -399,6 +396,14 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + ++find_package (Eigen3 3.3 REQUIRED NO_MODULE) ++TARGET_LINK_LIBRARIES (lightgbm Eigen3::Eigen) ++TARGET_LINK_LIBRARIES (_lightgbm Eigen3::Eigen) ++ ++find_package(fmt) ++TARGET_LINK_LIBRARIES(lightgbm fmt::fmt) ++TARGET_LINK_LIBRARIES(_lightgbm fmt::fmt) ++ + if(USE_MPI) + TARGET_LINK_LIBRARIES(lightgbm ${MPI_CXX_LIBRARIES}) + TARGET_LINK_LIBRARIES(_lightgbm ${MPI_CXX_LIBRARIES}) +diff --git a/include/LightGBM/utils/common.h b/include/LightGBM/utils/common.h +index 4357357..00101a5 100644 +--- a/include/LightGBM/utils/common.h ++++ b/include/LightGBM/utils/common.h +@@ -32,9 +32,9 @@ + + #if (!((defined(sun) || defined(__sun)) && (defined(__SVR4) || defined(__svr4__)))) + #define FMT_HEADER_ONLY +-#include "../../../external_libs/fmt/include/fmt/format.h" ++#include "fmt/format.h" + #endif +-#include "../../../external_libs/fast_double_parser/include/fast_double_parser.h" ++#include "fast_double_parser.h" + + #ifdef _MSC_VER + #include <intrin.h> +-- +2.31.1 + -- 2.30.2 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v4 3/3] gnu: Add python-lightgbm. 2021-10-31 1:40 ` [bug#48785] [PATCH v4 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-10-31 1:40 ` [bug#48785] [PATCH v4 2/3] gnu: lightgbm: Update to 3.3.1 Vinicius Monego @ 2021-10-31 1:40 ` Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2021-10-31 1:40 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (python-lightgbm): New variable. --- gnu/packages/machine-learning.scm | 59 +++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index c7613ebe4e..59e9fdc75b 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1268,6 +1268,65 @@ the following advantages: @end itemize\n") (license license:expat))) +(define-public python-lightgbm + (package + (inherit lightgbm) + (name "python-lightgbm") + (source (package-source lightgbm)) + (build-system python-build-system) + (arguments + `(#:configure-flags (list "--precompile") ; use precompiled library + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'preparations + (lambda _ + (setenv "HOME" "/tmp") ;to write a log file + ;; Move Python files to source root to silence some warnings. + (rename-file "python-package/setup.py" "setup.py") + (rename-file "python-package/README.rst" "README.rst") + (rename-file "python-package/lightgbm" "lightgbm") + ;; Install version file to read the library's version. + (with-output-to-file "lightgbm/VERSION.txt" + (lambda () + (display ,(package-version lightgbm)))) + ;; Skip building the shared library. + (substitute* "setup.py" + (("'install_lib': CustomInstallLib,") "")))) + (add-after 'install 'install-liblightgbm + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (pylib (string-append out "/lib/python" + ,(version-major+minor + (package-version python)) + "/site-packages")) + (lgbdir (string-append pylib "/lightgbm")) + (lib-lightgbm (string-append (assoc-ref inputs "lightgbm") + "/lib/lib_lightgbm.so"))) + ;; This symlink is necessary for tests. + (symlink lib-lightgbm "lightgbm/lib_lightgbm.so") + ;; And this one for runtime. + (mkdir-p (string-append lgbdir "/lib")) + (symlink lib-lightgbm (string-append lgbdir + "/lib_lightgbm.so"))))) + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest" + "tests/python_package_test"))))))) + (native-inputs + `(("python-psutil" ,python-psutil) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) + (inputs + `(("lightgbm" ,lightgbm))) + (propagated-inputs + `(("python-numpy" ,python-numpy) + ("python-pandas" ,python-pandas) + ("python-scikit-learn" ,python-scikit-learn) + ("python-scipy" ,python-scipy))) + (synopsis "Python frontend for LightGBM"))) + (define-public vowpal-wabbit ;; Language bindings not included. (package -- 2.30.2 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v5 1/3] gnu: Add fast-double-parser. 2021-06-01 22:05 [bug#48785] [PATCH 0/3] Update LightGBM and split frontends Vinicius Monego ` (3 preceding siblings ...) 2021-10-31 1:40 ` [bug#48785] [PATCH v4 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2023-04-07 15:17 ` Vinicius Monego 2023-04-07 15:17 ` [bug#48785] [PATCH v5 2/3] gnu: lightgbm: Update to 3.3.5 Vinicius Monego 2023-04-07 15:17 ` [bug#48785] [PATCH v5 3/3] gnu: Add python-lightgbm Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 1/3] gnu: Add fast-double-parser Vinicius Monego 5 siblings, 2 replies; 19+ messages in thread From: Vinicius Monego @ 2023-04-07 15:17 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/cpp.scm (fast-double-parser): New variable. --- gnu/packages/cpp.scm | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm index ed6ae69198..af1d436bc2 100644 --- a/gnu/packages/cpp.scm +++ b/gnu/packages/cpp.scm @@ -10,7 +10,7 @@ ;;; Copyright © 2020 Roel Janssen <roel@gnu.org> ;;; Copyright © 2020, 2021, 2023 Ricardo Wurmus <rekado@elephly.net> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re> -;;; Copyright © 2020, 2021, 2022 Vinicius Monego <monego@posteo.net> +;;; Copyright © 2020, 2021, 2022, 2023 Vinicius Monego <monego@posteo.net> ;;; Copyright © 2020, 2022 Marius Bakke <marius@gnu.org> ;;; Copyright © 2020 Michael Rohleder <mike@rohleder.de> ;;; Copyright © 2020 Alexandros Theodotou <alex@zrythm.org> @@ -445,6 +445,39 @@ (define-public xsimd operating on batches.") (license license:bsd-3))) +(define-public fast-double-parser + (package + (name "fast-double-parser") + (version "0.7.0") + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/lemire/fast_double_parser") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "0c3ajwm65yip61iq3ybvrkx5d3136r9dc68j1c1fmrv61n5nfmlb")))) + (build-system cmake-build-system) + (arguments + (list #:phases + #~(modify-phases %standard-phases + ;; There is no install target, we have to do it manually. + (replace 'install + (lambda _ + (mkdir-p (string-append #$output "/include")) + (copy-file + (string-append #$(package-source this-package) + "/include/fast_double_parser.h") + (string-append #$output + "/include/fast_double_parser.h"))))))) + (home-page "https://github.com/lemire/fast_double_parser") + (synopsis "Parse strings into double floating-point values") + (description "@code{fast-double-parser} provides a fast function to parse +strings into double (binary64) floating-point values, enforces the RFC 7159 +(JSON standard) grammar.") + (license license:asl2.0))) + (define-public google-highway (package (name "google-highway") base-commit: 2853a4dd5eee7eedb1c19a5fd2d0f2ca468a4509 -- 2.34.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v5 2/3] gnu: lightgbm: Update to 3.3.5. 2023-04-07 15:17 ` [bug#48785] [PATCH v5 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2023-04-07 15:17 ` Vinicius Monego 2023-04-07 15:17 ` [bug#48785] [PATCH v5 3/3] gnu: Add python-lightgbm Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2023-04-07 15:17 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (lightgbm): Update to 3.3.5. [source]: Add patch. [native-inputs]: Add opencl-headers. [inputs]: Add boost, eigen, fast-double-parser, fmt, ocl-icd. [propagated-inputs]: Remove python-numpy, python-scipy. [arguments]: Add #:tests?, #:test-target. Pass "-DBUILD_CPP_TEST=OFF" and "-DUSE_GPU=ON" to #:configure-flags. Don't override the check phase. Use G-expressions. * gnu/packages/patches/lightgbm-use-system-libraries.patch: New patch. * gnu/local.mk (dist_patch_DATA): Add it. --- gnu/local.mk | 1 + gnu/packages/machine-learning.scm | 46 +++++++-------- .../lightgbm-use-system-libraries.patch | 58 +++++++++++++++++++ 3 files changed, 81 insertions(+), 24 deletions(-) create mode 100644 gnu/packages/patches/lightgbm-use-system-libraries.patch diff --git a/gnu/local.mk b/gnu/local.mk index b7e19b6bc2..2ed70c98bf 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1489,6 +1489,7 @@ dist_patch_DATA = \ %D%/packages/patches/lierolibre-newer-libconfig.patch \ %D%/packages/patches/lierolibre-remove-arch-warning.patch \ %D%/packages/patches/lierolibre-try-building-other-arch.patch \ + %D%/packages/patches/lightgbm-use-system-libraries.patch \ %D%/packages/patches/linbox-fix-pkgconfig.patch \ %D%/packages/patches/linphone-desktop-without-sdk.patch \ %D%/packages/patches/linux-libre-infodocs-target.patch \ diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 37d4ef78ad..248e318eea 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -83,9 +83,11 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages ninja) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages parallel) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages pretty-print) #:use-module (gnu packages protobuf) #:use-module (gnu packages pulseaudio) #:use-module (gnu packages python) @@ -1472,31 +1474,27 @@ (define-public python-autograd (define-public lightgbm (package (name "lightgbm") - (version "2.0.12") - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/Microsoft/LightGBM") - (commit (string-append "v" version)))) - (sha256 - (base32 - "0jlvyn7k81dzrh9ij3zw576wbgiwmmr26rzpdxjn1dbpc3njpvzi")) - (file-name (git-file-name name version)))) - (native-inputs - (list python-pytest python-nose)) - (inputs - (list openmpi)) - (propagated-inputs - (list python-numpy python-scipy)) + (version "3.3.5") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/Microsoft/LightGBM") + (commit (string-append "v" version)))) + (patches + (search-patches "lightgbm-use-system-libraries.patch")) + (sha256 + (base32 "0fljcnlj02pn2y5lxq16rsg6hm0wq4adzrl52i0s8whpcdrgsc0a")) + (file-name (git-file-name name version)))) (arguments - `(#:configure-flags - '("-DUSE_MPI=ON") - #:phases - (modify-phases %standard-phases - (replace 'check - (lambda _ - (with-directory-excursion "../source" - (invoke "pytest" "tests/c_api_test/test_.py"))))))) + (list + #:tests? #f ; FIXME: googletest is not found by CMake + #:test-target "testlightgbm" + #:configure-flags #~(list "-DBUILD_CPP_TEST=OFF" + "-DUSE_GPU=ON" ;using OpenCL + "-DUSE_MPI=ON"))) + (native-inputs (list opencl-headers)) + (inputs (list boost eigen fast-double-parser fmt ocl-icd openmpi)) (build-system cmake-build-system) (home-page "https://github.com/Microsoft/LightGBM") (synopsis "Gradient boosting framework based on decision tree algorithms") diff --git a/gnu/packages/patches/lightgbm-use-system-libraries.patch b/gnu/packages/patches/lightgbm-use-system-libraries.patch new file mode 100644 index 0000000000..115ca02775 --- /dev/null +++ b/gnu/packages/patches/lightgbm-use-system-libraries.patch @@ -0,0 +1,58 @@ +From 894648291b9629d51785e4fd1cae820d2858e039 Mon Sep 17 00:00:00 2001 +From: Vinicius Monego <monego@posteo.net> +Date: Mon, 31 May 2021 21:23:52 -0300 +Subject: [PATCH] Use system libraries. + +--- + CMakeLists.txt | 11 ++++++++--- + include/LightGBM/utils/common.h | 4 ++-- + 2 files changed, 10 insertions(+), 5 deletions(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index b845f36..560e477 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -93,9 +93,6 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + +-SET(EIGEN_DIR "${PROJECT_SOURCE_DIR}/external_libs/eigen") +-include_directories(${EIGEN_DIR}) +- + # See https://gitlab.com/libeigen/eigen/-/blob/master/COPYING.README + ADD_DEFINITIONS(-DEIGEN_MPL2_ONLY) + +@@ -399,6 +396,14 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + ++find_package (Eigen3 3.3 REQUIRED NO_MODULE) ++TARGET_LINK_LIBRARIES (lightgbm Eigen3::Eigen) ++TARGET_LINK_LIBRARIES (_lightgbm Eigen3::Eigen) ++ ++find_package(fmt) ++TARGET_LINK_LIBRARIES(lightgbm fmt::fmt) ++TARGET_LINK_LIBRARIES(_lightgbm fmt::fmt) ++ + if(USE_MPI) + TARGET_LINK_LIBRARIES(lightgbm ${MPI_CXX_LIBRARIES}) + TARGET_LINK_LIBRARIES(_lightgbm ${MPI_CXX_LIBRARIES}) +diff --git a/include/LightGBM/utils/common.h b/include/LightGBM/utils/common.h +index 4357357..00101a5 100644 +--- a/include/LightGBM/utils/common.h ++++ b/include/LightGBM/utils/common.h +@@ -32,9 +32,9 @@ + + #if (!((defined(sun) || defined(__sun)) && (defined(__SVR4) || defined(__svr4__)))) + #define FMT_HEADER_ONLY +-#include "../../../external_libs/fmt/include/fmt/format.h" ++#include "fmt/format.h" + #endif +-#include "../../../external_libs/fast_double_parser/include/fast_double_parser.h" ++#include "fast_double_parser.h" + + #ifdef _MSC_VER + #include <intrin.h> +-- +2.31.1 + -- 2.34.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v5 3/3] gnu: Add python-lightgbm. 2023-04-07 15:17 ` [bug#48785] [PATCH v5 1/3] gnu: Add fast-double-parser Vinicius Monego 2023-04-07 15:17 ` [bug#48785] [PATCH v5 2/3] gnu: lightgbm: Update to 3.3.5 Vinicius Monego @ 2023-04-07 15:17 ` Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2023-04-07 15:17 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (python-lightgbm): New variable. --- gnu/packages/machine-learning.scm | 55 +++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 248e318eea..1585f09992 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1511,6 +1511,61 @@ (define-public lightgbm @end itemize\n") (license license:expat))) +(define-public python-lightgbm + (package + (inherit lightgbm) + (name "python-lightgbm") + (source (package-source lightgbm)) + (build-system python-build-system) + (arguments + (list #:configure-flags #~(list "--precompile") ; use our library + #:phases + #~(modify-phases %standard-phases + (add-after 'unpack 'preparations + (lambda _ + (setenv "HOME" "/tmp") ; to write a log file + (rename-file "python-package/setup.py" "setup.py") + (rename-file "python-package/README.rst" "README.rst") + (rename-file "python-package/lightgbm" "lightgbm") + ;; Write the version so the module identifies it. + (with-output-to-file "lightgbm/VERSION.txt" + (lambda () (display #$(package-version lightgbm)))) + ;; Manually install the lib. + (substitute* "setup.py" + (("'install_lib': CustomInstallLib,") "")))) + (add-after 'install 'install-liblightgbm + ;; LightGBM's setup.py hardcodes LIB_PATH to look for + ;; lib_lightgbm.so in this source, which we can't modify. + ;; So, we symlink it in the install phase. + (lambda _ + (let* ((pylib (string-append #$output + "/lib/python" + #$(version-major+minor + (package-version + python)) + "/site-packages")) + (lgbdir (string-append pylib "/lightgbm")) + (lib-lightgbm (string-append + #$(this-package-input "lightgbm") + "/lib/lib_lightgbm.so"))) + ;; For whatever reason, the first symlink is required for + ;; the check phase and the second for sanity-check. + (symlink lib-lightgbm "lightgbm/lib_lightgbm.so") + (mkdir-p (string-append lgbdir "/lib")) + (symlink lib-lightgbm + (string-append lgbdir + "/lib_lightgbm.so"))))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (invoke "python" "-m" "pytest" + "tests/python_package_test"))))))) + (native-inputs (list python-psutil python-pytest python-wheel)) + (inputs (list lightgbm)) + (propagated-inputs (list python-numpy python-pandas python-scikit-learn + python-scipy)) + (synopsis "Python interface for LightGBM"))) + (define-public vowpal-wabbit ;; Language bindings not included. (package -- 2.34.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v6 1/3] gnu: Add fast-double-parser. 2021-06-01 22:05 [bug#48785] [PATCH 0/3] Update LightGBM and split frontends Vinicius Monego ` (4 preceding siblings ...) 2023-04-07 15:17 ` [bug#48785] [PATCH v5 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2023-06-26 1:06 ` Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 2/3] gnu: lightgbm: Update to 3.3.5 Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 3/3] gnu: Add python-lightgbm Vinicius Monego 5 siblings, 2 replies; 19+ messages in thread From: Vinicius Monego @ 2023-06-26 1:06 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/cpp.scm (fast-double-parser): New variable. --- gnu/packages/cpp.scm | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm index 39a34c20dd..65e040700c 100644 --- a/gnu/packages/cpp.scm +++ b/gnu/packages/cpp.scm @@ -10,7 +10,7 @@ ;;; Copyright © 2020 Roel Janssen <roel@gnu.org> ;;; Copyright © 2020, 2021, 2023 Ricardo Wurmus <rekado@elephly.net> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re> -;;; Copyright © 2020, 2021, 2022 Vinicius Monego <monego@posteo.net> +;;; Copyright © 2020, 2021, 2022, 2023 Vinicius Monego <monego@posteo.net> ;;; Copyright © 2020, 2022 Marius Bakke <marius@gnu.org> ;;; Copyright © 2020 Michael Rohleder <mike@rohleder.de> ;;; Copyright © 2020 Alexandros Theodotou <alex@zrythm.org> @@ -448,6 +448,40 @@ (define-public xsimd operating on batches.") (license license:bsd-3))) +(define-public fast-double-parser + (package + (name "fast-double-parser") + (version "0.7.0") + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/lemire/fast_double_parser") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "0c3ajwm65yip61iq3ybvrkx5d3136r9dc68j1c1fmrv61n5nfmlb")))) + (build-system cmake-build-system) + (arguments + (list #:phases + #~(modify-phases %standard-phases + ;; There is no install target, we have to do it manually. + (replace 'install + (lambda _ + (mkdir-p (string-append #$output "/include")) + (copy-file + (string-append #$(package-source this-package) + "/include/fast_double_parser.h") + (string-append #$output + "/include/fast_double_parser.h"))))))) + (home-page "https://github.com/lemire/fast_double_parser") + (synopsis "Parse strings into double floating-point values") + (description "@code{fast-double-parser} provides a fast function to parse +strings into double (binary64) floating-point values, enforces the RFC 7159 +(JSON standard) grammar.") + ;; Dual-licensed, either license applies. + (license (list license:asl2.0 license:boost1.0)))) + (define-public google-highway (package (name "google-highway") base-commit: 41e8726c8924e9f5b79a9f44fad94ab5586bfc50 -- 2.34.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v6 2/3] gnu: lightgbm: Update to 3.3.5. 2023-06-26 1:06 ` [bug#48785] [PATCH v6 1/3] gnu: Add fast-double-parser Vinicius Monego @ 2023-06-26 1:06 ` Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 3/3] gnu: Add python-lightgbm Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2023-06-26 1:06 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (lightgbm): Update to 3.3.5. [source]: Add patch. [native-inputs]: Add opencl-headers. [inputs]: Add boost, eigen, fast-double-parser, fmt, ocl-icd. [propagated-inputs]: Remove python-numpy, python-scipy. [arguments]: Add #:tests?, #:test-target. Pass "-DBUILD_CPP_TEST=OFF" and "-DUSE_GPU=ON" to #:configure-flags. Don't override the check phase. Use G-expressions. * gnu/packages/patches/lightgbm-use-system-libraries.patch: New patch. * gnu/local.mk (dist_patch_DATA): Add it. --- gnu/local.mk | 1 + gnu/packages/machine-learning.scm | 46 +++++++-------- .../lightgbm-use-system-libraries.patch | 57 +++++++++++++++++++ 3 files changed, 80 insertions(+), 24 deletions(-) create mode 100644 gnu/packages/patches/lightgbm-use-system-libraries.patch diff --git a/gnu/local.mk b/gnu/local.mk index 4566f1b4a4..4445a08fb7 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1527,6 +1527,7 @@ dist_patch_DATA = \ %D%/packages/patches/lierolibre-remove-arch-warning.patch \ %D%/packages/patches/lierolibre-try-building-other-arch.patch \ %D%/packages/patches/libcdio-glibc-compat.patch \ + %D%/packages/patches/lightgbm-use-system-libraries.patch \ %D%/packages/patches/linbox-fix-pkgconfig.patch \ %D%/packages/patches/linphone-desktop-without-sdk.patch \ %D%/packages/patches/linux-libre-infodocs-target.patch \ diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 8fbb0274d4..17de467f46 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -88,9 +88,11 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages ninja) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages parallel) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages pretty-print) #:use-module (gnu packages protobuf) #:use-module (gnu packages pulseaudio) #:use-module (gnu packages python) @@ -1800,31 +1802,27 @@ (define-public python-autograd (define-public lightgbm (package (name "lightgbm") - (version "2.0.12") - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/Microsoft/LightGBM") - (commit (string-append "v" version)))) - (sha256 - (base32 - "0jlvyn7k81dzrh9ij3zw576wbgiwmmr26rzpdxjn1dbpc3njpvzi")) - (file-name (git-file-name name version)))) - (native-inputs - (list python-pytest python-nose)) - (inputs - (list openmpi)) - (propagated-inputs - (list python-numpy python-scipy)) + (version "3.3.5") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/Microsoft/LightGBM") + (commit (string-append "v" version)))) + (patches + (search-patches "lightgbm-use-system-libraries.patch")) + (sha256 + (base32 "0fljcnlj02pn2y5lxq16rsg6hm0wq4adzrl52i0s8whpcdrgsc0a")) + (file-name (git-file-name name version)))) (arguments - `(#:configure-flags - '("-DUSE_MPI=ON") - #:phases - (modify-phases %standard-phases - (replace 'check - (lambda _ - (with-directory-excursion "../source" - (invoke "pytest" "tests/c_api_test/test_.py"))))))) + (list + #:tests? #f ; FIXME: googletest is not found by CMake + #:test-target "testlightgbm" + #:configure-flags #~(list "-DBUILD_CPP_TEST=OFF" + "-DUSE_GPU=ON" ;using OpenCL + "-DUSE_MPI=ON"))) + (native-inputs (list opencl-headers)) + (inputs (list boost eigen fast-double-parser fmt ocl-icd openmpi)) (build-system cmake-build-system) (home-page "https://github.com/Microsoft/LightGBM") (synopsis "Gradient boosting framework based on decision tree algorithms") diff --git a/gnu/packages/patches/lightgbm-use-system-libraries.patch b/gnu/packages/patches/lightgbm-use-system-libraries.patch new file mode 100644 index 0000000000..c0665cfff9 --- /dev/null +++ b/gnu/packages/patches/lightgbm-use-system-libraries.patch @@ -0,0 +1,57 @@ +From 894648291b9629d51785e4fd1cae820d2858e039 Mon Sep 17 00:00:00 2001 +From: Vinicius Monego <monego@posteo.net> +Date: Mon, 31 May 2021 21:23:52 -0300 +Subject: [PATCH] Use system libraries. + +--- + CMakeLists.txt | 11 ++++++++--- + include/LightGBM/utils/common.h | 4 ++-- + 2 files changed, 10 insertions(+), 5 deletions(-) + +diff --git a/CMakeLists.txt b/CMakeLists.txt +index b845f36..560e477 100644 +--- a/CMakeLists.txt ++++ b/CMakeLists.txt +@@ -93,9 +93,6 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + +-SET(EIGEN_DIR "${PROJECT_SOURCE_DIR}/external_libs/eigen") +-include_directories(${EIGEN_DIR}) +- + # See https://gitlab.com/libeigen/eigen/-/blob/master/COPYING.README + ADD_DEFINITIONS(-DEIGEN_MPL2_ONLY) + +@@ -399,6 +396,14 @@ if(USE_SWIG) + endif() + endif(USE_SWIG) + ++find_package (Eigen3 3.3 REQUIRED NO_MODULE) ++TARGET_LINK_LIBRARIES (lightgbm Eigen3::Eigen) ++TARGET_LINK_LIBRARIES (_lightgbm Eigen3::Eigen) ++ ++find_package(fmt) ++TARGET_LINK_LIBRARIES(lightgbm fmt::fmt) ++TARGET_LINK_LIBRARIES(_lightgbm fmt::fmt) ++ + if(USE_MPI) + TARGET_LINK_LIBRARIES(lightgbm ${MPI_CXX_LIBRARIES}) + TARGET_LINK_LIBRARIES(_lightgbm ${MPI_CXX_LIBRARIES}) +diff --git a/include/LightGBM/utils/common.h b/include/LightGBM/utils/common.h +index 4357357..00101a5 100644 +--- a/include/LightGBM/utils/common.h ++++ b/include/LightGBM/utils/common.h +@@ -32,9 +32,9 @@ + + #if (!((defined(sun) || defined(__sun)) && (defined(__SVR4) || defined(__svr4__)))) + #define FMT_HEADER_ONLY +-#include "../../../external_libs/fmt/include/fmt/format.h" ++#include "fmt/format.h" + #endif +-#include "../../../external_libs/fast_double_parser/include/fast_double_parser.h" ++#include "fast_double_parser.h" + + #ifdef _MSC_VER + #include <intrin.h> +-- +2.31.1 -- 2.34.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [bug#48785] [PATCH v6 3/3] gnu: Add python-lightgbm. 2023-06-26 1:06 ` [bug#48785] [PATCH v6 1/3] gnu: Add fast-double-parser Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 2/3] gnu: lightgbm: Update to 3.3.5 Vinicius Monego @ 2023-06-26 1:06 ` Vinicius Monego 1 sibling, 0 replies; 19+ messages in thread From: Vinicius Monego @ 2023-06-26 1:06 UTC (permalink / raw) To: 48785; +Cc: Vinicius Monego * gnu/packages/machine-learning.scm (python-lightgbm): New variable. --- Removed symlinks in favor of substituting lib_path. gnu/packages/machine-learning.scm | 39 +++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 17de467f46..bba89d789d 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1839,6 +1839,45 @@ (define-public lightgbm @end itemize\n") (license license:expat))) +(define-public python-lightgbm + (package + (inherit lightgbm) + (name "python-lightgbm") + (source (package-source lightgbm)) + (build-system python-build-system) + (arguments + (list #:configure-flags #~(list "--precompile") ; use our library + #:phases + #~(modify-phases %standard-phases + (add-after 'unpack 'preparations + (lambda _ + (setenv "HOME" "/tmp") ; to write a log file + (rename-file "python-package/setup.py" "setup.py") + (rename-file "python-package/README.rst" "README.rst") + (rename-file "python-package/lightgbm" "lightgbm") + ;; Write the version so the module identifies it. + (with-output-to-file "lightgbm/VERSION.txt" + (lambda () (display #$(package-version lightgbm)))) + ;; Manually install the lib. + (substitute* "setup.py" + (("'install_lib': CustomInstallLib,") "")) + ;; Change lib_path to look for our package's lib. + (substitute* "lightgbm/libpath.py" + (("lib_path = \\[str\\(p\\) for p in dll_path if p\\.is_file\\(\\)]") + ;; lib_path is a list of paths. + (format #f "lib_path = ['~a/lib/lib_lightgbm.so']" + #$(this-package-input "lightgbm")))))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (invoke "python" "-m" "pytest" + "tests/python_package_test"))))))) + (native-inputs (list python-psutil python-pytest python-wheel)) + (inputs (list lightgbm)) + (propagated-inputs (list python-numpy python-pandas python-scikit-learn + python-scipy)) + (synopsis "Python interface for LightGBM"))) + (define-public vowpal-wabbit ;; Language bindings not included. (package -- 2.34.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
end of thread, other threads:[~2023-06-26 1:08 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-06-01 22:05 [bug#48785] [PATCH 0/3] Update LightGBM and split frontends Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego 2021-06-01 22:06 ` [bug#48785] [PATCH 3/3] gnu: Add python-lightgbm Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego 2021-06-30 20:09 ` [bug#48785] [PATCH v2 3/3] gnu: Add python-lightgbm Vinicius Monego 2021-10-08 2:54 ` [bug#48785] [PATCH v3 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-10-08 2:54 ` [bug#48785] [PATCH v3 2/3] gnu: lightgbm: Update to 3.2.1 Vinicius Monego 2021-10-08 2:54 ` [bug#48785] [PATCH v3 3/3] gnu: Add python-lightgbm Vinicius Monego 2021-10-31 1:40 ` [bug#48785] [PATCH v4 1/3] gnu: Add fast-double-parser Vinicius Monego 2021-10-31 1:40 ` [bug#48785] [PATCH v4 2/3] gnu: lightgbm: Update to 3.3.1 Vinicius Monego 2021-10-31 1:40 ` [bug#48785] [PATCH v4 3/3] gnu: Add python-lightgbm Vinicius Monego 2023-04-07 15:17 ` [bug#48785] [PATCH v5 1/3] gnu: Add fast-double-parser Vinicius Monego 2023-04-07 15:17 ` [bug#48785] [PATCH v5 2/3] gnu: lightgbm: Update to 3.3.5 Vinicius Monego 2023-04-07 15:17 ` [bug#48785] [PATCH v5 3/3] gnu: Add python-lightgbm Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 1/3] gnu: Add fast-double-parser Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 2/3] gnu: lightgbm: Update to 3.3.5 Vinicius Monego 2023-06-26 1:06 ` [bug#48785] [PATCH v6 3/3] gnu: Add python-lightgbm Vinicius Monego
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.