From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marius Bakke Subject: Re: [PATCH] gnu: Add dlib. Date: Tue, 16 Aug 2016 11:45:16 +0100 Message-ID: <8760r10z7n.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> References: <8737m837zk.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <20160814172544.GA28736@jasmine> <87twengmbh.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <87a8gexyrz.fsf@gmail.com> <87oa4ugsi4.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <20160815201528.GA23963@jasmine> <87a8gd22ug.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <20160815222840.GA10735@jasmine> <4752bc68-5466-6c26-a7b4-e53aec400ff5@uq.edu.au> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:47752) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bZbsF-00013s-8a for guix-devel@gnu.org; Tue, 16 Aug 2016 06:45:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bZbsA-0003Jn-5g for guix-devel@gnu.org; Tue, 16 Aug 2016 06:45:34 -0400 Received: from out2-smtp.messagingengine.com ([66.111.4.26]:37735) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bZbs7-0003BC-K7 for guix-devel@gnu.org; Tue, 16 Aug 2016 06:45:30 -0400 In-Reply-To: <4752bc68-5466-6c26-a7b4-e53aec400ff5@uq.edu.au> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ben Woodcroft , Leo Famulari Cc: guix-devel , Alex Kost --=-=-= Content-Type: text/plain Ben Woodcroft writes: > On 16/08/16 08:28, Leo Famulari wrote: >> On Mon, Aug 15, 2016 at 09:29:11PM +0100, Marius Bakke wrote: >>> Leo Famulari writes: >>>> Running test_empirical_kernel_map / phase `check' failed after 2043.7 seconds >>> Strange, I've built this dozens of times now with no test failures >>> (GuixSD on amd64). Are you memory constrained by any chance? Any OOM or >>> segfaults? I don't think the tests have taken more than ~15 minutes to >>> run either, once they are built. >> I don't think I'm particularly resource constrained. I have ~7 GiB RAM >> available to build with, plenty of disk space, and the processor is this >> thing: >> >> http://ark.intel.com/products/64896/Intel-Core-i5-3320M-Processor-3M-Cache-up-to-3_30-GHz >> >> It appears to exit cleanly (no segfault or OOM). >> >> It fails the same test on this machine as well as a slower x86_64 >> (Thinkpad x200s). One machine is Guix / Debian, the other is GuixSD. > I built this patch on a x86_64 server with 40 cores and 256G of memory > twice, here's what I got: > > Running test_one_vs_all_trainer \ > > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > !!!!!!!!!!!!!!!!!!!!!!!!!!!!! TEST FAILED: test_one_vs_all_trainer > !!!!!!!!!!!!!!!!!!!!!!!!!!!!! > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! > > Failure message from test: > > Error occurred at line 141. > Error occurred in file > /tmp/nix-build-dlib-19.1.drv-0/dlib-19.1/dlib/test/one_vs_all_trainer.cpp. > Failing expression was ans == res. > res: > 60 0 0 > 0 69 1 > 0 0 80 > > > > > Testing Finished > Total number of individual testing statements executed: 471078543 > Number of failed tests: 1 > Number of passed tests: 133 I can reproduce Leos error on a foreign distro (Ubuntu 16.04, 32GiB RAM) and see this in dmesg: [497259.758549] dtest[27102]: segfault at b50 ip 00007ffff68270e0 sp 00007fffffffa978 error 4 in libopenblasp-r0.2.15.so[7ffff5715000+18c6000] That memory address is 128 TiB. Though it consistently builds fine on a 64GiB GuixSD machine as verified with `guix build --check`. I initially made this package on a foreign distro without "lapack" in inputs and have verified that dropping LAPACK makes the tests pass. I also found some other optional dependencies after digging around the source, as well as a recommendation to disable/enable asserts: http://dlib.net/dlib/config.h.html Ben, Leo: Can you try the following patch and see if that works for you? --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-gnu-Add-dlib.patch Content-Transfer-Encoding: quoted-printable >From 7cc190763e1312536fe3ac9cd91ea3756cf66b80 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 13 Aug 2016 11:26:10 +0100 Subject: [PATCH] gnu: Add dlib. * gnu/packages/machine-learning.scm (dlib): New variable. --- gnu/packages/machine-learning.scm | 63 +++++++++++++++++++++++++++++++++++= +++- 1 file changed, 62 insertions(+), 1 deletion(-) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learn= ing.scm index f96672c..ebf3307 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2015, 2016 Ricardo Wurmus ;;; Copyright =C2=A9 2016 Efraim Flashner +;;; Copyright =C2=A9 2016 Marius Bakke ;;; ;;; This file is part of GNU Guix. ;;; @@ -27,18 +28,21 @@ #:use-module (guix build-system gnu) #:use-module (guix build-system r) #:use-module (gnu packages) + #:use-module (gnu packages algebra) #:use-module (gnu packages autotools) #:use-module (gnu packages boost) #:use-module (gnu packages compression) #:use-module (gnu packages dejagnu) #:use-module (gnu packages gcc) + #:use-module (gnu packages image) #:use-module (gnu packages maths) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) #:use-module (gnu packages statistics) #:use-module (gnu packages swig) - #:use-module (gnu packages xml)) + #:use-module (gnu packages xml) + #:use-module (gnu packages xorg)) =20 (define-public libsvm (package @@ -467,3 +471,60 @@ geometric models.") "This package provides functions for feed-forward neural networks wit= h a single hidden layer, and for multinomial log-linear models.") (license (list license:gpl2+ license:gpl3+)))) + +(define-public dlib + (package + (name "dlib") + (version "19.1") + (source (origin + (method url-fetch) + (uri (string-append + "http://dlib.net/files/dlib-" version ".tar.bz2")) + (sha256 + (base32 + "0p2pvcdalc6jhb6r99ybvjd9x74sclr0ngswdg9j2xl5pj7knbr4")) + (modules '((guix build utils))) + (snippet + '(begin + ;; Delete ~13MB of bundled dependencies. + (delete-file-recursively "dlib/external") + (delete-file-recursively "docs/dlib/external"))))) + (build-system cmake-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'disable-asserts + (lambda _ + ;; config.h recommends explicitly enabling or disabling asser= ts + ;; when building as a shared library. By default neither is s= et. + (substitute* "dlib/config.h" + (("^//#define DLIB_DISABLE_ASSERTS") "#define DLIB_DISABLE_= ASSERTS")))) + (replace 'check + (lambda _ + ;; No test target, so we build and run the unit tests here. + (let ((test-dir (string-append "../dlib-" ,version "/dlib/tes= t/build"))) + (mkdir-p test-dir) + (with-directory-excursion test-dir + (and (zero? (system* "cmake" "..")) + (zero? (system* "cmake" "--build" "." "--config" "Re= lease")) + (zero? (system* "./dtest" "--runall")))))))))) + (native-inputs + `(("pkg-config" ,pkg-config))) + (inputs + `(("fftw" ,fftw) + ("giflib" ,giflib) + ;("lapack" ,lapack) XXX lapack here causes test failures in some se= tups. + ("libjpeg" ,libjpeg) + ("libpng" ,libpng) + ("libx11" ,libx11) + ("openblas" ,openblas) + ("zlib" ,zlib))) + (synopsis + "Toolkit for making machine learning and data analysis applications i= n C++") + (description + "Dlib is a modern C++ toolkit containing machine learning algorithms = and +tools. It is used in both industry and academia in a wide range of domains +including robotics, embedded devices, mobile phones, and large high perfor= mance +computing environments.") + (home-page "http://dlib.net") + (license license:boost1.0))) --=20 2.9.2 --=-=-=--