From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: CPU-specific builds Date: Tue, 11 Oct 2016 23:20:13 +0200 Message-ID: <87insyshv6.fsf@gnu.org> References: <5985e2f0-edd3-c40b-bb10-f84b233bb48d@uq.edu.au> <871szyho3h.fsf@gnu.org> <612094dc-bcdb-b9a2-292c-501b897bcf21@uq.edu.au> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:57223) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bu4TV-00073E-BG for guix-devel@gnu.org; Tue, 11 Oct 2016 17:20:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bu4TP-0002wo-As for guix-devel@gnu.org; Tue, 11 Oct 2016 17:20:36 -0400 In-Reply-To: <612094dc-bcdb-b9a2-292c-501b897bcf21@uq.edu.au> (Ben Woodcroft's message of "Mon, 10 Oct 2016 23:45:53 +1000") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ben Woodcroft Cc: "guix-devel@gnu.org" Ben Woodcroft skribis: > On 02/10/16 23:33, Ludovic Court=C3=A8s wrote: [...] >>> So I'm wondering if there is some way to specify a system more >>> specific than 'X86_64'? I tried simply adding '--with-arch=3Dhaswell' as >>> a configure argument in gcc-4.9 so that flag became the default for >>> gcc usage and saw some performance improvements, though I did have to >>> disable tests in gnutls. >> Do you have performance figures for some CPU-intensive applications? >> >> What software are you most interested in? > We tend to run software whose runtime is dependent on the input data, > so it is hard to say. But up to days or weeks of walltime in some > cases. It takes a lot of power researching climate change.. > > As an anecdote, adding "-march=3Dhaswell" shaved 13% off the runtime of > diamond, ~20% if the CPUs were contended. That=E2=80=99s more than I expected. It=E2=80=99d be useful to have a way = to specify this. >> Ideally, software for which using these CPU extensions makes a >> significant difference would do what glibc does, which is to provide >> several implementations of the relevant functions (one for SSE2, one for >> AVX, etc.) and have the right one be selected at load time via an IFUNC >> or similar mechanism. > That sounds useful in some cases, but it is probably too much of a > stretch for most bioinformatics packages. Yeah. > In the end I think I'll just compile the specific packages we are > specifically interested in. I attached some example code in case > anyone is interested. But this brought up a few questions: > > 1) I also tried using --expression e.g. guix build --expression '(@@ > (my packages cpu-specific) diamond-cpu-specific)' but that fails to > compile as if the GUIX_PACKAGE_PATH is ignored, is that unexpected? What error did you get? > 2) Is something amiss with gcc-toolchain-6? Compiling with it, diamond > complains of a missing stdlib.h. Everything=E2=80=99s fine AFAICS. However, note that =E2=80=98gnu-build-sy= stem=E2=80=99 pulls in GCC 4.9, glibc, etc. If you add =E2=80=98gcc-toolchain=E2=80=99 to the = inputs, that surely conflicts, but I=E2=80=99m unsure which one =E2=80=9Cwins=E2=80=9D; = could you check the =E2=80=98environment-variables=E2=80=99 file in a build tree? > (define-public gcc-cpu-specific > (let ((base gcc-5)) ; gcc-6 does not seem to work. > (package > (inherit base) > (name "gcc-cpu-specific") > (arguments > (substitute-keyword-arguments (package-arguments base) > ((#:configure-flags configure-flags) > `(append ,configure-flags > (list (string-append > "--with-arch=3D" ,cpu))))))))) > > (define-public (cpu-specific-package base-package) > (package > (inherit base-package) > (name (package-name base-package)) > ;; We must set a higher package version so this package is used inste= ad of > ;; the package in Guix proper. > (version (string-append (package-version base-package) "-cpu-specific= ")) > (inputs > `(,@(package-inputs base-package) > ("gcc" ,((@@ (gnu packages commencement) > gcc-toolchain) gcc-cpu-specific)))))) > > (define-public diamond-cpu-specific (cpu-specific-package diamond)) > (define-public fasttree-cpu-specific (cpu-specific-package fasttree)) > (define-public blast+-cpu-specific (cpu-specific-package blast+)) > (define-public bwa-cpu-specific (cpu-specific-package bwa)) > (define-public metabat-cpu-specific (cpu-specific-package metabat)) Looks like the right way to start. We should start thinking about what the ideal interface would look like, though. At the highest level, I imagine we=E2=80=99d want something like: guix package --tune=3Dhaswell -i diamond Under the hood that could work by having an arbitrary list of key/value options passed to build systems, rather than just the system type and cross-compilation target as is currently the case. Food for thought=E2=80=A6 Ludo=E2=80=99.