From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id UNUZJV52lGS8+QAASxT56A (envelope-from ) for ; Thu, 22 Jun 2023 18:27:10 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id oB9JJF52lGTH1AAAG6o9tA (envelope-from ) for ; Thu, 22 Jun 2023 18:27:10 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3D00CCF0F for ; Thu, 22 Jun 2023 18:27:10 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCN8a-0005Za-TY; Thu, 22 Jun 2023 12:26:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCN8F-0005LG-CI for bug-guix@gnu.org; Thu, 22 Jun 2023 12:26:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qCN8E-0002J7-Rx for bug-guix@gnu.org; Thu, 22 Jun 2023 12:26:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qCN8E-0004HM-2f for bug-guix@gnu.org; Thu, 22 Jun 2023 12:26:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#63986: Julia is very slow Resent-From: Efraim Flashner Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 22 Jun 2023 16:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 63986 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Simon Tournier Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 63986@debbugs.gnu.org, Nicolas Graves , csantosb@inventati.org Received: via spool by 63986-submit@debbugs.gnu.org id=B63986.168745111616373 (code B ref 63986); Thu, 22 Jun 2023 16:26:02 +0000 Received: (at 63986) by debbugs.gnu.org; 22 Jun 2023 16:25:16 +0000 Received: from localhost ([127.0.0.1]:36567 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qCN7T-0004G0-U5 for submit@debbugs.gnu.org; Thu, 22 Jun 2023 12:25:16 -0400 Received: from mail-wr1-f50.google.com ([209.85.221.50]:56653) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qCN7R-0004Fl-Qi for 63986@debbugs.gnu.org; Thu, 22 Jun 2023 12:25:15 -0400 Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-3111547c8f9so8510750f8f.1 for <63986@debbugs.gnu.org>; Thu, 22 Jun 2023 09:25:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687451108; x=1690043108; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender:from:to :cc:subject:date:message-id:reply-to; bh=vvhWHnCHM+ij5aVKhS0iJmCdHC/FRM5UQ9cey6hHr8c=; b=CBEyCOiJGUqP3TqETuTYsOwy5I3seQKO5kPF3KbE2QfH9uaJprwokpSjfExp9lpq4L BFLIjYIJdLKpdwB8FtYGBm+dDiMru+B3aYgzQjN8HvQ3hiupzx039wFGAGJi35HIIt1P 7Czpmy+Zy+ey9BLbd4nqUBtOcnynKoQYPdDaoabF1nVHzFGQ+sglgNLaOhi5ZUIT/T68 uXs30yWa9s8gevkhvfacap61T7lNV+V7Ny+3ehBDG6KJ/tM+phG+vdmMCAntqWfLi4B1 L9Q++1qb/v4olc9hHPzhX5iP1ZNHNo0c7EoLjwpeq9sNKJ/XzjN9nryg78HK9YKgKVl3 lsdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687451108; x=1690043108; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vvhWHnCHM+ij5aVKhS0iJmCdHC/FRM5UQ9cey6hHr8c=; b=JxJ8csYmPGjka525b4CrImab8qe97r+Y+2PTl6WXc1wI5u6xXUnwW+0gR6JBEf7Bn6 qbHWgHyRoHJCQK9hMsfv+xUJSuLVuHpbi8K/+L/tkJdaXzCV2SrdSwAvcAFlussI7EBc T6S1s4OyUS8qlqw2APB1yJjMVK5ftU431Cy4B6vXoGqM3roxfbW3Lhpx8PECCAVT9J7j a9kf2mSN2BAFuMdKL6DBKCmw27BKi1CbcqNl7ONyO/FPqmkrnQ0ykPrwSnXcnGGot2g5 7Nd7uxHVJll7jHL09BgeGaxl+1C4ujYC1MP0fWgvzUKrXS0npdDn8SN090VKCfP8OETj 35QQ== X-Gm-Message-State: AC+VfDwcDcxGal20TdfbNbHsR3UUJ87W08y124zEk4FGqAnqPP0s+QqE k+nQo6+B9Acklz7Tqhvc3oo= X-Google-Smtp-Source: ACHHUZ6hn0nziJbQiLpQ0uiOP6JVy3qZSenhvZF9FdQjbJTq3+M/ZXm5mgo59aM8DK8GO4hjGQ/Djg== X-Received: by 2002:a5d:4711:0:b0:311:19df:dad2 with SMTP id y17-20020a5d4711000000b0031119dfdad2mr16569276wrq.47.1687451107638; Thu, 22 Jun 2023 09:25:07 -0700 (PDT) Received: from localhost ([188.120.129.124]) by smtp.gmail.com with ESMTPSA id m7-20020a7bca47000000b003f90b9b2c31sm13233255wml.28.2023.06.22.09.25.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Jun 2023 09:25:06 -0700 (PDT) Date: Thu, 22 Jun 2023 19:25:05 +0300 From: Efraim Flashner Message-ID: Mail-Followup-To: Efraim Flashner , Simon Tournier , csantosb@inventati.org, Ludovic =?UTF-8?Q?Court=C3=A8s?= , 63986@debbugs.gnu.org, Nicolas Graves References: <87legsl3cm.fsf@inventati.org> <87jzw6rq48.fsf@gnu.org> <871qi7a0qr.fsf@gmail.com> <87pm5o7uh9.fsf@gnu.org> <87ttv0mth3.fsf@inventati.org> <87wmzvmxt8.fsf@inventati.org> <87h6qz8pg0.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="ddGpKuQheAwz6IMG" Content-Disposition: inline In-Reply-To: <87h6qz8pg0.fsf@gmail.com> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1687451230; a=rsa-sha256; cv=none; b=K1T01fV6B54+KXGZEOHbhLy0a7bfVih14aqbauQ/WHg2kICAKEfUmxHfX/YBT9d7MBrplQ /YAEc9QEXEPfjwRTgPVDuF3xWdGeylQqjZ4CRQEgLt/ohVfy+4aDZSqkCGcMW/pbXoengc MNITO3BNic2i9tFkqg6HrGv3IgVAiZKu41MxOFivhWiVdwKQcTAR+nS1qPAPrYPuGtZHlD kW29+3pnVAxkDxVVKu6mo/r4ujC+Ndbk+2Q0QatMMA2yqK/deNyW8N5CDCYURMQq+JP4u4 v/4KedoWso5M3DVXu4qvXqS+8ofWkkYA/60baeEaPw7ALsekzrz5WtR1eOOfDg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=CBEyCOiJ; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1687451230; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=vvhWHnCHM+ij5aVKhS0iJmCdHC/FRM5UQ9cey6hHr8c=; b=ILVWii1e9LP+iw/QmhRX0eM6uI4HTh53XmrbhRFNQH4XWf9abnz9m+qCysHFSDWNiBp6fV psjCIdK/66s6j7QWvbwmtEjVmvwq1ecIuZImfyn24TIcy9RnEAZnH1a6hxaN8/34gf+oyB Cu2ayM7FGn4Ui/3TZgOFZmSOChP0X2Nx/4VTCgI6fO63swnF6k+g2+3IGN9w4hY3f4EqWo RS+ajipAIByY6prL+Wyi39Gl66qsBj+qJUh42qUB4vGbs1mopzG0EXNzkequUUigyGk+aQ djY6L1KwgkbF8EY+38cWbE2PZyqIQzpEcjMox+TT1sqFCFWWdPNrwg3ySJ3cPg== Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=CBEyCOiJ; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Scanner: scn0.migadu.com X-Migadu-Spam-Score: -0.85 X-Spam-Score: -0.85 X-Migadu-Queue-Id: 3D00CCF0F X-TUID: 0eJ2gqRpTRAN --ddGpKuQheAwz6IMG Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 22, 2023 at 05:52:15PM +0200, Simon Tournier wrote: > Hi, >=20 > On Thu, 22 Jun 2023 at 15:26, Cayetano Santos via Bug reports for GNU Gui= x wrote: > >> Are we following all instructions here ? > >> > >> https://docs.julialang.org/en/v1.8/devdocs/build/distributing/#Notes= -on-BLAS-and-LAPACK >=20 > [...] >=20 > > Base.USE_BLAS64 > > > > gives "true" when running fast. Guix julia gives "false". >=20 > When I try USE_BLAS64=3D1, then I get: >=20 > --8<---------------cut here---------------start------------->8--- > =E2=94=8C Error: No loaded BLAS libraries were built with ILP64 support > =E2=94=94 @ LinearAlgebra.BLAS /tmp/guix-build-julia-1.8.3.drv-0/julia-1.= 8.3/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/blas.jl:155 > Quitting. > --8<---------------cut here---------------end--------------->8--- >=20 > And from the documentation above, it reads: >=20 > [...] while on 64-bit architectures, Julia builds OpenBLAS to > use 64-bit integers (ILP64). It is essential that all Julia > functions that call BLAS and LAPACK API routines use integers of > the correct width. >=20 > Well using the patch attached, I get: >=20 > 6.884 ms (2 allocations: 7.63 MiB) >=20 > compared to the previous >=20 > 494.345 ms (2 allocations: 7.63 MiB) >=20 > WDYT about this patch? (ins)efraim@3900XT ~/workspace/guix$ cat /gnu/store/v6z5ykkjfzbc72x1x900xfl= spqc5wd5r-openblas-ilp64-0.3.20/lib/pkgconfig/openblas.pc libdir=3D/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/= lib includedir=3D/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3= =2E20/include openblas_config=3D USE_64BITINT=3D DYNAMIC_ARCH=3D1 DYNAMIC_OLDER=3D1 NO_CB= LAS=3D NO_LAPACK=3D NO_LAPACKE=3D NO_AFFINITY=3D1 USE_OPENMP=3D generic MAX= _THREADS=3D128 version=3D0.3.20 extralib=3D-lm -lpthread -lgfortran -lm -lpthread -lgfortran Name: openblas Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 = BSD version Version: ${version} URL: https://github.com/xianyi/OpenBLAS Libs: -L${libdir} -lopenblas Libs.private: ${extralib} Cflags: -I${includedir} Looks like it should be "LIBBLAS=3D-lopenblas" It might need some tuning anyway, currently we have julia for i686 and switching to solely openblas-ilp64 we'd lose 32-bit support. I also noticed the julia expects the 64-bit openblas to be libopenblas64 (which happens to be what Debian=C2=B9 has). Would we need to adapt anything in stdlib/OpenBLAS_jll/src/OpenBLAS_jll.jl? Also, are we supposed to build lapack with our openblas as an input? =C2=B9 https://sources.debian.org/src/openblas/0.3.21%2Bds-4/debian/rules/#= L71 > From 024c92fac091f59dcdbd3a78eb6ea77bb15b2170 Mon Sep 17 00:00:00 2001 > Message-Id: <024c92fac091f59dcdbd3a78eb6ea77bb15b2170.1687449033.git.zimo= n.toutoune@gmail.com> > From: Simon Tournier > Date: Thu, 22 Jun 2023 17:45:50 +0200 > Subject: [PATCH] gnu: julia: Use openblas with ILP64 support. >=20 > Fixes . > Reported by Cayetano Santos . >=20 > * gnu/packages/julia.scm (julia)[arguments]<#:make-flags>: Use OpenBLAS w= ith > ILP64 support. > [inputs]: Replace openblas by openblas-ilp64. > --- > gnu/packages/julia.scm | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) >=20 > diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm > index ba54175822..a034cbf543 100644 > --- a/gnu/packages/julia.scm > +++ b/gnu/packages/julia.scm > @@ -481,10 +481,10 @@ (define-public julia > =20 > ,@(if (target-aarch64?) > `("USE_BLAS64=3D0") > - '()) > + `("USE_BLAS64=3D1")) > =20 > - "LIBBLAS=3D-lopenblas" > - "LIBBLASNAME=3Dlibopenblas" > + "LIBBLAS=3D-lopenblas_ilp64" > + "LIBBLASNAME=3Dlibopenblas_ilp64" > =20 > (string-append "UTF8PROC_INC=3D" > (assoc-ref %build-inputs "utf8proc") > @@ -513,7 +513,7 @@ (define-public julia > ("llvm" ,llvm-julia) > ("mbedtls-apache" ,mbedtls-apache) > ("mpfr" ,mpfr) > - ("openblas" ,openblas) > + ("openblas" ,openblas-ilp64) > ("openlibm" ,openlibm) > ("p7zip" ,p7zip) > ("pcre2" ,pcre2) >=20 > base-commit: 37c2e94cec6cb8b5e0e93e7b6c712c3b187ca5db > --=20 > 2.38.1 >=20 >=20 > Well, I need to do more tests but I guess that=E2=80=99s the good directi= on. :-) >=20 > Cheers, > simon --=20 Efraim Flashner =D7=A8=D7=A0=D7=A9=D7=9C=D7=A4 = =D7=9D=D7=99=D7=A8=D7=A4=D7=90 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --ddGpKuQheAwz6IMG Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmSUdd4ACgkQQarn3Mo9 g1ESrRAAgXQ6J4sf6gpxitgDtYAgogbuyH71S/VJyIDyPDZK0LCrEFvRwPThlD39 W/2ZPjjvtriPtUChiRvNPrykVNugzMJ7uUAOwUflKIFq+b30B/2uZt2P8EgAUkuE FEbgpWOoT2CvmN80LHMl5vlOEsd4wd1W8ZexIspZeOIF11CjmxDbSXrftir0gLTs koSplycsYnfpp6WJAlk4ATYkkTy6QO24ZRLz5WpXYD4DNQNLRbMa3bkVrxZmScGE ZQ8gawgKojceyyhOYu39cXk36EiSLxH1ok0jKm1n5nHVWeSDm3Xi8rlMhE9e/Lkm 6DZUAylZ2sRu+qqCYyjIw9pEptv98crDoh+1tmWo44S3Vx6ezNgb/ltyTf2gGa1z eh0UxunoLRGun03On5r4QDKSFsSpkLo6WzHV/mNxIlP5Ew/P7dasbT1+lR4T5x4Y KvERCHsgKggn2U3bUvxvH5pzaOva4WtpwF6I7xStdQ2VX3HQu9VVFfL97YurC+lr SDjwrHri3MqGY7CqEetlizNEyK0nUQdrAt9D9QfjWCf6GpgE/xfPGVNcy5oqoLAb lM0O/RI1f6d11xn5aELyJBZ5tIHrTpM+8ZIINHGEfslk+kBel85MYT0VtLMtiGtx 8woKv9uBaKjJOmyLwy8rnbN8UmlCOok2NGFgEaijVWQ5QiPA2gM= =/GU9 -----END PGP SIGNATURE----- --ddGpKuQheAwz6IMG--