From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id 4JhdM3KplWQOfgAASxT56A (envelope-from ) for ; Fri, 23 Jun 2023 16:17:22 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id mE0zMnKplWR8bwEAG6o9tA (envelope-from ) for ; Fri, 23 Jun 2023 16:17:22 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5EC345A7B for ; Fri, 23 Jun 2023 16:17:22 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qChaz-00076a-2o; Fri, 23 Jun 2023 10:17:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qChaw-00075l-Lb for bug-guix@gnu.org; Fri, 23 Jun 2023 10:17:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qChaw-0003qI-BG for bug-guix@gnu.org; Fri, 23 Jun 2023 10:17:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qChaw-0008Hy-1F for bug-guix@gnu.org; Fri, 23 Jun 2023 10:17:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#63986: Julia is very slow Resent-From: Efraim Flashner Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 23 Jun 2023 14:17:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 63986 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Simon Tournier Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 63986@debbugs.gnu.org, Nicolas Graves , csantosb@inventati.org Received: via spool by 63986-submit@debbugs.gnu.org id=B63986.168752978231811 (code B ref 63986); Fri, 23 Jun 2023 14:17:01 +0000 Received: (at 63986) by debbugs.gnu.org; 23 Jun 2023 14:16:22 +0000 Received: from localhost ([127.0.0.1]:39288 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qChaH-0008H1-RF for submit@debbugs.gnu.org; Fri, 23 Jun 2023 10:16:22 -0400 Received: from mail-wm1-f46.google.com ([209.85.128.46]:50259) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qChaE-0008Gn-Gk for 63986@debbugs.gnu.org; Fri, 23 Jun 2023 10:16:20 -0400 Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-3fa798cf201so8742945e9.0 for <63986@debbugs.gnu.org>; Fri, 23 Jun 2023 07:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687529772; x=1690121772; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender:from:to :cc:subject:date:message-id:reply-to; bh=Vx3xwOWE0Z/5txufR2s/EzT2VZUyhMtocCYsXjhex5c=; b=eLwNcviplugoywecPR8GV1f8FcIWl4KdWcLOc2msNy5G8Fyx8K5iuTXr7qyHDlOowo Mnlm+i3zUOtQ+SHIDqRBeILfSkfA1rcn7O6EJaBVxou2OKqSmuE9ik4ZMk6iGluF/F6t 0fPG8pF3MbeEjwVdgVnR7omRIyD3Hfz64jgOuFm9x7PDGoRBe/m/YMP/TUZXUvTAVysM WDg1MssCTW8Ta35UJ7J2urTSd4YZ173yJZ2ypnWu0FlFNJs9sp+TNMy1aypLcv5JWc6e JVwNZ5FfBqT7jvmng80I4V51Ov51XdORLqQrAUPkw8F82lnrF83YwK1YOl4rT6ApQhN/ PlrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687529772; x=1690121772; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Vx3xwOWE0Z/5txufR2s/EzT2VZUyhMtocCYsXjhex5c=; b=Omrp1+xglybYG1DqTmSpEe5VG/QNLj+Sr2s+QmhrUjuJXzHmkbaDfP/spp2ungVxVB KbMZw1L8G+c4pZu7mj06Bm92yZc3JESPkspz70eKAWZBs7dp3X7OfWoJaHmZGFWGaScI nkQigNfUXtanGEcdnet3VU0FMiZwuThpmCCUMWj3d+e/FBy+Og2MfjNOVTHcrb7RoLZd wZp0nnhYGPYZLfeY4uKBmZERb1d316l+v1LNe1rBK3oPM48y95ckc5ESLmnrlilqPNe7 E4PIR+EEW+7sYcoGQp6iCYnT5WVrnPjkOKBfb5Km/hAGS9j2i9oZBTzhNV6F5g0pVmjk zOzg== X-Gm-Message-State: AC+VfDwbqNEula4RkT9yb9sBPrv7da5Tt5aF9jHUw5iV/DRc/pfpJBUL aBPQO5VdWTV/FhKJlUbKww8= X-Google-Smtp-Source: ACHHUZ7q5MuGeOrlwV6ONsJFY6cll2GYVxWzAd+kE4XOtHK4WtU94lmBNe3qtoYTh/L9zaeBPFeQLw== X-Received: by 2002:a1c:4c05:0:b0:3fa:787b:5185 with SMTP id z5-20020a1c4c05000000b003fa787b5185mr2915767wmf.13.1687529772286; Fri, 23 Jun 2023 07:16:12 -0700 (PDT) Received: from localhost ([188.120.129.124]) by smtp.gmail.com with ESMTPSA id l21-20020a1c7915000000b003f605566610sm2493121wme.13.2023.06.23.07.16.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Jun 2023 07:16:11 -0700 (PDT) Date: Fri, 23 Jun 2023 17:16:10 +0300 From: Efraim Flashner Message-ID: Mail-Followup-To: Efraim Flashner , Simon Tournier , Ludovic =?UTF-8?Q?Court=C3=A8s?= , 63986@debbugs.gnu.org, Nicolas Graves , csantosb@inventati.org References: <871qi7a0qr.fsf@gmail.com> <87pm5o7uh9.fsf@gnu.org> <87ttv0mth3.fsf@inventati.org> <87wmzvmxt8.fsf@inventati.org> <87h6qz8pg0.fsf@gmail.com> <87cz1n8lq3.fsf@gmail.com> <871qi24k9z.fsf@inria.fr> <87h6qy71oa.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="zcDsr1hIZVUVbILN" Content-Disposition: inline In-Reply-To: <87h6qy71oa.fsf@gmail.com> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1687529842; a=rsa-sha256; cv=none; b=q7ZuNXKs1ANk60bxXmrkhdq8LjwygGECzMkY1Zpf1wpQ2B6FdH2m7R/f4Ulu4wIG65aTuS hoMuvFYXPHK+pHEud6Q8QckQ3hpUvGLcpOT1SfC30jftjbaDNEm/K8obwVP8a7q6gVja0r QKQr07JGQX61C+e3ywT/FeUMWgU27XxaPr2IaDj9rxKiztHsggtFWy2EMJ6VO+pbXrroMT 3LdgATh9M7Nw7pUMdnmrqrx8ctPc0/ZMIRksigZ4006T/cDap6YhL8nTlW3GKulZP2mWFm Ps/PX9q3F0Pe03O54hLd8Qj5PJprQSkGjLmuHFVEdud6ZzGmiC9pFfDWikIdNQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=eLwNcvip; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1687529842; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=Vx3xwOWE0Z/5txufR2s/EzT2VZUyhMtocCYsXjhex5c=; b=Fd31l9NMWnmKwp7ToLxJ6Psm3sjcYD/ktn2iESIYDnTWQO3sRwGFr+sQInYi6gODXcFGIC eE5kxtwqBUBFUsSL391QrjXO9ZyxUtRyBVKcHc38X+W3XcmWpb3R+8nOsqWD2N/2uo3tHb EJcgHyQ5Ml5ue2xpoU4Fg1Q56SyrOLERNUA0BAhfJnz5V48dZ8cw/LhcjjKHcwgb+reGWg BifAn26eOUUmb6Y5St0WZBkihX9oJZF/E7V5gJg94v+8VTgJ9J7Weau8WY21ZvbTWltfEp l6MODwTyATKXWKJaG6bcIsVk1UxNKUVhmjui+MxT1TqxP/RXTDJYTI8hxXKozA== Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=eLwNcvip; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Scanner: scn0.migadu.com X-Migadu-Spam-Score: -2.15 X-Spam-Score: -2.15 X-Migadu-Queue-Id: 5EC345A7B X-TUID: t682ULG/5VYT --zcDsr1hIZVUVbILN Content-Type: multipart/mixed; boundary="P1A0iMc2RBMfbKZV" Content-Disposition: inline --P1A0iMc2RBMfbKZV Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 23, 2023 at 03:23:17PM +0200, Simon Tournier wrote: > Hi Efraim, >=20 > On Fri, 23 Jun 2023 at 12:16, Efraim Flashner wro= te: >=20 > > It should be, but julia fails the precompile stage with openblas-ilp64 > > > > Precompilation complete. Summary: > > Total =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 1= 006.632176 seconds > > Generation =E2=94=80=E2=94=80 690.274180 seconds 68.5726% > > Execution =E2=94=80=E2=94=80=E2=94=80 316.357997 seconds 31.4274% > > > > signal (15): Terminated > > in expression starting at none:0 > > unknown function (ip: 0xfffff4d81b24) > > unknown function (ip: 0xfffff4ed10bb) > > unknown function (ip: 0xfffff4eda16f) > > _ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /gnu/sto= re/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknow= n line) > > _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /gnu/store/wa2= 8l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line) > > _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /gnu/store/wa28l47= 6c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line) > > _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /gnu/store/wa28l47= 6c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line) > > operator() at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/aotcomp= ile.cpp:580 [inlined] > > jl_dump_native_impl at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/sr= c/aotcompile.cpp:592 > > jl_write_compiler_output at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8= =2E3/src/precompile.c:94 > > ijl_atexit_hook at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/in= it.c:207 > > jl_repl_entrypoint at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src= /jlapi.c:720 > > main at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/cli/loader_exe.c:= 59 > > __libc_start_call_main at /gnu/store/a19xbynxc3sg25xpkwmx7g0mdl7g31hx-g= libc-2.35/lib/libc.so.6 (unknown line) > > __libc_start_main at /gnu/store/a19xbynxc3sg25xpkwmx7g0mdl7g31hx-glibc-= 2.35/lib/libc.so.6 (unknown line) > > _start at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/bin/julia (= unknown line) > > unknown function (ip: (nil)) > > Allocations: 145655328 (Pool: 145582643; Big: 72685); GC: 120 > > *** This error is usually fixed by running `make clean`. If the error p= ersists, try `make cleanall`. *** > > make[1]: *** [sysimage.mk:89: /tmp/guix-build-julia-1.8.3.drv-0/julia-1= =2E8.3/usr/lib/julia/sys-o.a] Error 1 > > make[1]: Leaving directory '/tmp/guix-build-julia-1.8.3.drv-0/julia-1.8= =2E3' > > make: *** [Makefile:88: julia-sysimg-release] Error 2 >=20 > Using my patch, it fails with: >=20 > --8<---------------cut here---------------start------------->8--- > Error in testset OpenBLAS_jll: > Test Failed at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/ju= lia/stdlib/v1.8/OpenBLAS_jll/test/runtests.jl:16 > Expression: dlsym(OpenBLAS_jll.libopenblas_handle, #=3D /tmp/guix-build= -julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/OpenBLAS_jll/tes= t/runtests.jl:16 =3D# @blasfunc(openblas_set_num_threads); throw_error =3D = false) !=3D nothing > Evaluated: nothing !=3D nothing > ERROR: LoadError: Test run finished with errors > in expression starting at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/t= est/runtests.jl:93 > --8<---------------cut here---------------end--------------->8--- Mine was on aarch64 > Well, I am still missing why the expression >=20 > dlsym(OpenBLAS_jll.libopenblas_handle, > #=3D /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdl= ib/v1.8/OpenBLAS_jll/test/runtests.jl:16 =3D# > @blasfunc(openblas_set_num_threads); throw_error =3D false) >=20 > is evaluated to nothing. Any idea? It is something about the wrong internal interface in openblas, I don't remember exactly what that error came from. Try with the attached diff. --=20 Efraim Flashner =D7=A8=D7=A0=D7=A9=D7=9C=D7=A4 = =D7=9D=D7=99=D7=A8=D7=A4=D7=90 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --P1A0iMc2RBMfbKZV Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="julia-openblas-ilp64.patch" Content-Transfer-Encoding: quoted-printable diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm index ba54175822..8ba2b480ce 100644 --- a/gnu/packages/julia.scm +++ b/gnu/packages/julia.scm @@ -269,7 +269,10 @@ (define-public julia (substitute* (jlpath "nghttp2") (((from "libnghttp2")) (to "libnghttp2" "libnghttp2"))) (substitute* (jlpath "OpenBLAS") - (((from "libopenblas")) (to "openblas" "libopenblas"))) + (((from "libopenblas")) + ,@(if (target-x86-64?) + `((to "openblas" "libopenblas64_" "libopenblas")) + `((to "openblas" "libopenblas"))))) (substitute* (jlpath "OpenLibm") (((from "libopenlibm")) (to "openlibm" "libopenlibm"))) (substitute* (jlpath "PCRE2") @@ -479,12 +482,13 @@ (define-public julia "NO_GIT=3D1" ; build from release tarball. "USE_GPL_LIBS=3D1" ; proudly =20 - ,@(if (target-aarch64?) - `("USE_BLAS64=3D0") - '()) - - "LIBBLAS=3D-lopenblas" - "LIBBLASNAME=3Dlibopenblas" + ,@(if (target-x86-64?) + `("USE_BLAS64=3D1" + "LIBBLAS=3D-lopenblas64_" + "LIBBLASNAME=3Dlibopenblas64_") + `("USE_BLAS64=3D0" + "LIBBLAS=3D-lopenblas" + "LIBBLASNAME=3Dlibopenblas")) =20 (string-append "UTF8PROC_INC=3D" (assoc-ref %build-inputs "utf8proc") @@ -513,7 +517,9 @@ (define-public julia ("llvm" ,llvm-julia) ("mbedtls-apache" ,mbedtls-apache) ("mpfr" ,mpfr) - ("openblas" ,openblas) + ,@(if (target-x86-64?) + `(("openblas" ,openblas-ilp64)) + `(("openblas" ,openblas))) ("openlibm" ,openlibm) ("p7zip" ,p7zip) ("pcre2" ,pcre2) diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm index f5a2181905..2d3ce41cb7 100644 --- a/gnu/packages/maths.scm +++ b/gnu/packages/maths.scm @@ -4645,7 +4645,9 @@ (define-public openblas-ilp64 (arguments (substitute-keyword-arguments (package-arguments openblas) ((#:make-flags flags #~'()) - #~(append (list "INTERFACE64=3D1" "LIBNAMESUFFIX=3Dilp64") + #~(append (list "INTERFACE64=3D1" + "SYMBOLSUFFIX=3D64_" + "LIBPREFIX=3Dlibopenblas64_") #$flags)))) (synopsis "Optimized BLAS library based on GotoBLAS (ILP64 version)") (license license:bsd-3))) --P1A0iMc2RBMfbKZV-- --zcDsr1hIZVUVbILN Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmSVqScACgkQQarn3Mo9 g1Hf5BAAvmP9BmsscY/Dc+6fLF8WhpBwIr7TSh/SKi73ON0gumUJcaISQaEnhQGp QD11JVj3l0OBsf2OuQYaxfmhUa4WKIEFl5XYDhE8QzX1wetw10Ai7qy8TaqYv0Po KN1TKdMy0pLj0JLfbRkfkeYYYQSIUILspnwzygxcYUYhgG3tUfKpYAJFlf7BpYDj NvFoxQfv+Wct2ea87feCANfNc+MbADnKCvmS5KAr2jm0i7INmp/KZs+KosmE77fU CxHPDnDvh42yhvR5ezDlazXNXjrRVYm3twh3hKPxVEKG/ZpchThd5Zs0o3EFdv6K 0AeXQtNJXUqSfCwxVEA6jFoenHsOkAJJftVs9Ml8N8e768rrpZOSXKkaG8g75cNH OF5rTWxordYr5DGHg4U5uZuhnKj9C38V37wUebEah/xAsBqF+fm2TdATP147SRdn HX4ulpnE5YO57ME7ZHAVB9WYO3Ekh7HUbUVHdkHAU6yaIb70ECIkgUItlCj9T6BE JHcm7A4ZzE/qtFV+9f0K3bo9p4TZ8ekKmqU2HIUtB20VyDmDaLqxSx7R4uvmWBCO x5HjJRi1bqbJSqm/v8mepfTR1ZtcsNgh43a9XePVdCpcvYKrYSFkA8GZXEEJWTbR 33/41BYgDp+YveqFfzZjjLxKHncVJUYbfh+yVGQz4iRhS1v+8+k= =VWxD -----END PGP SIGNATURE----- --zcDsr1hIZVUVbILN--