bug#60200: Incompatibilities between gcc-toolchain and R packages

bug#60200: Incompatibilities between gcc-toolchain and R packages

[Lars-Dominik Braun]

Hi,

I ran into issues with the package r-brms. Take the following reproducer as an example:

    $ guix shell r-brms r make sed gcc-toolchain bash -C --no-cwd --share=/tmp
	$ R
	> library(brms)
	> fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient), data = epilepsy, family = poisson())
	Compiling Stan program...
    Error in dyn.load(libLFile) :
	  unable to load shared object '/tmp/RtmpKqzbYg/file3245e787c.so':
	  /gnu/store/vqhamsanmlm8v6f90a635zc6gmhwlphp-gfortran-10.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /tmp/RtmpKqzbYg/file3245e787c.so)
	Error in sink(type = "output") : invalid connection

The same code works well with gcc-toolchain@10 instead of gcc-toolchain
(@12, which is the default). As we can see the generated shared library
above depends on GCC 12, 10 and GFortran 10 at the same time:

	$ ldd /tmp/RtmpKqzbYg/file3245e787c.so | grep -e gcc -e fortran
	libstdc++.so.6 => /gnu/store/4zvswpr2h3b7dvqpvjcdam8vfhyjrmgl-gcc-12.2.0-lib/lib/libstdc++.so.6 (0x00007f69bd9fc000)
	libgcc_s.so.1 => /gnu/store/4zvswpr2h3b7dvqpvjcdam8vfhyjrmgl-gcc-12.2.0-lib/lib/libgcc_s.so.1 (0x00007f69bd898000)
	libgfortran.so.5 => /gnu/store/vqhamsanmlm8v6f90a635zc6gmhwlphp-gfortran-10.3.0-lib/lib/libgfortran.so.5 (0x00007f69bd3e6000)
	libquadmath.so.0 => /gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/libquadmath.so.0 (0x00007f69bd39b000)
	libgomp.so.1 => /gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/libgomp.so.1 (0x00007f69bccf9000)

The command used to link that .so is revealed by strace to be (filenames
are random and may differ between runs):

	["g++", "-std=gnu++14", "-shared", "-L/gnu/store/145dmr8drw3yzrdhzbsksi05p599hrs6-r-minimal-4.2.2/lib/R/lib", "-L/usr/local/lib", "-o", "file3373276d0.so", "file3373276d0.o", "/gnu/store/5rdg6sv1nrg1ikqxcykvdh9g4yd2xjvy-r-rstan-2.21.7/site-library/rstan/lib//libStanServices.a", "-L/gnu/store/y7rvp47mz3hcnpvnqsx42qz5yvs10pnl-r-stanheaders-2.21.0-7/site-library/StanHeaders/lib/", "-lStanHeaders", "-L/gnu/store/01lhjrrkvrnsjd8xh4ssdn3bgxc422jl-r-rcppparallel-5.1.5/site-library/RcppParallel/lib/", "-ltbb", "-L/gnu/store/145dmr8drw3yzrdhzbsksi05p599hrs6-r-minimal-4.2.2/lib/R/lib", "-lR"]

So it links against libStanServices.a, libStanHeaders(.a), libtbb(.so) and
libR(.so) of which only libR has a reference to gfortran:

	$ ldd /gnu/store/145dmr8drw3yzrdhzbsksi05p599hrs6-r-minimal-4.2.2/lib/R/lib/libR.so | grep -i fortran
	libgfortran.so.5 => /gnu/store/vqhamsanmlm8v6f90a635zc6gmhwlphp-gfortran-10.3.0-lib/lib/libgfortran.so.5 (0x00007f5be49db000)

That means every software linking against R is also incompatible with
the current default gcc-toolchain (when using the command line or
specifications; different story when writing package recipes).

Possible solutions:

- Make gcc-toolchain@10 the default and rename gcc-toolchain@12 to
  gcc-toolchain-next@12, like we do for Haskell and (sometimes) Python.
- Update, both, the default GCC and GFortran to version 12.
- Explicitly depend on the correct gcc-toolchain in packages that need
  a compiler.

Cheers,
Lars

bug#60200: Incompatibilities between gcc-toolchain and R packages

[zimoun]

Hi Lars,

On Mon, 19 Dec 2022 at 13:00, Lars-Dominik Braun <lars@6xq.net> wrote:
>     $ guix shell r-brms r make sed gcc-toolchain bash -C --no-cwd --share=/tmp
> 	$ R
> 	> library(brms)
> 	> fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient), data = epilepsy, family = poisson())
> 	Compiling Stan program...
>     Error in dyn.load(libLFile) :
> 	  unable to load shared object '/tmp/RtmpKqzbYg/file3245e787c.so':
> 	  /gnu/store/vqhamsanmlm8v6f90a635zc6gmhwlphp-gfortran-10.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /tmp/RtmpKqzbYg/file3245e787c.so)
> 	Error in sink(type = "output") : invalid connection
>
> The same code works well with gcc-toolchain@10 instead of gcc-toolchain
> (@12, which is the default). As we can see the generated shared library
> above depends on GCC 12, 10 and GFortran 10 at the same time:

[...]

> Possible solutions:
>
> - Make gcc-toolchain@10 the default and rename gcc-toolchain@12 to
>   gcc-toolchain-next@12, like we do for Haskell and (sometimes) Python.
> - Update, both, the default GCC and GFortran to version 12.
> - Explicitly depend on the correct gcc-toolchain in packages that need
>   a compiler.

It depends on what you are naming default. :-)

The default GCC toolchain for compiling is provided by the package
gcc-toolchain@10.

Without any specification about the version, if a package name is
defined at several versions, then the command-line uses the higher
version of this package.  To avoid the kind of confusing error as you
are observing, usually, the suffix ’-next’ is applied to the package
name for higher versions.  For instance, python-numpy-next.

To add a data point to the confusion, consider the package ’gcc’ instead
of ’gcc-toolchain’.  As noticed earlier [1], there is a bug.  Consider:

--8<---------------cut here---------------start------------->8---
$ guix show gcc@6
guix show: error: gcc@6: package not found

$ guix show gcc@7
name: gcc
version: 7.5.0
outputs:
+ lib: shared libraries
+ debug: debug information
+ out: everything else
systems: x86_64-linux i686-linux
dependencies: gmp@6.2.1 isl@0.23 libelf@0.8.13 libstdc++-headers@10.3.0 mpc@1.2.1 mpfr@4.1.0 perl@5.34.0 texinfo@6.7 zlib@1.2.11
location: gnu/packages/gcc.scm:601:2
homepage: https://gcc.gnu.org/
license: GPL 3+
synopsis: GNU Compiler Collection  
description: GCC is the GNU Compiler Collection.  It provides compiler front-ends for several languages, including C, C++, Objective-C, Fortran, Ada, and
+ Go.  It also includes runtime support libraries for these languages.
--8<---------------cut here---------------end--------------->8---

It is because the ’properties’ field of gcc@7 is rewritten,

--8<---------------cut here---------------start------------->8---
(define-public gcc-7
  (package
    (inherit gcc-6)
    (version "7.5.0")
[...]
    (properties
     `((compiler-cpu-architectures
        ("aarch64" ,@%gcc-7.5-aarch64-micro-architectures)
        ("armhf" ,@%gcc-7.5-armhf-micro-architectures)
        ("x86_64" ,@%gcc-7.5-x86_64-micro-architectures))))))
--8<---------------cut here---------------end--------------->8---

and thus, it looses ’hidden’, as shown here:

--8<---------------cut here---------------start------------->8---
$ guix repl
GNU Guile 3.0.8
Copyright (C) 1995-2021 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> ,use(gnu packages gcc)
scheme@(guix-user)> (package-properties gcc-6)
$1 = ((hidden? . #t) (gcc-libc . #f))
scheme@(guix-user)> (package-properties gcc-7)
$2 = ((compiler-cpu-architectures ("aarch64" "armv8-a" "armv8.1-a" "armv8.2-a" "armv8.3-a") ("armhf" "armv7" "armv7-a" "armv7-m" "armv7-r" "armv7e-m" "armv7ve" "armv8-a" "armv8-a+crc" "armv8.1-a" "armv8.1-a+crc" "armv8-m.base"
"armv8-m.main" "armv8-m.main+dsp" "iwmmxt" "iwmmxt2") ("x86_64" "core2" "nehalem" "westmere" "sandybridge" "ivybridge" "haswell" "broadwell" "skylake" "bonnell" "silvermont" "knl" "skylake-avx512" "k8" "k8-sse3" "barcelona" "bdver1"
"bdver2" "bdver3" "bdver4" "znver1" "btver1" "btver2" "geode")))
scheme@(guix-user)>
--8<---------------cut here---------------end--------------->8---

Because the intent of,

--8<---------------cut here---------------start------------->8---
(define-public gcc-toolchain-aka-gcc
  ;; It's natural for users to try "guix install gcc".  This package
  ;; automatically "redirects" them to 'gcc-toolchain'.
  (deprecated-package "gcc" gcc-toolchain-10))
--8<---------------cut here---------------end--------------->8---

is to return the default GCC toolchain when typing ’gcc’ at the command
line.  And error for any other version than @10.

1: <https://yhetil.org/guix/87h6yz46wp.fsf@gmail.com>


Update the default GCC toolchain from 10 to 12 is a core-updates change
and a (almost) full world rebuild.  When the issue you are describing is
an issue about an inconsistency with the command-line.


From my point of view, an option would to rename the packages
gcc-toolchain@11 and gcc-toolchain@12 as gcc-toolchain-next@11 and
gcc-toolchain-next@12.  Which would be consistent with the rest.

And also fix the bug with the package ’gcc’. :-)


Cheers,
simon

bug#60200: Incompatibilities between gcc-toolchain and R packages

[Lars-Dominik Braun]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

Hi Simon,

> Without any specification about the version, if a package name is
> defined at several versions, then the command-line uses the higher
> version of this package.
minor nit-pick: Not the command-line, but everything that uses
specifications. So manifests via SPECIFICATIONS->MANIFEST are also
affected, see:

    (use-modules (gnu packages) (guix profiles))
    (manifest-entries (specifications->manifest '("gcc-toolchain")))
    $2 = (#<<manifest-entry> name: "gcc-toolchain" version: "12.2.0" …)

The -next suffix has the obvious disadvantage that specifications may
become invalid as we move -next to the “regular” package. So maybe
marking packages “default” like the attached patch does could improve
the current situation. Not just for gcc, but also Haskell and Python
come to mind.

Cheers,
Lars


[-- Attachment #2: default-gcc-toolchain.patch --]
[-- Type: text/plain, Size: 2317 bytes --]

diff --git a/gnu/packages.scm b/gnu/packages.scm
index 61345f75a9..7e5a6d49c2 100644
--- a/gnu/packages.scm
+++ b/gnu/packages.scm
@@ -356,20 +356,24 @@ (define cache
            (find-packages-by-name/direct name version))))
 
 (define (find-best-packages-by-name name version)
-  "If version is #f, return the list of packages named NAME with the highest
-version numbers; otherwise, return the list of packages named NAME and at
-VERSION."
+  "If version is #f, return the list of packages named NAME with only
+packages marked default? or, if none exist, the highest version numbers;
+otherwise, return the list of packages named NAME and at VERSION."
   (if version
       (find-packages-by-name name version)
       (match (find-packages-by-name name)
         (()
          '())
         ((matches ...)
-         ;; Return the subset of MATCHES with the higher version number.
-         (let ((highest (package-version (first matches))))
-           (take-while (lambda (p)
-                         (string=? (package-version p) highest))
-                       matches))))))
+         ;; Return the subset of MATCHES which are marked default or those with
+         ;; the higher version number.
+         (let ((highest (package-version (first matches)))
+               (default (filter (lambda (p) (assoc-ref (package-properties p) 'default?)) matches)))
+           (if (not (null? default))
+               default
+               (take-while (lambda (p)
+                             (string=? (package-version p) highest))
+                           matches)))))))
 
 ;; Prevent Guile 3 from inlining this procedure so we can mock it in tests.
 (set! find-best-packages-by-name find-best-packages-by-name)
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index b4566b41cc..2d5e0add26 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -3855,7 +3855,10 @@ (define* (make-gcc-toolchain gcc
                 ("libc-static" ,libc "static"))))))
 
 (define-public gcc-toolchain
-  (make-gcc-toolchain gcc-final))
+  (let ((parent (make-gcc-toolchain gcc-final)))
+    (package
+      (inherit parent)
+      (properties (alist-cons 'default? #t (package-properties parent))))))
 
 (define-public gcc-toolchain-4.8
   (make-gcc-toolchain gcc-4.8))