Aiko Kyle writes: > On the latest master I can run 'guix pull', and the latest guix seems > to build just fine, however when I go to do 'guix system reconfigure', > building guix fails the check phase. Aiko has reported that the fix for 52908 did indeed fix the "tests/guix-system.sh" problem on aarch64-linux. This means the "service 'xorg-server' provided more than once" error has been fixed. However, fixing that issue has revealed another. Aiko said: > guix system reconfigure is still failing for me on aarch64 due to the > test 'file-needed/recursive' in tests/gremlin.scm failing. So the current status is that Aiko still can't do "guix reconfigure" on master after running "guix pull". For this reason, I have re-opened this bug and unmerged it from 52908, since it's not exactly the same issue any more. At the moment, the file-needed/recursive bug that is preventing Aiko from running "guix reconfigure" appears to be specific to aarch64-linux. The problem was reported on guix-devel here: https://lists.gnu.org/archive/html/guix-devel/2022-01/msg00019.html Pierre Langlois did some investigation and found this information, posted in the email thread above: > I've also been trying to fix this test but without much luck. It > does look similar to this issue for ppc64 [0], where the `ldd/ld.so' > beaviour isn't the same as what the gremlin guile module does. However > the patch proposed there isn't fixing the issue for me either on > aarch64. > > [0]: https://issues.guix.gnu.org/52940, > > Maybe we can compare notes and a solution will come up :-). So the test > fails because 'truth', the result from `ldd', has ld-linux-aarch64.so > listed while 'needed', from the gremlin guile module doesn't: > > --8<---------------cut here---------------start------------->8--- > (truth ;; result from `ldd libguile.so' > ("/gnu/store/...-gcc-10.3.0-lib/lib/libgcc_s.so.1" > "/gnu/store/...-glibc-2.33/lib/ld-linux-aarch64.so.1" ;; This isn't > ;; in gremlins > "/gnu/store/...-glibc-2.33/lib/libc.so.6" > "/gnu/store/...-glibc-2.33/lib/libcrypt.so.1" > "/gnu/store/...-glibc-2.33/lib/libdl.so.2" > "/gnu/store/...-glibc-2.33/lib/libm.so.6" > "/gnu/store/...-glibc-2.33/lib/libpthread.so.0" > "/gnu/store/...-guile-3.0.7/lib/libguile-3.0.so.1" > "/gnu/store/...-libffi-3.3/lib/libffi.so.7" > "/gnu/store/...-libgc-8.0.4/lib/libgc.so.1" > "/gnu/store/...-libunistring-0.9.10/lib/libunistring.so.2")) > > (needed ;; result from gremlin > ("/gnu/store/...-gcc-10.3.0-lib/lib/libgcc_s.so.1" > "/gnu/store/...-glibc-2.33/lib/libc.so.6" > "/gnu/store/...-glibc-2.33/lib/libcrypt.so.1" > "/gnu/store/...-glibc-2.33/lib/libdl.so.2" > "/gnu/store/...-glibc-2.33/lib/libm.so.6" > "/gnu/store/...-glibc-2.33/lib/libpthread.so.0" > "/gnu/store/...-guile-3.0.7/lib/libguile-3.0.so.1" > "/gnu/store/...-libffi-3.3/lib/libffi.so.7" > "/gnu/store/...-libgc-8.0.4/lib/libgc.so.1" > "/gnu/store/...-libunistring-0.9.10/lib/libunistring.so.2")) > --8<---------------cut here---------------end--------------->8--- > > Digging a bit more I started comparing x86_64 and aarch64 binaries and > noticed that libguile.so didn't have the same dynamic section: > > --8<---------------cut here---------------start------------->8--- > # On aarch64: > $ objdump -x > /gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0 > > > > /gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0: > file format elf64-littleaarch64 > ... > Dynamic Section: > > NEEDED libgc.so.1 > > NEEDED libpthread.so.0 > > NEEDED libffi.so.7 > > NEEDED libunistring.so.2 > > NEEDED libcrypt.so.1 > > NEEDED libdl.so.2 > > NEEDED libm.so.6 > > NEEDED libgcc_s.so.1 > > NEEDED libc.so.6 > > SONAME libguile-3.0.so.1 > ... > > # On x86_64: > $ objdump -x > /gnu/store/3h3jn0745ngd87zp83k5smwhykxvdfgf-guile-3.0.7/lib/libguile-3.0.so.1.4.0 > > /gnu/store/3h3jn0745ngd87zp83k5smwhykxvdfgf-guile-3.0.7/lib/libguile-3.0.so.1.4.0: > file format elf64-x86-64 > ... > Dynamic Section: > NEEDED libgc.so.1 > NEEDED libpthread.so.0 > NEEDED libffi.so.7 > NEEDED libunistring.so.2 > NEEDED libcrypt.so.1 > NEEDED libdl.so.2 > NEEDED libm.so.6 > NEEDED libgcc_s.so.1 > NEEDED libc.so.6 > NEEDED ld-linux-x86-64.so.2 # ld-linux-.so is here > SONAME libguile-3.0.so.1 > ... > --8<---------------cut here---------------end--------------->8--- > > On aarch64, ld-linux- is missing. I'm not sure if this is an > issue with our aarch64 port or if that's OK. It's interesting though > that if you run `ldd' on libguile on aarch64, the dynamic linker is > added to the list, even though it's not in the dynamic section: > > --8<---------------cut here---------------start------------->8--- > # On aarch64 > $ ldd > /gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0 > linux-vdso.so.1 (0x0000ffff96fab000) > libgc.so.1 => > /gnu/store/1gkpbfxjx2sbchjhf19yjm4a8vkir0nm-libgc-8.0.4/lib/libgc.so.1 > (0x0000ffff96d88000) > libpthread.so.0 => > /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libpthread.so.0 > (0x0000ffff96d59000) > libffi.so.7 => > /gnu/store/hjirgm7pwmc2biqz6d0fc1ajr3ha4v14-libffi-3.3/lib/libffi.so.7 > (0x0000ffff96d40000) > libunistring.so.2 => > /gnu/store/4k552fq1p6q73mr9ww7g5y3m77p7cfbm-libunistring-0.9.10/lib/libunistring.so.2 > (0x0000ffff96bb4000) > libcrypt.so.1 => > /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libcrypt.so.1 > (0x0000ffff96b6d000) > libdl.so.2 => > /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libdl.so.2 > (0x0000ffff96b59000) > libm.so.6 => > /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libm.so.6 > (0x0000ffff96ab0000) > libgcc_s.so.1 => > /gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/libgcc_s.so.1 > (0x0000ffff96a8b000) > libc.so.6 => > /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libc.so.6 > (0x0000ffff96917000) > > /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/ld-linux-aarch64.so.1 > (0x0000ffff96f79000) > # ^ > --8<---------------cut here---------------end--------------->8--- > > We could adapt the test to add the dynamic linker, emulating `ldd', but > I'm curious if anybody deeply familiar with ELF and dynamic linking > might have an idea what's going on. To summarize, this seems to be the problem: On aarch64-linux, when an ELF file's dynamic section does not contain a NEEDED entry for ld-linux-aarch64.so.1, file-needed/recursive dutifully omits that entry, but ldd prints it (even though it is actually missing from the file's dynamic section). This causes the file-needed/recursive test in tests/gremlin.scm to fail because the set of entries returned by file-needed/recursive differs from the set of entries returned by ldd. Although this may sound similar to 52940, it is different. Bug 52940 was an issue where, when an ELF file's dynamic section contains two entries in RUNPATH that are different strings but refer to the same directory (which does happen on powerpc64le-linux), it causes file-needed/recursive to include in its result two entries that are different strings but refer to the same file. We decided that such redundant entries are probably benign, so we resolved 52940 by changing the test to compare entries by file equality, rather than string equality. However, this is not the same issue as described by Aiko and Pierre above, so it makes sense that the fix for 52940 did not fix the test for aarch64-linux. So, the separate aarch64-linux problem with file-needed/recursive persists. For comparison, on powerpc64le-linux, when an ELF file's dynamic section does not contain a NEEDED entry for "ld64.so.2" (I believe this is the powerpc64le-linux equivalent of ld-linux-aarch64.so.1), both ldd and file-needed/recursive include ld64.so.2, despite the fact that there is no actual NEEDED entry for it in the ELF file. Here is what I see on a powerpc64le-linux system: --8<---------------cut here---------------start------------->8--- marusich@suzaku ~/guix-master [env]$ type -P guile /gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/guile marusich@suzaku ~/guix-master [env]$ type -P ldd /gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/ldd marusich@suzaku ~/guix-master [env]$ ldd /gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/guile linux-vdso64.so.1 (0x00007fff89220000) libguile-3.0.so.1 => /gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib/libguile-3.0.so.1 (0x00007fff89050000) libgc.so.1 => /gnu/store/csqz3w2axyci2qm79xsj11cpfxqh7zb1-libgc-8.0.4/lib/libgc.so.1 (0x00007fff88fb0000) libpthread.so.0 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libpthread.so.0 (0x00007fff88f60000) libffi.so.7 => /gnu/store/kmgva3hbpk1bv0gvx5s4w01n0fdvd2l9-libffi-3.3/lib/../lib/libffi.so.7 (0x00007fff88f30000) libunistring.so.2 => /gnu/store/zf87w2xccli6yvxpplfwd82gcgm6jyrd-libunistring-0.9.10/lib/libunistring.so.2 (0x00007fff88d80000) libcrypt.so.1 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libcrypt.so.1 (0x00007fff88d30000) libdl.so.2 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libdl.so.2 (0x00007fff88d00000) libm.so.6 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libm.so.6 (0x00007fff88bb0000) libgcc_s.so.1 => /gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib/libgcc_s.so.1 (0x00007fff88b70000) libc.so.6 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libc.so.6 (0x00007fff88950000) /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/ld64.so.2 (0x00007fff89240000) marusich@suzaku ~/guix-master [env]$ readelf -d /gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/guile Dynamic section at offset 0xfc60 contains 37 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [libguile-3.0.so.1] 0x0000000000000001 (NEEDED) Shared library: [libgc.so.1] 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] 0x0000000000000001 (NEEDED) Shared library: [libffi.so.7] 0x0000000000000001 (NEEDED) Shared library: [libunistring.so.2] 0x0000000000000001 (NEEDED) Shared library: [libcrypt.so.1] 0x0000000000000001 (NEEDED) Shared library: [libdl.so.2] 0x0000000000000001 (NEEDED) Shared library: [libm.so.6] 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000001d (RUNPATH) Library runpath: [/gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib:/gnu/store/csqz3w2axyci2qm79xsj11cpfxqh7zb1-libgc-8.0.4/lib:/gnu/store/kmgva3hbpk1bv0gvx5s4w01n0fdvd2l9-libffi-3.3/lib/../lib:/gnu/store/zf87w2xccli6yvxpplfwd82gcgm6jyrd-libunistring-0.9.10/lib:/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib:/gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib:/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/../lib:/gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib/gcc/powerpc64le-unknown-linux-gnu/10.3.0/../../../../lib] 0x000000000000000c (INIT) 0x100009e0 0x000000000000000d (FINI) 0x10000f30 0x0000000000000019 (INIT_ARRAY) 0x1001fc50 0x000000000000001b (INIT_ARRAYSZ) 8 (bytes) 0x000000000000001a (FINI_ARRAY) 0x1001fc58 0x000000000000001c (FINI_ARRAYSZ) 8 (bytes) 0x0000000000000004 (HASH) 0x100002a0 0x000000006ffffef5 (GNU_HASH) 0x100002f8 0x0000000000000005 (STRTAB) 0x100004a8 0x0000000000000006 (SYMTAB) 0x10000328 0x000000000000000a (STRSZ) 894 (bytes) 0x000000000000000b (SYMENT) 24 (bytes) 0x0000000000000015 (DEBUG) 0x0 0x0000000000000003 (PLTGOT) 0x10020000 0x0000000000000002 (PLTRELSZ) 216 (bytes) 0x0000000000000014 (PLTREL) RELA 0x0000000000000017 (JMPREL) 0x100008e8 0x0000000070000000 (PPC64_GLINK) 0x10000eec 0x0000000070000003 (PPC64_OPT) 0x0 0x0000000000000007 (RELA) 0x10000888 0x0000000000000008 (RELASZ) 96 (bytes) 0x0000000000000009 (RELAENT) 24 (bytes) 0x000000006ffffffe (VERNEED) 0x10000848 0x000000006fffffff (VERNEEDNUM) 2 0x000000006ffffff0 (VERSYM) 0x10000826 0x0000000000000000 (NULL) 0x0 marusich@suzaku ~/guix-master [env]$ ./pre-inst-env guix repl GNU Guile 3.0.7 Copyright (C) 1995-2021 Free Software Foundation, Inc. Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. This program is free software, and you are welcome to redistribute it under certain conditions; type `,show c' for details. Enter `,help' for help. scheme@(guix-user)> ,use (guix build gremlin) scheme@(guix-user)> ,pp (file-needed/recursive "/gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/guile") $1 = ("/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libc.so.6" "/gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib/libgcc_s.so.1" "/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libm.so.6" "/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libdl.so.2" "/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libcrypt.so.1" "/gnu/store/zf87w2xccli6yvxpplfwd82gcgm6jyrd-libunistring-0.9.10/lib/libunistring.so.2" "/gnu/store/kmgva3hbpk1bv0gvx5s4w01n0fdvd2l9-libffi-3.3/lib/../lib/libffi.so.7" "/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libpthread.so.0" "/gnu/store/csqz3w2axyci2qm79xsj11cpfxqh7zb1-libgc-8.0.4/lib/libgc.so.1" "/gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib/libguile-3.0.so.1" "/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/ld64.so.2" "/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/../lib/libc.so.6") $2 = ("ld64.so.2") scheme@(guix-user)> --8<---------------cut here---------------end--------------->8--- I don't know if it's related, but I just noticed this: it's a little strange that in the above output (for powerpc64le-linux), ld64.so.2 is included in the second value ($2). This is supposedly the list of .so file names that could not be found. It's strange that ld64.so.2 shows up in $2 because it seems to contradict the fact that it is included in $1, which is the list of files that were found successfully. Since Pierre shared information about the libguile shared object specifically, I'll do the same here. On powerpc64le-linux, the "ld64.so.2" entry is present in the dynamic section of the /gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib/libguile-3.0.so.1 ELF file: --8<---------------cut here---------------start------------->8--- marusich@suzaku ~/guix-master [env]$ readelf -d /gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib/libguile-3.0.so.1| grep -e NEEDED -e RUNPATH 0x0000000000000001 (NEEDED) Shared library: [libgc.so.1] 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] 0x0000000000000001 (NEEDED) Shared library: [libffi.so.7] 0x0000000000000001 (NEEDED) Shared library: [libunistring.so.2] 0x0000000000000001 (NEEDED) Shared library: [libcrypt.so.1] 0x0000000000000001 (NEEDED) Shared library: [libdl.so.2] 0x0000000000000001 (NEEDED) Shared library: [libm.so.6] 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x0000000000000001 (NEEDED) Shared library: [ld64.so.2] 0x000000000000001d (RUNPATH) Library runpath: [/gnu/store/csqz3w2axyci2qm79xsj11cpfxqh7zb1-libgc-8.0.4/lib:/gnu/store/kmgva3hbpk1bv0gvx5s4w01n0fdvd2l9-libffi-3.3/lib/../lib:/gnu/store/zf87w2xccli6yvxpplfwd82gcgm6jyrd-libunistr ing-0.9.10/lib:/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib:/gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib:/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/../lib:/gnu/store/l0hnwrv8ma03shjg84a03s05 wmj7a0sk-gcc-10.3.0-lib/lib/gcc/powerpc64le-unknown-linux-gnu/10.3.0/../../../../lib] --8<---------------cut here---------------end--------------->8--- Maybe this information can help somehow. It seems fishy that on aarch64-linux, there is no NEEDED entry for ld-linux-aarch64.so.1 in the ELF files under consideration. As shown above, a similar entry DOES exist on both x86_64-linux and powerpc64le-linux. For this reason, it seems plausible that maybe the missing NEEDED entry is bad. However, I don't really know enough to say whether it's indicative of a problem with our aarch64-linux port. Is there an aarch64-linux or ELF expert in the room who can help? :-) It also seems fishy that, on powerpc64le-linux, file-needed/recursive apparently resolves ld64.so.2 successfully, even though it simultaneously includes it in the "failed to resolve" list. Confusing as that is to me, I don't know if that's really related to the aarch64-linux issue. More investigation is needed... -- Chris PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836