guix fails to build on aarch64

guix fails to build on aarch64

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

Ever since commit:

  224d437fb4147fe9eae277cbc8d39e1ef54bf5c9 gnu: guix: Update to 2a621f1.

I've been unable to build "guix" on aarch64, several test suites
failed... I'll try to dig up the failures, but figured it was worth
mentioning in case anyone else saw it too and had the failures more
handy.

Reverting that commit gets guix building fine again, so that's a bit of
an ugly workaround, but ... works okish.

Makes it a bit hard to update Guix System on aarch64!


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: guix fails to build on aarch64

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2377 bytes --]

On 2021-12-16, Vagrant Cascadian wrote:
> Ever since commit:
>
>   224d437fb4147fe9eae277cbc8d39e1ef54bf5c9 gnu: guix: Update to 2a621f1.
>
> I've been unable to build "guix" on aarch64, several test suites
> failed... I'll try to dig up the failures

Here are the failed tests...

test-name: maybe-remove-expired-cache-entries, first cleanup
location: /tmp/guix-build-guix-1.3.0-14.2a621f1.drv-0/source/tests/cache.scm:58
source:
+ (test-equal
+   "maybe-remove-expired-cache-entries, first cleanup"
+   '("a" "b" "c")
+   (test-cache-cleanup cache))
expected-value: ("a" "b" "c")
actual-value: ()
result: FAIL

test-name: maybe-remove-expired-cache-entries, cleanup needed
location: /tmp/guix-build-guix-1.3.0-14.2a621f1.drv-0/source/tests/cache.scm:70
source:
+ (test-equal
+   "maybe-remove-expired-cache-entries, cleanup needed"
+   '("a" "b" "c")
+   (test-cache-cleanup
+     cache
+     (call-with-output-file
+       (string-append cache "/last-expiry-cleanup")
+       (lambda (port) (display 0 port)))))
expected-value: ("a" "b" "c")
actual-value: ()
result: FAIL

test-name: file-needed/recursive
location: /tmp/guix-build-guix-1.3.0-14.2a621f1.drv-0/source/tests/gremlin.scm:70
source:
+ (test-assert
+   "file-needed/recursive"
+   (let* ((needed
+            (file-needed/recursive %guile-executable))
+          (pipe (dynamic-wind
+                  (lambda ()
+                    (setenv "LD_TRACE_LOADED_OBJECTS" "yup"))
+                  (lambda ()
+                    (open-pipe* OPEN_READ %guile-executable))
+                  (lambda () (unsetenv "LD_TRACE_LOADED_OBJECTS")))))
+     (define ldd-rx
+       (make-regexp
+         "^[[:blank:]]+([[:graph:]]+ => )?([[:graph:]]+) .*$"))
+     (define (read-ldd-output port)
+       (let loop ((result '()))
+         (match (read-line port)
+                ((? eof-object?) (reverse result))
+                ((= (cut regexp-exec ldd-rx <>) m)
+                 (if m
+                   (loop (cons (match:substring m 2) result))
+                   (loop result))))))
+     (define ground-truth
+       (remove
+         (cut string-prefix? "linux-vdso.so" <>)
+         (read-ldd-output pipe)))
+     (and (zero? (close-pipe pipe))
+          (lset= string=?
+                 (pk 'truth ground-truth)
+                 (pk 'needed needed)))))
actual-value: #f
result: FAIL



live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: guix fails to build on aarch64

[Ricardo Wurmus]

Vagrant Cascadian <vagrant@debian.org> writes:

> test-name: file-needed/recursive
> location: /tmp/guix-build-guix-1.3.0-14.2a621f1.drv-0/source/tests/gremlin.scm:70
> source:
…
> +     (and (zero? (close-pipe pipe))
> +          (lset= string=?
> +                 (pk 'truth ground-truth)
> +                 (pk 'needed needed)))))
> actual-value: #f
> result: FAIL

Did the logs not contain the values for truth and needed?  That would
mean that the test already failed to close the pipe.

-- 
Ricardo

guix fails to build on aarch64

[Akira Kyle]

I think this is the related issue: https://issues.guix.gnu.org/52943

Re: guix fails to build on aarch64

[Akira Kyle]

Ricardo Wurmus <rekado@elephly.net> writes:
> > test-name: file-needed/recursive
> > location:
> > /tmp/guix-build-guix-1.3.0-14.2a621f1.drv-0/source/tests/gremlin.scm:70
> > source:
> …
> > +     (and (zero? (close-pipe pipe))
> > +          (lset= string=?
> > +                 (pk 'truth ground-truth)
> > +                 (pk 'needed needed)))))
> > actual-value: #f
> > result: FAIL

> Did the logs not contain the values for truth and needed?  That would
> mean that the test already failed to close the pipe.

I've been trying to debug failing guix tests on aarch64. At the end of
logfile for the gremlin test suite there's the following which may be
related to why the truth and needed values were not printed:

a.out: stripping RUNPATH to
("/gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib"
"/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib"
"/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/gcc/aarch64-unknown-linux-gnu/10.3.0/../../.."
"/gnu/store/40lx91wz35qci25qzi9xfqvxwby85xha-profile/lib") (removed
("/foo" "/bar"))
a.out: warning: RUNPATH contains bogus entries:
("/gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib"
"/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib"
"/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/gcc/aarch64-unknown-linux-gnu/10.3.0/../../.."
"/gnu/store/40lx91wz35qci25qzi9xfqvxwby85xha-profile/lib")
a.out: error: depends on 'libgcc_s.so.1', which cannot be found in RUNPATH ()
WARNING: (test-gremlin): imported module (guix build utils) overrides
core binding `delete'

There are other tests failing as well and the full test-suite.log I
posted here: https://issues.guix.gnu.org/52943

I'm a newcomer to guix so I feel a bit out of my depth trying to debug
these failures. I'd really like to be able to use guix as my daily
driver but so far it's been difficult since my machine is aarch64!

Re: guix fails to build on aarch64

[Pierre Langlois]

[-- Attachment #1: Type: text/plain, Size: 8824 bytes --]

Hi there,

Akira Kyle <akira@akirakyle.com> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>> > test-name: file-needed/recursive
>> > location:
>> > /tmp/guix-build-guix-1.3.0-14.2a621f1.drv-0/source/tests/gremlin.scm:70
>> > source:
>> …
>> > +     (and (zero? (close-pipe pipe))
>> > +          (lset= string=?
>> > +                 (pk 'truth ground-truth)
>> > +                 (pk 'needed needed)))))
>> > actual-value: #f
>> > result: FAIL
>
>> Did the logs not contain the values for truth and needed?  That would
>> mean that the test already failed to close the pipe.
>
> I've been trying to debug failing guix tests on aarch64. At the end of
> logfile for the gremlin test suite there's the following which may be
> related to why the truth and needed values were not printed:
>
> a.out: stripping RUNPATH to
> ("/gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib"
> "/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib"
> "/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/gcc/aarch64-unknown-linux-gnu/10.3.0/../../.."
> "/gnu/store/40lx91wz35qci25qzi9xfqvxwby85xha-profile/lib") (removed
> ("/foo" "/bar"))
> a.out: warning: RUNPATH contains bogus entries:
> ("/gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib"
> "/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib"
> "/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/gcc/aarch64-unknown-linux-gnu/10.3.0/../../.."
> "/gnu/store/40lx91wz35qci25qzi9xfqvxwby85xha-profile/lib")
> a.out: error: depends on 'libgcc_s.so.1', which cannot be found in RUNPATH ()
> WARNING: (test-gremlin): imported module (guix build utils) overrides
> core binding `delete'

I've also been trying to fix this test but without much luck.  It
does look similar to this issue for ppc64 [0], where the `ldd/ld.so'
beaviour isn't the same as what the gremlin guile module does. However
the patch proposed there isn't fixing the issue for me either on
aarch64.

[0]: https://issues.guix.gnu.org/52940,

Maybe we can compare notes and a solution will come up :-).  So the test
fails because 'truth', the result from `ldd', has ld-linux-aarch64.so
listed while 'needed', from the gremlin guile module doesn't:

--8<---------------cut here---------------start------------->8---
(truth ;; result from `ldd libguile.so'
 ("/gnu/store/...-gcc-10.3.0-lib/lib/libgcc_s.so.1"
  "/gnu/store/...-glibc-2.33/lib/ld-linux-aarch64.so.1"   ;; This isn't
                                                          ;; in gremlins
  "/gnu/store/...-glibc-2.33/lib/libc.so.6"
  "/gnu/store/...-glibc-2.33/lib/libcrypt.so.1"
  "/gnu/store/...-glibc-2.33/lib/libdl.so.2"
  "/gnu/store/...-glibc-2.33/lib/libm.so.6"
  "/gnu/store/...-glibc-2.33/lib/libpthread.so.0"
  "/gnu/store/...-guile-3.0.7/lib/libguile-3.0.so.1"
  "/gnu/store/...-libffi-3.3/lib/libffi.so.7"
  "/gnu/store/...-libgc-8.0.4/lib/libgc.so.1"
  "/gnu/store/...-libunistring-0.9.10/lib/libunistring.so.2"))

(needed  ;; result from gremlin
 ("/gnu/store/...-gcc-10.3.0-lib/lib/libgcc_s.so.1"
  "/gnu/store/...-glibc-2.33/lib/libc.so.6"
  "/gnu/store/...-glibc-2.33/lib/libcrypt.so.1"
  "/gnu/store/...-glibc-2.33/lib/libdl.so.2"
  "/gnu/store/...-glibc-2.33/lib/libm.so.6"
  "/gnu/store/...-glibc-2.33/lib/libpthread.so.0"
  "/gnu/store/...-guile-3.0.7/lib/libguile-3.0.so.1"
  "/gnu/store/...-libffi-3.3/lib/libffi.so.7"
  "/gnu/store/...-libgc-8.0.4/lib/libgc.so.1"
  "/gnu/store/...-libunistring-0.9.10/lib/libunistring.so.2"))
--8<---------------cut here---------------end--------------->8---

Digging a bit more I started comparing x86_64 and aarch64 binaries and
noticed that libguile.so didn't have the same dynamic section:

--8<---------------cut here---------------start------------->8---
# On aarch64:
$ objdump -x /gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0                        
                                                                                                                                        
/gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0:     file format elf64-littleaarch64    
...
Dynamic Section:                                                                                                                        
  NEEDED               libgc.so.1                                                                                                       
  NEEDED               libpthread.so.0                                                                                                  
  NEEDED               libffi.so.7                                                                                                      
  NEEDED               libunistring.so.2                                                                                                
  NEEDED               libcrypt.so.1                                                                                                    
  NEEDED               libdl.so.2                                                                                                       
  NEEDED               libm.so.6                                                                                                        
  NEEDED               libgcc_s.so.1                                                                                                    
  NEEDED               libc.so.6                                                                                                        
  SONAME               libguile-3.0.so.1    
...

# On x86_64:
$ objdump -x /gnu/store/3h3jn0745ngd87zp83k5smwhykxvdfgf-guile-3.0.7/lib/libguile-3.0.so.1.4.0

/gnu/store/3h3jn0745ngd87zp83k5smwhykxvdfgf-guile-3.0.7/lib/libguile-3.0.so.1.4.0:     file format elf64-x86-64
...
Dynamic Section:
  NEEDED               libgc.so.1
  NEEDED               libpthread.so.0
  NEEDED               libffi.so.7
  NEEDED               libunistring.so.2
  NEEDED               libcrypt.so.1
  NEEDED               libdl.so.2
  NEEDED               libm.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libc.so.6
  NEEDED               ld-linux-x86-64.so.2  # ld-linux-<arch>.so is here
  SONAME               libguile-3.0.so.1
...
--8<---------------cut here---------------end--------------->8---

On aarch64, ld-linux-<arch> is missing.  I'm not sure if this is an
issue with our aarch64 port or if that's OK. It's interesting though
that if you run `ldd' on libguile on aarch64, the dynamic linker is
added to the list, even though it's not in the dynamic section:

--8<---------------cut here---------------start------------->8---
# On aarch64
$ ldd /gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0
        linux-vdso.so.1 (0x0000ffff96fab000)
        libgc.so.1 => /gnu/store/1gkpbfxjx2sbchjhf19yjm4a8vkir0nm-libgc-8.0.4/lib/libgc.so.1 (0x0000ffff96d88000)
        libpthread.so.0 => /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libpthread.so.0 (0x0000ffff96d59000)
        libffi.so.7 => /gnu/store/hjirgm7pwmc2biqz6d0fc1ajr3ha4v14-libffi-3.3/lib/libffi.so.7 (0x0000ffff96d40000)
        libunistring.so.2 => /gnu/store/4k552fq1p6q73mr9ww7g5y3m77p7cfbm-libunistring-0.9.10/lib/libunistring.so.2 (0x0000ffff96bb4000)
        libcrypt.so.1 => /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libcrypt.so.1 (0x0000ffff96b6d000)
        libdl.so.2 => /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libdl.so.2 (0x0000ffff96b59000)
        libm.so.6 => /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libm.so.6 (0x0000ffff96ab0000)
        libgcc_s.so.1 => /gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/libgcc_s.so.1 (0x0000ffff96a8b000)
        libc.so.6 => /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libc.so.6 (0x0000ffff96917000)
        /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/ld-linux-aarch64.so.1 (0x0000ffff96f79000)
        # ^
--8<---------------cut here---------------end--------------->8---

We could adapt the test to add the dynamic linker, emulating `ldd', but
I'm curious if anybody deeply familiar with ELF and dynamic linking
might have an idea what's going on.

>
> There are other tests failing as well and the full test-suite.log I
> posted here: https://issues.guix.gnu.org/52943
>
> I'm a newcomer to guix so I feel a bit out of my depth trying to debug
> these failures. I'd really like to be able to use guix as my daily
> driver but so far it's been difficult since my machine is aarch64!

Thanks for highlighting this issue! Hopefully we can get this fixed
soon.

Thanks,
Pierre

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

Re: guix fails to build on aarch64

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]

Hi Pierre,

Thank you very much for the details!  I have added this information to
the existing bug report:

https://issues.guix.gnu.org/52943

Please direct further discussion regarding this bug there, so that
people looking at the bug report will see an accurate record of the
investigation so far.

-- 
Chris

PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]