bug#43513: json-c build failure (on armhf-linux) while trying to build u-boot

[Danny Milosavljevic <dannym@scratchpost.org>]

[-- Attachment #1.1: Type: text/plain, Size: 6193 bytes --]

Hi,

I found the underlying cause of the problem with qemu transparent emulation:

* qemu transparent emulator has 64 bit registers
* the thing it's emulating has 32 bit registers
* The glibc in the distro that is running in the emulator is using getdents64
(on 32 bits!) and then (rightfully) checking whether d_off and the inode number
fit into their own (32 bits/entry) struct, which they don't (the thing they get
from the kernel is 64 bits/entry).

See also https://lore.kernel.org/lkml/20181229015453.GA6310@bombadil.infradead.org/T/
for an analysis.

See also https://sourceware.org/bugzilla/show_bug.cgi?id=23960 for a discussion.

Least-shitty workaround: Use a 32-bit qemu (yes, a qemu compiled on 32 bit)
on a 64 bit machine for transparent emulation of ANOTHER 32-bit machine.
That way, the kernel can know that there's a 32 bit user lurking somewhere up
the call chain that is calling getdents64 and is not actually able to process the
result.  "The truth?  It can't handle the truth."

The right fix: One could also tell all the packages in the emulated
system to use the large file size API (-D_FILE_OFFSET_BITS=64 and co).  In this
case cmake is affected--but it could be any number of things.  I think that that
is the only good fix (we could also add a compile-time check whether <dirent.h>
has been included without anyone specifying -D_FILE_OFFSET_BITS=64--that would
make finding these problems a LOT easier; if possible, emit that compile-time
error only if readdir is actually called anywhere).

For the workaround, we could adapt Guix system's gnu/services/virtualization.scm
so it uses a qemu built for the 32-bit variant of the host architecture if it's
emulating a 32 bit system.

So qemu could be compiled for i686 if the host is x86_64 and the emulated
system is armhf,
qemu could be compiled for armhf if the host is aarch64 and the emulated system
is armhf and so on.

That also means that if a host system is 64 bits and DOES NOT HAVE a 32 bit
pendant target, then the emulation of a 32 bit system is going to be imperfect.

One way to fix that would be to fix the kernel to accept an argument on
the getdents64 syscall (and similar ones) that tells it whether the user wants
to have 64 bit offsets or 32 bit offsets[1].  Right now, from user space, that
is only possible via process personality flags.  And those would switch the
entire qemu executable over to 32 bits, which we don't want (qemu itself has
to do stuff using kernel syscalls, so it needs to be capable of 64 bits
if it itself is 64 bits).  And I think that even that case is not being
handled in the kernel correctly.  So this fix cannot be done.

@Ludo:

Anyway, I have a question on how to replicate what "guix build --target=i686..."
does, in the guix service definition.  How can I make it use
package-cross-derivation instead of package-derivation ?
See attachment.

With the attachment and the service definition

            (service qemu-binfmt-service-type
             (qemu-binfmt-configuration
              (platforms (lookup-qemu-platforms "arm"))
              (guix-support? #t)))

in order to emulate ARM on x86_64 I eventually get:

>Building qemu-minimal for i686

(That's what I want!)

>[...]
>starting phase `configure'
>
>ERROR: pkg-config binary 'i686-unknown-linux-gnu-pkg-config' not found

pkg-config has not been cross-compiled...

That's because it's not using package-cross-derivation, I guess.

>command "./configure" "--cc=/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/bin/gcc" "--host-cc=/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/bin/gcc" "--disable-debug-info" "--enable-virtfs" "--prefix=/gnu/store/80ljf47lrh8arrzjmkrrqxghc0k67b3s-qemu-minimal-5.1.0" "--sysconfdir=/etc" "--cross-prefix=i686-unknown-linux-gnu-" "--target-list=i386-softmmu,x86_64-softmmu" failed with status 1

I think that "--cc" should use ,(cc-for-target) at all times.

Better this:

diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index 53e9dde125..bf712afd4a 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -227,7 +227,7 @@
                (setenv "LDFLAGS" "-lrt")
                (apply invoke
                       `("./configure"
-                        ,(string-append "--cc=" (which "gcc"))
+                        ,(string-append "--cc=" ,(cc-for-target))
                         ;; Some architectures insist on using HOST_CC
                         ,(string-append "--host-cc=" (which "gcc"))
                         "--disable-debug-info" ; save build space

[1] A way for userspace to tell the kernel that is to use getdents instad of
getdents64 on 32 bits, like it used to.  But they don't want to do that anymore.

Another way would be for qemu to translate a syscall getdents64 from the guest
to getdents on the host if the machine they are emulating is 32 bits.  But that
would mean that getdents on a 64 bit host would still have to act like it's
32 bits and translate the d_off accordingly.  That's not guaranteed[2].

Or even using the old readdir syscall.

[2] Linux-5.8.8:

[... CONFIG_COMPAT ...]
SYSCALL_DEFINE3(getdents, unsigned int, fd,
                struct linux_dirent __user *, dirent, unsigned int, count)
{
        struct fd f;
        struct getdents_callback buf = {
                .ctx.actor = filldir,
                .count = count,
                .current_dir = dirent
        };
        int error;

        f = fdget_pos(fd);
        if (!f.file)
                return -EBADF;

        error = iterate_dir(f.file, &buf.ctx);
        if (error >= 0)
                error = buf.error;
        if (buf.prev_reclen) {
                struct linux_dirent __user * lastdirent;
                lastdirent = (void __user *)buf.current_dir - buf.prev_reclen;

                if (put_user(buf.ctx.pos, &lastdirent->d_off))
                        error = -EFAULT;
                else
                        error = count - buf.count;
        }
        fdput_pos(f);
        return error;
}
It's not guaranteed.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: services-qemu-transparent-emulation-make-more-exact.patch --]
[-- Type: text/x-patch, Size: 11665 bytes --]

diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 20e104f48c..77f15ba07b 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -23,6 +23,7 @@
   #:use-module (gnu bootloader grub)
   #:use-module (gnu image)
   #:use-module (gnu packages admin)
+  #:use-module (gnu packages cross-base)
   #:use-module (gnu packages ssh)
   #:use-module (gnu packages virtualization)
   #:use-module (gnu services base)
@@ -554,12 +555,13 @@ potential infinite waits blocking libvirt."))
 
 ;; Platforms that QEMU can emulate.
 (define-record-type <qemu-platform>
-  (qemu-platform name family magic mask)
+  (qemu-platform name family register-width magic mask)
   qemu-platform?
-  (name     qemu-platform-name)                   ;string
-  (family   qemu-platform-family)                 ;string
-  (magic    qemu-platform-magic)                  ;bytevector
-  (mask     qemu-platform-mask))                  ;bytevector
+  (name     qemu-platform-name)                                 ;string
+  (family   qemu-platform-family)                               ;string
+  (register-width    qemu-platform-register-width)              ;int, in bits
+  (magic    qemu-platform-magic)                                ;bytevector
+  (mask     qemu-platform-mask))                                ;bytevector
 
 (define-syntax bv
   (lambda (s)
@@ -576,123 +578,123 @@ potential infinite waits blocking libvirt."))
 ;;; 'scripts/qemu-binfmt-conf.sh' in QEMU.
 
 (define %i386
-  (qemu-platform "i386" "i386"
+  (qemu-platform "i386" "i386" 32
                  (bv "\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00")
                  (bv "\xff\xff\xff\xff\xff\xfe\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %i486
-  (qemu-platform "i486" "i386"
+  (qemu-platform "i486" "i386" 32
                  (bv "\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x06\x00")
                  (bv "\xff\xff\xff\xff\xff\xfe\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %alpha
-  (qemu-platform "alpha" "alpha"
+  (qemu-platform "alpha" "alpha" 64
                  (bv "\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x26\x90")
                  (bv "\xff\xff\xff\xff\xff\xfe\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %arm
-  (qemu-platform "arm" "arm"
+  (qemu-platform "arm" "arm" 32
                  (bv "\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %armeb
-  (qemu-platform "armeb" "arm"
+  (qemu-platform "armeb" "arm" 32
                  (bv "\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %sparc
-  (qemu-platform "sparc" "sparc"
+  (qemu-platform "sparc" "sparc" 32 ; FIXME check
                  (bv "\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %sparc32plus
-  (qemu-platform "sparc32plus" "sparc"
+  (qemu-platform "sparc32plus" "sparc" 32 ; FIXME check
                  (bv "\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x12")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %ppc
-  (qemu-platform "ppc" "ppc"
+  (qemu-platform "ppc" "ppc" 32
                  (bv "\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x14")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %ppc64
-  (qemu-platform "ppc64" "ppc"
+  (qemu-platform "ppc64" "ppc" 64
                  (bv "\x7fELF\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x15")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %ppc64le
-  (qemu-platform "ppc64le" "ppcle"
+  (qemu-platform "ppc64le" "ppcle" 64
                  (bv "\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x15\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\x00")))
 
 (define %m68k
-  (qemu-platform "m68k" "m68k"
+  (qemu-platform "m68k" "m68k" 32
                  (bv "\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x04")
                  (bv "\xff\xff\xff\xff\xff\xff\xfe\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 ;; XXX: We could use the other endianness on a MIPS host.
 (define %mips
-  (qemu-platform "mips" "mips"
+  (qemu-platform "mips" "mips" 32
                  (bv "\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %mipsel
-  (qemu-platform "mipsel" "mips"
+  (qemu-platform "mipsel" "mips" 32
                  (bv "\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %mipsn32
-  (qemu-platform "mipsn32" "mips"
+  (qemu-platform "mipsn32" "mips" 32 ; FIXME check
                  (bv "\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %mipsn32el
-  (qemu-platform "mipsn32el" "mips"
+  (qemu-platform "mipsn32el" "mips" 32 ; FIXME check
                  (bv "\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %mips64
-  (qemu-platform "mips64" "mips"
+  (qemu-platform "mips64" "mips" 64
                  (bv "\x7fELF\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %mips64el
-  (qemu-platform "mips64el" "mips"
+  (qemu-platform "mips64el" "mips" 64
                  (bv "\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %riscv32
-  (qemu-platform "riscv32" "riscv"
+  (qemu-platform "riscv32" "riscv" 32 ; FIXME
                  (bv "\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %riscv64
-  (qemu-platform "riscv64" "riscv"
+  (qemu-platform "riscv64" "riscv" 64
                  (bv "\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %sh4
-  (qemu-platform "sh4" "sh4"
+  (qemu-platform "sh4" "sh4" 32
                  (bv "\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x2a\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %sh4eb
-  (qemu-platform "sh4eb" "sh4"
+  (qemu-platform "sh4eb" "sh4" 32
                  (bv "\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x2a")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %s390x
-  (qemu-platform "s390x" "s390x"
+  (qemu-platform "s390x" "s390x" 64
                  (bv "\x7fELF\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x16")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
 (define %aarch64
-  (qemu-platform "aarch64" "arm"
+  (qemu-platform "aarch64" "arm" 64
                  (bv "\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff")))
 
 (define %hppa
-  (qemu-platform "hppa" "hppa"
+  (qemu-platform "hppa" "hppa" 32
                  (bv "\x7f\x45\x4c\x46\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x0f")
                  (bv "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff")))
 
@@ -712,12 +714,48 @@ potential infinite waits blocking libvirt."))
   qemu-binfmt-configuration make-qemu-binfmt-configuration
   qemu-binfmt-configuration?
   (qemu        qemu-binfmt-configuration-qemu
-               (default qemu))
+               (default qemu-minimal))
   (platforms   qemu-binfmt-configuration-platforms
                (default '()))                     ;safest default
   (guix-support? qemu-binfmt-configuration-guix-support?
                  (default #f)))
 
+(define (register-width system)
+  (match (%current-system)
+   ("i686-linux" 32)
+   ("armhf-linux" 32)
+   ("aarch64-linux" 64)
+   ("x86_64-linux" 64)))
+
+(define (closest-cross-compiled-qemu qemu target-bits)
+  "Cross-compile QEMU for the given TARGET-BITS platform that is closest to
+the actual host architecture, if possible.  This is in order to prevent
+https://lore.kernel.org/lkml/20181229015453.GA6310@bombadil.infradead.org/T/"
+  (define (cross-compiled-qemu target)
+    (package
+      (inherit qemu)
+      (arguments
+       (substitute-keyword-arguments (package-arguments qemu)
+        ((#:configure-flags flags)
+         `(cons ,(string-append "--cross-prefix=" target "-")
+                ,flags))))
+      (native-inputs
+        `(("cross-gcc" ,(cross-gcc target))
+          ("cross-binutils" ,(cross-binutils target))
+          ,@(package-native-inputs qemu)))))
+  (match target-bits
+   (64 qemu)
+   (32 (match (register-width (%current-system))
+        (32 qemu)
+        (64 (match (%current-system)
+             ("x86_64-linux"
+              (cross-compiled-qemu (nix-system->gnu-triplet "i686-linux")))
+             ("aarch64-linux"
+              (cross-compiled-qemu "arm-linux-gnueabihf"))
+             (_ (begin
+                   ;; TODO: Print warning
+                   qemu))))))))
+
 (define (qemu-platform->binfmt qemu platform)
   "Return a gexp that evaluates to a binfmt string for PLATFORM, using the
 given QEMU package."
@@ -732,12 +770,13 @@ given QEMU package."
           (bytevector->u8-list bv))))
 
   (match platform
-    (($ <qemu-platform> name family magic mask)
+    (($ <qemu-platform> name family register-width magic mask)
      ;; See 'Documentation/binfmt_misc.txt' in the kernel.
      #~(string-append ":qemu-" #$name ":M::"
                       #$(bytevector->binfmt-string magic)
                       ":" #$(bytevector->binfmt-string mask)
-                      ":" #$(file-append qemu "/bin/qemu-" name)
+                      ":" #$(file-append (closest-cross-compiled-qemu qemu register-width)
+                                         "/bin/qemu-" name)
                       ":"                         ;FLAGS go here
                       ))))
 

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]