bug#40650: guix test suite failures building Debian package

[Vagrant Cascadian <vagrant@debian.org>]

[-- Attachment #1.1: Type: text/plain, Size: 6716 bytes --]

With a few patches/workarounds applied, we got better results overall:

  # TOTAL: 1011
  # PASS:  954 (was 950)
  # SKIP:  51
  # XFAIL: 2
  # FAIL:  4 (was 8)

FAIL: tests/gexp
FAIL: tests/size
FAIL: tests/syscalls
FAIL: tests/guix-build-branch

This time tests/build-utils, tests/channels and
tests/git all passed!

Seems tests/gexp and tests/size are new failures? Maybe revealed by
fixing other issues.

Not sure how that math works out... went from 8 failures to 4, but I see
3 fixed and two added?

More details below and full test-suite.log attached for more meaningful
analysis.


On 2020-04-17, Ludovic Courtès wrote:
> Vagrant Cascadian <vagrant@debian.org> skribis:
>> On 2020-04-16, Ludovic Courtès wrote:
>>> Vagrant Cascadian <vagrant@debian.org> skribis:
>>>> Any of the test suites that require networking will need to be disabled
>>>> for Debian packages.
>>>
>>> That should be fine.
>>
>> Well, they need to be disabled even if networking is available...
>
> Would it be an option for you to run:
>
>   unshare -Un make check
>
> or similar?

Not sure; will look into it.


> If not we would need to identify all the places that check for
> networking and replace them with a constant.  For tests/*.scm, you can
> change ‘network-reachable?’ to return #f.  For tests/*.sh, you have to
> grep/sed for ‘getaddrinfo’.

This certainly sounds more explicit... :)



>>> Here’s it’s ‘search-bootstrap-binary’ from (guix tests) that tries to
>>> create the file above, having downloaded it earlier.
>>>
>>> For the ‘guix’ package in Guix, what we do is that this and related
>>> files are inputs to the package; that way, they are not downloaded at
>>> all upon ‘make check’ since they’re already there.
>>>
>>> All you need is to ensure that
>>> gnu/packages/bootstrap/*-linux/{bash,mkdir,tar,xz} exist, are
>>> executable, and are statically-linked.  You could provide Debian
>>> binaries if that’s more appropriate.
>>
>> Regarding providing Debian binaries, I could Build-Depend (the debian
>> equivalent of "inputs") on bash-static and symlink to that, and maybe
>> use busybox-static and symlinks for mkdir and tar, but I don't think
>> there's a static implementation of xz in Debian.
>>
>> Do they strictly need to be statically linked? These are just for test
>> suites, not actually used in the eventual packaged guix?
>
> Actually no, it would also work with dynamically-linked binaries since
> we’re not doing chroot builds.
>
> So you could copy (not symlink) binaries from Debian, even simply those
> that happen to be in /bin.
>
>> I notice it's also checking for i686-linux/bash above even though the
>> build was done on amd64/x86_64 ... and it would be non-trivial to enable
>> cross-architecture checks for all architectures across all of Debian's
>> architectures.
>
> It’s simply that we use i686 binaries on x86_64.

So, copying the system /bin/bash, /bin/mkdir, /bin/tar and /usr/bin/xz
into gnu/packages/bootstrap/i686-linux/ seems to have passed the test.


> At any rate, you’ll restrict the package to the subset of architectures
> that Guix supports, right?

In Debian the norm is to assume it *might* build on any architecture,
and only restrict it if it's impossible for it to ever build for some
theoretical reason, or some other practical reason. That it doesn't yet
build on a particular architecture is generally fine.

So my intention was to let it try to build on any architecture...
Consider it outreach to the Debian porter community!


>>> We may need to provide a dummy .gitconfig file or something here so we
>>> can run these tests.  See also <https://issues.guix.gnu.org/issue/37679>.
>>> (The same applies to several test failures.)
>>>
>>> Thoughts?
>>
>> I could try a couple things ... basically re-implement the patch from
>> #37679 in Debian's packaging (e.g. HOME=/some/path and populate
>> $HOME/.gitconfig), or apply the patch and try it. :)
>
> Yeah, let’s do something about it.  :-)

I tried the patch in the mentioned bug, and it seems to have worked!


>>>> test-name: scandir*, properties
>>>> location: /build/guix-jgTHmh/guix-1.1.0/tests/syscalls.scm:257
>>>> source:
>>>> + (test-assert
>>>> +   "scandir*, properties"
>>>> +   (let ((directory
>>>> +           (dirname
>>>> +             (search-path %load-path "guix/base32.scm"))))
>>>> +     (every (lambda (entry name)
>>>> +              (match entry
>>>> +                     ((name2 . properties)
>>>> +                      (and (string=? name2 name)
>>>> +                           (let* ((full (string-append directory "/" name))
>>>> +                                  (stat (lstat full))
>>>> +                                  (inode (assoc-ref properties 'inode))
>>>> +                                  (type (assoc-ref properties 'type)))
>>>> +                             (and (= inode (stat:ino stat))
>>>> +                                  (or (eq? type 'unknown)
>>>> +                                      (eq? type (stat:type stat)))))))))
>>>> +            (scandir* directory)
>>>> +            (scandir directory (const #t) string<?))))
>>>> actual-value: #f
>>>> result: FAIL
>>>
>>> Could you wrap the ‘scandir*’ and ‘scandir’ calls in ‘pk’, run:
>>
>> like this?
>>                 (pk (scandir* directory))
>>                 (pk (scandir directory (const #t) string<?)))))
>
> Yes.

Patched and test-suite.log attached.


>>>> ++ guix build guile-gcrypt --with-branch=guile-gcrypt=master -d
>>>> accepted connection from pid 17231, user vagrant
>>>> updating checkout of 'https://notabug.org/cwebber/guile-gcrypt.git'...
>>>> guix build: error: cannot fetch branch 'master' from https://notabug.org/cwebber/guile-gcrypt.git: the SSL certificate is invalid: 0x08 - The certificate is not correctly signed by the trusted CA
>>>> + latest_drv=
>>>> FAIL tests/guix-build-branch.sh (exit status: 1)
>>>
>>> This test relies on network access + HTTPS certificates.  It does check
>>> for the former but not the latter.  Any suggestions on how to do that?
>>
>> This might be a candidate for "autopackagetests" described earlier.
>>
>> Alternately/Additionally, ship or create a git repository from the
>> source tree, run git daemon on a random free port, and try it there.
>
> Uh, that would be better… I’d gladly accept patches.  :-)

We'll see...


> Thanks for your feedback!  Hopefully we can easily address the
> remaining issues.

Yeah, it was nice to see fixes already on the second attempt!


live well,
  vagrant


[-- Attachment #1.2: test-suite.log --]
[-- Type: application/octet-stream, Size: 494597 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]