Re: Problems running 'check-system'

[ludo@gnu.org (Ludovic Courtès)]

Hi Chris,

Chris Marusich <cmmarusich@gmail.com> skribis:

>   make -j check-system TESTS=installed-os

[...]

> srfi/srfi-1.scm:575:27: Throw to key `srfi-34' with args `(#<condition &message [message: "could not find bootstrap binary 'guile-2.0.9.tar.xz' for system 'x86_64-linux'"] 4b62d80>)'.
> make: *** [Makefile:5066: check-system] Error 1
> [2] [env] marusich@garuda:~/guix
> $ 
>
>
> However, if I invoke 'make' first, then the "could not find bootstrap
> binary" message does not show up, and the test proceeds to be run:

That’s not surprising: ‘make’ triggers a download of guile-2.0.9.tar.xz
in gnu/packages/bootstrap.  See Makefile.am.

However, ‘check-system’ should depend on this target.  Fixed in
693f12ce2326f82020e90e58e69cf2b54808c19b.

> Issue #2: even when I run 'make' first, the test fails.  It fails with
> this message:
>
> [... some output omitted for brevity ...]
>
> starting phase `copy-bootstrap-guile'
> Backtrace:
> In ice-9/boot-9.scm:
>  160: 13 [catch #t #<catch-closure 8c5dc0> ...]
> In unknown file:
>    ?: 12 [apply-smob/1 #<catch-closure 8c5dc0>]
> In ice-9/boot-9.scm:
>   66: 11 [call-with-prompt prompt0 ...]
> In ice-9/eval.scm:
>  432: 10 [eval # #]
> In ice-9/boot-9.scm:
> 2412: 9 [save-module-excursion #<procedure 8e6840 at ice-9/boot-9.scm:4084:3 ()>]
> 4089: 8 [#<procedure 8e6840 at ice-9/boot-9.scm:4084:3 ()>]
> 1734: 7 [%start-stack load-stack #<procedure 8f6e20 at ice-9/boot-9.scm:4080:10 ()>]
> 1739: 6 [#<procedure 8f8960 ()>]
> In unknown file:
>    ?: 5 [primitive-load "/gnu/store/wgh83kqjif20wfdg56iz7bxk9d4xmlk0-guix-0.12.0-9.25a4+-guile-builder"]
> In ice-9/eval.scm:
>  387: 4 [eval # ()]
> In srfi/srfi-1.scm:
>  827: 3 [every1 #<procedure f9fa40 at /gnu/store/a42pfdz8w5qxdkp6xz8783ydywmp0p8p-module-import/guix/build/gnu-build-system.scm:649:9 (expr)> ...]
> In /gnu/store/a42pfdz8w5qxdkp6xz8783ydywmp0p8p-module-import/guix/build/gnu-build-system.scm:
>  653: 2 [#<procedure f9fa40 at /gnu/store/a42pfdz8w5qxdkp6xz8783ydywmp0p8p-module-import/guix/build/gnu-build-system.scm:649:9 (expr)> #]
> In ice-9/eval.scm:
>  432: 1 [eval # #]
> In unknown file:
>    ?: 0 [copy-file "/gnu/store/dgncc5wmw8prxq09y71hqjc6g7rxqvvb-guile-2.0.9.tar.xz" ...]
>
> ERROR: In procedure copy-file:
> ERROR: In procedure copy-file: Permission denied

What this means is that the target of ‘copy-file’ is read-only.

> Any idea why that's failing?  The file
> /gnu/store/dgncc5wmw8prxq09y71hqjc6g7rxqvvb-guile-2.0.9.tar.xz is
> readable by all users.  I tried to insert a "pk" in the file
> guix/build/gnu-build-system.scm to see what the target of the copy
> command was, but after about 24 hours (!!) of waiting for the subsequent
> build to finish (for some reason this single-line change caused many
> things to be rebuilt), I found that the build failed for a different,
> apparently unrelated reason.  Quite unfortunate.

(guix build gnu-build-system) is used by everything, which is why
modifying it triggers a rebuild of everything.

> This brings me to Issue #3: the system tests appear to be unreliable and
> impractical for day-to-day development.  Let me be clear: I really,
> really want to like these system tests.  It's fantastic that the entire
> system, its installation process, and other features can be tested
> automatically.  I was hoping to hack around with them and eventually add
> a system test for booting and installing from an ISO-9669 image.  But
> with an iteration time of over 24 hours (like above) for a single-line
> change, I don't know how helpful these tests really are for day-to-day
> development.  I feel like I must be missing some kind of trick here to
> speed up the iteration time.  Does anyone have any tips on how to speed
> it up?  Maybe I'm missing something that's obvious to everyone else.

No, I think you stumbled upon a bunch of rough edges, and I can
understand your frustration.

> Finally, it seems like the system tests are not being run automatically
> during the build of the Guix package, so I wonder if they're ever being
> run by anyone at all.  Every time I've tried to run them recently,
> they've failed, and my attempts to troubleshoot are frustrated by the
> incredibly slow iteration time.  I have yet to find a commit on which
> the system tests succeed, including the commit tagged for the v0.12.0
> release.  I was hoping to find one so I could automatically bisect to
> find out which commit introduced the I saw above, but at this rate I
> doubt I'll be able to find one soon.

The system tests are being run on Hydra at each commit:

  https://hydra.gnu.org/job/gnu/master/test.installed-os.x86_64-linux
  https://hydra.gnu.org/job/gnu/master/test.encrypted-root-os.i686-linux
  https://hydra.gnu.org/job/gnu/master/test.dicod.x86_64-linux
  …

(See build-aux/hydra/gnu-system.scm for details.)

I run some of the tests locally or check the results online from
time-to-time, but I admit that checking all of them is still mostly
manual.

However, while the “normal” system tests (mcron, dicod, nginx, openssh,
etc.) run quickly, the installation tests are expensive: they always
start by rebuilding Guix from the current source, proceed with
installation in a VM, and finally boot the resulting system.  Thus,
working with “normal” system tests is convenient, but working with
installation tests is more tiresome.

I think it’s inherent to the nature of installation tests, though if
anyone has ideas on how to make them run faster, that’d be great!

Good news: as of v0.12.0-3433-g4aabc8eaa, “installed-os” passes for me.
:-)

HTH!

Ludo’.