tests/builders.scm hangs

tests/builders.scm hangs

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 4203 bytes --]

Hello,

after downloading the lastest git version (a0f16bb), "./autoreconf -vi" and 
configuring with
   ./configure --prefix=/usr/local/guix-git --with-nix-
prefix=/usr/local/nix-1.1
(by the way, would not "--with-nix" be a more standard choice than "--with-
nix-prefix"?) on Debian testing, "make" passes, but "make check" poses 
problems. Some tests pass without the nix-worker daemon running:
PASS: tests/base32.scm
FAIL: tests/builders.scm
FAIL: tests/derivations.scm
PASS: tests/utils.scm
PASS: tests/build-utils.scm
FAIL: tests/packages.scm
PASS: tests/snix.scm
PASS: tests/union.scm
FAIL: tests/guix-build.sh
FAIL: tests/guix-download.sh
FAIL: tests/guix-package.sh

When the nix-worker daemon is running, builders.scm hangs.
"ps -ef|grep nix" shows the following list of processes:
root     13947  4755  0 20:03 pts/0    00:00:00 nix-worker --daemon
privat   14031 14026  0 20:06 pts/1    00:00:00 /bin/bash -c list='' 
list2='tests/base32.log tests/builders.log tests/derivations.log 
tests/utils.log tests/build-utils.log tests/packages.log tests/snix.log 
tests/union.log tests/guix-build.log tests/guix-download.log tests/guix-
package.log'; for f in $list2; do \   test .log = $f && continue; \   if 
test check-TESTS = recheck; then \     test -f $f || continue; \     if 
test -r $f && read line < $f; then \       case $line in FAIL*|XPASS*) : ;; 
*) continue;; esac; \     fi; \   fi; \   if test -z "$list"; then list=$f; 
else list="$list $f"; fi; \ done; \ if test check-TESTS = recheck && test -
n "$list"; then \   { am__dry=no; case $MAKEFLAGS in *\\[\ \?]*) echo 'am--
echo: ; @echo "AM"  OK' | make -f - 2>/dev/null | grep '^AM OK$' >/dev/null 
|| am__dry=yes;; *) for am__flg in $MAKEFLAGS; do case $am__flg in *=*|--*) 
;; *n*) am__dry=yes; break;; esac; done;; esac; test $am__dry = yes; } || 
rm -f $list || exit 1; \ fi; \ make  test-suite.log TEST_LOGS="$list"
privat   14032 14031  0 20:06 pts/1    00:00:00 make test-suite.log 
TEST_LOGS=tests/base32.log tests/builders.log tests/derivations.log 
tests/utils.log tests/build-utils.log tests/packages.log tests/snix.log 
tests/union.log tests/guix-build.log tests/guix-download.log tests/guix-
package.log
root     14061 13947  3 20:06 ?        00:00:00 nix-worker 14060   
root     14062 14061  0 20:06 ?        00:00:00 /usr/bin/perl -w -
I/usr/local/nix-1.1/lib/perl5/site_perl/5.14.2/x86_64-linux-gnu-thread-
multi /usr/local/nix-1.1/libexec/nix/substituters/copy-from-other-stores.pl 
--query
root     14063 14061  0 20:06 ?        00:00:00 /usr/bin/perl -w -
I/usr/local/nix-1.1/lib/perl5/site_perl/5.14.2/x86_64-linux-gnu-thread-
multi /usr/local/nix-1.1/libexec/nix/substituters/download-using-
manifests.pl --query
root     14064 14061  0 20:06 ?        00:00:00 guile --no-auto-compile -L 
/nix/store/7asnahbrlirf5dwyd16w0qpwciv3kj3z-module-import 
/nix/store/q41qjkxi1h8w6qjgv6j2vcv9rf9ka4pg-make-3.82.tar.bz2-guile-builder

I tried to execute the last few ones by hand.
/usr/local/nix-1.1/libexec/nix/substituters/copy-from-other-stores.pl --
query
and
/usr/local/nix-1.1/libexec/nix/substituters/download-using-manifests.pl --
query
do not return. So maybe there is a problem with my nix installation?

guile --no-auto-compile -L /nix/store/7asnahbrlirf5dwyd16w0qpwciv3kj3z-
module-import /nix/store/q41qjkxi1h8w6qjgv6j2vcv9rf9ka4pg-
make-3.82.tar.bz2-guile-builder
results in
starting download of `#f' from 
`http://ftpmirror.gnu.org/make/make-3.82.tar.bz2'...
following redirection to 
`http://ftp.igh.cnrs.fr/pub/gnu/make/make-3.82.tar.bz2'...
ERROR: Throw to key `bad-response' with args `("EOF while reading response 
body: ~a bytes of ~a" (17295 1242186))'.
starting download of `#f' from `ftp://ftp.cs.tu-
berlin.de/pub/gnu/make/make-3.82.tar.bz2'...
ERROR: Wrong type (expecting string): #f
starting download of `#f' from 
`ftp://ftp.chg.ru/pub/gnu/make/make-3.82.tar.bz2'...
which is a very suspicious progress message; is the `#f' normal?

I can download the file 
http://ftp.igh.cnrs.fr/pub/gnu/make/make-3.82.tar.bz2
by hand without problem. The ftp access to 
ftp://ftp.chg.ru/pub/gnu/make/make-3.82.tar.bz2 seems to be blocked by my 
current isp.

Thank you very much for your help,

Andreas


[-- Attachment #2: Type: text/html, Size: 12015 bytes --]

Re: tests/builders.scm hangs

[Ludovic Courtès]

Hi Andreas,

Andreas Enge <andreas@enge.fr> skribis:

> after downloading the lastest git version (a0f16bb), "./autoreconf -vi" and 
> configuring with
>    ./configure --prefix=/usr/local/guix-git --with-nix-
> prefix=/usr/local/nix-1.1
> (by the way, would not "--with-nix" be a more standard choice than "--with-
> nix-prefix"?)

I think --with-FOO-prefix is quite common too.  But anyway, the
dependency on Nix will soon be removed, as Guix will ship the subset of
Nix it depends on (that’s currently in the ‘nix-integration’ branch, if
you’re curious ;-)).

> on Debian testing, "make" passes, but "make check" poses 
> problems. Some tests pass without the nix-worker daemon running:

All the tests that failed actually require a running nix-worker, as you
apparently figured out.

> When the nix-worker daemon is running, builders.scm hangs.
> "ps -ef|grep nix" shows the following list of processes:
> root     13947  4755  0 20:03 pts/0    00:00:00 nix-worker --daemon
> privat   14031 14026  0 20:06 pts/1    00:00:00 /bin/bash -c list='' 
> list2='tests/base32.log tests/builders.log tests/derivations.log 
> tests/utils.log tests/build-utils.log tests/packages.log tests/snix.log 
> tests/union.log tests/guix-build.log tests/guix-download.log tests/guix-
> package.log'; for f in $list2; do \   test .log = $f && continue; \   if 
> test check-TESTS = recheck; then \     test -f $f || continue; \     if 
> test -r $f && read line < $f; then \       case $line in FAIL*|XPASS*) : ;; 
> *) continue;; esac; \     fi; \   fi; \   if test -z "$list"; then list=$f; 
> else list="$list $f"; fi; \ done; \ if test check-TESTS = recheck && test -
> n "$list"; then \   { am__dry=no; case $MAKEFLAGS in *\\[\ \?]*) echo 'am--
> echo: ; @echo "AM"  OK' | make -f - 2>/dev/null | grep '^AM OK$' >/dev/null 
> || am__dry=yes;; *) for am__flg in $MAKEFLAGS; do case $am__flg in *=*|--*) 
> ;; *n*) am__dry=yes; break;; esac; done;; esac; test $am__dry = yes; } || 
> rm -f $list || exit 1; \ fi; \ make  test-suite.log TEST_LOGS="$list"
> privat   14032 14031  0 20:06 pts/1    00:00:00 make test-suite.log 
> TEST_LOGS=tests/base32.log tests/builders.log tests/derivations.log 
> tests/utils.log tests/build-utils.log tests/packages.log tests/snix.log 
> tests/union.log tests/guix-build.log tests/guix-download.log tests/guix-
> package.log
> root     14061 13947  3 20:06 ?        00:00:00 nix-worker 14060   
> root     14062 14061  0 20:06 ?        00:00:00 /usr/bin/perl -w -
> I/usr/local/nix-1.1/lib/perl5/site_perl/5.14.2/x86_64-linux-gnu-thread-
> multi /usr/local/nix-1.1/libexec/nix/substituters/copy-from-other-stores.pl 
> --query
> root     14063 14061  0 20:06 ?        00:00:00 /usr/bin/perl -w -
> I/usr/local/nix-1.1/lib/perl5/site_perl/5.14.2/x86_64-linux-gnu-thread-
> multi /usr/local/nix-1.1/libexec/nix/substituters/download-using-
> manifests.pl --query
> root     14064 14061  0 20:06 ?        00:00:00 guile --no-auto-compile -L 
> /nix/store/7asnahbrlirf5dwyd16w0qpwciv3kj3z-module-import 
> /nix/store/q41qjkxi1h8w6qjgv6j2vcv9rf9ka4pg-make-3.82.tar.bz2-guile-builder

(Note that ‘make check’ is expected to take some time the first time you
run it (5–10 mn), because it downloads and builds a few things.)

My guess is that it’s one of the substituters above that’s hanging, but
I don’t know why exactly.  Could you try running nix-worker like this:

  NIX_SUBSTITUTERS="" nix-worker --daemon &

> I tried to execute the last few ones by hand.
> /usr/local/nix-1.1/libexec/nix/substituters/copy-from-other-stores.pl --
> query
> and
> /usr/local/nix-1.1/libexec/nix/substituters/download-using-manifests.pl --
> query
> do not return. So maybe there is a problem with my nix installation?

No, it’s normal: these scripts normally interact with nix-worker over
stdin/stdout.

> guile --no-auto-compile -L /nix/store/7asnahbrlirf5dwyd16w0qpwciv3kj3z-
> module-import /nix/store/q41qjkxi1h8w6qjgv6j2vcv9rf9ka4pg-
> make-3.82.tar.bz2-guile-builder
> results in
> starting download of `#f' from 

Yes, that’s expected too: this script expects the ‘out’ environment
variable to be defined to the name of the output file.

Thanks!

Ludo’.