bug#47018: core-updates: make check fails when guix-daemon is running

bug#47018: core-updates: make check fails when guix-daemon is running

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 4679 bytes --]

Hi,

Lars-Dominik, I'm CCing you on this email because you introduced the
code discussed below, so I'm hoping you might know something about it.
If you could please take a look, I'd really appreciate it!

Starting with commit 09448c0994390697e876db235a3b773311795238, "make
check" fails when a guix-daemon is running.  (It doesn't fail when
guix-daemon is not running, since the problematic code below is only
executed when an "external store" is available.)  A guix-daemon is
usually running when one manually invokes "make check" on a system where
Guix has already been installed.

One specific test failure is test/guix-package.sh, which fails as
follows:

--8<---------------cut here---------------start------------->8---
test-name: python-build-system: python-dummy-ok
location: /home/marusich/guix-core-updates/tests/builders.scm:174
source:
+ (test-assert
+   (string-append
+     "python-build-system: "
+     (package-name p))
+   (let* ((drv (package-derivation store p)))
+     (build-derivations store (list drv))))
actual-value: #f
actual-error:
+ (out-of-range
+   #f
+   "Value out of range ~S to ~S: ~S"
+   (0 59 81)
+   (81))
result: FAIL

test-name: python-build-system: python-dummy-no-setuptools
location: /home/marusich/guix-core-updates/tests/builders.scm:174
source:
+ (test-assert
+   (string-append
+     "python-build-system: "
+     (package-name p))
+   (let* ((drv (package-derivation store p)))
+     (build-derivations store (list drv))))
actual-value: #f
actual-error:
+ (out-of-range
+   #f
+   "Value out of range ~S to ~S: ~S"
+   (0 59 81)
+   (81))
result: FAIL
--8<---------------cut here---------------end--------------->8---

The test succeeds on the previous commit.

Upon closer inspection, it seems the failure occurs when calling
package-derivation.  By putting the following code into the
tests/builders.scm file...

--8<---------------cut here---------------start------------->8---
(pk 'beforerun)
(with-external-store store
  (package-derivation store python-dummy-ok))
(pk 'afterrun)
--8<---------------cut here---------------end--------------->8---

...I was able to get this error message when running "make check
TESTS=tests/builders.scm":

--8<---------------cut here---------------start------------->8---
;;; (beforerun)
Backtrace:
In guix/packages.scm:
  1519:22 19 (thunk)
  1448:25 18 (bag->derivation #<store-connection 256.99 7fff865c5190> …)
In srfi/srfi-1.scm:
   586:29 17 (map1 (("source" #<origin "https://data.iana.org/t…>) …))
   586:29 16 (map1 (("tzcode" #<origin "https://data.iana.org/t…>) …))
   586:17 15 (map1 (("tar" #<package tar@1.34 guix/build-system…>) …))
In guix/packages.scm:
  1224:16 14 (expand-input #<store-connection 256.99 7fff865c5190> # …)
  1181:16 13 (package-derivation _ #<package tar@1.34 guix/build-sy…> …)
  1519:22 12 (thunk)
  1448:25 11 (bag->derivation #<store-connection 256.99 7fff865c5190> …)
In srfi/srfi-1.scm:
   586:17 10 (map1 (("source" #<origin "mirror://gnu/tar/tar-1.…>) …))
In guix/packages.scm:
   1243:5  9 (expand-input _ _ _ _ _)
In ice-9/boot-9.scm:
  1736:10  8 (with-exception-handler _ _ #:unwind? _ # _)
In guix/packages.scm:
  1248:18  7 (_)
In guix/store.scm:
  2062:24  6 (run-with-store #<store-connection 256.99 7fff865c5190> …)
   1896:8  5 (_ _)
In guix/gexp.scm:
   258:18  4 (_ _)
In guix/packages.scm:
   1635:7  3 (_ _)
    616:2  2 (patch-and-repack #<derivation /gnu/store/z66sd90j5ya4…> …)
In unknown file:
           1 (string-drop "/gnu/store/4y8yhyxhvh725bp3bk7wzr3jn8r5h…" …)
In ice-9/boot-9.scm:
  1669:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Value out of range 0 to 59: 74
--8<---------------cut here---------------end--------------->8---

I'm not sure exactly what the problem is.  I'm inclined to believe that
it represents some sort of test logic bug, rather than a real python
packaging bug, I'm not really sure.

I see that the problematic forms are using macros.  Specifically, the
tests/builders.scm code is attempting to simultaneously mix invocations
of lambdas, the test-assert syntax (i.e., macro), and the
with-external-store syntax (i.e., macro).  I wonder if there's a
macro-related bug lurking here...  Any ideas?

For now, the failures can be worked around by stopping guix-daemon when
running "make check."  Naturally, this bug isn't a problem when guix
builds guix, since an external store is not available in the build
environment, so the problematic code doesn't get executed there.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#47018: core-updates: make check fails when guix-daemon is running

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 244 bytes --]

Chris Marusich <cmmarusich@gmail.com> writes:

> One specific test failure is test/guix-package.sh, which fails as
> follows:

I misspoke.  The failing test is tests/builders.scm.  The rest of my
message should be correct.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#47018: core-updates: make check fails when guix-daemon is running

[Lars-Dominik Braun]

Hi Chris,

> Lars-Dominik, I'm CCing you on this email because you introduced the
> code discussed below, so I'm hoping you might know something about it.
> If you could please take a look, I'd really appreciate it!
I’m pretty sure it worked when I submitted the patch. Looking at the
untruncated backtrace and `git blame guix/packages.scm` I’d guess that the
recent changes to patch-and-repack somehow broke this. But that’s really all I
can say unfortunately. Maybe CC Maxim Cournoyer, who made that change
(cfcead2e515c0dae02127e5a76496463898be6b6)?

---snip---
Backtrace:
In guix/packages.scm:
  1522:22 19 (thunk)
  1451:25 18 (bag->derivation #<store-connection 256.99 7fa2f699daa0> #<<bag> name: "tzdata-2020d" system: "x86_64-linux" target: #f build-inputs: (("source" #<origin "https://data.iana.org/time-zones/releases/tzdata2020d.tar.gz" #<content-hash sha256:1wxskk9mh1x2073n99qna2mq58mgi648mbq5dxlqfcrnvrbkk0cd> () 7fa2f96e7360>) ("tzcode" #<origin "https://data.iana.org/time-zones/releases/tzcode2020d.tar.gz" #<content-hash sha256:1mgsckixmmk9qxzsflfxnp3999qi3ls72bgksclk01g852x51w3c> () 7fa2fee73b40>) ("tar" #<package tar@1.34 guix/build-system/gnu.scm:143 7fa2fee78a00>) ("gzip" #<package gzip@1.10 guix/build-system/gnu.scm:143 7fa2fee78820>) ("bzip2" #<package bzip2@1.0.8 guix/build-system/gnu.scm:143 7fa2fee78640>) ("xz" #<package xz@5.2.5 guix/build-system/gnu.scm:143 7fa2fee78460>) ("file" #<package file@5.39 guix/build-system/gnu.scm:143 7fa2fee78280>) ("diffutils" #<package diffutils@3.7 guix/build-system/gnu.scm:143 7fa2fee780a0>) ("patch" #<package patch@2.7.6 guix/build-system/g…> …)
In srfi/srfi-1.scm:
   586:29 17 (map1 (("source" #<origin "https://data.iana.org/time-zones/releases/tzdata2020d.tar.gz" #<content-hash sha256:1wxskk9mh1x2073n99qna2mq58mgi648mbq5dxlqfcrnvrbkk0cd> () 7fa2f96e7360>) ("tzcode" #<origin "https://data.iana.org/time-zones/releases/tzcode2020d.tar.gz" #<content-hash sha256:1mgsckixmmk9qxzsflfxnp3999qi3ls72bgksclk01g852x51w3c> () 7fa2fee73b40>) ("tar" #<package tar@1.34 guix/build-system/gnu.scm:143 7fa2fee78a00>) ("gzip" #<package gzip@1.10 guix/build-system/gnu.scm:143 7fa2fee78820>) ("bzip2" #<package bzip2@1.0.8 guix/build-system/gnu.scm:143 7fa2fee78640>) ("xz" #<package xz@5.2.5 guix/build-system/gnu.scm:143 7fa2fee78460>) ("file" #<package file@5.39 guix/build-system/gnu.scm:143 7fa2fee78280>) ("diffutils" #<package diffutils@3.7 guix/build-system/gnu.scm:143 7fa2fee780a0>) ("patch" #<package patch@2.7.6 guix/build-system/gnu.scm:143 7fa2fee7ae60>) ("findutils" #<package findutils@4.8.0 guix/build-system/gnu.scm:143 7fa2fee7ac80>) ("gawk" #<pack…>) …))
   586:29 16 (map1 (("tzcode" #<origin "https://data.iana.org/time-zones/releases/tzcode2020d.tar.gz" #<content-hash sha256:1mgsckixmmk9qxzsflfxnp3999qi3ls72bgksclk01g852x51w3c> () 7fa2fee73b40>) ("tar" #<package tar@1.34 guix/build-system/gnu.scm:143 7fa2fee78a00>) ("gzip" #<package gzip@1.10 guix/build-system/gnu.scm:143 7fa2fee78820>) ("bzip2" #<package bzip2@1.0.8 guix/build-system/gnu.scm:143 7fa2fee78640>) ("xz" #<package xz@5.2.5 guix/build-system/gnu.scm:143 7fa2fee78460>) ("file" #<package file@5.39 guix/build-system/gnu.scm:143 7fa2fee78280>) ("diffutils" #<package diffutils@3.7 guix/build-system/gnu.scm:143 7fa2fee780a0>) ("patch" #<package patch@2.7.6 guix/build-system/gnu.scm:143 7fa2fee7ae60>) ("findutils" #<package findutils@4.8.0 guix/build-system/gnu.scm:143 7fa2fee7ac80>) ("gawk" #<package gawk@5.1.0 guix/build-system/gnu.scm:143 7fa2fee7aaa0>) ("sed" #<package sed@4.8 gnu/packages/commencement.scm:3648 7fa2fee78be0>) ("grep" #<package grep@3.6 gnu/packages/c…>) …))
   586:17 15 (map1 (("tar" #<package tar@1.34 guix/build-system/gnu.scm:143 7fa2fee78a00>) ("gzip" #<package gzip@1.10 guix/build-system/gnu.scm:143 7fa2fee78820>) ("bzip2" #<package bzip2@1.0.8 guix/build-system/gnu.scm:143 7fa2fee78640>) ("xz" #<package xz@5.2.5 guix/build-system/gnu.scm:143 7fa2fee78460>) ("file" #<package file@5.39 guix/build-system/gnu.scm:143 7fa2fee78280>) ("diffutils" #<package diffutils@3.7 guix/build-system/gnu.scm:143 7fa2fee780a0>) ("patch" #<package patch@2.7.6 guix/build-system/gnu.scm:143 7fa2fee7ae60>) ("findutils" #<package findutils@4.8.0 guix/build-system/gnu.scm:143 7fa2fee7ac80>) ("gawk" #<package gawk@5.1.0 guix/build-system/gnu.scm:143 7fa2fee7aaa0>) ("sed" #<package sed@4.8 gnu/packages/commencement.scm:3648 7fa2fee78be0>) ("grep" #<package grep@3.6 gnu/packages/commencement.scm:3632 7fa2fee78e60>) ("coreutils" #<package coreutils@8.32 guix/build-system/gnu.scm:143 7fa2fee74140>) ("make" #<package make@4.3 gnu/packages/commencement.scm:…>) …))
In guix/packages.scm:
  1227:16 14 (expand-input #<store-connection 256.99 7fa2f699daa0> #<package tzdata@2020d guix/packages.scm:448 7fa2f7ccc5a0> ("tar" #<package tar@1.34 guix/build-system/gnu.scm:143 7fa2fee78a00>) "x86_64-linux" _)
  1184:16 13 (package-derivation _ #<package tar@1.34 guix/build-system/gnu.scm:143 7fa2fee78a00> _ #:graft? _)
  1522:22 12 (thunk)
  1451:25 11 (bag->derivation #<store-connection 256.99 7fa2f699daa0> #<<bag> name: "tar-1.34" system: "x86_64-linux" target: #f build-inputs: (("source" #<origin "mirror://gnu/tar/tar-1.34.tar.xz" #<content-hash sha256:0a0x87anh9chbi2cgcyy7pmnm5hzk4yd1w2j8gm1wplwhwkbvgk3> ("…/gnu/packages/patches/tar-skip-unreliable-tests.patch" "…/gnu/packages/patches/tar-remove-wholesparse-check.patch") 7fa2f7b41360>) ("coreutils" #<package coreutils@8.32 guix/build-system/gnu.scm:143 7fa2fee74140>) ("grep" #<package grep@3.6 gnu/packages/commencement.scm:3632 7fa2fee78e60>) ("locales" #<package glibc-utf8-locales@2.32 gnu/packages/commencement.scm:3573 7fa2fee74640>) ("bash" #<package bash-minimal@5.0.16 gnu/packages/commencement.scm:3541 7fa2fee74960>) ("gcc" #<package gcc@7.5.0 gnu/packages/commencement.scm:3453 7fa2fee74aa0>) ("ld-wrapper" #<package ld-wrapper-boot3@0 gnu/packages/base.scm:606 7fa2f68acc80>) (…> …)
In srfi/srfi-1.scm:
   586:17 10 (map1 (("source" #<origin "mirror://gnu/tar/tar-1.34.tar.xz" #<content-hash sha256:0a0x87anh9chbi2cgcyy7pmnm5hzk4yd1w2j8gm1wplwhwkbvgk3> ("…/gnu/packages/patches/tar-skip-unreliable-tests.patch" "…/gnu/packages/patches/tar-remove-wholesparse-check.patch") 7fa2f7b41360>) ("coreutils" #<package coreutils@8.32 guix/build-system/gnu.scm:143 7fa2fee74140>) ("grep" #<package grep@3.6 gnu/packages/commencement.scm:3632 7fa2fee78e60>) ("locales" #<package glibc-utf8-locales@2.32 gnu/packages/commencement.scm:3573 7fa2fee74640>) ("bash" #<package bash-minimal@5.0.16 gnu/packages/commencement.scm:3541 7fa2fee74960>) ("gcc" #<package gcc@7.5.0 gnu/packages/commencement.scm:3453 7fa2fee74aa0>) ("ld-wrapper" #<package ld-wrapper-boot3@0 gnu/packages/base.scm:606 7fa2f68acc80>) ("libc" #<package glibc@2.32 gnu/packages/commencement.scm:3348 7fa2fee74dc0>) ("libc:static" #<package glibc@2.32 gnu/p…> …) …))
In guix/packages.scm:
   1246:5  9 (expand-input _ _ _ _ _)
In ice-9/boot-9.scm:
  1736:10  8 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In guix/packages.scm:
  1251:18  7 (_)
In guix/store.scm:
  2066:24  6 (run-with-store #<store-connection 256.99 7fa2f699daa0> #<procedure 7fa2f69c6540 at guix/gexp.scm:246:2 (state)> #:guile-for-build _ #:system _ #:target _)
   1900:8  5 (_ _)
In guix/gexp.scm:
   256:18  4 (_ _)
In guix/packages.scm:
   1638:7  3 (_ _)
    619:2  2 (patch-and-repack #<derivation /gnu/store/l9nzv7lmznp2y22i2n3j7mccz5jjhlv1-tar-1.34.tar.xz.drv => /gnu/store/4y8yhyxhvh725bp3bk7wzr3jn8r5hjm0-tar-1.34.tar.xz 7fa2f699d280> ("…/gnu/packages/patches/tar-skip-unreliable-tests.patch" "…/gnu/packages/patches/tar-remove-wholesparse-check.patch") #:inputs _ #:snippet _ #:flags _ #:modules _ #:guile-for-build _ #:system _)
In unknown file:
           1 (string-drop "/gnu/store/4y8yhyxhvh725bp3bk7wzr3jn8r5hjm0-tar-1.34.tar.xz" 91)
In ice-9/boot-9.scm:
  1669:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Value out of range 0 to 59: 91
---snap---

Cheers,
Lars

bug#47018: core-updates: make check fails when guix-daemon is running

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1181 bytes --]

Hi Lars-Dominik and Maxim,

Lars-Dominik, thank you for the quick reply!  Maxim, do you have time to
take a look at this bug?  Lars-Dominik mentioned that it's possible that
your recent changes to patch-and-repack might be related somehow.

Lars-Dominik Braun <lars@6xq.net> writes:

> I’m pretty sure it worked when I submitted the patch. Looking at the
> untruncated backtrace and `git blame guix/packages.scm` I’d guess that the
> recent changes to patch-and-repack somehow broke this. But that’s really all I
> can say unfortunately. Maybe CC Maxim Cournoyer, who made that change
> (cfcead2e515c0dae02127e5a76496463898be6b6)?

Thank you for the tip.  I haven't checked this yet due to lack of time,
but I will eventually.

I am posting mainly to note that this problem also affects
tests/pack.scm, which appears to be the only other test (besides
tests/builders.scm) using the with-external-store form.  Both tests fail
when a guix-daemon is running.  So, I think something about the
with-external-form is not playing well with the rest of the code, but I
don't fully understand the problem yet.

More investigation is required...

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#47018: core-updates: make check fails when guix-daemon is running

[Ludovic Courtès]

Hi,

(+Cc: Maxim.)

Chris Marusich <cmmarusich@gmail.com> wrote:

> test-name: python-build-system: python-dummy-ok
> location: /home/marusich/guix-core-updates/tests/builders.scm:174
> source:
> + (test-assert
> +   (string-append
> +     "python-build-system: "
> +     (package-name p))
> +   (let* ((drv (package-derivation store p)))
> +     (build-derivations store (list drv))))
> actual-value: #f
> actual-error:
> + (out-of-range
> +   #f
> +   "Value out of range ~S to ~S: ~S"
> +   (0 59 81)
> +   (81))
> result: FAIL

Lars-Dominik Braun <lars@6xq.net> skribis:

>    586:17 10 (map1 (("source" #<origin "mirror://gnu/tar/tar-1.34.tar.xz" #<content-hash sha256:0a0x87anh9chbi2cgcyy7pmnm5hzk4yd1w2j8gm1wplwhwkbvgk3> ("…/gnu/packages/patches/tar-skip-unreliable-tests.patch" "…/gnu/packages/patches/tar-remove-wholesparse-check.patch") 7fa2f7b41360>) ("coreutils" #<package coreutils@8.32 guix/build-system/gnu.scm:143 7fa2fee74140>) ("grep" #<package grep@3.6 gnu/packages/commencement.scm:3632 7fa2fee78e60>) ("locales" #<package glibc-utf8-locales@2.32 gnu/packages/commencement.scm:3573 7fa2fee74640>) ("bash" #<package bash-minimal@5.0.16 gnu/packages/commencement.scm:3541 7fa2fee74960>) ("gcc" #<package gcc@7.5.0 gnu/packages/commencement.scm:3453 7fa2fee74aa0>) ("ld-wrapper" #<package ld-wrapper-boot3@0 gnu/packages/base.scm:606 7fa2f68acc80>) ("libc" #<package glibc@2.32 gnu/packages/commencement.scm:3348 7fa2fee74dc0>) ("libc:static" #<package glibc@2.32 gnu/p…> …) …))
> In guix/packages.scm:
>    1246:5  9 (expand-input _ _ _ _ _)
> In ice-9/boot-9.scm:
>   1736:10  8 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
> In guix/packages.scm:
>   1251:18  7 (_)
> In guix/store.scm:
>   2066:24  6 (run-with-store #<store-connection 256.99 7fa2f699daa0> #<procedure 7fa2f69c6540 at guix/gexp.scm:246:2 (state)> #:guile-for-build _ #:system _ #:target _)
>    1900:8  5 (_ _)
> In guix/gexp.scm:
>    256:18  4 (_ _)
> In guix/packages.scm:
>    1638:7  3 (_ _)
>     619:2  2 (patch-and-repack #<derivation /gnu/store/l9nzv7lmznp2y22i2n3j7mccz5jjhlv1-tar-1.34.tar.xz.drv => /gnu/store/4y8yhyxhvh725bp3bk7wzr3jn8r5hjm0-tar-1.34.tar.xz 7fa2f699d280> ("…/gnu/packages/patches/tar-skip-unreliable-tests.patch" "…/gnu/packages/patches/tar-remove-wholesparse-check.patch") #:inputs _ #:snippet _ #:flags _ #:modules _ #:guile-for-build _ #:system _)
> In unknown file:
>            1 (string-drop "/gnu/store/4y8yhyxhvh725bp3bk7wzr3jn8r5hjm0-tar-1.34.tar.xz" 91)
> In ice-9/boot-9.scm:
>   1669:16  0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> Value out of range 0 to 59: 91

Maxim, does that ring a bell?

I think this has to do with ‘%store-prefix’, ‘%storedir’,
etc. shenanigans and their impact on the python-build-system tests you
added in ‘core-updates’ a while back.

Thanks,
Ludo’.

bug#47018: core-updates: make check fails when guix-daemon is running

[Ludovic Courtès]

Hi,

Lars-Dominik Braun <lars@6xq.net> skribis:

> In guix/packages.scm:
>    1638:7  3 (_ _)
>     619:2  2 (patch-and-repack #<derivation /gnu/store/l9nzv7lmznp2y22i2n3j7mccz5jjhlv1-tar-1.34.tar.xz.drv => /gnu/store/4y8yhyxhvh725bp3bk7wzr3jn8r5hjm0-tar-1.34.tar.xz 7fa2f699d280> ("…/gnu/packages/patches/tar-skip-unreliable-tests.patch" "…/gnu/packages/patches/tar-remove-wholesparse-check.patch") #:inputs _ #:snippet _ #:flags _ #:modules _ #:guile-for-build _ #:system _)
> In unknown file:
>            1 (string-drop "/gnu/store/4y8yhyxhvh725bp3bk7wzr3jn8r5hjm0-tar-1.34.tar.xz" 91)
> In ice-9/boot-9.scm:
>   1669:16  0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> Value out of range 0 to 59: 91

Fixed in 99ea6a2996a388134c6ea8fdce34f70d446b1450!

Ludo’.