bug#44370: Cuirass fails building (some) packages from a custom-packages input

bug#44370: Cuirass fails building (some) packages from a custom-packages input

[david larsson]

Hi!

I am unable to build certain packages from a custom-packages input in my 
cuirass-specs below, and am triggering the below backtrace with below 
config. The packages that trigger the errors can successfully be 
installed otherwise by pointing to them with a guix channel. Failure 
happens when adding python-pydotplus, perl-ical-data packages etc from 
the custom-packages input to the subset list in #:proc-args.

The config is instantiated (after guix pull to 
commit=594b2a116ea4267d88a294dd05f8dbbb8ce7bcc0 but Im having the same 
error on current master too) by running: herd stop cuirass-web ; herd 
stop cuirass ; rm /var/lib/cuirass/cuirass.db* ; guix system reconfigure 
/home/user1/src/my-guixsd-config.sh/VM-configs/bare-bones2.scm ;  herd 
start cuirass ; sleep 1 ; herd restart cuirass-web


================================
BACKTRACE
================================

Backtrace:
In ice-9/boot-9.scm:
   1736:10 15 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
           14 (apply-smob/0 #<thunk 7f168d1195c0>)
In ice-9/boot-9.scm:
...skipping...
Uncaught exception in fiber ##f:
In cuirass/base.scm:
    793:13  3 (_)
In ice-9/boot-9.scm:
   1736:10  2 (with-exception-handler _ _ #:unwind? _ # _)
   1669:16  1 (raise-exception _ #:continuable? _)
   1667:16  0 (raise-exception _ #:continuable? _)
ice-9/boot-9.scm:1667:16: In procedure raise-exception:
ERROR:
   1. &evaluation-error:
       name: "my-pkgs"
       id:


================================
CONFIG:
================================


(use-modules (gnu) (gnu system nss))
;; additions: spice, cuirass, virtualization (qemu-binfmt-service-type)
(use-service-modules networking ssh desktop spice cuirass 
virtualization)
;; additions: bash
(use-package-modules screen ssh gnome certs bash)

(define %my-special-files
   `(("/bin/sh" ,(file-append bash "/bin/sh"))
     ("/bin/bash" ,(file-append bash "/bin/bash"))
     ;; gitolite update hook needs it
     ;;    ("/usr/bin/perl" ,(file-append perl "/bin/perl"))
     ("/usr/bin/env" ,(file-append coreutils "/bin/env"))))

(define %cuirass-specs
   #~(
      list
   '((#:name . "my-pkgs")
     (#:load-path-inputs . ("guix"))
     (#:package-path-inputs . ("my-guix-packages"))
     (#:proc-input . "guix")
     (#:proc-file . "build-aux/cuirass/gnu-system.scm")
     (#:proc . cuirass-jobs)
     (#:proc-args .
      (;;(subset . manifests)
       ;;(subset . ("hello" "cowsay" "orgmk" "fwknop" "python-on-guile" 
"libxml2-xpath0" "guile-bash-parallel" "guile-base64")) ;; works!
       (subset . ("hello" "cowsay" "orgmk" "fwknop" "python-on-guile" 
"libxml2-xpath0" "guile-bash-parallel" "guile-base64" 
"python-pydotplus")) ;; fails with stacktrace
       (systems . ("x86_64-linux"))
       ;;(manifests . (("my-guix-packages" . "manifest.scm")))
       ))
     (#:inputs . (
                  ((#:name . "guix")
                   (#:url . "git://git.savannah.gnu.org/guix.git")
                   (#:load-path . ".")
                   ;;(#:branch . "master")

                   ;; d7 commit reports failed builds in 
/var/log/cuirass.log and cuirass-web reports them as scheduled, even 
when outputs in the store are successfully built
                   ;;(#:commit . 
"d7e033b9a153a9e60f52ff64f4eb355c1c3d0a6e")

                   ;; 594b commit works - success - until adding say 
perl-ical-data, python-pydotplus or bash-coding-utils.sh package from 
my-guix-packages
                   (#:commit . 
"594b2a116ea4267d88a294dd05f8dbbb8ce7bcc0")
                   (#:no-compile? . #t))

                  ((#:name . "my-guix-packages")
                   (#:url . 
"https://github.com/methuselah-0/my-guix-packages.git")
                   (#:load-path . "packages")
                   (#:branch . "master")
                   (#:no-compile? . #t))

                  ))
     (#:build-outputs . ()))

))

(operating-system
   (host-name "librem13v3guixsd")
   (timezone "Europe/Stockholm")
   (locale "en_US.utf8")

   ;; Boot in "legacy" BIOS mode, assuming /dev/sdX is the
   ;; target hard disk, and "my-root" is the label of the target
   ;; root file system.
   (bootloader (bootloader-configuration
                 (bootloader grub-bootloader)
                 (target "/dev/vda")))
   (file-systems (cons (file-system
                         (device (file-system-label "fsroot"))
                         (mount-point "/")
                         (type "ext4"))
                       %base-file-systems))

   ;; This is where user accounts are specified.  The "root"
   ;; account is implicit, and is initially created with the
   ;; empty password.
   (users (cons* (user-account
                  (name "user1")
                  (group "users")

                  ;; Adding the account to the "wheel" group
                  ;; makes it a sudoer.  Adding it to "audio"
                  ;; and "video" allows the user to play sound
                  ;; and access the webcam.
                  (supplementary-groups '("wheel"
                                          "audio" "video"))
                  (home-directory "/home/user1"))

                 (user-account
                  (name "cuirass")
                  (group "cuirass")
                  (supplementary-groups '("netdev"
                                          "audio" "video"))
                  (home-directory "/home/cuirass"))

                %base-user-accounts))

   ;; Globally-installed packages.
   ;; added nss-certs and le-certs globally for e.g. cuirass to be able 
to fetch stuff
   (packages (cons* screen openssh nss-certs le-certs %base-packages))

   ;; Add services to the baseline: a DHCP client and
   ;; an SSH server.
   (services (cons* (service gnome-desktop-service-type)
		   (service xfce-desktop-service-type)
                    (spice-vdagent-service)
                     ;; 
https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html
                     (service qemu-binfmt-service-type
                              (qemu-binfmt-configuration
                               (platforms (lookup-qemu-platforms "arm" 
"aarch64"))
                               (guix-support? #t)))

                     (service openssh-service-type
                              (openssh-configuration
                               (permit-root-login #t)
                               ;;(allow-empty-passwords? #t)
                               (x11-forwarding? #t)
                               (port-number 2223)))
                     ;; 
https://lists.gnu.org/archive/html/help-guix/2017-08/msg00095.html
                     (simple-service 'store-my-config
                                     etc-service-type
                                     `(("config.scm"
                                        ,(local-file (assoc-ref
                                                      
(current-source-location)
                                                      'filename)))))
		    (service special-files-service-type %my-special-files)
                     ;; 
https://lists.gnu.org/archive/html/help-guix/2019-06/msg00116.html
                     (service cuirass-service-type
                              (cuirass-configuration
                               (interval 3600)
                               (fallback? #f) ;; default is #f
                               (host "0.0.0.0")
                               (port 8082)
                               (database "/var/lib/cuirass/cuirass.db") 
;; default is /var/lib/cuirass/cuirass.db
                               (use-substitutes? #t)
                               (specifications %cuirass-specs)))
                     %desktop-services))
   ;; Allow resolution of '.local' host names with mDNS.
   (name-service-switch %mdns-host-lookup-nss))


Best regards,
David

bug#44370: Cuirass fails building (some) packages from a custom-packages input

[Mathieu Othacehe]

Hello David,

> ice-9/boot-9.scm:1667:16: In procedure raise-exception:
> ERROR:
>   1. &evaluation-error:
>       name: "my-pkgs"
>       id:

Thanks for the bug report. Could you please send the associated log
trace in /var/log/cuirass/evaluations/xxx.gz?

Mathieu

bug#44370: Cuirass fails building (some) packages from a custom-packages input

[david larsson]

On 2020-11-01 19:53, Mathieu Othacehe wrote:
> Hello David,
> 
>> ice-9/boot-9.scm:1667:16: In procedure raise-exception:
>> ERROR:
>>   1. &evaluation-error:
>>       name: "my-pkgs"
>>       id:
> 
> Thanks for the bug report. Could you please send the associated log
> trace in /var/log/cuirass/evaluations/xxx.gz?
> 
> Mathieu

Yes, it looks like this:

---------------------------

root@librem13v3guixsd /var/log/cuirass/evaluations# gzip --decompress -c 
1.gz
error: specifications->manifest: unbound variable
hint: Did you forget `(use-modules (gnu packages))'?

warning:
building things during evaluation
'build-things' arguments: 
(("/gnu/store/3pynx6589v9a6j6fz0svsps1wj22bnll-compute-guix-derivation.drv") 
0)
Computing Guix derivation for 'x86_64-linux'...
warning:
building things during evaluation
'build-things' arguments: 
(("/gnu/store/pjj934izli1bvz3mymzvfa3iwg2crdmx-profile.drv") 0)
Backtrace:
In ice-9/boot-9.scm:
   1736:10  8 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
            7 (apply-smob/0 #<thunk 7f1b50d125c0>)
In ice-9/boot-9.scm:
     718:2  6 (call-with-prompt _ _ #<procedure default-prompt-handle?>)
In ice-9/eval.scm:
     619:8  5 (_ #(#(#<directory (guile-user) 7f1b50934f00>)))
    293:34  4 (_ #(#(#(#(#(#(#(#(#(#(#(?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
     159:9  3 (_ _)
     159:9  2 (_ #(#(#(#(#<module (#{ g19}#) 7f1b50934460>) # #) ?) #))
In guix/utils.scm:
     705:8  1 (call-with-temporary-directory #<procedure 7f1b47905ac0?>)
In guix/inferior.scm:
     247:2  0 (_ _)

guix/inferior.scm:247:2: ERROR:
   1. &inferior-exception:
       arguments: (quit 1)
       inferior: #<<inferior> pid: pipe socket: #<input-output: string 
7f1b3d1dabd0> close: #<procedure close-pipe (p)> version: (0 1 1) 
packages: #<promise #<procedure 7f1b3d1de940 at guix/inferior.scm:162:32 
()>> table: #<promise #<procedure 7f1b46b3a780 at 
guix/inferior.scm:163:32 ()>>>
       stack: ((#f ("ice-9/boot-9.scm" 1763 13)) (raise-exception 
("ice-9/boot-9.scm" 1666 16)) (raise-exception ("ice-9/boot-9.scm" 1668 
16)) (map1 ("srfi/srfi-1.scm" 585 17)) (map1 ("srfi/srfi-1.scm" 585 29)) 
(map1 ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) 
(map1 ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) 
(map1 ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) 
(map1 ("srfi/srfi-1.scm" 585 29)) (#f ("gnu/ci.scm" 552 38)) (map1 
("srfi/srfi-1.scm" 585 17)) (append-map ("srfi/srfi-1.scm" 672 15)) 
(hydra-jobs ("gnu/ci.scm" 516 4)) (#f ("ice-9/eval.scm" 158 9)) (#f 
("ice-9/eval.scm" 158 9)) (with-exception-handler ("ice-9/boot-9.scm" 
1735 10)) (call-with-prompt ("ice-9/boot-9.scm" 717 2)) (dynamic-wind 
("ice-9/boot-9.scm" 141 2)) (#f (#f #f #f)) (#f ("guix/repl.scm" 92 21)) 
(with-exception-handler ("ice-9/boot-9.scm" 1735 10)) 
(with-exception-handler ("ice-9/boot-9.scm" 1730 15)) (#f 
("guix/repl.scm" 119 7)))

---------------------------

Best regards,
David

bug#44370: Cuirass fails building (some) packages from a custom-packages input

[Mathieu Othacehe]

Hello David,

> error: specifications->manifest: unbound variable

This error is caused by the evaluation of "packages/manifest.scm" that
misses some includes.

> guix/inferior.scm:247:2: ERROR:
>   1. &inferior-exception:
>       arguments: (quit 1)
>       inferior: #<<inferior> pid: pipe socket: #<input-output: string
>       7f1b3d1dabd0> close: #<procedure close-pipe (p)> version: (0 1 1) 
> packages: #<promise #<procedure 7f1b3d1de940 at guix/inferior.scm:162:32 ()>>
> table: #<promise #<procedure 7f1b46b3a780 at guix/inferior.scm:163:32 ()>>>
>       stack: ((#f ("ice-9/boot-9.scm" 1763 13)) (raise-exception
>       ("ice-9/boot-9.scm" 1666 16)) (raise-exception ("ice-9/boot-9.scm" 1668 
> 16)) (map1 ("srfi/srfi-1.scm" 585 17)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
> ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
> ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
> ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
> ("srfi/srfi-1.scm" 585 29)) (#f ("gnu/ci.scm" 552 38)) (map1
> ("srfi/srfi-1.scm" 585 17)) (append-map ("srfi/srfi-1.scm" 672 15)) 
> (hydra-jobs ("gnu/ci.scm" 516 4)) (#f ("ice-9/eval.scm" 158 9)) (#f
> ("ice-9/eval.scm" 158 9)) (with-exception-handler ("ice-9/boot-9.scm" 
> 1735 10)) (call-with-prompt ("ice-9/boot-9.scm" 717 2)) (dynamic-wind
> ("ice-9/boot-9.scm" 141 2)) (#f (#f #f #f)) (#f ("guix/repl.scm" 92 21)) 
> (with-exception-handler ("ice-9/boot-9.scm" 1735 10)) (with-exception-handler
> ("ice-9/boot-9.scm" 1730 15)) (#f ("guix/repl.scm" 119 7)))

That one is caused by packages with "#f" licenses. I've pushed
7c02eb6c8d7d050ecb8f3082c4a754ad32211403 to ignore package licenses.

Thanks,

Mathieu

bug#44370: Cuirass fails building (some) packages from a custom-packages input

[Ludovic Courtès]

Hi Mathieu,

Seems to me that 7c02eb6c8d7d050ecb8f3082c4a754ad32211403 +
44daec7fa4ae97a3c4490adf461322ec9f7a780a are not quite correct since
they can let lists through (for packages where ‘license’ is a list of
<license> objects).

Now, in Guix proper, the ‘license’ field should always be either a
<license> or a list of <license> records.  The original code would
“enforce” that (by failing hard if a package doesn’t follow the rule
:-)), which I think is good.

Now, why is David’s use case influenced by the way (gnu ci) handles
licenses?

Thanks,
Ludo’.

bug#44370: Cuirass fails building (some) packages from a custom-packages input

[Mathieu Othacehe]

Hey Ludo,

> Now, in Guix proper, the ‘license’ field should always be either a
> <license> or a list of <license> records.  The original code would
> “enforce” that (by failing hard if a package doesn’t follow the rule
> :-)), which I think is good.

You are right, missed that.

> Now, why is David’s use case influenced by the way (gnu ci) handles
> licenses?

David has some custom packages with the license set to "#f", causing the
following backtrace:

--8<---------------cut here---------------start------------->8---
guix/inferior.scm:247:2: ERROR:
  1. &inferior-exception:
      arguments: (quit 1)
      inferior: #<<inferior> pid: pipe socket: #<input-output: string
      7f1b3d1dabd0> close: #<procedure close-pipe (p)> version: (0 1 1) 
packages: #<promise #<procedure 7f1b3d1de940 at guix/inferior.scm:162:32 ()>>
table: #<promise #<procedure 7f1b46b3a780 at guix/inferior.scm:163:32 ()>>>
      stack: ((#f ("ice-9/boot-9.scm" 1763 13)) (raise-exception
      ("ice-9/boot-9.scm" 1666 16)) (raise-exception ("ice-9/boot-9.scm" 1668 
16)) (map1 ("srfi/srfi-1.scm" 585 17)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
("srfi/srfi-1.scm" 585 29)) (#f ("gnu/ci.scm" 552 38)) (map1
("srfi/srfi-1.scm" 585 17)) (append-map ("srfi/srfi-1.scm" 672 15)) 
(hydra-jobs ("gnu/ci.scm" 516 4)) (#f ("ice-9/eval.scm" 158 9)) (#f
("ice-9/eval.scm" 158 9)) (with-exception-handler ("ice-9/boot-9.scm" 
1735 10)) (call-with-prompt ("ice-9/boot-9.scm" 717 2)) (dynamic-wind
("ice-9/boot-9.scm" 141 2)) (#f (#f #f #f)) (#f ("guix/repl.scm" 92 21)) 
(with-exception-handler ("ice-9/boot-9.scm" 1735 10)) (with-exception-handler
("ice-9/boot-9.scm" 1730 15)) (#f ("guix/repl.scm" 119 7)))
--8<---------------cut here---------------end--------------->8---

As the alist fields such as "description", "long-description",
"license", "home-page", "maintainers" are ignored by Cuirass anyway,
what would you think of removing them?

This should allow custom packages to be built by Cuirass even if they
are not fully defined.

Thanks,

Mathieu

bug#44370: Cuirass fails building (some) packages from a custom-packages input

[Ludovic Courtès]

Hi Mathieu,

Mathieu Othacehe <othacehe@gnu.org> skribis:

>> Now, why is David’s use case influenced by the way (gnu ci) handles
>> licenses?
>
> David has some custom packages with the license set to "#f", causing the
> following backtrace:
>
> guix/inferior.scm:247:2: ERROR:
>   1. &inferior-exception:
>       arguments: (quit 1)
>       inferior: #<<inferior> pid: pipe socket: #<input-output: string
>       7f1b3d1dabd0> close: #<procedure close-pipe (p)> version: (0 1 1) 
> packages: #<promise #<procedure 7f1b3d1de940 at guix/inferior.scm:162:32 ()>>
> table: #<promise #<procedure 7f1b46b3a780 at guix/inferior.scm:163:32 ()>>>
>       stack: ((#f ("ice-9/boot-9.scm" 1763 13)) (raise-exception
>       ("ice-9/boot-9.scm" 1666 16)) (raise-exception ("ice-9/boot-9.scm" 1668 
> 16)) (map1 ("srfi/srfi-1.scm" 585 17)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
> ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
> ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
> ("srfi/srfi-1.scm" 585 29)) (map1 ("srfi/srfi-1.scm" 585 29)) (map1
> ("srfi/srfi-1.scm" 585 29)) (#f ("gnu/ci.scm" 552 38)) (map1
> ("srfi/srfi-1.scm" 585 17)) (append-map ("srfi/srfi-1.scm" 672 15)) 
> (hydra-jobs ("gnu/ci.scm" 516 4)) (#f ("ice-9/eval.scm" 158 9)) (#f
> ("ice-9/eval.scm" 158 9)) (with-exception-handler ("ice-9/boot-9.scm" 
> 1735 10)) (call-with-prompt ("ice-9/boot-9.scm" 717 2)) (dynamic-wind
> ("ice-9/boot-9.scm" 141 2)) (#f (#f #f #f)) (#f ("guix/repl.scm" 92 21)) 
> (with-exception-handler ("ice-9/boot-9.scm" 1735 10)) (with-exception-handler
> ("ice-9/boot-9.scm" 1730 15)) (#f ("guix/repl.scm" 119 7)))
>
> As the alist fields such as "description", "long-description",
> "license", "home-page", "maintainers" are ignored by Cuirass anyway,
> what would you think of removing them?
>
> This should allow custom packages to be built by Cuirass even if they
> are not fully defined.

Yes, I agree it would be nice to allow for that, but I think (gnu ci) is
maybe not the right place.  As I see it, (gnu ci) is meant to be used
for the packages that live in (gnu packages …).

Perhaps we could provide a more generic (guix ci jobs) module that would
convert packages and manifests to the alists that Cuirass expects,
without imposing much policy?

In fact, thinking about it, it might make sense for Cuirass to interpret
its inputs as channels so that it can do the directly build channels
similar to what ‘gnu-system.scm’ currently does.

Thanks,
Ludo’.