Configuration for Cuirass

Configuration for Cuirass

[divoplade]

Hello,

I would like to use the cuirass service, but I cannot seem to make it
work. It either fails with an exception, or it builds nothing.

I would like to build all the packages from my repository (
https://framagit.org/divoplade/pomdappi-ci) and nothing from guix git.

Here is a specification that builds nothing but that lets cuirass run:

(define %cuirass-specs
  #~(list
     '((#:name . "pomdappi-master")
       (#:load-path-inputs . ())
       (#:package-path-inputs . ("."))
       (#:proc-input . "pomdappi-packages")
       (#:proc . cuirass-jobs)
       (#:proc-args . ())
       (#:inputs . (((#:name . "pomdappi-packages")
                     (#:url . "
https://framagit.org/divoplade/pomdappi-ci.git")
                     (#:load-path . ".")
                     (#:branch . "master")
                     (#:no-compile? . #t))))
       (#:build-outputs . ()))))

How should I do it?

Best regards,

divoplade

Re: Configuration for Cuirass

[Mathieu Othacehe]

Hello divoplade,

> How should I do it?

Writing the appropriate specification is quite tricky and I plan to
write a shepherd service to make it easier. In the meantime, something
like that should get you closer:

--8<---------------cut here---------------start------------->8---
(define my-packages
  '((#:name . "my-packages")
    (#:load-path-inputs . ("guix"))
    (#:package-path-inputs . ())
    (#:proc-input . "guix")
    (#:proc-file . "build-aux/cuirass/gnu-system.scm")
    (#:proc . cuirass-jobs)
    (#:proc-args (subset "pomdappi")) ;name of the packages you want to build
    (#:inputs . (((#:name . "pomdappi")
                  (#:url . "https://framagit.org/divoplade/pomdappi-ci.git")
                  (#:load-path . ".")
                  (#:branch . "master")
                  (#:no-compile? . #t))
                 ((#:name . "guix")
                  (#:url . "git://git.savannah.gnu.org/guix.git")
                  (#:load-path . ".")
                  (#:branch . "master")
                  (#:no-compile? . #t))))
    (#:build-outputs . ())))
(list my-packages)
--8<---------------cut here---------------end--------------->8---

If you want to use a fixed guix input, you can use the #:commit key.

Thanks,

Mathieu

Re: Configuration for Cuirass

[divoplade]

Hello Mathieu,

Le jeudi 15 octobre 2020 à 10:40 +0200, Mathieu Othacehe a écrit :
> --8<---------------cut here---------------start------------->8---
> (define my-packages
>   '((#:name . "my-packages")
>     (#:load-path-inputs . ("guix"))
>     (#:package-path-inputs . ())
>     (#:proc-input . "guix")
>     (#:proc-file . "build-aux/cuirass/gnu-system.scm")
>     (#:proc . cuirass-jobs)
>     (#:proc-args (subset "pomdappi")) ;name of the packages you want
> to build
>     (#:inputs . (((#:name . "pomdappi")
>                   (#:url . "
> https://framagit.org/divoplade/pomdappi-ci.git")
>                   (#:load-path . ".")
>                   (#:branch . "master")
>                   (#:no-compile? . #t))
>                  ((#:name . "guix")
>                   (#:url . "git://git.savannah.gnu.org/guix.git")
>                   (#:load-path . ".")
>                   (#:branch . "master")
>                   (#:no-compile? . #t))))
>     (#:build-outputs . ())))
> (list my-packages)
> --8<---------------cut here---------------end--------------->8---

Thank you for your help.

I use this, but I still have no builds (https://ci.divoplade.fr/)

My log file alternates between "building 0 derivations" and "failed to
connect to git.savannah.gnu.org: Connection timed out".

2020-10-15T11:08:12 next evaluation in 60 seconds
2020-10-15T11:09:12 fetching input 'pomdappi' of spec 'my-packages'
2020-10-15T11:09:12 fetching input 'guix' of spec 'my-packages'
WARNING: (guile-user): imported module (fibers) overrides core binding
`sleep'
2020-10-15T11:10:14 performing database optimizations
2020-10-15T11:10:14 running Fibers on 4 kernel threads
WARNING: (cuirass watchdog): imported module (fibers) overrides core
binding `sleep'
2020-10-15T11:10:14 marking stale builds as "scheduled"...
2020-10-15T11:10:14 retrieving list of pending builds...
2020-10-15T11:10:14 Updating metric builds-per-day (1602633600) to 0.
2020-10-15T11:10:14 Updating metric new-derivations-per-day
(1602633600) to 0.
2020-10-15T11:10:14 heap: 11.93 MiB; threads: 16; file descriptors: 65
2020-10-15T11:10:14 canceling 0 stale builds
2020-10-15T11:10:14 restarting 0 pending builds
2020-10-15T11:10:14 Updating metric pending-builds (1602720000) to 0.
2020-10-15T11:10:14 Updating metric pending-builds (1602720000) to 0.
2020-10-15T11:10:14 building 0 derivations in batches of 200
2020-10-15T11:10:14 done with 0 derivations
2020-10-15T11:10:14 done with restarted builds
2020-10-15T11:10:14 Failed to compute metric average-10-last-eval-
duration-per-spec (my-packages).
2020-10-15T11:10:14 Failed to compute metric average-100-last-eval-
duration-per-spec (my-packages).
2020-10-15T11:10:14 Failed to compute metric average-eval-duration-per-
spec (my-packages).
2020-10-15T11:10:14 Failed to compute metric percentage-failure-10-
last-eval-per-spec (my-packages).
2020-10-15T11:10:14 Failed to compute metric percentage-failure-100-
last-eval-per-spec (my-packages).
2020-10-15T11:10:14 Failed to compute metric percentage-failed-eval-
per-spec (my-packages).
2020-10-15T11:10:14 Metrics update took 0.020688 seconds
2020-10-15T11:10:14 fetching input 'pomdappi' of spec 'my-packages'
2020-10-15T11:10:14 fetching input 'guix' of spec 'my-packages'
2020-10-15T11:14:35 Git error while fetching inputs of 'my-packages':
"failed to connect to git.savannah.gnu.org: Connection timed out"

2020-10-15T11:14:35 next evaluation in 60 seconds

Using ah HTTPS URI for guix git does not help. Do you know what is
going on?

Best regards,

divoplade

Re: Configuration for Cuirass

[Mathieu Othacehe]

Hello,

> My log file alternates between "building 0 derivations" and "failed to
> connect to git.savannah.gnu.org: Connection timed out".

The first message is probably a consequence of the second. Make sure
that nss-certs is available in the root profile.

You can also try to setup a Guix mirror to see if it helps.

Mathieu

Re: Configuration for Cuirass

[divoplade]

Le jeudi 15 octobre 2020 à 14:32 +0200, Mathieu Othacehe a écrit :
> You can also try to setup a Guix mirror to see if it helps.

That's what I did. It seems that the configuration is ignored if the
/var/lib/cuirass/cuirass.db file exists, and that cuirass cannot
restart if I remove it. So I had to reboot a few times.

Now it's better, in the sense that it actually tries to build
something:

https://ci.divoplade.fr/jobset/my-packages

The only problem is the exception, logged as:

Uncaught exception in fiber ##f:
In cuirass/base.scm:
   785:13  3 (_)
In ice-9/boot-9.scm:
  1736:10  2 (with-exception-handler _ _ #:unwind? _ # _)
  1669:16  1 (raise-exception _ #:continuable? _)
  1667:16  0 (raise-exception _ #:continuable? _)
ice-9/boot-9.scm:1667:16: In procedure raise-exception:
ERROR:
  1. &evaluation-error:
      name: "my-packages"
      id: 1

See https://ci.divoplade.fr/eval/1/log/raw

Re: Configuration for Cuirass

[Mathieu Othacehe]

Hello,

> See https://ci.divoplade.fr/eval/1/log/raw

Sadly the log isn't helpful, could you share your specification?

Thanks,

Mathieu

Re: Configuration for Cuirass

[divoplade]

Hi,

Le dimanche 18 octobre 2020 à 17:46 +0200, Mathieu Othacehe a écrit :
> > See https://ci.divoplade.fr/eval/1/log/raw
> 
> Sadly the log isn't helpful, could you share your specification?

Sorry, I tried a few different things and it might have changed a bit,
but the error is the same. The config is here:

https://code.divoplade.fr/divoplade-site.git/tree/divoplade/services/cuirass.scm

The /srv/git path is the root of the repos at 
https://code.divoplade.fr/.

The evaluations are here:

https://ci.divoplade.fr/jobset/my-packages

divoplade

Re: Configuration for Cuirass

[Mathieu Othacehe]

Running this:

--8<---------------cut here---------------start------------->8---
mathieu@cervin:~/tmp$ GUIX_PACKAGE_PATH=pomdappi/ guix repl 
scheme@(guix-user)> ,use (gnu) (guix store)  (gnu ci)
scheme@(guix-user)> (define s (hydra-jobs (open-connection) '((subset "pomdappi") (systems "x86_64-linux"))))
--8<---------------cut here---------------end--------------->8---

gives several errors:

--8<---------------cut here---------------start------------->8---
evaluating for 'x86_64-linux' (heap size: 43.0 MiB)...
guix repl: warning: failed to load '(gen-hall)':
no code for module (gen-hall)
hint: File `pomdappi//gen-hall.scm' should probably start with:

     (define-module (gen-hall))

guix repl: warning: failed to load '(guix divoplade services pomdappi)':
no code for module (divoplade packages pomdappi)
pomdappi//guix/divoplade/services/pomdappi.scm:1:0: warning: module name (divoplade services pomdappi) does not match file name 'guix/divoplade/services/pomdappi.scm'
hint: File `pomdappi//guix/divoplade/services/pomdappi.scm' should probably start with:

     (define-module (guix divoplade services pomdappi))

;;; note: source file pomdappi/guix.scm
;;;       newer than compiled /gnu/store/j447qlziqmkmlq50kxsrcww2nhhkj1vv-guix-module-union/lib/guile/3.0/site-ccache/guix.go
;;; note: source file pomdappi/guix.scm
;;;       newer than compiled /home/mathieu/.guix-profile//lib/guile/3.0/site-ccache/guix.go
guix repl: warning: failed to load '(pomdappi access-token)':
In procedure dynamic-link: file: "libpomdappi", message: "file not found"
guix repl: warning: failed to load '(pomdappi authorization)':
no code for module (turtle tordf)
guix repl: warning: failed to load '(pomdappi client)':
no code for module (rdf rdf)
guix repl: warning: failed to load '(tests access-token)': No such file or directory
guix repl: warning: failed to load '(tests cache)': No such file or directory
guix repl: warning: failed to load '(tests client-manifest)': No such file or directory
guix repl: warning: failed to load '(tests dpop)': No such file or directory
guix repl: warning: failed to load '(tests flow)': No such file or directory
%%%% Starting test JKT
error: make-rsa-public-key: unbound variable
hint: Did you forget a `use-modules' form?

error: make-rsa-public-key: unbound variable
hint: Did you forget a `use-modules' form?

guix repl: warning: failed to load '(tests jws)': No such file or directory
guix repl: warning: failed to load '(tests jwt)': No such file or directory
guix repl: warning: failed to load '(tests refresh)': No such file or directory
guix repl: error: pomdappi: unknown package
--8<---------------cut here---------------end--------------->8---

Cuirass evaluation is probably broken because of those errors.

Mathieu

Re: Configuration for Cuirass

[divoplade]

Thank you for your answer.

The package I want to run guix in should be this one:
https://code.divoplade.fr/divoplade-site.git/

It is a channel aggregating my packages. The one you tried is just the
source code of one of them, and it's not supposed to be used as a guix
channel (thus the errors).

If I clone it and run your commands I do not get an error (I edited the
notes about source file newer than compiled):

--8<---------------cut here---------------start------------->8---
divoplade-site$ GUIX_PACKAGE_PATH=./ guix repl
GNU Guile 3.0.4
Copyright (C) 1995-2020 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> ,use (gnu) (guix store)  (gnu ci)
scheme@(guix-user)> (define s (hydra-jobs (open-connection) '((subset
"pomdappi") (systems "x86_64-linux"))))
evaluating for 'x86_64-linux' (heap size: 43.0 MiB)...
scheme@(guix-user)> s
$1 = ((pomdappi-0.0.1-87-g2d75550.x86_64-linux . #<procedure
7fdb62a012d0 at gnu/ci.scm:114:19 ()>))
scheme@(guix-user)>
--8<---------------cut here---------------end--------------->8---

Le dimanche 18 octobre 2020 à 18:24 +0200, Mathieu Othacehe a écrit :
> Running this:
> 
> --8<---------------cut here---------------start------------->8---
> mathieu@cervin:~/tmp$ GUIX_PACKAGE_PATH=pomdappi/ guix repl 
> scheme@(guix-user)> ,use (gnu) (guix store)  (gnu ci)
> scheme@(guix-user)> (define s (hydra-jobs (open-connection) '((subset
> "pomdappi") (systems "x86_64-linux"))))
> --8<---------------cut here---------------end--------------->8---
> 
> gives several errors:
> 
> --8<---------------cut here---------------start------------->8---
> evaluating for 'x86_64-linux' (heap size: 43.0 MiB)...
> guix repl: warning: failed to load '(gen-hall)':
> no code for module (gen-hall)
> hint: File `pomdappi//gen-hall.scm' should probably start with:
> 
>      (define-module (gen-hall))
> 
> guix repl: warning: failed to load '(guix divoplade services
> pomdappi)':
> no code for module (divoplade packages pomdappi)
> pomdappi//guix/divoplade/services/pomdappi.scm:1:0: warning: module
> name (divoplade services pomdappi) does not match file name
> 'guix/divoplade/services/pomdappi.scm'
> hint: File `pomdappi//guix/divoplade/services/pomdappi.scm' should
> probably start with:
> 
>      (define-module (guix divoplade services pomdappi))
> 
> ;;; note: source file pomdappi/guix.scm
> ;;;       newer than compiled
> /gnu/store/j447qlziqmkmlq50kxsrcww2nhhkj1vv-guix-module-
> union/lib/guile/3.0/site-ccache/guix.go
> ;;; note: source file pomdappi/guix.scm
> ;;;       newer than compiled /home/mathieu/.guix-
> profile//lib/guile/3.0/site-ccache/guix.go
> guix repl: warning: failed to load '(pomdappi access-token)':
> In procedure dynamic-link: file: "libpomdappi", message: "file not
> found"
> guix repl: warning: failed to load '(pomdappi authorization)':
> no code for module (turtle tordf)
> guix repl: warning: failed to load '(pomdappi client)':
> no code for module (rdf rdf)
> guix repl: warning: failed to load '(tests access-token)': No such
> file or directory
> guix repl: warning: failed to load '(tests cache)': No such file or
> directory
> guix repl: warning: failed to load '(tests client-manifest)': No such
> file or directory
> guix repl: warning: failed to load '(tests dpop)': No such file or
> directory
> guix repl: warning: failed to load '(tests flow)': No such file or
> directory
> %%%% Starting test JKT
> error: make-rsa-public-key: unbound variable
> hint: Did you forget a `use-modules' form?
> 
> error: make-rsa-public-key: unbound variable
> hint: Did you forget a `use-modules' form?
> 
> guix repl: warning: failed to load '(tests jws)': No such file or
> directory
> guix repl: warning: failed to load '(tests jwt)': No such file or
> directory
> guix repl: warning: failed to load '(tests refresh)': No such file or
> directory
> guix repl: error: pomdappi: unknown package
> --8<---------------cut here---------------end--------------->8---
> 
> Cuirass evaluation is probably broken because of those errors.
> 
> Mathieu

Re: Configuration for Cuirass

[Mathieu Othacehe]

> If I clone it and run your commands I do not get an error (I edited the
> notes about source file newer than compiled):

You also need to add "divoplade-site" to "#:package-path-inputs" in the
specifications.

See the following snippet from Cuirass info page:

--8<---------------cut here---------------start------------->8---
‘package_path_inputs’
     This field holds a list of input names whose load path is prepended
     to ‘GUIX_PACKAGE_PATH’ when evaluating ‘proc_file’.
--8<---------------cut here---------------end--------------->8---

Mathieu

Re: Configuration for Cuirass

[divoplade]

Le dimanche 18 octobre 2020 à 19:38 +0200, Mathieu Othacehe a écrit :
> You also need to add "divoplade-site" to "#:package-path-inputs" in
> the
> specifications.
> 
> See the following snippet from Cuirass info page:
> 
> --8<---------------cut here---------------start------------->8---
> ‘package_path_inputs’
>      This field holds a list of input names whose load path is
> prepended
>      to ‘GUIX_PACKAGE_PATH’ when evaluating ‘proc_file’.
> --8<---------------cut here---------------end--------------->8---

I was missing that, thank you. Unfortunately, it did not change the
error.

Re: Configuration for Cuirass

[Mathieu Othacehe]

Hello,

> I was missing that, thank you. Unfortunately, it did not change the
> error.

That's unfortunate. I tried the following specification:

--8<---------------cut here---------------start------------->8---
(list '((#:name . "my-packages"
       (#:load-path-inputs . ("guix"))
       (#:package-path-inputs . ("divoplade-site"))
       (#:proc-input . "guix")
       (#:proc-file . "build-aux/cuirass/gnu-system.scm")
       (#:proc . cuirass-jobs)
       (#:proc-args (subset "pomdappi" "update-gandi" "index-page" "orange-livebox-ipv4"))
       (#:inputs . (((#:name . "divoplade-site")
                     (#:url . "file:///home/mathieu/tmp/divoplade-site/")
                     (#:load-path . ".")
                     (#:branch . "master")
                     (#:no-compile? . #t))
                    ((#:name . "guix")
                     (#:url . "git://git.savannah.gnu.org/guix.git")
                     (#:load-path . ".")
                     (#:branch . "master")
                     (#:no-compile? . #t))))
       (#:build-outputs . ())))
--8<---------------cut here---------------end--------------->8---

by running Cuirass with the following command:

--8<---------------cut here---------------start------------->8---
./pre-inst-env cuirass -S ~/tmp/test.scm -D test.db
--8<---------------cut here---------------end--------------->8---

directly from a Cuirass checkout. It works fine here, not sure what's
wrong with the service.

Mathieu

Re: Configuration for Cuirass

[divoplade]

Le lundi 19 octobre 2020 à 12:27 +0200, Mathieu Othacehe a écrit :
> --8<---------------cut here---------------start------------->8---
> (list '((#:name . "my-packages"

(for future reference: there's a little problem with parens, the reader
will know how to fix it)

> It works fine here, not sure what's
> wrong with the service.
You were right: I just needed to nuke the DB and it started to work!

Thank you so much for your involvement in helping me fix the problem. I
really appreciate it.

Best regards,

divoplade