Gitlab CI

Gitlab CI

[Jérémy Korwin-Zmijowski]

Hey Guixters !

I am experimenting with Gitlab CI for a Guile project I am working on.
It is managed with Hall and I would like to benefit the Guix features.

Currently, I struggle to run Guix inside the default Debian stable
docker image (framagit.org).

Here is my .gitlab-ci.yml :

   before_script:
    - apt -qq update
    - apt -qq install -y wget gpg xz-utils
    - wget 
   https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
 - chmod +x guix-install.sh
 - wget https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - |
gpg --import -
 - printf '\n' | ./guix-install.sh
 - /root/.config/guix/current/bin/guix-daemon --build-users-
group=guixbuild
 - guix pull

stages:
  - bootstrap
  - configure
  - build
  - test

job:bootstrap:
  stage: bootstrap
  script: autoreconf -vfi

What I get :

   [1592641832.155]: [ FAIL ] unsupported init system; run the daemon
   manually:
      /root/.config/guix/current/bin/guix-daemon --build-users-
   group=guixbuild
   [1592641832.155]: [ INFO ] making the guix command available to
   other users

Does anyone has done that before ?
Is there a way to get a Guix docker image instead of the Debian one ?

Have a nice weekend !

Jérémy

Re: Gitlab CI

[Ricardo Wurmus]

Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> writes:

> Hey Guixters !
>
> I am experimenting with Gitlab CI for a Guile project I am working on.
> It is managed with Hall and I would like to benefit the Guix features.
>
> Currently, I struggle to run Guix inside the default Debian stable
> docker image (framagit.org).

Can you specify a different Docker image?  Then you could create one
containing Guix with “guix pack -f docker”.

-- 
Ricardo

Re: Gitlab CI

[Jérémy Korwin-Zmijowski]

Le samedi 20 juin 2020 à 15:40 +0200, Ricardo Wurmus a écrit :
> 
> Can you specify a different Docker image?  Then you could create one
> containing Guix with “guix pack -f docker”.
> 

I am not the owmer of the Gitlab instance, so I think I can't... But I
can ask to the admin haha. Worth the shot !

Thank you Ricardo !

Re: Gitlab CI

[divoplade]

Hello Jérémy,

The problem is that guix-install.sh asks you whether you want to
authorize the substitutes, but it can't because it is not used
interactively, so it fails.

There are other traps to avoid (do not forget to install ntp, call
guix-daemon with --disable-chroot because you are in an unprivileged
container, cache things to avoid pulling the world at each commit). I
have a solution based on caching a guix pack containing all your
packages: https://framagit.org/divoplade/guix-project

variables:
  GUIX_REF: "--branch=master"
  PACKAGES: "guile-hall hello"

before_script:
  - test -f cache.tar.gz || apt -qq update
  - test -f cache.tar.gz || apt -qq install -y wget gpg xz-utils ntp
  - test -f cache.tar.gz || wget 
https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - | gpg --
import -
  - test -f cache.tar.gz || wget 
https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
  - test -f cache.tar.gz || chmod ugo+x guix-install.sh
  - test -f cache.tar.gz || (echo "" ; echo "y") | ./guix-install.sh ||
echo "Assume OK"
  - test -f cache.tar.gz || source
/root/.config/guix/current/etc/profile
  - test -f cache.tar.gz || guix-daemon --build-users-group=guixbuild
--disable-chroot &
  - test -f cache.tar.gz || guix pull $GUIX_REF
  - test -f cache.tar.gz || cp $(guix pack -C gzip $PACKAGES
-S/opt/bin=bin) cache.tar.gz
  - cp cache.tar.gz /
  - (cd / && tar xf cache.tar.gz)
  - export PATH=/opt/bin:$PATH

cache:
  key: "$GUIX_REF $PACKAGES"
  paths:
    - cache.tar.gz

job:
  script:
    - hello
    - hall --help

After a very long set up (1/2 hour with a somewhat fast runner), you
will leverage the cached guix pack to run the jobs instantly, with the
default gitlab runner (unprivileged, debian-based).

If you want to update, clear the runner caches, or change the GUIX_REF
variable. You can pin a specific commit (better for reproducibility,
not so great for releases) with --commit=COMMIT.

I could not get guix to run within the job, even with "guix" added to
PACKAGES: it would need this in the script:

    - apt -qq update
    - apt -qq install -y ntp
    - (groupadd --system guixbuild && for i in $(seq 1 10) ; do useradd
-g guixbuild -G guixbuild -d /var/empty -s $(which nologin) -c "Guix
build user $i" --system guixbuilder$i; done) || true
    - guix-daemon --build-users-group=guixbuild --disable-chroot &
    - guix archive --authorize < $(dirname $(readlink
/usr/local/bin/guix))/../share/guix/ci.guix.info.pub

However, if you try it, you will notice a crash.

I am looking forward to having guix shipped as a debian package, it
will be way easier.

divoplade

Le samedi 20 juin 2020 à 10:42 +0200, Jérémy Korwin-Zmijowski a écrit :
> Hey Guixters !
> 
> I am experimenting with Gitlab CI for a Guile project I am working
> on.
> It is managed with Hall and I would like to benefit the Guix
> features.
> 
> Currently, I struggle to run Guix inside the default Debian stable
> docker image (framagit.org).
> 
> Here is my .gitlab-ci.yml :
> 
>    before_script:
>     - apt -qq update
>     - apt -qq install -y wget gpg xz-utils
>     - wget 
>    
> https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
>  - chmod +x guix-install.sh
>  - wget https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - |
> gpg --import -
>  - printf '\n' | ./guix-install.sh
>  - /root/.config/guix/current/bin/guix-daemon --build-users-
> group=guixbuild
>  - guix pull
> 
> stages:
>   - bootstrap
>   - configure
>   - build
>   - test
> 
> job:bootstrap:
>   stage: bootstrap
>   script: autoreconf -vfi
> 
> What I get :
> 
>    [1592641832.155]: [ FAIL ] unsupported init system; run the daemon
>    manually:
>       /root/.config/guix/current/bin/guix-daemon --build-users-
>    group=guixbuild
>    [1592641832.155]: [ INFO ] making the guix command available to
>    other users
> 
> Does anyone has done that before ?
> Is there a way to get a Guix docker image instead of the Debian one ?
> 
> Have a nice weekend !
> 
> Jérémy
> 
> 

Re: Gitlab CI

[Jérémy Korwin-Zmijowski]

Hey divoplade !

Thank you very much for sharing your hack ! The caching section is very
interesting... Does it save time between runs ?

After a lot of attempts, I ended up with a green pipeline but it takes
43 minute to complete haha 
https://framagit.org/Jeko/guile-dummy/-/pipelines

Jérémy

Re: Gitlab CI

[Jérémy Korwin-Zmijowski]

I tried to apply the caching as you did, but I error during project
configuration.

https://framagit.org/Jeko/guile-dummy/-/jobs/1029866

Jérémy

Le mardi 23 juin 2020 à 12:15 +0200, Jérémy Korwin-Zmijowski a écrit :
> Hey divoplade !
> 
> Thank you very much for sharing your hack ! The caching section is
> very
> interesting... Does it save time between runs ?
> 
> After a lot of attempts, I ended up with a green pipeline but it
> takes
> 43 minute to complete haha 
> https://framagit.org/Jeko/guile-dummy/-/pipelines
> 
> Jérémy
> 
> 
> 
> 
> 
> 

Re: Gitlab CI

[zimoun]

Hi Jérémy,

On Sat, 20 Jun 2020 at 10:42, Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> wrote:

>  - printf '\n' | ./guix-install.sh

[...]

>    [1592641832.155]: [ FAIL ] unsupported init system; run the daemon
>    manually:
>       /root/.config/guix/current/bin/guix-daemon --build-users-
>    group=guixbuild
>    [1592641832.155]: [ INFO ] making the guix command available to
>    other users
>
> Does anyone has done that before ?

It is because the guix-install.sh script does not recognize the init
system of the Docker image, see 'sys_enable_guix_daemon()'.  Roughly
speaking, it does not fall into upstart) neither systemd) neither
sysv-init) so it falls into NA|*).

I do not know how the Docker image you use is built.

> Is there a way to get a Guix docker image instead of the Debian one ?

The easiest is, IMHO, instead of "./guix-install.sh", you should run all
the 8 steps described in [1] adapting the step #5.

1: https://guix.gnu.org/manual/devel/en/guix.html#Binary-Installation


Cheers,
simon

Re: Gitlab CI

[Jérémy Korwin-Zmijowski]

Hi Zimoun !

Thank you for taking time to help over here !

Thanks to divoplade, we succedded to make something up for relative
simple projects (I don't know how it goes for more complicated ones).
If anybody feel interested :
https://framagit.org/Jeko/guile-dummy

Cheers,
Jérémy

Re: Gitlab CI

[divoplade]

[-- Attachment #1: Type: text/plain, Size: 1850 bytes --]

Hello Jérémy,

I have a new solution (it is quite different from the previous one) in
which it is possible to run guix commands within a job, while still
caching most of the files (see attached). I am sorry, my fork of your
project is already a mess so I prefer to put the file in its entirety
here.

Anyways, everything will be way easier once we have a proper debian
package for guix:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644

divoplade

Le samedi 20 juin 2020 à 10:42 +0200, Jérémy Korwin-Zmijowski a écrit :
> Hey Guixters !
> 
> I am experimenting with Gitlab CI for a Guile project I am working
> on.
> It is managed with Hall and I would like to benefit the Guix
> features.
> 
> Currently, I struggle to run Guix inside the default Debian stable
> docker image (framagit.org).
> 
> Here is my .gitlab-ci.yml :
> 
>    before_script:
>     - apt -qq update
>     - apt -qq install -y wget gpg xz-utils
>     - wget 
>    
> https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
>  - chmod +x guix-install.sh
>  - wget https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - |
> gpg --import -
>  - printf '\n' | ./guix-install.sh
>  - /root/.config/guix/current/bin/guix-daemon --build-users-
> group=guixbuild
>  - guix pull
> 
> stages:
>   - bootstrap
>   - configure
>   - build
>   - test
> 
> job:bootstrap:
>   stage: bootstrap
>   script: autoreconf -vfi
> 
> What I get :
> 
>    [1592641832.155]: [ FAIL ] unsupported init system; run the daemon
>    manually:
>       /root/.config/guix/current/bin/guix-daemon --build-users-
>    group=guixbuild
>    [1592641832.155]: [ INFO ] making the guix command available to
>    other users
> 
> Does anyone has done that before ?
> Is there a way to get a Guix docker image instead of the Debian one ?
> 
> Have a nice weekend !
> 
> Jérémy
> 
> 

[-- Attachment #2: .gitlab-ci.yml --]
[-- Type: application/x-yaml, Size: 2088 bytes --]

Re: Gitlab CI

[Jérémy Korwin-Zmijowski]

Le jeudi 25 juin 2020 à 23:32 +0200, divoplade a écrit :
> Hello Jérémy,
> 
> I have a new solution (it is quite different from the previous one)
> in
> which it is possible to run guix commands within a job, while still
> caching most of the files (see attached). I am sorry, my fork of your
> project is already a mess so I prefer to put the file in its entirety
> here.
> 
> Anyways, everything will be way easier once we have a proper debian
> package for guix:
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644
> 
> divoplade

Hi divoplade !

Thank you for the new yaml file ! Unfortunately, I didn't manage to use
this one yet. I have to do more experiments I guess.

I the meantime I would like to try to put a Guix's docker image on
dockerhub and use it in Framagit/Gitlab if it is possible...

Cheers

Jérémy

Re: Debian package with Gitlab CI

[divoplade]

Hello,

Le lundi 06 juillet 2020 à 14:25 +0200, Jérémy Korwin-Zmijowski a
écrit :
> Thank you for the new yaml file ! Unfortunately, I didn't manage to
> use
> this one yet. I have to do more experiments I guess.

I do, too. In the mean time, I think we would be very happy to have a
debian package. There is some work done for that (
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644), but I'm not
familiar enough with the debian process to understand if we can use
anything of it yet. In any case, there is no guix package in the docker
image debian:experimental for now...

The best thing would be to just apt-get install guix (caching the
packages so as not to download it again) and simply start the daemon.

Re: Gitlab CI

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 318 bytes --]

Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> writes:

> I the meantime I would like to try to put a Guix's docker image on
> dockerhub and use it in Framagit/Gitlab if it is possible...

This would be fantastic, please let us know about your progress :)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Gitlab CI

[Ludovic Courtès]

Hello!

I’m late to the party but I’m interested in this discussion…  :-)

Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> skribis:

> I am experimenting with Gitlab CI for a Guile project I am working on.
> It is managed with Hall and I would like to benefit the Guix features.

GitLab CI can run jobs directly in a Docker image:

  https://docs.gitlab.com/ce/ci/docker/using_docker_images.html

So it would be interesting to have any easy way for people to create a
Docker image for this purpose.

Colleagues of mine tried ‘guix system docker-image’ for that but that
turns out to not quite work (IIRC gitlab-runner tries to do ‘docker exec
your build command’, which fails here because those images have shepherd
as their entry point).

So presumably a better option is to use an image created like this:

  guix pack -f docker -S /bin=bin guix --localstatedir

and then, in the image, you first need to spawn guix-daemon, perhaps
with ‘--disable-chroot’ because the image lacks build users.

It would be useful also to set the user under which commands run in the
image, similar to the ‘USER’ directive in Dockerfiles.

Thoughts?

It would be nice to make that as smooth as possible and to add a recipe
in the cookbook!

Ludo’.

Re: Gitlab CI

[divoplade]

Hello,

Le vendredi 24 juillet 2020 à 16:30 +0200, Ludovic Courtès a écrit :
> 
> So presumably a better option is to use an image created like this:
> 
>   guix pack -f docker -S /bin=bin guix --localstatedir
> 
As you may know, most gitlab server will not let you use the provided
docker registry, and so the only option will be to push to docker hub.
Moreover, running this command as part of a CI job just to push it to
docker hub will require a privileged container. So you will need to
either provide your own runner, or provide you own docker registry
(docker hub or else).

I prefer the solution in which the pack is a .tar.gz that is directly
untarred to /; this way it can be run on a shared unprivileged runner
and the storage can be achieved in the runner itself, via the gitlab CI
cache.

divoplade

Re: Gitlab CI

[Ludovic Courtès]

Hi,

divoplade <d@divoplade.fr> skribis:

> Le vendredi 24 juillet 2020 à 16:30 +0200, Ludovic Courtès a écrit :
>> 
>> So presumably a better option is to use an image created like this:
>> 
>>   guix pack -f docker -S /bin=bin guix --localstatedir
>> 
> As you may know, most gitlab server will not let you use the provided
> docker registry, and so the only option will be to push to docker hub.

Yes.  What I had in mind was that you would create that image, upload it
to a registry GitLab-CI can pull from, and then refer to that image in
your ‘.gitlab-ci.yml’.

> I prefer the solution in which the pack is a .tar.gz that is directly
> untarred to /; this way it can be run on a shared unprivileged runner
> and the storage can be achieved in the runner itself, via the gitlab CI
> cache.

Ah yes, I guess that would also work, and it’s in fact less tedious.
That amounts to running https://guix.gnu.org/install.sh every time
though, no?

Thanks,
Ludo’.

Re: Gitlab CI

[divoplade]

Le lundi 27 juillet 2020 à 09:46 +0200, Ludovic Courtès a écrit :
> > I prefer the solution in which the pack is a .tar.gz that is
> > directly
> > untarred to /; this way it can be run on a shared unprivileged
> > runner
> > and the storage can be achieved in the runner itself, via the
> > gitlab CI
> > cache.
> 
> Ah yes, I guess that would also work, and it’s in fact less tedious.
> That amounts to running https://guix.gnu.org/install.sh every time
> though, no?

Pretty much, except "every time" is really less often than expected,
because the gitlab runner can cache files between jobs. So with an
appropriate tar command, it is possible to pack the whole store plus
the /var/guix and untar it at the beginning of the job (or re-run the
guix install if the cache has been dropped). Or run "guix pack".

There is a very unfortunate problem that running "guix pull" or guix
installing something will require the "ntp" debian package, so each job
requires apt-get update and apt-get install -y ntp... I am
investigating how to cache the ntp package (and its dependencies) to
avoid running apt-get update, but that's quite disappointing.

divoplade

Re: Gitlab CI

[Stephen Scheck]

Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> writes:

> Hey Guixters !
>
> I am experimenting with Gitlab CI for a Guile project I am working on.
> It is managed with Hall and I would like to benefit the Guix features.
>
> Does anyone has done that before ?
> Is there a way to get a Guix docker image instead of the Debian one ?

Hello Jérémy,

I have been trying to accomplish a similar goal - using Guix in GitLab CI for a Guile project. To that end, I’ve built some Docker images you might find interesting:

Guix in Alpine Linux
https://hub.docker.com/r/singularsyntax/guix-bootstrap/tags
https://gitlab.com/singularsyntax-docker-hub/guix-bootstrap (the Dockerfile and pipeline for deployment to Docker Hub)

My goal is to enable typical Docker workflows with Guix, for example something like this Dockerfile:

FROM gnu/guix:1.1.0
RUN guix package —install <some-build-and-test-package-dependencies>

Then use the resulting Docker image in the build, test, etc. stages of a GitLab CI/CD pipeline.

Unfortunately, Guix does not have a natural fit with Docker containerization since `guix-daemon` must be running for any Guix commands to work. I haven’t been able to get any Dockerfile workflows like above working, since `docker build` seems to skip execution of the regular image entry point command which starts `guix-daemon`. I think to make it work, you’d need to provide some kind of shell wrapper around the `guix` command which first starts `guix-daemon`, waits for it to be ready, then runs Guix commands. But you can use the Alpine image for Docker-in-Docker workflows where you first start a container with `docker run`, then issue successive `docker exec` commands to perform work inside the container, then extract the results from the container for use in other stages of a CI/CD pipeline.

I also attempted to build a “pure” Guix container not requiring installation on top of another Linux distribution by using `guix system docker-image <…>`. I even had a GitLab pipeline making daily builds which you might find interesting (but I recently disabled it as I’ll explain below):

https://hub.docker.com/r/singularsyntax/guix/tags
https://gitlab.com/singularsyntax-docker-hub/guix (GitLab CI/CD pipeline for deployment to Docker Hub)

It works fine for obtaining a sandbox for which to experiment interactively with Guix, but has the same problems as the Alpine image, and is even a little worse because there are no “anchor” binaries inside the container, not even `/bin/sh`, until the container init process establishes symlinks into the `/gnu/store` tree. If you try to use it in scripts or CI/CD pipelines, they often fail due to race conditions with the init script, and without `/bin/sh` it’s difficult to reliably assert any pre-conditions inside the container. So it’s a real pain to work with, and I think of little utility for scripting as it currently stands.

Anyway, hopefully this provides some more insight, and good luck!

Re: Gitlab CI

[Stephen Scheck]

Ludovic Courtès writes:

> Hello! I’m late to the party but I’m interested in this discussion… :-)

> GitLab CI can run jobs directly in a Docker image:
>   https://docs.gitlab.com/ce/ci/docker/using_docker_images.html

> So it would be interesting to have any easy way for people to create a Docker image for this purpose.

I’ve updated my Alpine-based Docker image, and I think it should work now for many common use cases,
but it would be great if it saw some more testing. Details here:

   https://hub.docker.com/r/singularsyntax/guix-bootstrap

Cheers -ss