unofficial mirror of help-guix@gnu.org 
 help / color / Atom feed
* Gitlab CI
@ 2020-06-20  8:42 Jérémy Korwin-Zmijowski
  2020-06-20 13:40 ` Ricardo Wurmus
                   ` (3 more replies)
  0 siblings, 4 replies; 9+ messages in thread
From: Jérémy Korwin-Zmijowski @ 2020-06-20  8:42 UTC (permalink / raw)
  To: Guix Help

Hey Guixters !

I am experimenting with Gitlab CI for a Guile project I am working on.
It is managed with Hall and I would like to benefit the Guix features.

Currently, I struggle to run Guix inside the default Debian stable
docker image (framagit.org).

Here is my .gitlab-ci.yml :

   before_script:
    - apt -qq update
    - apt -qq install -y wget gpg xz-utils
    - wget 
   https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
 - chmod +x guix-install.sh
 - wget https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - |
gpg --import -
 - printf '\n' | ./guix-install.sh
 - /root/.config/guix/current/bin/guix-daemon --build-users-
group=guixbuild
 - guix pull

stages:
  - bootstrap
  - configure
  - build
  - test

job:bootstrap:
  stage: bootstrap
  script: autoreconf -vfi

What I get :

   [1592641832.155]: [ FAIL ] unsupported init system; run the daemon
   manually:
      /root/.config/guix/current/bin/guix-daemon --build-users-
   group=guixbuild
   [1592641832.155]: [ INFO ] making the guix command available to
   other users

Does anyone has done that before ?
Is there a way to get a Guix docker image instead of the Debian one ?

Have a nice weekend !

Jérémy



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Gitlab CI
  2020-06-20  8:42 Gitlab CI Jérémy Korwin-Zmijowski
@ 2020-06-20 13:40 ` Ricardo Wurmus
  2020-06-23  8:54   ` Jérémy Korwin-Zmijowski
  2020-06-20 15:05 ` divoplade
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 9+ messages in thread
From: Ricardo Wurmus @ 2020-06-20 13:40 UTC (permalink / raw)
  To: Jérémy Korwin-Zmijowski; +Cc: help-guix


Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> writes:

> Hey Guixters !
>
> I am experimenting with Gitlab CI for a Guile project I am working on.
> It is managed with Hall and I would like to benefit the Guix features.
>
> Currently, I struggle to run Guix inside the default Debian stable
> docker image (framagit.org).

Can you specify a different Docker image?  Then you could create one
containing Guix with “guix pack -f docker”.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Gitlab CI
  2020-06-20  8:42 Gitlab CI Jérémy Korwin-Zmijowski
  2020-06-20 13:40 ` Ricardo Wurmus
@ 2020-06-20 15:05 ` divoplade
  2020-06-23 10:15   ` Jérémy Korwin-Zmijowski
  2020-06-25 12:32 ` zimoun
  2020-06-25 21:32 ` divoplade
  3 siblings, 1 reply; 9+ messages in thread
From: divoplade @ 2020-06-20 15:05 UTC (permalink / raw)
  To: Jérémy Korwin-Zmijowski, Guix Help

Hello Jérémy,

The problem is that guix-install.sh asks you whether you want to
authorize the substitutes, but it can't because it is not used
interactively, so it fails.

There are other traps to avoid (do not forget to install ntp, call
guix-daemon with --disable-chroot because you are in an unprivileged
container, cache things to avoid pulling the world at each commit). I
have a solution based on caching a guix pack containing all your
packages: https://framagit.org/divoplade/guix-project

variables:
  GUIX_REF: "--branch=master"
  PACKAGES: "guile-hall hello"

before_script:
  - test -f cache.tar.gz || apt -qq update
  - test -f cache.tar.gz || apt -qq install -y wget gpg xz-utils ntp
  - test -f cache.tar.gz || wget 
https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - | gpg --
import -
  - test -f cache.tar.gz || wget 
https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
  - test -f cache.tar.gz || chmod ugo+x guix-install.sh
  - test -f cache.tar.gz || (echo "" ; echo "y") | ./guix-install.sh ||
echo "Assume OK"
  - test -f cache.tar.gz || source
/root/.config/guix/current/etc/profile
  - test -f cache.tar.gz || guix-daemon --build-users-group=guixbuild
--disable-chroot &
  - test -f cache.tar.gz || guix pull $GUIX_REF
  - test -f cache.tar.gz || cp $(guix pack -C gzip $PACKAGES
-S/opt/bin=bin) cache.tar.gz
  - cp cache.tar.gz /
  - (cd / && tar xf cache.tar.gz)
  - export PATH=/opt/bin:$PATH

cache:
  key: "$GUIX_REF $PACKAGES"
  paths:
    - cache.tar.gz

job:
  script:
    - hello
    - hall --help

After a very long set up (1/2 hour with a somewhat fast runner), you
will leverage the cached guix pack to run the jobs instantly, with the
default gitlab runner (unprivileged, debian-based).

If you want to update, clear the runner caches, or change the GUIX_REF
variable. You can pin a specific commit (better for reproducibility,
not so great for releases) with --commit=COMMIT.

I could not get guix to run within the job, even with "guix" added to
PACKAGES: it would need this in the script:

    - apt -qq update
    - apt -qq install -y ntp
    - (groupadd --system guixbuild && for i in $(seq 1 10) ; do useradd
-g guixbuild -G guixbuild -d /var/empty -s $(which nologin) -c "Guix
build user $i" --system guixbuilder$i; done) || true
    - guix-daemon --build-users-group=guixbuild --disable-chroot &
    - guix archive --authorize < $(dirname $(readlink
/usr/local/bin/guix))/../share/guix/ci.guix.info.pub

However, if you try it, you will notice a crash.

I am looking forward to having guix shipped as a debian package, it
will be way easier.

divoplade

Le samedi 20 juin 2020 à 10:42 +0200, Jérémy Korwin-Zmijowski a écrit :
> Hey Guixters !
> 
> I am experimenting with Gitlab CI for a Guile project I am working
> on.
> It is managed with Hall and I would like to benefit the Guix
> features.
> 
> Currently, I struggle to run Guix inside the default Debian stable
> docker image (framagit.org).
> 
> Here is my .gitlab-ci.yml :
> 
>    before_script:
>     - apt -qq update
>     - apt -qq install -y wget gpg xz-utils
>     - wget 
>    
> https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
>  - chmod +x guix-install.sh
>  - wget https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - |
> gpg --import -
>  - printf '\n' | ./guix-install.sh
>  - /root/.config/guix/current/bin/guix-daemon --build-users-
> group=guixbuild
>  - guix pull
> 
> stages:
>   - bootstrap
>   - configure
>   - build
>   - test
> 
> job:bootstrap:
>   stage: bootstrap
>   script: autoreconf -vfi
> 
> What I get :
> 
>    [1592641832.155]: [ FAIL ] unsupported init system; run the daemon
>    manually:
>       /root/.config/guix/current/bin/guix-daemon --build-users-
>    group=guixbuild
>    [1592641832.155]: [ INFO ] making the guix command available to
>    other users
> 
> Does anyone has done that before ?
> Is there a way to get a Guix docker image instead of the Debian one ?
> 
> Have a nice weekend !
> 
> Jérémy
> 
> 



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Gitlab CI
  2020-06-20 13:40 ` Ricardo Wurmus
@ 2020-06-23  8:54   ` Jérémy Korwin-Zmijowski
  0 siblings, 0 replies; 9+ messages in thread
From: Jérémy Korwin-Zmijowski @ 2020-06-23  8:54 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: help-guix

Le samedi 20 juin 2020 à 15:40 +0200, Ricardo Wurmus a écrit :
> 
> Can you specify a different Docker image?  Then you could create one
> containing Guix with “guix pack -f docker”.
> 

I am not the owmer of the Gitlab instance, so I think I can't... But I
can ask to the admin haha. Worth the shot !

Thank you Ricardo !



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Gitlab CI
  2020-06-20 15:05 ` divoplade
@ 2020-06-23 10:15   ` Jérémy Korwin-Zmijowski
  2020-06-23 15:24     ` Jérémy Korwin-Zmijowski
  0 siblings, 1 reply; 9+ messages in thread
From: Jérémy Korwin-Zmijowski @ 2020-06-23 10:15 UTC (permalink / raw)
  To: divoplade, Guix Help

Hey divoplade !

Thank you very much for sharing your hack ! The caching section is very
interesting... Does it save time between runs ?

After a lot of attempts, I ended up with a green pipeline but it takes
43 minute to complete haha 
https://framagit.org/Jeko/guile-dummy/-/pipelines

Jérémy







^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Gitlab CI
  2020-06-23 10:15   ` Jérémy Korwin-Zmijowski
@ 2020-06-23 15:24     ` Jérémy Korwin-Zmijowski
  0 siblings, 0 replies; 9+ messages in thread
From: Jérémy Korwin-Zmijowski @ 2020-06-23 15:24 UTC (permalink / raw)
  To: divoplade, Guix Help

I tried to apply the caching as you did, but I error during project
configuration.

https://framagit.org/Jeko/guile-dummy/-/jobs/1029866

Jérémy

Le mardi 23 juin 2020 à 12:15 +0200, Jérémy Korwin-Zmijowski a écrit :
> Hey divoplade !
> 
> Thank you very much for sharing your hack ! The caching section is
> very
> interesting... Does it save time between runs ?
> 
> After a lot of attempts, I ended up with a green pipeline but it
> takes
> 43 minute to complete haha 
> https://framagit.org/Jeko/guile-dummy/-/pipelines
> 
> Jérémy
> 
> 
> 
> 
> 
> 



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Gitlab CI
  2020-06-20  8:42 Gitlab CI Jérémy Korwin-Zmijowski
  2020-06-20 13:40 ` Ricardo Wurmus
  2020-06-20 15:05 ` divoplade
@ 2020-06-25 12:32 ` zimoun
  2020-06-25 14:10   ` Jérémy Korwin-Zmijowski
  2020-06-25 21:32 ` divoplade
  3 siblings, 1 reply; 9+ messages in thread
From: zimoun @ 2020-06-25 12:32 UTC (permalink / raw)
  To: Jérémy Korwin-Zmijowski, Guix Help

Hi Jérémy,

On Sat, 20 Jun 2020 at 10:42, Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> wrote:

>  - printf '\n' | ./guix-install.sh

[...]

>    [1592641832.155]: [ FAIL ] unsupported init system; run the daemon
>    manually:
>       /root/.config/guix/current/bin/guix-daemon --build-users-
>    group=guixbuild
>    [1592641832.155]: [ INFO ] making the guix command available to
>    other users
>
> Does anyone has done that before ?

It is because the guix-install.sh script does not recognize the init
system of the Docker image, see 'sys_enable_guix_daemon()'.  Roughly
speaking, it does not fall into upstart) neither systemd) neither
sysv-init) so it falls into NA|*).

I do not know how the Docker image you use is built.

> Is there a way to get a Guix docker image instead of the Debian one ?

The easiest is, IMHO, instead of "./guix-install.sh", you should run all
the 8 steps described in [1] adapting the step #5.

1: https://guix.gnu.org/manual/devel/en/guix.html#Binary-Installation


Cheers,
simon


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Gitlab CI
  2020-06-25 12:32 ` zimoun
@ 2020-06-25 14:10   ` Jérémy Korwin-Zmijowski
  0 siblings, 0 replies; 9+ messages in thread
From: Jérémy Korwin-Zmijowski @ 2020-06-25 14:10 UTC (permalink / raw)
  To: zimoun, Guix Help

Hi Zimoun !

Thank you for taking time to help over here !

Thanks to divoplade, we succedded to make something up for relative
simple projects (I don't know how it goes for more complicated ones).
If anybody feel interested :
https://framagit.org/Jeko/guile-dummy

Cheers,
Jérémy



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Gitlab CI
  2020-06-20  8:42 Gitlab CI Jérémy Korwin-Zmijowski
                   ` (2 preceding siblings ...)
  2020-06-25 12:32 ` zimoun
@ 2020-06-25 21:32 ` divoplade
  3 siblings, 0 replies; 9+ messages in thread
From: divoplade @ 2020-06-25 21:32 UTC (permalink / raw)
  To: Jérémy Korwin-Zmijowski, Guix Help

[-- Attachment #1: Type: text/plain, Size: 1850 bytes --]

Hello Jérémy,

I have a new solution (it is quite different from the previous one) in
which it is possible to run guix commands within a job, while still
caching most of the files (see attached). I am sorry, my fork of your
project is already a mess so I prefer to put the file in its entirety
here.

Anyways, everything will be way easier once we have a proper debian
package for guix:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644

divoplade

Le samedi 20 juin 2020 à 10:42 +0200, Jérémy Korwin-Zmijowski a écrit :
> Hey Guixters !
> 
> I am experimenting with Gitlab CI for a Guile project I am working
> on.
> It is managed with Hall and I would like to benefit the Guix
> features.
> 
> Currently, I struggle to run Guix inside the default Debian stable
> docker image (framagit.org).
> 
> Here is my .gitlab-ci.yml :
> 
>    before_script:
>     - apt -qq update
>     - apt -qq install -y wget gpg xz-utils
>     - wget 
>    
> https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
>  - chmod +x guix-install.sh
>  - wget https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - |
> gpg --import -
>  - printf '\n' | ./guix-install.sh
>  - /root/.config/guix/current/bin/guix-daemon --build-users-
> group=guixbuild
>  - guix pull
> 
> stages:
>   - bootstrap
>   - configure
>   - build
>   - test
> 
> job:bootstrap:
>   stage: bootstrap
>   script: autoreconf -vfi
> 
> What I get :
> 
>    [1592641832.155]: [ FAIL ] unsupported init system; run the daemon
>    manually:
>       /root/.config/guix/current/bin/guix-daemon --build-users-
>    group=guixbuild
>    [1592641832.155]: [ INFO ] making the guix command available to
>    other users
> 
> Does anyone has done that before ?
> Is there a way to get a Guix docker image instead of the Debian one ?
> 
> Have a nice weekend !
> 
> Jérémy
> 
> 

[-- Attachment #2: .gitlab-ci.yml --]
[-- Type: application/x-yaml, Size: 2088 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, back to index

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-20  8:42 Gitlab CI Jérémy Korwin-Zmijowski
2020-06-20 13:40 ` Ricardo Wurmus
2020-06-23  8:54   ` Jérémy Korwin-Zmijowski
2020-06-20 15:05 ` divoplade
2020-06-23 10:15   ` Jérémy Korwin-Zmijowski
2020-06-23 15:24     ` Jérémy Korwin-Zmijowski
2020-06-25 12:32 ` zimoun
2020-06-25 14:10   ` Jérémy Korwin-Zmijowski
2020-06-25 21:32 ` divoplade

unofficial mirror of help-guix@gnu.org 

Archives are clonable:
	git clone --mirror https://yhetil.org/guix-user/0 guix-user/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 guix-user guix-user/ https://yhetil.org/guix-user \
		help-guix@gnu.org
	public-inbox-index guix-user

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.yhetil.org/yhetil.gnu.guix.user
	nntp://news.gmane.io/gmane.comp.gnu.guix.user


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git