Building a Docker image for GitLab-CI

Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hello Guix!

Has anyone succeeded in building a Docker image suitable for use in
GitLab-CI?  I haven’t.  Here’s what I tried.

Initially, I built an image with ‘guix system image -t docker …’ but
that doesn’t work because then the image’s “entry point” is shepherd,
but shepherd never returns.  Thus, GitLab-CI would spawn the image and
eventually time out.

So I tried this instead:

  guix pack guix bash-minimal coreutils-minimal grep net-base \
    --save-provenance -S /bin=bin -S /share=share -S /etc=etc \
    -f docker --max-layers=100

… with ‘.gitlab-ci.yml’ doing something like this:

--8<---------------cut here---------------start------------->8---
build:
  image: registry.gitlab.inria.fr/…
  tags: ["ci.inria.fr", "linux"]
  before_script:
    - echo "nameserver 10.0.2.3 # XXX" > /etc/resolv.conf
    - guix archive --authorize < /share/guix/ci.guix.gnu.org.pub
    - guix archive --authorize < /share/guix/bordeaux.guix.gnu.org.pub
    - guix-daemon --disable-chroot &
  script:
    - guix shell -m manifest.scm -- rubber --pdf article.tex
  artifacts:
    paths:
      - article.pdf
--8<---------------cut here---------------end--------------->8---

Problem is, name resolution appears to fail in the container image; the
‘resolv.conf’ trick was a crude attempt to work around it, but it
failed.  I guess the problem is that I don’t know how GitLab-CI or
Docker is supposed to set up networking inside those containers.

Thoughts?

Neat tip to upload your Guix-built image to a registry: use Skopeo.

  guix shell skopeo -- skopeo login registry.gitlab.inria.fr
  guix shell skopeo -- skopeo copy \
    docker-archive:///gnu/store/…-docker-image.tar.gz \
    docker://registry.gitlab.inria.fr/… \
    --insecure-policy

(“Insecure policy”, what could possibly go wrong?)

Ludo’.

Re: Building a Docker image for GitLab-CI

[Suhail]

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Initially, I built an image with ‘guix system image -t docker …’ but
> that doesn’t work because then the image’s “entry point” is shepherd,
> but shepherd never returns.

Did you try resetting the entrypoint in .gitlab-ci.yml using the
image:entrypoint keyword? [1]

[1]: <https://docs.gitlab.com/ee/ci/yaml/#imageentrypoint>

-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an
intention to enter into an agreement, has no operative effect until a
definitive agreement is signed in writing by both parties, and that no
party should act in reliance on the email or any representations of the
sender until a definitive agreement is signed in writing by both
parties.

Re: Building a Docker image for GitLab-CI

[Andreas Enge]

Am Tue, Feb 13, 2024 at 11:31:28AM +0100 schrieb Ludovic Courtès:
> Has anyone succeeded in building a Docker image suitable for use in
> GitLab-CI?  I haven’t.  Here’s what I tried.

A colleague of mine just found this:
   https://gitlab.com/daym/guix-on-docker/

:-)

Andreas

Re: Building a Docker image for GitLab-CI

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 638 bytes --]

On Tue, Feb 13, 2024 at 11:31:28AM +0100, Ludovic Courtès wrote:
> Hello Guix!
> 
> Has anyone succeeded in building a Docker image suitable for use in
> GitLab-CI?  I haven’t.  Here’s what I tried.

In the past I used a script to install guix using the shell script and
then ran guix pull before building my package.  I suppose you could use
a Debian image and run 'guix pull' first before building something.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hi,

Efraim Flashner <efraim@flashner.co.il> skribis:

> In the past I used a script to install guix using the shell script and
> then ran guix pull before building my package.  I suppose you could use
> a Debian image and run 'guix pull' first before building something.

I could… but that’d be cheating.  :-)

Ludo’.

Re: Building a Docker image for GitLab-CI

[Reza Housseini]

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

sorry forgot to include the list...

Hi Ludo

> Has anyone succeeded in building a Docker image suitable for use in
> GitLab-CI?

I normally do the following and it seems to work fine with our gitlab
instance:

registry=registry.gitlab.ost.ch:45023/sciceg/teaching/eeu_mlds
archive=$(guix time-machine -C channels.scm -- pack -f docker -S /bin=bin -S /lib=lib -S /share=share -m manifest.scm)
tag=$(docker load -i $archive)
docker tag ${tag##*"Loaded image: "} $registry
docker push $registry

what seems to be crucial is to add the following packages to the
manifest file:

"bash" 
"coreutils"
"git"

than I can use the image in the gitlab-ci.yml file

generate-exercises:
  image: registry.gitlab.ost.ch:45023/sciceg/teaching/eeu_mlds:latest
  ...

Hope that helps!

Best,
Reza

PS: It would be really nice if one could provide a docker tag directly
to `guix pack -f docker`

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hi Reza,

Reza Housseini <reza.housseini@gmail.com> skribis:

>> Has anyone succeeded in building a Docker image suitable for use in
>> GitLab-CI?
>
> I normally do the following and it seems to work fine with our gitlab
> instance:
>
> registry=registry.gitlab.ost.ch:45023/sciceg/teaching/eeu_mlds
> archive=$(guix time-machine -C channels.scm -- pack -f docker -S /bin=bin -S /lib=lib -S /share=share -m manifest.scm)
> tag=$(docker load -i $archive)
> docker tag ${tag##*"Loaded image: "} $registry
> docker push $registry
>
> what seems to be crucial is to add the following packages to the
> manifest file:
>
> "bash" 
> "coreutils"
> "git"

That’s nice, but unless I’m mistaken, Guix is missing from the image,
right?

My goal would be to be able to use Guix within the image, so I can have
GitLab-CI spawn ‘guix build’ commands (or similar).

> PS: It would be really nice if one could provide a docker tag directly
> to `guix pack -f docker`

I think it’s already possible, see ‘guix pack --help-docker-format’.

Thanks,
Ludo’.

Re: Building a Docker image for GitLab-CI

[Andreas Enge]

Hello,

Am Tue, Jun 04, 2024 at 01:29:22PM +0200 schrieb Ludovic Courtès:
> My goal would be to be able to use Guix within the image, so I can have
> GitLab-CI spawn ‘guix build’ commands (or similar).

with a colleague we have set up such a system. He has started from a Debian
image and written a docker script to install Guix "manually", which provides
our base image. Then Gitlab CI creates a new image from a channels file in
the git repository we want to monitor. The command we use is
"guix system image -t docker" and not "guix pack"; I am not really familiar
with the second command, and as far as I understand the first one has the
difference of running the shepherd as the docker command, which in our case
starts the guix-daemon and a guix-build-coordinator-agent.
(One addition: We use the patch in
   https://issues.guix.gnu.org/70933
to enable chroot in the container and then run the container in privileged
mode.)

Maybe we should write up a little blog post once everything is settled.

Andreas

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hi,

Andreas Enge <andreas@enge.fr> skribis:

> Am Tue, Jun 04, 2024 at 01:29:22PM +0200 schrieb Ludovic Courtès:
>> My goal would be to be able to use Guix within the image, so I can have
>> GitLab-CI spawn ‘guix build’ commands (or similar).
>
> with a colleague we have set up such a system. He has started from a Debian
> image and written a docker script to install Guix "manually", which provides
> our base image. Then Gitlab CI creates a new image from a channels file in
> the git repository we want to monitor. The command we use is
> "guix system image -t docker" and not "guix pack";

To be able to do that, you need to have Guix already up and running in
the Docker image you pass to GitLab-CI: that’s the problem I’m trying to
solve.

Or did I misunderstand?

(It’s possible to sidestep that problem for instance by having GitLab-CI
offload to a runner in a machine that you control, where Guix is
installed.  But I’m looking for a solution that could work
out-of-the-box on any GitLab-CI or similar instance.)

Thanks,
Ludo’.

Re: Building a Docker image for GitLab-CI

[Reza Housseini]

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> That’s nice, but unless I’m mistaken, Guix is missing from the image,
> right?
>
> My goal would be to be able to use Guix within the image, so I can have
> GitLab-CI spawn ‘guix build’ commands (or similar).

Oh sorry this was a misunderstanding from my side. Have you tried to use
the guix system image command to create a docker container?

Best,
Reza

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Reza Housseini <reza.housseini@gmail.com> skribis:

> Ludovic Courtès <ludovic.courtes@inria.fr> writes:
>
>> That’s nice, but unless I’m mistaken, Guix is missing from the image,
>> right?
>>
>> My goal would be to be able to use Guix within the image, so I can have
>> GitLab-CI spawn ‘guix build’ commands (or similar).
>
> Oh sorry this was a misunderstanding from my side. Have you tried to use
> the guix system image command to create a docker container?

I did, but that’s not suitable:

  https://mail.gnu.org/archive/html/help-guix/2024-02/msg00066.html

Ludo’.

Re: Building a Docker image for GitLab-CI

[Andreas Enge]

Am Thu, Jun 06, 2024 at 11:23:20AM +0200 schrieb Ludovic Courtès:
> (It’s possible to sidestep that problem for instance by having GitLab-CI
> offload to a runner in a machine that you control, where Guix is
> installed.  But I’m looking for a solution that could work
> out-of-the-box on any GitLab-CI or similar instance.)

It turns out that our Gitlab instance enables docker containers in
privileged mode, so it just happened to work in my case...

Andreas

Re: Building a Docker image for GitLab-CI

[Cayetano Santos]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]


I just setup a toy project, here:

    https://gitlab.com/csantosb/gitlabci

Here, I reproduce the steps to build a docker image using guix system
image, and set te entrypoint to "", as you can see.

Still, a piece in the puzzle is missing, ideas ?

    https://gitlab.com/csantosb/gitlabci/-/jobs/8648358846

--
Cayetano Santos
GnuPG Key:   https://meta.sr.ht/~csantosb.pgp
FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: Building a Docker image for GitLab-CI

[Cayetano Santos]

[-- Attachment #1: Type: text/plain, Size: 351 bytes --]


In the test-entry branch, I’m testing the trick in the guix-on-docker
repository, where the entrypoint is given a guix script.

    https://gitlab.com/csantosb/gitlabci/-/jobs/8648429390

Getting closer ...

--
Cayetano Santos
GnuPG Key:   https://meta.sr.ht/~csantosb.pgp
FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: Building a Docker image for GitLab-CI

[Simon Josefsson via]

[-- Attachment #1: Type: text/plain, Size: 6187 bytes --]

All,

I am trying to get a Guix container usable in GitLab, and thought I'd
share my status.  I have established working networking in the resulting
Guix container, which seems like progress (whoohoo!).  tl;dr:

https://gitlab.com/debdistutils/guix/container/-/jobs/8652014833

The problem seems to be that GitLab (I suppose they use docker?) picks
the wrong container file system from the resulting image.  It doesn't
use the same file system as a local 'podman run', see below.  Could this
be a 'guix pack' OCI container bug/problem?  Maybe 'guix pack' produces
too many blobs and GitLab/docker stops looking after while?  Or the
container configuration is incorrect?  Some bad interaction with 'podman
load -i'?  My OCI-fu is too weak here, help appreciated.

Here is my work:

https://gitlab.com/debdistutils/guix/container/
https://gitlab.com/debdistutils/guix/container/-/blob/main/.gitlab-ci.yml

(The system-* jobs are inspired by Cayetano's work, they aren't
necessary unless there is something wrong with the 'guix pack' approach
below...)

The idea is:

1) First job 'debian-with-guix' creates a Debian trixie container with
Guix and podman, and a 'guix pull'.  This is for caching, it takes 25
minutes to run on a large compute node, using a GitLab-local mirror of
Guix because Savannah access makes it even slower.

https://gitlab.com/debdistutils/guix/container/-/blob/main/debian-with-guix/Containerfile?ref_type=heads

https://gitlab.com/debdistutils/guix/container/-/jobs/8649009536

Output container (quite useful on its own):

registry.gitlab.com/debdistutils/guix/container:debian-with-guix

2) Second job 'pack' uses the 'debian-with-guix' image, runs a 'guix
pack' and creates a container using 'podman' and uploads it into:

registry.gitlab.com/debdistutils/guix/container:pack

I suspect the problem is happening in the 'guix pack' or 'podman load'
commands here.  Output from last successful run:

https://gitlab.com/debdistutils/guix/container/-/jobs/8649183646

3) Third job 'pack-test' tries to use the pack image in a GitLab job, as
a normal GitLab CI/CD build would work.

Job output is here:

https://gitlab.com/debdistutils/guix/container/-/jobs/8652014833

It fails with networking errors just like Ludo's earlier e-mail:

fping: icmp: unknown protocol

What is really weird is this root directory:

Using docker image sha256:57160f1c13ce56799d6e3e83dd97da4c929993ac008404ac38c67317cded25d1 for registry.gitlab.com/debdistutils/guix/container:pack with digest registry.gitlab.com/debdistutils/guix/container@sha256:be1ad3a7af69669cf3d138c6ec2b1201a64294aad33320246212c6689a1e5c9d ...
...
$ ls -la /etc
total 20
drwxr-xr-x 2 0 0 4096 Dec 16 10:15 .
drwxr-xr-x 1 0 0 4096 Dec 16 10:15 ..
-rw-r--r-- 1 0 0   46 Dec 16 10:15 hostname
-rw-r--r-- 1 0 0  283 Dec 16 10:15 hosts
lrwxrwxrwx 1 0 0   12 Dec 16 10:15 mtab -> /proc/mounts
-rw-r--r-- 1 0 0  841 Dec 16 10:15 resolv.conf

There is no /etc/protocols!  No wonder things doesn't work.

However if I run the same container locally, it looks different:

jas@kaka:~/src/guix-container$ podman images|grep pack
registry.gitlab.com/debdistutils/guix/container                   pack                  57160f1c13ce  54 years ago   1.23 GB
jas@kaka:~/src/guix-container$ podman run --entrypoint /bin/bash -it --rm registry.gitlab.com/debdistutils/guix/container:pack
bash-5.1# ls -la /etc
lrwxrwxrwx 1 0 0 55 Jan  1  1970 /etc -> /gnu/store/kcpr09dgqnr8q29d167cjk3wsmgizzkb-profile/etc
bash-5.1# ls -la /etc/
total 92
...
lrwxrwxrwx 1 0 0   70 Jan  1  1970 protocols -> /gnu/store/bfp25w47fxn8z0fdwj45prx2609sx59j-net-base-5.3/etc/protocols
...
bash-5.1# 

What is going on here?  Seems like GitLab docker is finding another
container file system compared to my local podman?  The hashes are the
same, so I should be working on the same file.

Does anyone have docker installed?  What does the equivalent of this
command do with docker?  Does it work like podman or GitLab?

podman run registry.gitlab.com/debdistutils/guix/container:pack ls -l /etc/

Networking seems to be set up fine, except the missing /etc/protocols:

$ ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
17: eth0@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc noqueue state UP group default 
    link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.17.0.3/16 brd 172.17.255.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fd76:5338:4685:1:0:242:ac11:3/64 scope global nodad 
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe11:3/64 scope link tentative 
       valid_lft forever preferred_lft forever
$ ip -4 route
default via 172.17.0.1 dev eth0 
172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.3 
$ ip -6 route
fd76:5338:4685:1::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fd76:5338:4685:1::1 dev eth0 metric 1024 pref medium
...
$ fping -c3 -D 130.237.72.201
fping: icmp: unknown protocol

Adding a small /etc/protocols makes networking work:

$ printf 'icmp\t1\tICMP\nipv6-icmp\t58\tIPv6-ICMP\n' > /etc/protocols
$ fping -c3 -D 130.237.72.201
[1734345646.83678] 130.237.72.201 : [0], 64 bytes, 112 ms (112 avg, 0% loss)
[1734345647.83701] 130.237.72.201 : [1], 64 bytes, 111 ms (112 avg, 0% loss)
[1734345648.83694] 130.237.72.201 : [2], 64 bytes, 111 ms (112 avg, 0% loss)
130.237.72.201 : xmt/rcv/%loss = 3/3/0%, min/avg/max = 111/112/112
$ fping -c3 -D gnu.org
[1734345648.90485] gnu.org : [0], 64 bytes, 25.1 ms (25.1 avg, 0% loss)
[1734345649.86358] gnu.org : [1], 64 bytes, 24.2 ms (24.6 avg, 0% loss)
[1734345650.86371] gnu.org : [2], 64 bytes, 24.3 ms (24.5 avg, 0% loss)
gnu.org : xmt/rcv/%loss = 3/3/0%, min/avg/max = 24.2/24.5/25.1

/Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: Building a Docker image for GitLab-CI

[Andreas Enge]

Hello Simon,

Am Mon, Dec 16, 2024 at 11:42:34AM +0100 schrieb Simon Josefsson via:
> I am trying to get a Guix container usable in GitLab, and thought I'd
> share my status.  I have established working networking in the resulting
> Guix container, which seems like progress (whoohoo!).  tl;dr:

at work we are using gitlab CI to build guix docker containers and run a
node on openshift for the bordeaux build farm:
   https://gitlab.inria.fr/enge/plm-guix
The README.md is completely outdated and serves mainly as a reminder to
myself on how this docker thing works; every time I look at it after a
break of a few months I have forgotten how to use a docker container...

And of course I have already forgotten the details; probably we should
write a little blog post. I will talk about it with my colleague when I
meet him next year ;-)

We also start with a Debian image and use a Dockerfile to install Guix
in it, as described in the Guix manual. Then for CI, we use this fixed
docker image to create a new one every time our repository (with a
channels.scm file and the plmshift.scm OS configuration file) changes.
In our case, this second docker image is the artefact that we then deploy.
We use "docker in docker" to create the images, and if I understood
correctly, this requires some privileges; these may not be given on
gitlab.com, but are available in our self-hosted instance.

Andreas

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hi Simon,

Simon Josefsson <simon@josefsson.org> skribis:

> https://gitlab.com/debdistutils/guix/container/-/blob/main/.gitlab-ci.yml

Yay, great that you went this far!

> It fails with networking errors just like Ludo's earlier e-mail:
>
> fping: icmp: unknown protocol
>
> What is really weird is this root directory:
>
> Using docker image sha256:57160f1c13ce56799d6e3e83dd97da4c929993ac008404ac38c67317cded25d1 for registry.gitlab.com/debdistutils/guix/container:pack with digest registry.gitlab.com/debdistutils/guix/container@sha256:be1ad3a7af69669cf3d138c6ec2b1201a64294aad33320246212c6689a1e5c9d ...
> ...
> $ ls -la /etc
> total 20
> drwxr-xr-x 2 0 0 4096 Dec 16 10:15 .
> drwxr-xr-x 1 0 0 4096 Dec 16 10:15 ..
> -rw-r--r-- 1 0 0   46 Dec 16 10:15 hostname
> -rw-r--r-- 1 0 0  283 Dec 16 10:15 hosts
> lrwxrwxrwx 1 0 0   12 Dec 16 10:15 mtab -> /proc/mounts
> -rw-r--r-- 1 0 0  841 Dec 16 10:15 resolv.conf
>
> There is no /etc/protocols!  No wonder things doesn't work.

And that’s in spite of you running ‘guix pack … net-base -S /etc=etc’.

Could it be that something in podman/Docker/GitLab-CI overrides /etc, or
overrides it specifically because it’s a symlink?  I’m not sure where to
look for that.

Ludo’.

Re: Building a Docker image for GitLab-CI

[Simon Josefsson via]

[-- Attachment #1: Type: text/plain, Size: 5858 bytes --]

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

>> What is really weird is this root directory:
>>
>> Using docker image
>> sha256:57160f1c13ce56799d6e3e83dd97da4c929993ac008404ac38c67317cded25d1
>> for registry.gitlab.com/debdistutils/guix/container:pack with digest
>> registry.gitlab.com/debdistutils/guix/container@sha256:be1ad3a7af69669cf3d138c6ec2b1201a64294aad33320246212c6689a1e5c9d
>> ...
>> ...
>> $ ls -la /etc
>> total 20
>> drwxr-xr-x 2 0 0 4096 Dec 16 10:15 .
>> drwxr-xr-x 1 0 0 4096 Dec 16 10:15 ..
>> -rw-r--r-- 1 0 0   46 Dec 16 10:15 hostname
>> -rw-r--r-- 1 0 0  283 Dec 16 10:15 hosts
>> lrwxrwxrwx 1 0 0   12 Dec 16 10:15 mtab -> /proc/mounts
>> -rw-r--r-- 1 0 0  841 Dec 16 10:15 resolv.conf
>>
>> There is no /etc/protocols!  No wonder things doesn't work.
>
> And that’s in spite of you running ‘guix pack … net-base -S /etc=etc’.
>
> Could it be that something in podman/Docker/GitLab-CI overrides /etc, or
> overrides it specifically because it’s a symlink?  I’m not sure where to
> look for that.

Yes it seems like a GitLab/docker-specific problem since I don't get the
same /etc when running the generated image locally.

There is this interesting entry in /proc/mounts:

https://gitlab.com/debdistutils/guix/container/-/jobs/8652014833#L343

overlay / overlay rw,relatime,lowerdir=l/ZPVAK6UICUAWUUE4GUD6AYFCEM:l/HI6HL2SJWTZDQHPTA3SGR4PWNE:l/OAJJQ5NKJAJLIAOEWBNLI6WRNC:l/MRDSZ2V6PLTEQGMEDSOPX6FKEY:l/FG4SISAU6TZNHB6CQR5X5GNEJB:l/EZGDP6A5CMVPA5O6IKOOKPDMBE:l/DA5NZCY6NVIGU2X6U5XQQXV54M:l/P4MIVQ3I7VCYFTQ3AG6RCXZVW5:l/FGPCINKKCYRDHZI5BAAU7HEETW:l/MUYJFZRJLR4Z3BNBFRBRTGP4S5:l/UPGZHVDAILBRLLEH6T5RKWIFG7:l/YNTBNOTPU7QRK5K6RD63VSXG5Q:l/XUPHFGGN36OPHNU334M3V7HDXP:l/PVA2QRQE4D5MAKI6BMGVPETNZN:l/VTGADKUDL4KA4HPA2YCCQ2JQNI:l/WZHWY243PTUZTQIZJM26PVSYG4:l/7YLLGUSIRSUFPXIFI57F2UQSFQ:l/ZEOCYJR44JGRBRKGEM4AMWUHQE:l/YXFWBRXLFSBMYDDCTNODGJWWUV:l/NQK5YT5BWDWSTKFRB3KOGVGYLC:l/6CC2D5S3LSZOOKLULC2JJ5BUHG:l/OMQ7M7FSULZ2WHTQPIOIYP7HYQ:l/VRGMNYJNRPOBPP5IZCJR3YV7FP:l/5L2O6RAVRTGGTB7I2YKS6RMX64:l/57QINIJWW7ECMX3DKLCDL5UMUS:l/HTIS4VRXRVO24AWC6AYPQ4LPRG:l/DOTULUURRI6Z4XR2B4LPQTP33T:l/LZYLE6JUKQFJBIAEMBQMQIWZEE:l/2WEGBKQG6D3VAWLXJ5NCZPFGNP:l/QYVPGN6K2A3Y4VVVPKYLR5JVL7:l/IPS4YGXCRZ47O4AOKMZ4TYD2N3:l/2MZU46ZBHYS5IQI4NAVFD3PNYR:l/HW6B6GDN33YB7GBY3LEOW2XXB4:l/FRINUFWGYICMVPLOJULIHQ3XKV:l/HSHVRY3DQIT5LUS3EONQQCKNL3:l/CRUDGYNQRSSDKB4TYALBBVFIL3:l/QJRZZB6NOMXWO46YCAJ4U53VSH:l/BZPO5MYEYX3YFX5NXYR32E6VRM:l/FDWSYXJR7RNKG42AMXGSC6NUQE:l/Z6BTLASGE6ZXGQZUFIHG4QXQLU:l/DWTG7Y6N4DNP5AZLI6MIDZEBHQ:l/TY4DSPRKETTLFE5WB7KFBO2VSM:l/SAE7PNZP6FFK2NVDOQQTZMO2BV:l/ZCL7CRORSVNYVFKNW2WT7TMGFZ:l/KVH3MQTKJ6SH46B2FEHW7UCYGK:l/XMDKZU7KD755BFINQOBKPLKMZQ:l/MFKZPIVLWDKG5PIVI3UQUU3ILT:l/I24GPDX2SRV3Y4YSJDYVKEBDQE:l/W6OZHVZW2NCSQOJEGMP45P2D6W:l/ZD4RI6B4WQ65QO7EMDQZSCZFHS:l/ZS7D35LTVLE6NSGCCY4SQQANE5:l/2ZIJ6PAUHDPBOXR72A6HGU6L4B:l/ZL5XOQ6XMF6ZYQSXVAK7744SX7:l/A3WXIZD22NL62JYHBZVL5K36IK:l/KUDHAVWVBDXKHHQQVF33KDPHF4:l/6O5OF37ORZTEUV2JXNOPO3WNTQ:l/YVGTC6PVMDS2GVOF57TJWCN5DM:l/VYM3JKFOWAY7UIYGU6TPJTZVD5:l/2BT7MWUS2JMZMQXQ3MRLO5AH6I:l/CNZTAOCVMGIFCNP4IF77OQ5MU6:l/KQCUQ7H6EY7U423TWBX6N6QNU7:l/VNZNN2P4U26XHEDRSNIQ656GRE:l/7YG6BVIVDJYCCGLISSU4APAR4S:l/ELR3M2R3NLUI4U2YCKYKG6MWUV:l/PK34AGBR7JY4PUJIVEAO4J7UAK:l/ELIDWT3IMYDRR5L5VTA3REN3LH:l/DK7VEQCTNIWW4BOYCOVXZX2GQY:l/6BCYXFR5B6S3EYCMVCXKQPEP3M:l/GAF6PUKMPKABSXDZCMVE3NM2K5:l/ASVMZMXSKHAVGH5UFXXRFE7TUX:l/25QORLMIGZEEIBQGTV6UBNZEAH:l/FH5SA2MBXXRRAMDYI72FPK7RXU:l/77IRT3TXX3H7XH66YGR7O5AIYK:l/FIQQSP7XQLUH3IWBXF4DZXWTFN:l/NG6ZAASCTOH6SQVUBR2FR6YE4T:l/QAHEWNHBILTWWWJ4QMX4ISWIT4:l/VCSGZSH4SQRVHK4EWSLGFECG2S:l/7FAMLBB6DJW7VWYEOIN6FLBI2E:l/C4ZXSOLVY36PYMTRL2E2YIKTKY:l/NVH5IIZJ5SO422GGMFGNTFCAOA:l/E6EE3L3EB2B36E5A5PL5KF32FL:l/RPXH632CD3Y54UYAMMULLEZOE4:l/662KE3GHNLWZEUTFCRZXOKVA3P:l/3ZDSIX7ZFSLT2FIEYD6TB6GA7R:l/ZS7SJOXF7XF6OFTLVHYMRYZANK:l/F3CEOKDDPOK72QLUWGKHDIJTPG:l/65CFVU5ZFM4XVSTLDO2WGQE3GU:l/3C2OJ5ZVICH5QP733HTS2DHPJM:l/B37PUETDZKC3MUZZX6CT3M6ZEC:l/LTVIXAHIS7O45NFIVNPDVGYMR4:l/GGBS4BWCAD6UMV3755MJ3BQRI5:l/4XGXVIO5QQ2ARZTTG74M4UDK4V:l/X673P4Q5TGPWBLQIXOQ5JXKG2C:l/3K2Q6NGZ5EEIKCFBRMEWWZFWQK:l/ALTG2QI6YZVKXIP4UCRDMQWS5Q:l/P4AE6X6G5GK66GHHRBZ27BTBF3:l/2UEJWLKTB2GNGT2G6UWFZVDFRN:l/OCTYS4BHPFX4HC3OXJJVGABEPX:l/M766VP6Y5QL2OGI6EYO5OEB6H4:l/DNITYWJFFEVGWQPFQ37S57EIB5:l/PQE5T2ER2OKPNC2YNHZEJXVIY6,upperdir=98dac307f50ae5da4c8f2cc5fdf024465f63600e30fc3bb434fca31191e2efdc/diff,workdir=98dac307f50ae5da4c8f2cc5fdf024465f63600e30fc3bb434fca31191e2efdc/work 0 0

Could those entries correspond to blobs generated by 'guix pack'?

Compare

https://gitlab.com/debdistutils/guix/container/-/jobs/8649183646#L130

$ podman load -i /gnu/store/*-docker-pack.tar.gz
Getting image source signatures
Copying blob sha256:26c7e7107d11a712095a4bf12ff26c8f39fb86c347af15ad50bec4d9536a4144
Copying blob sha256:3c21466d0d8255e7a1dfcbc206c891fdf6cdd6241f461cdb038ca6ef7b508bce
...
Copying blob sha256:b6ead463213fb7ec39911848da3b34e404ed184ee48373737c8b2eb2abd0730a
Copying blob sha256:72ca44ded2f166add396cc6a890a5e8a19c182603e4528a9bfef3012ab59b6b5
Copying blob sha256:092becdd45260f6b3d07626b2a39a738cc0a2d5a1c9f4a000cd61e762da8fe1b
Copying blob sha256:5397467e8b6a3b911d0e61f722622b995850be0721121a8e36b26ca7037b2622
Copying blob sha256:e3646d587f1665642b0077dc27a60b3dfed78dc21aa99eca4391b56c754f4aa7
Copying blob sha256:bff8143cb75389795d58b8a9ddbb572496e2e2ea1369a7f025a6b6e15b3a8074
Copying config sha256:57160f1c13ce56799d6e3e83dd97da4c929993ac008404ac38c67317cded25d1
Writing manifest to image destination
Loaded image: localhost/guix-bash-minimal-coreutils-minimal-git:latest

Notice that there are many more 'Copying blob' lines than overlay mounts
above.  Are we just seeing overlayfs mount truncation here?

Is it possible to make 'guix pack' create a merged container instead of
all these layers?

I'll experiment a bit more...

/Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Simon Josefsson <simon@josefsson.org> skribis:

> Yes it seems like a GitLab/docker-specific problem since I don't get the
> same /etc when running the generated image locally.
>
> There is this interesting entry in /proc/mounts:
>
> https://gitlab.com/debdistutils/guix/container/-/jobs/8652014833#L343
>
> overlay / overlay rw,relatime,lowerdir=l/ZPVAK6UICUAWUUE4GUD6AYFCEM:l/HI6HL2SJWTZDQHPTA3SGR4PWNE:l/OAJJQ5NKJAJLIAOEWBNLI6WRNC:l/MRDSZ2V6PLTEQGMEDSOPX6FKEY:l/FG4SISAU6TZNHB6CQR5X5GNEJB:l/EZGDP6A5CMVPA5O6IKOOKPDMBE:l/DA5NZCY6NVIGU2X6U5XQQXV54M:l/P4MIVQ3I7VCYFTQ3AG6RCXZVW5:l/FGPCINKKCYRDHZI5BAAU7HEETW:l/MUYJFZRJLR4Z3BNBFRBRTGP4S5:l/UPGZHVDAILBRLLEH6T5RKWIFG7:l/YNTBNOTPU7QRK5K6RD63VSXG5Q:l/XUPHFGGN36OPHNU334M3V7HDXP:l/PVA2QRQE4D5MAKI6BMGVPETNZN:l/VTGADKUDL4KA4HPA2YCCQ2JQNI:l/WZHWY243PTUZTQIZJM26PVSYG4:l/7YLLGUSIRSUFPXIFI57F2UQSFQ:l/ZEOCYJR44JGRBRKGEM4AMWUHQE:l/YXFWBRXLFSBMYDDCTNODGJWWUV:l/NQK5YT5BWDWSTKFRB3KOGVGYLC:l/6CC2D5S3LSZOOKLULC2JJ5BUHG:l/OMQ7M7FSULZ2WHTQPIOIYP7HYQ:l/VRGMNYJNRPOBPP5IZCJR3YV7FP:l/5L2O6RAVRTGGTB7I2YKS6RMX64:l/57QINIJWW7ECMX3DKLCDL5UMUS:l/HTIS4VRXRVO24AWC6AYPQ4LPRG:l/DOTULUURRI6Z4XR2B4LPQTP33T:l/LZYLE6JUKQFJBIAEMBQMQIWZEE:l/2WEGBKQG6D3VAWLXJ5NCZPFGNP:l/QYVPGN6K2A3Y4VVVPKYLR5JVL7:l/IPS4YGXCRZ47O4AOKMZ4TYD2N3:l/2MZU46ZBHYS5IQI4NAVFD3PNYR:l/HW6B6GDN33YB7GBY3LEOW2XXB4:l/FRINUFWGYICMVPLOJULIHQ3XKV:l/HSHVRY3DQIT5LUS3EONQQCKNL3:l/CRUDGYNQRSSDKB4TYALBBVFIL3:l/QJRZZB6NOMXWO46YCAJ4U53VSH:l/BZPO5MYEYX3YFX5NXYR32E6VRM:l/FDWSYXJR7RNKG42AMXGSC6NUQE:l/Z6BTLASGE6ZXGQZUFIHG4QXQLU:l/DWTG7Y6N4DNP5AZLI6MIDZEBHQ:l/TY4DSPRKETTLFE5WB7KFBO2VSM:l/SAE7PNZP6FFK2NVDOQQTZMO2BV:l/ZCL7CRORSVNYVFKNW2WT7TMGFZ:l/KVH3MQTKJ6SH46B2FEHW7UCYGK:l/XMDKZU7KD755BFINQOBKPLKMZQ:l/MFKZPIVLWDKG5PIVI3UQUU3ILT:l/I24GPDX2SRV3Y4YSJDYVKEBDQE:l/W6OZHVZW2NCSQOJEGMP45P2D6W:l/ZD4RI6B4WQ65QO7EMDQZSCZFHS:l/ZS7D35LTVLE6NSGCCY4SQQANE5:l/2ZIJ6PAUHDPBOXR72A6HGU6L4B:l/ZL5XOQ6XMF6ZYQSXVAK7744SX7:l/A3WXIZD22NL62JYHBZVL5K36IK:l/KUDHAVWVBDXKHHQQVF33KDPHF4:l/6O5OF37ORZTEUV2JXNOPO3WNTQ:l/YVGTC6PVMDS2GVOF57TJWCN5DM:l/VYM3JKFOWAY7UIYGU6TPJTZVD5:l/2BT7MWUS2JMZMQXQ3MRLO5AH6I:l/CNZTAOCVMGIFCNP4IF77OQ5MU6:l/KQCUQ7H6EY7U423TWBX6N6QNU7:l/VNZNN2P4U26XHEDRSNIQ656GRE:l/7YG6BVIVDJYCCGLISSU4APAR4S:l/ELR3M2R3NLUI4U2YCKYKG6MWUV:l/PK34AGBR7JY4PUJIVEAO4J7UAK:l/ELIDWT3IMYDRR5L5VTA3REN3LH:l/DK7VEQCTNIWW4BOYCOVXZX2GQY:l/6BCYXFR5B6S3EYCMVCXKQPEP3M:l/GAF6PUKMPKABSXDZCMVE3NM2K5:l/ASVMZMXSKHAVGH5UFXXRFE7TUX:l/25QORLMIGZEEIBQGTV6UBNZEAH:l/FH5SA2MBXXRRAMDYI72FPK7RXU:l/77IRT3TXX3H7XH66YGR7O5AIYK:l/FIQQSP7XQLUH3IWBXF4DZXWTFN:l/NG6ZAASCTOH6SQVUBR2FR6YE4T:l/QAHEWNHBILTWWWJ4QMX4ISWIT4:l/VCSGZSH4SQRVHK4EWSLGFECG2S:l/7FAMLBB6DJW7VWYEOIN6FLBI2E:l/C4ZXSOLVY36PYMTRL2E2YIKTKY:l/NVH5IIZJ5SO422GGMFGNTFCAOA:l/E6EE3L3EB2B36E5A5PL5KF32FL:l/RPXH632CD3Y54UYAMMULLEZOE4:l/662KE3GHNLWZEUTFCRZXOKVA3P:l/3ZDSIX7ZFSLT2FIEYD6TB6GA7R:l/ZS7SJOXF7XF6OFTLVHYMRYZANK:l/F3CEOKDDPOK72QLUWGKHDIJTPG:l/65CFVU5ZFM4XVSTLDO2WGQE3GU:l/3C2OJ5ZVICH5QP733HTS2DHPJM:l/B37PUETDZKC3MUZZX6CT3M6ZEC:l/LTVIXAHIS7O45NFIVNPDVGYMR4:l/GGBS4BWCAD6UMV3755MJ3BQRI5:l/4XGXVIO5QQ2ARZTTG74M4UDK4V:l/X673P4Q5TGPWBLQIXOQ5JXKG2C:l/3K2Q6NGZ5EEIKCFBRMEWWZFWQK:l/ALTG2QI6YZVKXIP4UCRDMQWS5Q:l/P4AE6X6G5GK66GHHRBZ27BTBF3:l/2UEJWLKTB2GNGT2G6UWFZVDFRN:l/OCTYS4BHPFX4HC3OXJJVGABEPX:l/M766VP6Y5QL2OGI6EYO5OEB6H4:l/DNITYWJFFEVGWQPFQ37S57EIB5:l/PQE5T2ER2OKPNC2YNHZEJXVIY6,upperdir=98dac307f50ae5da4c8f2cc5fdf024465f63600e30fc3bb434fca31191e2efdc/diff,workdir=98dac307f50ae5da4c8f2cc5fdf024465f63600e30fc3bb434fca31191e2efdc/work 0 0
>
> Could those entries correspond to blobs generated by 'guix pack'?

Not directly (the generated image doesn’t mount anything by itself), but
it could correspond to the image layers maybe?

> Is it possible to make 'guix pack' create a merged container instead of
> all these layers?

Maybe you could try different values of ‘guix pack --max-layers=…’?

When ‘--max-layers’ is omitted, the resulting image contains a single
layer AIUI (according to the default value of
‘%docker-image-max-layers’).

Ludo’.

Re: Building a Docker image for GitLab-CI

[Simon Josefsson via]

[-- Attachment #1: Type: text/plain, Size: 4067 bytes --]

I am happy to announce Guix container images:

https://gitlab.com/debdistutils/guix/container/

They are suitable for use in GitLab pipelines.

There are many things to continue discuss and resolve.  However it is
now possible to start a GitLab pipeline job that uses an 'image:'
pointing to a container file that were created by `guix pack` and it has
network access.  Andreas Enge posted a link on how to achieve this with
non-free 'docker' and some custom hosted gitlab runners with extra
permissions, but my variant uses only free software and the images can
be reproduced solely on the GitLab public runners (i.e., no image
uploading from a local laptop).  As far as I know this hasn't been done
before.

Successful example job that runs a Guix environment:

https://gitlab.com/debdistutils/guix/container/-/jobs/8670483694

Start the container like this:

sudo apt-get install podman || guix package -i podman
podman run -it registry.gitlab.com/debdistutils/guix/container:latest

The README mentions a few works that you will appreciate -- can you
suggest ways to resolve these matters?

- `export HOME=/`

- `export CWD=/`

- `guix-daemon --disable-chroot &`

- `GUIX_PROFILE=/root/.config/guix/current; . "$GUIX_PROFILE/etc/profile"`

- guix package -i fails: `guix perform-download: error: refusing to
  run with elevated privileges (UID 0)`

- GitLab pipeline job entrypoints: three possible entry-point usages
  behave somewhat different depending on how `guix pack` was invoked

- Adding `nss-certs` to the `guix pack` command breaks: `(symlink
  "NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny.pem" #) Throw to key
  encoding-error' with args ("scm_to_stringn" "cannot convert wide
  string to output locale" 84 #f #f)'.`

Those things aside, the main problem right now is that I would expect
'guix package -i gcc automake autoconf' to work.  However as you can see
in the `test-amd64-package-install` job things fail to download
seamingly from an intentional privilege check:

guix perform-download: error: refusing to run with elevated privileges (UID 0)

https://gitlab.com/debdistutils/guix/container/-/jobs/8670514009

What is the best way to resolve this?  Ugly insecure workarounds are
acceptable too, I didn't find any --insecure or similar.

It would be nice to publish a couple of different images, one minimal
and one more complete with, say, gcc on it.  Thoughts on what packages
to include and how to name the images?

Doing arm64 images should be straight forward but I get an error running
'guix pull' in the Debian image: https://issues.guix.gnu.org/74925 --
Also I do not know how to push a combined amd64+arm64 image like Debian
do with the 'debian:trixie' image doing the right thing for both amd64
and arm64 under the same name.  Ideas?

Can someone try to use them in some other environments, maybe GitHub
actions or Codeberg Woodpecker?

Finally, you may wonder why things didn't work before.  Some of the
major reasons: 1) --max-layer=100 and 2) -S /etc=etc and 3) Missing
/etc/protocols etc.  GitLab's docker setup doesn't handle many layers,
and it happens to just mount a sub-set of layers (see mount output,
missing a lot of layers).  Which files are put at which layer seems to
vary between `guix pack` runs for some reason, making it really hard to
debug (sometimes things worked partially, sometimes not, depending on
which files ended up visible).  I use --max-layers=8 now.  Re /etc=etc
it seems GitLab's docker setup bind-mounts things below /etc/ and it
cannot handle the root /etc symlink.  A workaround is to use `lndir`
which I use in the `test-amd64-package-install` job.  This is limitation
of GitLab's docker setup: I tried running a `-S /etc=etc` image on my
own GitLab runner based on Trisquel [1] and it worked fine, it mounted
things below the symlinked tree properly.  Could `guix pack` be teached
how to do a lndir-approach for /etc instead of symlink, perhaps?

Happy hacking,
/Simon

[1] https://gitlab.com/debdistutils/debdistreproduce/-/blob/main/gitlab-runner-with-podman-on-trisquel-aramo.md?ref_type=heads

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: Building a Docker image for GitLab-CI

[Simon Josefsson via]

[-- Attachment #1: Type: text/plain, Size: 4359 bytes --]

Andreas Enge <andreas@enge.fr> writes:

> Hello Simon,
>
> Am Mon, Dec 16, 2024 at 11:42:34AM +0100 schrieb Simon Josefsson via:
>> I am trying to get a Guix container usable in GitLab, and thought I'd
>> share my status.  I have established working networking in the resulting
>> Guix container, which seems like progress (whoohoo!).  tl;dr:
>
> at work we are using gitlab CI to build guix docker containers and run a
> node on openshift for the bordeaux build farm:
>    https://gitlab.inria.fr/enge/plm-guix
> The README.md is completely outdated and serves mainly as a reminder to
> myself on how this docker thing works; every time I look at it after a
> break of a few months I have forgotten how to use a docker container...
>
> And of course I have already forgotten the details; probably we should
> write a little blog post. I will talk about it with my colleague when I
> meet him next year ;-)
>
> We also start with a Debian image and use a Dockerfile to install Guix
> in it, as described in the Guix manual. Then for CI, we use this fixed
> docker image to create a new one every time our repository (with a
> channels.scm file and the plmshift.scm OS configuration file) changes.
> In our case, this second docker image is the artefact that we then deploy.
> We use "docker in docker" to create the images, and if I understood
> correctly, this requires some privileges; these may not be given on
> gitlab.com, but are available in our self-hosted instance.

Hi Andreas!  This all sounds quite similar to what I'm doing, although
using a different software stack.  I'm reading through your work now,
after actually finishing my work which I've announced here:

https://blog.josefsson.org/2024/12/18/guix-container-images-for-gitlab-ci-cd/
https://gitlab.com/debdistutils/guix/container

Looking into details, it seems you run this command to create the image:

https://gitlab.inria.fr/enge/plm-guix/-/blob/bf87f970c316f20cea2cf80f2511a280b5a71ed8/.gitlab-ci.yml#L44

docker run --privileged -v ./config:/config $CI_REGISTRY_IMAGE/builder:latest sh -c 'cd /config && /guix-daemon.sh guix time-machine -C channels.scm -- system image -t docker plmshift.scm  >/dev/null 2>&1 && cat /gnu/store/*docker-image.tar.gz' > image/docker-image.tar.gz
docker load -i image/docker-image.tar.gz

Your docker file is here:

https://gitlab.inria.fr/enge/plm-guix/-/blob/master/docker/Dockerfile?ref_type=heads

The guix-daemon.sh script is here:

https://gitlab.inria.fr/enge/plm-guix/-/blob/master/docker/guix-daemon.sh?ref_type=heads

Your plmshift.scm file is here:

https://gitlab.inria.fr/enge/plm-guix/-/blob/master/config/plmshift.scm?ref_type=heads

For comparison, I'm creating the image like this:

https://gitlab.com/debdistutils/guix/container/-/blob/main/.gitlab-ci.yml?ref_type=heads#L61

GUIX_PACKS_SLIM: guix bash-minimal coreutils-minimal net-base lndir
GUIX_PACKS_LATEST: $GUIX_PACKS_SLIM git-minimal findutils diffutils gcc-toolchain make automake autoconf tar grep sed gawk m4 gzip xz bzip2 iproute2 inetutils libcap shadow wget nss-certs
...
pack=$(guix pack $GUIX_PACKS --save-provenance -S /bin=bin -S /share=share -f docker --image-tag=guix --max-layers=8 --verbosity=0)
podman load -i $pack

My containerfile is here:

https://gitlab.com/debdistutils/guix/container/-/blob/main/debian-with-guix/Containerfile?ref_type=heads

Some of the stuff you resolve by using guix-daemon.sh and guix system
image on the plmshift.scm I instead push onto the consumer of my work,
as in these instructions:

https://gitlab.com/debdistutils/guix/container#how-to-use

One difference is that you are using your previous image as a basis for
the next one, which means you are using native Guix to build the image,
whereas I'm using Debian+Guix to build the image.  There is no
fundamental reason for this in my approach, so I opened an issue about
it:

https://gitlab.com/debdistutils/guix/container/-/issues/1

One more fundamental issue is that you are using 'guix system image' and
I was inspired by Ludo's e-mail and used 'guix pack'.  My experiments
with system images were that they ended up being larger, but maybe that
can be resolved by removing stuff.  Are there any general thoughts on
which is better to use?  Guix system vs Guix pack?  I kind of like the
idea of adding on top off guix-pack rather than removing from
guix-system.

/Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: Building a Docker image for GitLab-CI

[Cayetano Santos via]

n<#secure method=pgpmime mode=sign>

Hi Andreas,

  Nice work ! I’ve been playing a bit with it, but failed miserably to
  use your image, as you can see:

    https://gitlab.com/csantosb/gitlabci/-/jobs/8681019372

  This is my dummy yaml file:

    https://gitlab.com/csantosb/gitlabci/-/blob/test-entry/.gitlab-ci.yml

  Do you have an example file on how to use the image ?

Thanks !

--
Cayetano Santos
GnuPG Key:   https://meta.sr.ht/~csantosb.pgp
FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

Re: Building a Docker image for GitLab-CI

[Ludovic Courtès]

Hi Simon!

Simon Josefsson <simon@josefsson.org> skribis:

> I am happy to announce Guix container images:
>
> https://gitlab.com/debdistutils/guix/container/
>
> They are suitable for use in GitLab pipelines.

Yay!

> - guix package -i fails: `guix perform-download: error: refusing to
>   run with elevated privileges (UID 0)`

Should be fixed by running guix-daemon with
“--build-users-group=whatever” so that downloads run as one of the build
users, not as root.

> - GitLab pipeline job entrypoints: three possible entry-point usages
>   behave somewhat different depending on how `guix pack` was invoked

This image uses Guix installed on Debian though, no?

> - Adding `nss-certs` to the `guix pack` command breaks: `(symlink
>   "NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny.pem" #) Throw to key
>   encoding-error' with args ("scm_to_stringn" "cannot convert wide
>   string to output locale" 84 #f #f)'.`

You need to run guix-daemon in a UTF-8 locale like “C.UTF-8” (which is
supported out-of-the-box).

See also <https://issues.guix.gnu.org/75007>.

> Finally, you may wonder why things didn't work before.  Some of the
> major reasons: 1) --max-layer=100 and 2) -S /etc=etc and 3) Missing
> /etc/protocols etc.  GitLab's docker setup doesn't handle many layers,
> and it happens to just mount a sub-set of layers (see mount output,
> missing a lot of layers).  Which files are put at which layer seems to
> vary between `guix pack` runs for some reason, making it really hard to
> debug (sometimes things worked partially, sometimes not, depending on
> which files ended up visible).  I use --max-layers=8 now.

Interesting.  Did you find points in the code (Docker? GitLab?) about
this subset of layers being mounted?

> Re /etc=etc it seems GitLab's docker setup bind-mounts things below
> /etc/ and it cannot handle the root /etc symlink.  A workaround is to
> use `lndir` which I use in the `test-amd64-package-install` job.  This
> is limitation of GitLab's docker setup: I tried running a `-S
> /etc=etc` image on my own GitLab runner based on Trisquel [1] and it
> worked fine, it mounted things below the symlinked tree properly.
> Could `guix pack` be teached how to do a lndir-approach for /etc
> instead of symlink, perhaps?

It could symlink individual files and make /etc a directory.  (What’s
‘lndir’, if I may ask?)

Thanks for the investigation and all!

Ludo’.

Re: Building a Docker image for GitLab-CI

[Simon Josefsson via]

[-- Attachment #1: Type: text/plain, Size: 7133 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

>> - guix package -i fails: `guix perform-download: error: refusing to
>>   run with elevated privileges (UID 0)`
>
> Should be fixed by running guix-daemon with
> “--build-users-group=whatever” so that downloads run as one of the build
> users, not as root.

Yes, I discovered that too.

>> - GitLab pipeline job entrypoints: three possible entry-point usages
>>   behave somewhat different depending on how `guix pack` was invoked
>
> This image uses Guix installed on Debian though, no?

No - the resulting image is purely from 'guix pack', no Debian in the
final image at all.

I didn't test now but I think Debian images handle all three entrypoint
values, but the 'guix pack' image doesn't.

>> - Adding `nss-certs` to the `guix pack` command breaks: `(symlink
>>   "NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny.pem" #) Throw to key
>>   encoding-error' with args ("scm_to_stringn" "cannot convert wide
>>   string to output locale" 84 #f #f)'.`
>
> You need to run guix-daemon in a UTF-8 locale like “C.UTF-8” (which is
> supported out-of-the-box).
>
> See also <https://issues.guix.gnu.org/75007>.

Thanks - I finally figured this one out in the same way.

>> Finally, you may wonder why things didn't work before.  Some of the
>> major reasons: 1) --max-layer=100 and 2) -S /etc=etc and 3) Missing
>> /etc/protocols etc.  GitLab's docker setup doesn't handle many layers,
>> and it happens to just mount a sub-set of layers (see mount output,
>> missing a lot of layers).  Which files are put at which layer seems to
>> vary between `guix pack` runs for some reason, making it really hard to
>> debug (sometimes things worked partially, sometimes not, depending on
>> which files ended up visible).  I use --max-layers=8 now.
>
> Interesting.  Did you find points in the code (Docker? GitLab?) about
> this subset of layers being mounted?

No, I didn't go down chasing the root cause.  I wonder if maybe it isn't
as simple as some Linux kernel restriction on mount point sizes?  If you
try to load a --max-layer=100 image on GitLab you end up with a
/proc/mounts containing lines like this:

rw,relatime,lowerdir=l/ZPVAK6UICUAWUUE4GUD6AYFCEM:l/HI6HL2SJWTZDQHPTA3SGR4PWNE:l/OAJJQ5NKJAJLIAOEWBNLI6WRNC:l/MRDSZ2V6PLTEQGMEDSOPX6FKEY:l/FG4SISAU6TZNHB6CQR5X5GNEJB:l/EZGDP6A5CMVPA5O6IKOOKPDMBE:l/DA5NZCY6NVIGU2X6U5XQQXV54M:l/P4MIVQ3I7VCYFTQ3AG6RCXZVW5:l/FGPCINKKCYRDHZI5BAAU7HEETW:l/MUYJFZRJLR4Z3BNBFRBRTGP4S5:l/UPGZHVDAILBRLLEH6T5RKWIFG7:l/YNTBNOTPU7QRK5K6RD63VSXG5Q:l/XUPHFGGN36OPHNU334M3V7HDXP:l/PVA2QRQE4D5MAKI6BMGVPETNZN:l/VTGADKUDL4KA4HPA2YCCQ2JQNI:l/WZHWY243PTUZTQIZJM26PVSYG4:l/7YLLGUSIRSUFPXIFI57F2UQSFQ:l/ZEOCYJR44JGRBRKGEM4AMWUHQE:l/YXFWBRXLFSBMYDDCTNODGJWWUV:l/NQK5YT5BWDWSTKFRB3KOGVGYLC:l/6CC2D5S3LSZOOKLULC2JJ5BUHG:l/OMQ7M7FSULZ2WHTQPIOIYP7HYQ:l/VRGMNYJNRPOBPP5IZCJR3YV7FP:l/5L2O6RAVRTGGTB7I2YKS6RMX64:l/57QINIJWW7ECMX3DKLCDL5UMUS:l/HTIS4VRXRVO24AWC6AYPQ4LPRG:l/DOTULUURRI6Z4XR2B4LPQTP33T:l/LZYLE6JUKQFJBIAEMBQMQIWZEE:l/2WEGBKQG6D3VAWLXJ5NCZPFGNP:l/QYVPGN6K2A3Y4VVVPKYLR5JVL7:l/IPS4YGXCRZ47O4AOKMZ4TYD2N3:l/2MZU46ZBHYS5IQI4NAVFD3PNYR:l/HW6B6GDN33YB7GBY3LEOW2XXB4:l/FRINUFWGYICMVPLOJULIHQ3XKV:l/HSHVRY3DQIT5LUS3EONQQCKNL3:l/CRUDGYNQRSSDKB4TYALBBVFIL3:l/QJRZZB6NOMXWO46YCAJ4U53VSH:l/BZPO5MYEYX3YFX5NXYR32E6VRM:l/FDWSYXJR7RNKG42AMXGSC6NUQE:l/Z6BTLASGE6ZXGQZUFIHG4QXQLU:l/DWTG7Y6N4DNP5AZLI6MIDZEBHQ:l/TY4DSPRKETTLFE5WB7KFBO2VSM:l/SAE7PNZP6FFK2NVDOQQTZMO2BV:l/ZCL7CRORSVNYVFKNW2WT7TMGFZ:l/KVH3MQTKJ6SH46B2FEHW7UCYGK:l/XMDKZU7KD755BFINQOBKPLKMZQ:l/MFKZPIVLWDKG5PIVI3UQUU3ILT:l/I24GPDX2SRV3Y4YSJDYVKEBDQE:l/W6OZHVZW2NCSQOJEGMP45P2D6W:l/ZD4RI6B4WQ65QO7EMDQZSCZFHS:l/ZS7D35LTVLE6NSGCCY4SQQANE5:l/2ZIJ6PAUHDPBOXR72A6HGU6L4B:l/ZL5XOQ6XMF6ZYQSXVAK7744SX7:l/A3WXIZD22NL62JYHBZVL5K36IK:l/KUDHAVWVBDXKHHQQVF33KDPHF4:l/6O5OF37ORZTEUV2JXNOPO3WNTQ:l/YVGTC6PVMDS2GVOF57TJWCN5DM:l/VYM3JKFOWAY7UIYGU6TPJTZVD5:l/2BT7MWUS2JMZMQXQ3MRLO5AH6I:l/CNZTAOCVMGIFCNP4IF77OQ5MU6:l/KQCUQ7H6EY7U423TWBX6N6QNU7:l/VNZNN2P4U26XHEDRSNIQ656GRE:l/7YG6BVIVDJYCCGLISSU4APAR4S:l/ELR3M2R3NLUI4U2YCKYKG6MWUV:l/PK34AGBR7JY4PUJIVEAO4J7UAK:l/ELIDWT3IMYDRR5L5VTA3REN3LH:l/DK7VEQCTNIWW4BOYCOVXZX2GQY:l/6BCYXFR5B6S3EYCMVCXKQPEP3M:l/GAF6PUKMPKABSXDZCMVE3NM2K5:l/ASVMZMXSKHAVGH5UFXXRFE7TUX:l/25QORLMIGZEEIBQGTV6UBNZEAH:l/FH5SA2MBXXRRAMDYI72FPK7RXU:l/77IRT3TXX3H7XH66YGR7O5AIYK:l/FIQQSP7XQLUH3IWBXF4DZXWTFN:l/NG6ZAASCTOH6SQVUBR2FR6YE4T:l/QAHEWNHBILTWWWJ4QMX4ISWIT4:l/VCSGZSH4SQRVHK4EWSLGFECG2S:l/7FAMLBB6DJW7VWYEOIN6FLBI2E:l/C4ZXSOLVY36PYMTRL2E2YIKTKY:l/NVH5IIZJ5SO422GGMFGNTFCAOA:l/E6EE3L3EB2B36E5A5PL5KF32FL:l/RPXH632CD3Y54UYAMMULLEZOE4:l/662KE3GHNLWZEUTFCRZXOKVA3P:l/3ZDSIX7ZFSLT2FIEYD6TB6GA7R:l/ZS7SJOXF7XF6OFTLVHYMRYZANK:l/F3CEOKDDPOK72QLUWGKHDIJTPG:l/65CFVU5ZFM4XVSTLDO2WGQE3GU:l/3C2OJ5ZVICH5QP733HTS2DHPJM:l/B37PUETDZKC3MUZZX6CT3M6ZEC:l/LTVIXAHIS7O45NFIVNPDVGYMR4:l/GGBS4BWCAD6UMV3755MJ3BQRI5:l/4XGXVIO5QQ2ARZTTG74M4UDK4V:l/X673P4Q5TGPWBLQIXOQ5JXKG2C:l/3K2Q6NGZ5EEIKCFBRMEWWZFWQK:l/ALTG2QI6YZVKXIP4UCRDMQWS5Q:l/P4AE6X6G5GK66GHHRBZ27BTBF3:l/2UEJWLKTB2GNGT2G6UWFZVDFRN:l/OCTYS4BHPFX4HC3OXJJVGABEPX:l/M766VP6Y5QL2OGI6EYO5OEB6H4:l/DNITYWJFFEVGWQPFQ37S57EIB5:l/PQE5T2ER2OKPNC2YNHZEJXVIY6,upperdir=98dac307f50ae5da4c8f2cc5fdf024465f63600e30fc3bb434fca31191e2efdc/diff,workdir=98dac307f50ae5da4c8f2cc5fdf024465f63600e30fc3bb434fca31191e2efdc/work

That doesn't contain all of the 100 layers (although I didn't double
count..).  Something is truncating those lines.

That would explain why it works for me locally with 'podman': it seems
podman uses much shorter names, like:

rw,relatime,lowerdir=0:1:2:3:4..

instead of those longer strings above.

>> Re /etc=etc it seems GitLab's docker setup bind-mounts things below
>> /etc/ and it cannot handle the root /etc symlink.  A workaround is to
>> use `lndir` which I use in the `test-amd64-package-install` job.  This
>> is limitation of GitLab's docker setup: I tried running a `-S
>> /etc=etc` image on my own GitLab runner based on Trisquel [1] and it
>> worked fine, it mounted things below the symlinked tree properly.
>> Could `guix pack` be teached how to do a lndir-approach for /etc
>> instead of symlink, perhaps?
>
> It could symlink individual files and make /etc a directory.

Is this possible now?  How?

The --symlink is defined like this now:

‘-S SPEC’
     Add the symlinks specified by SPEC to the pack.  This option can
     appear several times.
     SPEC has the form ‘SOURCE=TARGET’, where SOURCE is the symlink that
     will be created and TARGET is the symlink target.
     For instance, ‘-S /opt/gnu/bin=bin’ creates a ‘/opt/gnu/bin’
     symlink pointing to the ‘bin’ sub-directory of the profile.

Could we extend that somehow to support the model of recursively
creating directories, but symlink files within them?

Or a new --symlink-mkdir parameter or similar?

> (What’s ‘lndir’, if I may ask?)

Exactly that :)

One of those ancient tools that still is surprisingly useful:

https://manpages.debian.org/bookworm/xutils-dev/lndir.1.en.html
https://gitlab.freedesktop.org/xorg/util/lndir/-/blob/master/lndir.c

/Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]