guix pack isolation

guix pack isolation

[Kyle Andrews]

Dear Guix,

I am wondering how I can/should go about isolating containers created
using `guix pack`. Right now they are inheriting configurations from my
personal user folder.

My system administrators are still quite skeptical about installing
Guix, but they have made it convenient to submit singularity container
jobs via SLURM. Otherwise, I would prefer to use `guix shell` because it
seems to make it easy to create a fully isolated container.

Is this a case where the best idea is to make a container first with
`guix shell` and then create the pack inside of that container? Are
there other approaches I should be trying?

Thanks,
Kyle

Re: guix pack isolation

[Simon Tournier]

Hi,

On ven., 10 févr. 2023 at 00:25, Kyle Andrews <kyle@posteo.net> wrote:

> I am wondering how I can/should go about isolating containers created
> using `guix pack`. Right now they are inheriting configurations from my
> personal user folder.
>
> My system administrators are still quite skeptical about installing
> Guix, but they have made it convenient to submit singularity container
> jobs via SLURM. Otherwise, I would prefer to use `guix shell` because it
> seems to make it easy to create a fully isolated container.

From my point of view, you have 2 options: relocatable or squashfs.

About relocatable pack, you might be interested by:

    https://hpc.guix.info/blog/2020/05/faster-relocatable-packs-with-fakechroot/


About squashfs (Singularity container), you can try:

    guix pack -f squashfs python python-numpy bash-minimal -S /bin=bin

which will produce a compressed squashfs that you then import with
Singularity.  I do not have Singularity at hand, let demo with Docker
which is very similar.

1. Produce the pack on the machine running Guix

--8<---------------cut here---------------start------------->8---
$ guix pack -f docker python python-numpy bash -S /bin=bin
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
The following derivations will be built:
  /gnu/store/nid89k59pcxkl6lg7mlppqk8qmizbn6p-python-python-numpy-bash-docker-pack.tar.gz.drv
  /gnu/store/jlib2ngiianwv6854c1kbl6zlgixxg48-module-import.drv
  /gnu/store/irsn92v5ykaxb4yrynrr45hp6jkd4cxc-profile.drv

1,2 MB will be downloaded
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
 guile-json-3.5.0  44KiB              1.1MiB/s 00:00 [##################] 100.0%
 module-import  1KiB                  508KiB/s 00:00 [##################] 100.0%
building profile with 3 packages...
 module-import-compiled  1.1MiB       5.4MiB/s 00:00 [##################] 100.0%
building /gnu/store/nid89k59pcxkl6lg7mlppqk8qmizbn6p-python-python-numpy-bash-docker-pack.tar.gz.drv...
/gnu/store/i3hkgkjq672hic4nkn4g718b9sggg4rh-python-python-numpy-bash-docker-pack.tar.gz
--8<---------------cut here---------------end--------------->8---

2. Transfer the produce pack (…-docker-pack.tar.gz) to the machine without Guix.
3. Load the pack and use it!

--8<---------------cut here---------------start------------->8---
$ docker load < my-docker-pack.tar.gz
Loaded image: python-python-numpy-bash:latest

$ docker run -ti python-python-numpy-bash:latest python3
Python 3.9.9 (main, Jan  1 1970, 00:00:01) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
import numpy as np
>>> A = np.array([[1,0,1],[0,1,0],[0,0,1]]);
A = np.array([[1,0,1],[0,1,0],[0,0,1]]);
>>> _, s, _ = np.linalg.svd(A); s; abs(s[0] - 1./s[2])
_, s, _ = np.linalg.svd(A); s; abs(s[0] - 1./s[2])
array([1.61803399, 1.        , 0.61803399])
0.0
>>> 
--8<---------------cut here---------------end--------------->8---

Cheers,
simon

    

Re: guix pack isolation

[Kyle Andrews]

Simon Tournier <zimon.toutoune@gmail.com> writes:

> About squashfs (Singularity container), you can try:
>
>     guix pack -f squashfs python python-numpy bash-minimal -S /bin=bin
>
> which will produce a compressed squashfs that you then import with
> Singularity.

This is pretty much what I had tried minus the `-S /bin=bin' part which
I don't understand. Thankfully, reading the manual informs me that:

```
‘guix pack -f squashfs’ always implies ‘-S /bin=bin’
```

So, atleast I don't have to understand it yet.

> I do not have Singularity at hand, let demo with Docker
> which is very similar.

Too bad! My system admistrators are against running Docker on the
cluster. The "relocatable" option didn't seem relevant to me since it wouldn't apply any file system isolation.

> $ docker run -ti python-python-numpy-bash:latest python3

The arguments in this command atleast gave me an idea. Maybe I am just
ignorant about how singularity works? I didn't know what -ti could
possibly mean without first looking them up. Maybe I just need to add
some additional arguments to singularity exec?

Arguments in the manual which look particularly interesting to me are:

--contain
--containall
--bind=/path/to/shared/file/system/location
--no-home
--workdir

=>
https://docs.sylabs.io/guides/3.1/user-guide/cli/singularity_exec.html

If you or anyone else have have any tips on how to best mimic the
behavior of the analogous `guix shell' command with an `singularity
exec' call I would use if guix was available, I am all ears.

Thanks,
Kyle

Re: guix pack isolation

[Simon Tournier]

Hi,

On Wed, 15 Feb 2023 at 23:56, Kyle Andrews <kyle@posteo.net> wrote:

>> I do not have Singularity at hand, let demo with Docker
>> which is very similar.
>
> Too bad! My system admistrators are against running Docker on the
> cluster. The "relocatable" option didn't seem relevant to me since it
> wouldn't apply any file system isolation.

Well, I think it should not be an issue for Singularity to reuse Docker
pack. ;-) Once the Docker pack imported inside Docker – say e.g., docker
load < $(guix pack -f pack …) – then something like,

    singularity run docker://python-python-numpy-bash:latest python3

should do the job.  Anyway! :-)

>> $ docker run -ti python-python-numpy-bash:latest python3
>
> The arguments in this command atleast gave me an idea. Maybe I am just
> ignorant about how singularity works? I didn't know what -ti could
> possibly mean without first looking them up. Maybe I just need to add
> some additional arguments to singularity exec?

Well, is this invocation

    singularity exec                                     \
       $(guix pack -f squashfs python python-numpy bash) \
       python3

working for you?

> Arguments in the manual which look particularly interesting to me are:
>
> --contain
> --containall
> --bind=/path/to/shared/file/system/location
> --no-home
> --workdir

This depends on what you would like to achieve. ;-)

> If you or anyone else have have any tips on how to best mimic the
> behavior of the analogous `guix shell' command with an `singularity
> exec' call I would use if guix was available, I am all ears.

Basically,

    guix shell -C -m manifest.scm -- something

is more or less similar to,

    singularity exec $(guix pack -f squashfs -m manifest.scm) something

But since Guix is not running on the cluster, you need to build the pack
on your machine running Guix – this will produce a self contained
archive, say using the format Squashfs (Singularity).

Hope that helps,
simon