From: "Ludovic Courtès" <ludo@gnu.org>
To: Tao Hansen <worldofgeese@riseup.net>
Cc: guix-devel@gnu.org
Subject: Re: Improving cgroups for fun and Kubernetes
Date: Wed, 04 Oct 2023 17:21:49 +0200 [thread overview]
Message-ID: <87bkdee7le.fsf@gnu.org> (raw)
In-Reply-To: <87zg1bplu4.fsf@riseup.net> (Tao Hansen's message of "Sun, 24 Sep 2023 18:39:15 +0200")
Hi Tao,
Tao Hansen <worldofgeese@riseup.net> skribis:
> This is my second posting to the mailing list but the first using Gnus
> and smtmpmail. If I've formatted anything poorly, don't hesitate to let
> me know.
Looks perfect to me. :-)
> I've been spending a silly amount of time trying to get a local flavor
> of Kubernetes running on Guix System. I wanted to share my experience
> and also solicit feedback from Guix's developers on how to improve the
> cgroups implementation such that those who follow me will have an easier
> time of it.
I’ve never used Kubernetes, but I’m confident you’re not the only
interested in using it on Guix System!
[...]
> The second problem is kind and minikube are both expecting Delegate=yes
> to be set, which is a systemd function that allows these tools to set
> cgroups limits. The limits it's expecting to control are cpu, cpuset,
> memory and pids. We can force these privileges like so, echo "+cpu
> +cpuset +memory +pids" >> /sys/fs/cgroup/cgroup.subtree_control
How about having a Shepherd service that does writes to that
‘cgroup.subtree_control’ file as you write above?
> To fix the first problem we can run
>
> g=users && sudo chgrp -R ${g} /sys/fs/cgroup/
> u=$USER && sudo chown -R ${u}: /sys/fs/cgroup
What does Debian do? Perhaps there’s a “cgroup” group (in /etc/groups)
that users who want to user podman need to belong to, similar to the
‘kvm’ group for those who want to access /dev/kvm?
Or maybe we should create a sub-tree specifically for podman usage?
At any rate, we could again have a Shepherd service that sets ownership
on the relevant file tree.
> Once we've addressed the first and second problem, the rest is
> relatively easy: we need to make iptables (and iptables' modules so just
> the package isn't enough: we need Guix's service) available. We need to
> set a range of user IDs and group IDs for Podman to make use of
> rootlessly, and finally we need to set a container policy otherwise Podman
> can't pull any image from anywhere. All of those can be done from inside
> our Guix System configuration file.
Right, we should populate /etc/subuid by default (I tried to use
subordinate UIDs in the past, by invoking ‘newuidmap’, but never managed
to get it to work.)
> Here's what that Guix System configuration looks like:
>
> ;; Rootless Podman requires the next 4 services
> ;; we're using the iptables service purely to make its resources
> ;; available to minikube and kind
>
> (service iptables-service-type
> (iptables-configuration
> (ipv4-rules (plain-file "iptables.rules" "*filter
> :INPUT ACCEPT
> :FORWARD ACCEPT
> :OUTPUT ACCEPT
> COMMIT
> "))
> (ipv6-rules (plain-file "ip6tables.rules" "*filter
> :INPUT ACCEPT
> :FORWARD ACCEPT
> :OUTPUT ACCEPT
> COMMIT
> "))))
> (simple-service 'etc-subuid etc-service-type
> (list `("subuid" ,(plain-file "subuid"
> (string-append "root:0:65536\n" username ":100000:65536\n")))))
> (simple-service 'etc-subgid etc-service-type
> (list `("subgid" ,(plain-file "subgid"
> (string-append "root:0:65536\n" username ":100000:65536\n")))))
> (service pam-limits-service-type
> (list
> (pam-limits-entry "*" 'both 'nofile 100000)))
> (simple-service 'etc-container-policy etc-service-type
> (list `("containers/policy.json", (plain-file
> "policy.json" "{\"default\": [{\"type\":
> \"insecureAcceptAnything\"}]}"))))
> %my-services
Looks great! We should probably consider /etc/{subuid,subgid} support
separately, but otherwise it looks like you already have the start of a
‘rootless-podman-service-type’ (or similar).
Thanks,
Ludo’.
prev parent reply other threads:[~2023-10-04 15:22 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-24 16:39 Improving cgroups for fun and Kubernetes Tao Hansen
2023-10-04 15:21 ` Ludovic Courtès [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87bkdee7le.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=guix-devel@gnu.org \
--cc=worldofgeese@riseup.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).