From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:403:478a::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id gGMGOTEKK2VxfAEA9RJhRA:P1 (envelope-from ) for ; Sat, 14 Oct 2023 23:37:54 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:478a::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id gGMGOTEKK2VxfAEA9RJhRA (envelope-from ) for ; Sat, 14 Oct 2023 23:37:54 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 18FE7706BF for ; Sat, 14 Oct 2023 23:37:52 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=autistici.org header.s=stigmate header.b=PFuhxJrg; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1697319473; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=opjZElKA+UIj+7axfLa4567GjTsChDqLcGZP+voXTlE=; b=ZBkRjxgDjDOtr7VWb/giI7N+BcdF9LjYn/y55HmerEj1fkdJLQoWYRr8vjf310FOnjSN5m rWdfoH4g7lIQca/UpuuEFMXg9Djo2UWavtc1Riwr3yffHprAVS8f7Wkno0xBCL6JbosMaU FzxiZylSGJ11SXPbkufcHsRPw7014y1TITQRRQXPNJyYSnfOvQAHGh9b0/H9M0n4g0/PPo g5zPaNN4qZtZfUkNihE4IGSjnu6GWqMdhCA7GgdVGQCLuZSWJOZbzgLvZ+gLDjTMmZor7V Ux/ibCyWlsc6+znuQXjx/b2ybCI2I4xriIjVQr2NJTIH0Wva11X7raGENvO1TQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1697319473; a=rsa-sha256; cv=none; b=rT10fWA5UR+WaWPsCUXQcWywz2KFzpF+E+gnqnPfutKFDQGjOj7nJcd90/9EaA1BBQR+wK xpOrPtjbhu81Ikk1awZxKcyOZdh7ww0iprhAQ7CCBI/Q9E5kpNZBNjOtdEijR9QGAwoFiX 0j1T9siYMplh4mAs9lntRP4HDVakSMjFmyXZBL2CKuJKGqx6KSMRLf1DNEyXVLaSv/xVA/ OJYkg3hvM2QnWQp8bPJNyMoHK/opH5e3Nae4ZdULBwKxpqpDlio9/BeSnG3+888gf5RlOD B23szJr05iuW4YlYuKS01g1vMPOD2QGuxip7te2e/mghIZP2tnr9HYsljUD+AQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=autistici.org header.s=stigmate header.b=PFuhxJrg; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrmKJ-0003x4-RG; Sat, 14 Oct 2023 17:37:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrmKI-0003wu-Rl for guix-patches@gnu.org; Sat, 14 Oct 2023 17:37:38 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrmKI-0005oo-JW for guix-patches@gnu.org; Sat, 14 Oct 2023 17:37:38 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qrmKg-0005wZ-Be for guix-patches@gnu.org; Sat, 14 Oct 2023 17:38:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#66160] [PATCH] gnu: Add oci-container-service-type. References: <650a02e2-9425-a7fd-2014-7cf6e17ee65b@autistici.org> In-Reply-To: <650a02e2-9425-a7fd-2014-7cf6e17ee65b@autistici.org> Resent-From: Giacomo Leidi Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 14 Oct 2023 21:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66160 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 66160@debbugs.gnu.org@debbugs.gnu.org Cc: Giacomo Leidi Received: via spool by 66160-submit@debbugs.gnu.org id=B66160.169731943822779 (code B ref 66160); Sat, 14 Oct 2023 21:38:02 +0000 Received: (at 66160) by debbugs.gnu.org; 14 Oct 2023 21:37:18 +0000 Received: from localhost ([127.0.0.1]:50760 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qrmJx-0005vK-AB for submit@debbugs.gnu.org; Sat, 14 Oct 2023 17:37:18 -0400 Received: from confino.investici.org ([93.190.126.19]:23381) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qrmJu-0005vB-Ie for 66160@debbugs.gnu.org; Sat, 14 Oct 2023 17:37:16 -0400 Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4S7Gss1R6xz112P; Sat, 14 Oct 2023 21:36:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1697319409; bh=SNhrLOp4rcsZ23W9XxBGRJjHILYPtJxp0yxV4pSQVsg=; h=From:To:Cc:Subject:Date:From; b=PFuhxJrgdygRmhHkynjHq/Hfmuh50gJ0ImcNgSph03Ge2IT4Kf02KoavC5Y6YolpO JhlFxfdjyuOlZKWTWg1AqqYXZP1G0b0U/KTbbFc0Nu5NlqQX5Z6MHUkLSdsOplt+GM uTmH0Kn2pNotZc7r5K12ppNI+cib2uOH9NdHHimk= Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4S7Gss0q91z10yD; Sat, 14 Oct 2023 21:36:48 +0000 (UTC) Date: Sat, 14 Oct 2023 23:36:12 +0200 Message-ID: X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Giacomo Leidi X-ACL-Warn: , Giacomo Leidi via Guix-patches From: Giacomo Leidi via Guix-patches via Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Spam-Score: -3.68 X-Migadu-Queue-Id: 18FE7706BF X-Migadu-Scanner: mx0.migadu.com X-Migadu-Spam-Score: -3.68 X-TUID: eMGuXySrhx+a * gnu/services/docker.scm (oci-container-configuration): New variable; (oci-container-shepherd-service): new variable; (oci-container-service-type): new variable. * doc/guix.texi: Document it. --- doc/guix.texi | 123 +++++++++++++++++++++ gnu/services/docker.scm | 237 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 359 insertions(+), 1 deletion(-) diff --git a/doc/guix.texi b/doc/guix.texi index 083504dcb8..54d5074bd7 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -39534,6 +39534,129 @@ Miscellaneous Services @command{singularity run} and similar commands. @end defvar +@cindex OCI-backed, Shepherd services +@subsubheading OCI backed services + +Should you wish to manage your Docker containers with the same consistent +interface you use for your other Shepherd services, +@var{oci-container-service-type} is the tool to use: given an +@acronym{Open Container Initiative, OCI} container image, it will run it in a +Shepherd service. One example where this is useful: it lets you run services +that are available as Docker/OCI images but not yet packaged for Guix. + +@defvar oci-container-service-type + +This is a thin wrapper around Docker's CLI that executes OCI images backed +processes as Shepherd Services. + +@lisp +(service oci-container-service-type + (list + (oci-container-configuration + (image "prom/prometheus") + (network "host") + (ports + '(("9000" . "9000") + ("9090" . "9090")))))) + +(service oci-container-service-type + (list + (oci-container-configuration + (image "grafana/grafana:10.0.1") + (network "host") + (ports + '(("3000" . "3000"))) + (volumes + '("/var/lib/grafana:/var/lib/grafana")))))) +@end lisp + +In this example two different Shepherd services are going be added to the +system. Each @code{oci-container-configuration} record translates to a +@code{docker run} invocation and its fields directly map to options. You can +refer to the +@url{https://docs.docker.com/engine/reference/commandline/run,upstream}, +documentation for the semantics of each value. If the images are not found they +will be +@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The +spawned services are going to be attached to the host network and are supposed +to behave like other processes. + +@end defvar + +@c %start of fragment + +@deftp {Data Type} oci-container-configuration +Available @code{oci-container-configuration} fields are: + +@table @asis +@item @code{user} (default: @code{"oci-container"}) (type: string) +The user under whose authority docker commands will be run. + +@item @code{group} (default: @code{"docker"}) (type: string) +The group under whose authority docker commands will be run. + +@item @code{command} (default: @code{()}) (type: list-of-strings) +Overwrite the default command (@code{CMD}) of the image. + +@item @code{entrypoint} (default: @code{""}) (type: string) +Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image. + +@item @code{environment} (default: @code{()}) (type: list) +Set environment variables. This can be a list of pairs or strings, even mixed: + +@lisp +(list '("LANGUAGE" . "eo:ca:eu") + "JAVA_HOME=/opt/java") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream} +documentation for semantics. + +@item @code{image} (type: string) +The image used to build the container. Images are resolved by the +Docker Engine, and follow the usual format +@code{myregistry.local:5000/testing/test-image:tag}. + +@item @code{name} (default: @code{""}) (type: string) +Set a name for the spawned container. + +@item @code{network} (default: @code{""}) (type: string) +Set a Docker network for the spawned container. + +@item @code{ports} (default: @code{()}) (type: list) +Set the port or port ranges to expose from the spawned container. This can be a +list of pairs or strings, even mixed: + +@lisp +(list '("8080" . "80") + "10443:443") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@uref{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream} +documentation for semantics. + +@item @code{volumes} (default: @code{()}) (type: list) +Set volume mappings for the spawned container. This can be a +list of pairs or strings, even mixed: + +@lisp +(list '("/root/data/grafana" . "/var/lib/grafana") + "/gnu/store:/gnu/store") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@uref{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream} +documentation for semantics. + +@end table + +@end deftp + + +@c %end of fragment + @cindex Audit @subsubheading Auditd Service diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm index c2023d618c..2d709bf2ce 100644 --- a/gnu/services/docker.scm +++ b/gnu/services/docker.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2020 Efraim Flashner ;;; Copyright © 2020 Jesse Dowell ;;; Copyright © 2021 Brice Waegeneire +;;; Copyright © 2023 Giacomo Leidi ;;; ;;; This file is part of GNU Guix. ;;; @@ -29,15 +30,34 @@ (define-module (gnu services docker) #:use-module (gnu services shepherd) #:use-module (gnu system setuid) #:use-module (gnu system shadow) + #:use-module (gnu packages admin) ;shadow #:use-module (gnu packages docker) #:use-module (gnu packages linux) ;singularity #:use-module (guix records) + #:use-module (guix diagnostics) #:use-module (guix gexp) + #:use-module (guix i18n) #:use-module (guix packages) + #:use-module (srfi srfi-1) + #:use-module (ice-9 format) + #:use-module (ice-9 match) #:export (docker-configuration docker-service-type - singularity-service-type)) + singularity-service-type + oci-container-configuration + oci-container-configuration? + oci-container-configuration-fields + oci-container-configuration-command + oci-container-configuration-entrypoint + oci-container-configuration-environment + oci-container-configuration-image + oci-container-configuration-name + oci-container-configuration-network + oci-container-configuration-ports + oci-container-configuration-volumes + oci-container-service-type + oci-container-shepherd-service)) (define-configuration docker-configuration (docker @@ -216,3 +236,218 @@ (define singularity-service-type (service-extension activation-service-type (const %singularity-activation)))) (default-value singularity))) + + +;;; +;;; OCI container. +;;; + +(define (oci-sanitize-pair pair delimiter) + (define (valid? member) + (or (string? member) + (gexp? member) + (file-like? member))) + (match pair + (((? valid? key) . (? valid? value)) + #~(string-append #$key #$delimiter #$value)) + (_ + (raise + (formatted-message + (G_ "pair members must contain only strings, gexps or file-like objects +but ~a was found") + pair))))) + +(define (oci-sanitize-mixed-list name value delimiter) + (map + (lambda (el) + (cond ((string? el) el) + ((pair? el) (oci-sanitize-pair el delimiter)) + (else + (raise + (formatted-message + (G_ "~a members must be either a string or a pair but ~a was +found!") + name el))))) + value)) + +(define (oci-sanitize-environment value) + ;; Expected spec format: + ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java") + (oci-sanitize-mixed-list "environment" value "=")) + +(define (oci-sanitize-ports value) + ;; Expected spec format: + ;; '(("8088" . "80") "2022:22") + (oci-sanitize-mixed-list "ports" value ":")) + +(define (oci-sanitize-volumes value) + ;; Expected spec format: + ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java") + (oci-sanitize-mixed-list "volumes" value ":")) + +(define-maybe/no-serialization string) + +(define-configuration/no-serialization oci-container-configuration + (user + (string "oci-container") + "The user under whose authority docker commands will be run.") + (group + (string "docker") + "The group under whose authority docker commands will be run.") + (command + (list-of-strings '()) + "Overwrite the default command (@code{CMD}) of the image.") + (entrypoint + (maybe-string) + "Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.") + (environment + (list '()) + "Set environment variables. This can be a list of pairs or strings, even +mixed: + +@lisp +(list '(\"LANGUAGE\" . \"eo:ca:eu\") + \"JAVA_HOME=/opt/java\") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream} +documentation for semantics." + (sanitizer oci-sanitize-environment)) + (image + (string) + "The image used to build the container. Images are resolved by the Docker +Engine, and follow the usual format +@code{myregistry.local:5000/testing/test-image:tag}.") + (name + (maybe-string) + "Set a name for the spawned container.") + (network + (maybe-string) + "Set a Docker network for the spawned container.") + (ports + (list '()) + "Set the port or port ranges to expose from the spawned container. This can +be a list of pairs or strings, even mixed: + +@lisp +(list '(\"8080\" . \"80\") + \"10443:443\") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream} +documentation for semantics." + (sanitizer oci-sanitize-ports)) + (volumes + (list '()) + "Set volume mappings for the spawned container. This can be a +list of pairs or strings, even mixed: + +@lisp +(list '(\"/root/data/grafana\" . \"/var/lib/grafana\") + \"/gnu/store:/gnu/store\") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream} +documentation for semantics." + (sanitizer oci-sanitize-volumes))) + +(define oci-container-configuration->options + (lambda (config) + (let ((entrypoint + (oci-container-configuration-entrypoint config)) + (network + (oci-container-configuration-network config))) + (apply append + (filter (compose not unspecified?) + `(,(if (maybe-value-set? entrypoint) + `("--entrypoint" ,entrypoint) + '()) + ,(append-map + (lambda (spec) + (list "--env" spec)) + (oci-container-configuration-environment config)) + ,(if (maybe-value-set? network) + `("--network" ,network) + '()) + ,(append-map + (lambda (spec) + (list "-p" spec)) + (oci-container-configuration-ports config)) + ,(append-map + (lambda (spec) + (list "-v" spec)) + (oci-container-configuration-volumes config)))))))) + +(define (oci-container-shepherd-service config) + (define (guess-name name image) + (if (maybe-value-set? name) + name + (string-append "docker-" + (basename (car (string-split image #\:)))))) + + (let* ((docker-command (file-append docker-cli "/bin/docker")) + (user (oci-container-configuration-user config)) + (group (oci-container-configuration-group config)) + (command (oci-container-configuration-command config)) + (config-name (oci-container-configuration-name config)) + (image (oci-container-configuration-image config)) + (options (oci-container-configuration->options config)) + (name (guess-name config-name image))) + + (shepherd-service (provision `(,(string->symbol name))) + (requirement '(dockerd user-processes)) + (respawn? #f) + (documentation + (string-append + "Docker backed Shepherd service for image: " image)) + (start + #~(make-forkexec-constructor + ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...] + (list #$docker-command "run" "--rm" + "--name" #$name + #$@options #$image #$@command) + #:user #$user + #:group #$group)) + (stop + #~(lambda _ + (invoke #$docker-command "rm" "-f" #$name))) + (actions + (list + (shepherd-action + (name 'pull) + (documentation + (format #f "Pull ~a's image (~a)." + name image)) + (procedure + #~(lambda _ + (invoke #$docker-command "pull" #$image))))))))) + +(define %oci-container-accounts + (list (user-account + (name "oci-container") + (comment "OCI services account") + (group "docker") + (system? #t) + (home-directory "/var/empty") + (shell (file-append shadow "/sbin/nologin"))))) + +(define (configs->shepherd-services configs) + (map oci-container-shepherd-service configs)) + +(define oci-container-service-type + (service-type (name 'oci-container) + (extensions (list (service-extension profile-service-type + (lambda _ (list docker-cli))) + (service-extension account-service-type + (const %oci-container-accounts)) + (service-extension shepherd-root-service-type + configs->shepherd-services))) + (default-value '()) + (extend append) + (compose concatenate) + (description + "This service allows the management of Docker and OCI +containers as Shepherd services."))) base-commit: 8aad7210ea06992ee3f36ca7f57678240949e063 -- 2.41.0