* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
@ 2023-09-22 20:32 paul via Guix-patches via
2023-09-22 20:34 ` Giacomo Leidi via Guix-patches via
` (5 more replies)
0 siblings, 6 replies; 17+ messages in thread
From: paul via Guix-patches via @ 2023-09-22 20:32 UTC (permalink / raw)
To: 66160
Dear Guixers,
following up on
https://lists.gnu.org/archive/html/guix-devel/2023-09/msg00468.html ,
I'm sending the implementation for the oci-backed shepherd services.
Thank you for your time,
giacomo
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-09-22 20:32 [bug#66160] [PATCH] gnu: Add oci-container-service-type paul via Guix-patches via
@ 2023-09-22 20:34 ` Giacomo Leidi via Guix-patches via
2023-10-05 14:30 ` Ludovic Courtès
2023-10-06 19:09 ` Giacomo Leidi via Guix-patches via
` (4 subsequent siblings)
5 siblings, 1 reply; 17+ messages in thread
From: Giacomo Leidi via Guix-patches via @ 2023-09-22 20:34 UTC (permalink / raw)
To: 66160; +Cc: Giacomo Leidi
* gnu/services/docker.scm (oci-container-configuration): New variable;
(oci-container-shepherd-service): new variable;
(oci-container-service-type): new variable.
* doc/guix.texi: Document it.
---
doc/guix.texi | 78 +++++++++++++++++++
gnu/services/docker.scm | 163 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 240 insertions(+), 1 deletion(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 617b8463e3..988ab64773 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -39349,6 +39349,84 @@ Miscellaneous Services
@command{singularity run} and similar commands.
@end defvar
+@cindex OCI-backed, Shepherd services
+@subsubheading OCI backed services
+
+Should you wish to manage your Docker containers with the same consistent
+interface you use for your other Shepherd services,
+@var{oci-container-service-type} is the tool to use.
+
+@defvar oci-container-service-type
+
+This is a thin wrapper around Docker's CLI that wraps OCI images backed
+processes as Shepherd Services.
+
+@lisp
+(simple-service 'oci-grafana-service
+ (list
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))))
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana"))))))
+@end lisp
+
+@end defvar
+
+@deftp {Data Type} oci-container-configuration
+Available @code{oci-container-configuration} fields are:
+
+@table @asis
+@item @code{command} (default: @code{()}) (type: list-of-strings)
+Overwrite the default CMD of the image.
+
+@item @code{entrypoint} (default: @code{""}) (type: string)
+Overwrite the default ENTRYPOINT of the image.
+
+@item @code{environment} (default: @code{()}) (type: list)
+Set environment variables. This can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '("LANGUAGE" . "eo:ca:eu")
+ "JAVA_HOME=/opt/java")
+@end lisp
+
+@item @code{image} (type: string)
+The image used to build the container.
+
+@item @code{name} (default: @code{""}) (type: string)
+Set a name for the spawned container.
+
+@item @code{network} (default: @code{""}) (type: string)
+Set a Docker network for the spawned container.
+
+@item @code{ports} (default: @code{()}) (type: list)
+Set the port or port ranges to expose from the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("8080" . "80")
+ "10443:443")
+@end lisp
+
+@item @code{volumes} (default: @code{()}) (type: list)
+Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("/root/data/grafana" . "/var/lib/grafana")
+ "/gnu/store:/gnu/store")
+@end lisp
+
+@end table
+@end deftp
+
@cindex Audit
@subsubheading Auditd Service
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
index c2023d618c..8a4fa2107e 100644
--- a/gnu/services/docker.scm
+++ b/gnu/services/docker.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2020 Jesse Dowell <jessedowell@gmail.com>
;;; Copyright © 2021 Brice Waegeneire <brice@waegenei.re>
+;;; Copyright © 2023 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -34,10 +35,25 @@ (define-module (gnu services docker)
#:use-module (guix records)
#:use-module (guix gexp)
#:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
#:export (docker-configuration
docker-service-type
- singularity-service-type))
+ singularity-service-type
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-name
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-service-type
+ oci-container-shepherd-service))
(define-configuration docker-configuration
(docker
@@ -216,3 +232,148 @@ (define singularity-service-type
(service-extension activation-service-type
(const %singularity-activation))))
(default-value singularity)))
+
+\f
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (cond ((file-like? (car pair))
+ (file-append (car pair) delimiter (cdr pair)))
+ ((gexp? (car pair))
+ (file-append (car pair) delimiter (cdr pair)))
+ ((string? (car pair))
+ (string-append (car pair) delimiter (cdr pair)))
+ (else
+ (error
+ (format #f "pair members must only contain gexps, file-like objects and strings but ~a was found" (car pair))))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (error
+ (format #f "~a members must be either a string or a pair but ~a was found!" name el)))))
+ value))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define-configuration/no-serialization oci-container-configuration
+ (command
+ (list-of-strings '())
+ "Overwrite the default CMD of the image.")
+ (entrypoint
+ (string "")
+ "Overwrite the default ENTRYPOINT of the image.")
+ (environment
+ (list '())
+ "Set environment variables."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string)
+ "The image used to build the container.")
+ (name
+ (string "")
+ "Set a name for the spawned container.")
+ (network
+ (string "")
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container."
+ (sanitizer oci-sanitize-volumes)))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(when (not (string-null? entrypoint))
+ (list "--entrypoint" entrypoint))
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(when (not (string-null? network))
+ (list "--network" network))
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (not (string-null? name))
+ name
+ (string-append "docker-"
+ (basename (car (string-split image #\:))))))
+
+ (let* ((docker-command (file-append docker-cli "/bin/docker"))
+ (config-name (oci-container-configuration-name config))
+ (image (oci-container-configuration-image config))
+ (name (guess-name config-name image)))
+
+ (shepherd-service (provision `(,(string->symbol name)))
+ (requirement '(dockerd))
+ (respawn? #f)
+ (documentation
+ (string-append
+ "Docker backed Shepherd service for image: " image))
+ (start
+ #~(make-forkexec-constructor
+ ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
+ (list #$docker-command
+ "run"
+ "--rm"
+ "--name" #$name
+ #$@(oci-container-configuration->options config)
+ #$(oci-container-configuration-image config)
+ #$@(oci-container-configuration-command config))
+ #:user "root"
+ #:group "root"))
+ (stop
+ #~(lambda _
+ (invoke #$docker-command "stop" #$name))))))
+
+(define (configs->shepherd-services configs)
+ (map oci-container-shepherd-service configs))
+
+(define oci-container-service-type
+ (service-type (name 'oci-container)
+ (extensions (list (service-extension profile-service-type
+ (lambda _ (list docker-cli)))
+ (service-extension shepherd-root-service-type
+ configs->shepherd-services)))
+ (default-value '())
+ (extend append)
+ (compose concatenate)
+ (description
+ "This service provides allows the management of Docker
+containers as Shepherd services.")))
base-commit: f45c0c82289d409b4fac00464ea8b323839ba53f
--
2.41.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-09-22 20:34 ` Giacomo Leidi via Guix-patches via
@ 2023-10-05 14:30 ` Ludovic Courtès
2023-10-05 17:30 ` paul via Guix-patches via
0 siblings, 1 reply; 17+ messages in thread
From: Ludovic Courtès @ 2023-10-05 14:30 UTC (permalink / raw)
To: Giacomo Leidi; +Cc: 66160
Hi,
Giacomo Leidi <goodoldpaul@autistici.org> skribis:
> * gnu/services/docker.scm (oci-container-configuration): New variable;
> (oci-container-shepherd-service): new variable;
> (oci-container-service-type): new variable.
> * doc/guix.texi: Document it.
Neat!
> +@cindex OCI-backed, Shepherd services
> +@subsubheading OCI backed services
> +
> +Should you wish to manage your Docker containers with the same consistent
> +interface you use for your other Shepherd services,
> +@var{oci-container-service-type} is the tool to use.
Perhaps expound a bit, like:
… is the tool to use: given an @acronym{Open Container Initiative,
OCI} container image, it will run it in a Shepherd service. One
example where this is useful: it lets you run services that are
available as Docker/OCI images but not yet packaged for Guix.
> +@defvar oci-container-service-type
> +
> +This is a thin wrapper around Docker's CLI that wraps OCI images backed
> +processes as Shepherd Services.
> +
> +@lisp
> +(simple-service 'oci-grafana-service
> + (list
> + (oci-container-configuration
The second argument to ‘simple-service’ is missing.
> + (image "prom/prometheus")
> + (network "host")
> + (ports
> + '(("9000" . "9000")
> + ("9090" . "9090"))))))
> + (oci-container-configuration
> + (image "grafana/grafana:10.0.1")
> + (network "host")
> + (volumes
> + '("/var/lib/grafana:/var/lib/grafana"))))))
> +@end lisp
Please explain the example in one or two sentences.
Personally, I’d like to know how the image names are resolved; would be
nice to mention it in the doc.
> +@table @asis
> +@item @code{command} (default: @code{()}) (type: list-of-strings)
> +Overwrite the default CMD of the image.
“… the default command (@code{CMD}) of the image.”
> +@item @code{entrypoint} (default: @code{""}) (type: string)
> +Overwrite the default ENTRYPOINT of the image.
Likewise.
> +@item @code{environment} (default: @code{()}) (type: list)
> +Set environment variables. This can be a list of pairs or strings, even mixed:
> +
> +@lisp
> +(list '("LANGUAGE" . "eo:ca:eu")
> + "JAVA_HOME=/opt/java")
I would choose one or the other, but not both.
> +@item @code{ports} (default: @code{()}) (type: list)
> +Set the port or port ranges to expose from the spawned container. This can be a
> +list of pairs or strings, even mixed:
> +
> +@lisp
> +(list '("8080" . "80")
> + "10443:443")
Likewise.
> +(define (oci-sanitize-pair pair delimiter)
> + (cond ((file-like? (car pair))
> + (file-append (car pair) delimiter (cdr pair)))
Please use ‘match’ instead of car/cdr (info "(guix) Data Types and
Pattern Matching").
> + (error
> + (format #f "pair members must only contain gexps, file-like objects and strings but ~a was found" (car pair))))))
Should be (raise (formatted-message (G_ …))). That way we get i18n
support and the message is presented like other error messages.
> + (error
> + (format #f "~a members must be either a string or a pair but ~a was found!" name el)))))
Ditto.
> + (shepherd-service (provision `(,(string->symbol name)))
> + (requirement '(dockerd))
Actually: (requirement '(dockerd user-processes)).
> + (description
> + "This service provides allows the management of Docker
> +containers as Shepherd services.")))
“Docker and OCI containers”
Could you send an updated patch?
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-10-05 14:30 ` Ludovic Courtès
@ 2023-10-05 17:30 ` paul via Guix-patches via
2023-10-13 22:53 ` paul via Guix-patches via
0 siblings, 1 reply; 17+ messages in thread
From: paul via Guix-patches via @ 2023-10-05 17:30 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 66160
Hi,
On 10/5/23 16:30, Ludovic Courtès wrote:
> Hi,
>
> Giacomo Leidi <goodoldpaul@autistici.org> skribis:
>
>> * gnu/services/docker.scm (oci-container-configuration): New variable;
>> (oci-container-shepherd-service): new variable;
>> (oci-container-service-type): new variable.
>> * doc/guix.texi: Document it.
> Neat!
>
>> +@cindex OCI-backed, Shepherd services
>> +@subsubheading OCI backed services
>> +
>> +Should you wish to manage your Docker containers with the same consistent
>> +interface you use for your other Shepherd services,
>> +@var{oci-container-service-type} is the tool to use.
> Perhaps expound a bit, like:
>
> … is the tool to use: given an @acronym{Open Container Initiative,
> OCI} container image, it will run it in a Shepherd service. One
> example where this is useful: it lets you run services that are
> available as Docker/OCI images but not yet packaged for Guix.
nice thank you, fixed.
>
>> +@defvar oci-container-service-type
>> +
>> +This is a thin wrapper around Docker's CLI that wraps OCI images backed
>> +processes as Shepherd Services.
>> +
>> +@lisp
>> +(simple-service 'oci-grafana-service
>> + (list
>> + (oci-container-configuration
> The second argument to ‘simple-service’ is missing.
Good catch, fixed.
>
>> + (image "prom/prometheus")
>> + (network "host")
>> + (ports
>> + '(("9000" . "9000")
>> + ("9090" . "9090"))))))
>> + (oci-container-configuration
>> + (image "grafana/grafana:10.0.1")
>> + (network "host")
>> + (volumes
>> + '("/var/lib/grafana:/var/lib/grafana"))))))
>> +@end lisp
> Please explain the example in one or two sentences.
>
> Personally, I’d like to know how the image names are resolved; would be
> nice to mention it in the doc.
[ ... ]
>
>> +@table @asis
>> +@item @code{command} (default: @code{()}) (type: list-of-strings)
>> +Overwrite the default CMD of the image.
> “… the default command (@code{CMD}) of the image.”
[ ... ]
>
>> +@item @code{entrypoint} (default: @code{""}) (type: string)
>> +Overwrite the default ENTRYPOINT of the image.
> Likewise.
Fixed, thank you.
>
>> +@item @code{environment} (default: @code{()}) (type: list)
>> +Set environment variables. This can be a list of pairs or strings, even mixed:
>> +
>> +@lisp
>> +(list '("LANGUAGE" . "eo:ca:eu")
>> + "JAVA_HOME=/opt/java")
> I would choose one or the other, but not both.
I would like to allow some kind of escape (the same way the nice Guix
configuration records provide an extra-content field which is literally
appended to the config) in case there's some something I didn't foresee
with this implementation. It may be paranoia, I don't have a strong
opinion. are you strongly against supporting the two formats?
>
>> +@item @code{ports} (default: @code{()}) (type: list)
>> +Set the port or port ranges to expose from the spawned container. This can be a
>> +list of pairs or strings, even mixed:
>> +
>> +@lisp
>> +(list '("8080" . "80")
>> + "10443:443")
> Likewise.
>
>> +(define (oci-sanitize-pair pair delimiter)
>> + (cond ((file-like? (car pair))
>> + (file-append (car pair) delimiter (cdr pair)))
> Please use ‘match’ instead of car/cdr (info "(guix) Data Types and
> Pattern Matching").
Thank you, fixed.
>
>> + (error
>> + (format #f "pair members must only contain gexps, file-like objects and strings but ~a was found" (car pair))))))
> Should be (raise (formatted-message (G_ …))). That way we get i18n
> support and the message is presented like other error messages.
[ ... ]
>
>> + (error
>> + (format #f "~a members must be either a string or a pair but ~a was found!" name el)))))
> Ditto.
[ ... ]
>
>> + (shepherd-service (provision `(,(string->symbol name)))
>> + (requirement '(dockerd))
> Actually: (requirement '(dockerd user-processes)).
[ ... ]
>
>> + (description
>> + "This service provides allows the management of Docker
>> +containers as Shepherd services.")))
> “Docker and OCI containers”
Fixed.
> Could you send an updated patch?
I should have addressed all of your comments besides the one on the
key-value format. I'm sending an updated patch.
Thank you for your time and effort,
giacomo
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-09-22 20:32 [bug#66160] [PATCH] gnu: Add oci-container-service-type paul via Guix-patches via
2023-09-22 20:34 ` Giacomo Leidi via Guix-patches via
@ 2023-10-06 19:09 ` Giacomo Leidi via Guix-patches via
2023-10-14 16:09 ` Ludovic Courtès
2023-10-13 22:57 ` Giacomo Leidi via Guix-patches via
` (3 subsequent siblings)
5 siblings, 1 reply; 17+ messages in thread
From: Giacomo Leidi via Guix-patches via @ 2023-10-06 19:09 UTC (permalink / raw)
To: 66160; +Cc: Giacomo Leidi
* gnu/services/docker.scm (oci-container-configuration): New variable;
(oci-container-shepherd-service): new variable;
(oci-container-service-type): new variable.
* doc/guix.texi: Document it.
---
doc/guix.texi | 108 ++++++++++++++++++++++++++
gnu/services/docker.scm | 167 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 274 insertions(+), 1 deletion(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 617b8463e3..5c3908f758 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -39349,6 +39349,114 @@ Miscellaneous Services
@command{singularity run} and similar commands.
@end defvar
+@cindex OCI-backed, Shepherd services
+@subsubheading OCI backed services
+
+Should you wish to manage your Docker containers with the same consistent
+interface you use for your other Shepherd services,
+@var{oci-container-service-type} is the tool to use: given an
+@acronym{Open Container Initiative, OCI} container image, it will run it in a
+Shepherd service. One example where this is useful: it lets you run services
+that are available as Docker/OCI images but not yet packaged for Guix.
+
+@defvar oci-container-service-type
+
+This is a thin wrapper around Docker's CLI that executes OCI images backed
+processes as Shepherd Services.
+
+@lisp
+(simple-service 'oci-grafana-service
+ oci-container-service-type
+ (list
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))))
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (ports
+ '(("3000" . "3000")))
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana"))))))
+@end lisp
+
+In this example two different Shepherd services are going be added to the
+system. Each @code{oci-container-configuration} record translates to a
+@code{docker run} invocation and its fields directly map to options. You can
+refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run,upstream},
+documentation for the semantics of each value. If the images are not found they
+will be
+@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The
+spawned services are going to be attached to the host network and are supposed
+to behave like other processes.
+
+@end defvar
+
+@deftp {Data Type} oci-container-configuration
+Available @code{oci-container-configuration} fields are:
+
+@table @asis
+@item @code{command} (default: @code{()}) (type: list-of-strings)
+Overwrite the default command (@code{CMD}) of the image.
+
+@item @code{entrypoint} (default: @code{""}) (type: string)
+Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
+
+@item @code{environment} (default: @code{()}) (type: list)
+Set environment variables. This can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '("LANGUAGE" . "eo:ca:eu")
+ "JAVA_HOME=/opt/java")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics.
+
+@item @code{image} (type: string)
+The image used to build the container. Images are resolved by the Docker Engine,
+and follow the usual format @code{myregistry.local:5000/testing/test-image:tag}.
+
+@item @code{name} (default: @code{""}) (type: string)
+Set a name for the spawned container.
+
+@item @code{network} (default: @code{""}) (type: string)
+Set a Docker network for the spawned container.
+
+@item @code{ports} (default: @code{()}) (type: list)
+Set the port or port ranges to expose from the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("8080" . "80")
+ "10443:443")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics.
+
+@item @code{volumes} (default: @code{()}) (type: list)
+Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("/root/data/grafana" . "/var/lib/grafana")
+ "/gnu/store:/gnu/store")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics.
+
+@end table
+@end deftp
+
@cindex Audit
@subsubheading Auditd Service
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
index c2023d618c..af87001143 100644
--- a/gnu/services/docker.scm
+++ b/gnu/services/docker.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2020 Jesse Dowell <jessedowell@gmail.com>
;;; Copyright © 2021 Brice Waegeneire <brice@waegenei.re>
+;;; Copyright © 2023 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -32,12 +33,30 @@ (define-module (gnu services docker)
#:use-module (gnu packages docker)
#:use-module (gnu packages linux) ;singularity
#:use-module (guix records)
+ #:use-module (guix diagnostics)
#:use-module (guix gexp)
+ #:use-module (guix i18n)
#:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
+ #:use-module (ice-9 match)
#:export (docker-configuration
docker-service-type
- singularity-service-type))
+ singularity-service-type
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-name
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-service-type
+ oci-container-shepherd-service))
(define-configuration docker-configuration
(docker
@@ -216,3 +235,149 @@ (define singularity-service-type
(service-extension activation-service-type
(const %singularity-activation))))
(default-value singularity)))
+
+\f
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (match pair
+ (((? string? key) . (? string? value))
+ (string-append key delimiter value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "pair members must contain only strings but ~a was found")
+ pair)))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (raise
+ (formatted-message
+ (G_ "~a members must be either a string or a pair but ~a was found!")
+ name el)))))
+ value))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define-configuration/no-serialization oci-container-configuration
+ (command
+ (list-of-strings '())
+ "Overwrite the default CMD of the image.")
+ (entrypoint
+ (string "")
+ "Overwrite the default ENTRYPOINT of the image.")
+ (environment
+ (list '())
+ "Set environment variables."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string)
+ "The image used to build the container.")
+ (name
+ (string "")
+ "Set a name for the spawned container.")
+ (network
+ (string "")
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container."
+ (sanitizer oci-sanitize-volumes)))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(when (not (string-null? entrypoint))
+ (list "--entrypoint" entrypoint))
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(when (not (string-null? network))
+ (list "--network" network))
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (not (string-null? name))
+ name
+ (string-append "docker-"
+ (basename (car (string-split image #\:))))))
+
+ (let* ((docker-command (file-append docker-cli "/bin/docker"))
+ (config-name (oci-container-configuration-name config))
+ (image (oci-container-configuration-image config))
+ (name (guess-name config-name image)))
+
+ (shepherd-service (provision `(,(string->symbol name)))
+ (requirement '(dockerd user-processes))
+ (respawn? #f)
+ (documentation
+ (string-append
+ "Docker backed Shepherd service for image: " image))
+ (start
+ #~(make-forkexec-constructor
+ ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
+ (list #$docker-command
+ "run"
+ "--rm"
+ "--name" #$name
+ #$@(oci-container-configuration->options config)
+ #$(oci-container-configuration-image config)
+ #$@(oci-container-configuration-command config))
+ #:user "root"
+ #:group "root"))
+ (stop
+ #~(lambda _
+ (invoke #$docker-command "stop" #$name))))))
+
+(define (configs->shepherd-services configs)
+ (map oci-container-shepherd-service configs))
+
+(define oci-container-service-type
+ (service-type (name 'oci-container)
+ (extensions (list (service-extension profile-service-type
+ (lambda _ (list docker-cli)))
+ (service-extension shepherd-root-service-type
+ configs->shepherd-services)))
+ (default-value '())
+ (extend append)
+ (compose concatenate)
+ (description
+ "This service allows the management of Docker and OCI
+containers as Shepherd services.")))
base-commit: f45c0c82289d409b4fac00464ea8b323839ba53f
--
2.41.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-10-05 17:30 ` paul via Guix-patches via
@ 2023-10-13 22:53 ` paul via Guix-patches via
0 siblings, 0 replies; 17+ messages in thread
From: paul via Guix-patches via @ 2023-10-13 22:53 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 66160
Hi,
I'm sending a patch rebased on current master. I also added a 'pull'
action to allow running docker pull for the image of an oci container.
Thank you for your time,
giacomo
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-09-22 20:32 [bug#66160] [PATCH] gnu: Add oci-container-service-type paul via Guix-patches via
2023-09-22 20:34 ` Giacomo Leidi via Guix-patches via
2023-10-06 19:09 ` Giacomo Leidi via Guix-patches via
@ 2023-10-13 22:57 ` Giacomo Leidi via Guix-patches via
2023-10-14 21:36 ` Giacomo Leidi via Guix-patches via
` (2 subsequent siblings)
5 siblings, 0 replies; 17+ messages in thread
From: Giacomo Leidi via Guix-patches via @ 2023-10-13 22:57 UTC (permalink / raw)
To: 66160; +Cc: Giacomo Leidi
* gnu/services/docker.scm (oci-container-configuration): New variable;
(oci-container-shepherd-service): new variable;
(oci-container-service-type): new variable.
* doc/guix.texi: Document it.
---
doc/guix.texi | 108 +++++++++++++++++++++++
gnu/services/docker.scm | 185 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 292 insertions(+), 1 deletion(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 083504dcb8..97c3515652 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -39534,6 +39534,114 @@ Miscellaneous Services
@command{singularity run} and similar commands.
@end defvar
+@cindex OCI-backed, Shepherd services
+@subsubheading OCI backed services
+
+Should you wish to manage your Docker containers with the same consistent
+interface you use for your other Shepherd services,
+@var{oci-container-service-type} is the tool to use: given an
+@acronym{Open Container Initiative, OCI} container image, it will run it in a
+Shepherd service. One example where this is useful: it lets you run services
+that are available as Docker/OCI images but not yet packaged for Guix.
+
+@defvar oci-container-service-type
+
+This is a thin wrapper around Docker's CLI that executes OCI images backed
+processes as Shepherd Services.
+
+@lisp
+(simple-service 'oci-grafana-service
+ oci-container-service-type
+ (list
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))))
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (ports
+ '(("3000" . "3000")))
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana"))))))
+@end lisp
+
+In this example two different Shepherd services are going be added to the
+system. Each @code{oci-container-configuration} record translates to a
+@code{docker run} invocation and its fields directly map to options. You can
+refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run,upstream},
+documentation for the semantics of each value. If the images are not found they
+will be
+@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The
+spawned services are going to be attached to the host network and are supposed
+to behave like other processes.
+
+@end defvar
+
+@deftp {Data Type} oci-container-configuration
+Available @code{oci-container-configuration} fields are:
+
+@table @asis
+@item @code{command} (default: @code{()}) (type: list-of-strings)
+Overwrite the default command (@code{CMD}) of the image.
+
+@item @code{entrypoint} (default: @code{""}) (type: string)
+Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
+
+@item @code{environment} (default: @code{()}) (type: list)
+Set environment variables. This can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '("LANGUAGE" . "eo:ca:eu")
+ "JAVA_HOME=/opt/java")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics.
+
+@item @code{image} (type: string)
+The image used to build the container. Images are resolved by the Docker Engine,
+and follow the usual format @code{myregistry.local:5000/testing/test-image:tag}.
+
+@item @code{name} (default: @code{""}) (type: string)
+Set a name for the spawned container.
+
+@item @code{network} (default: @code{""}) (type: string)
+Set a Docker network for the spawned container.
+
+@item @code{ports} (default: @code{()}) (type: list)
+Set the port or port ranges to expose from the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("8080" . "80")
+ "10443:443")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics.
+
+@item @code{volumes} (default: @code{()}) (type: list)
+Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("/root/data/grafana" . "/var/lib/grafana")
+ "/gnu/store:/gnu/store")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics.
+
+@end table
+@end deftp
+
@cindex Audit
@subsubheading Auditd Service
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
index c2023d618c..be954f7a27 100644
--- a/gnu/services/docker.scm
+++ b/gnu/services/docker.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2020 Jesse Dowell <jessedowell@gmail.com>
;;; Copyright © 2021 Brice Waegeneire <brice@waegenei.re>
+;;; Copyright © 2023 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -32,12 +33,30 @@ (define-module (gnu services docker)
#:use-module (gnu packages docker)
#:use-module (gnu packages linux) ;singularity
#:use-module (guix records)
+ #:use-module (guix diagnostics)
#:use-module (guix gexp)
+ #:use-module (guix i18n)
#:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
+ #:use-module (ice-9 match)
#:export (docker-configuration
docker-service-type
- singularity-service-type))
+ singularity-service-type
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-name
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-service-type
+ oci-container-shepherd-service))
(define-configuration docker-configuration
(docker
@@ -216,3 +235,167 @@ (define singularity-service-type
(service-extension activation-service-type
(const %singularity-activation))))
(default-value singularity)))
+
+\f
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (match pair
+ (((? valid? key) . (? valid? value))
+ #~(string-append #$key #$delimiter #$value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "pair members must contain only strings, gexps or file-like objects
+but ~a was found")
+ pair)))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (raise
+ (formatted-message
+ (G_ "~a members must be either a string or a pair but ~a was
+found!")
+ name el)))))
+ value))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define-configuration/no-serialization oci-container-configuration
+ (command
+ (list-of-strings '())
+ "Overwrite the default CMD of the image.")
+ (entrypoint
+ (string "")
+ "Overwrite the default ENTRYPOINT of the image.")
+ (environment
+ (list '())
+ "Set environment variables."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string)
+ "The image used to build the container.")
+ (name
+ (string "")
+ "Set a name for the spawned container.")
+ (network
+ (string "")
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container."
+ (sanitizer oci-sanitize-volumes)))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(when (not (string-null? entrypoint))
+ (list "--entrypoint" entrypoint))
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(when (not (string-null? network))
+ (list "--network" network))
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (not (string-null? name))
+ name
+ (string-append "docker-"
+ (basename (car (string-split image #\:))))))
+
+ (let* ((docker-command (file-append docker-cli "/bin/docker"))
+ (command (oci-container-configuration-command config))
+ (config-name (oci-container-configuration-name config))
+ (image (oci-container-configuration-image config))
+ (options (oci-container-configuration->options config))
+ (name (guess-name config-name image)))
+
+ (shepherd-service (provision `(,(string->symbol name)))
+ (requirement '(dockerd user-processes))
+ (respawn? #f)
+ (documentation
+ (string-append
+ "Docker backed Shepherd service for image: " image))
+ (start
+ #~(make-forkexec-constructor
+ ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
+ (list #$docker-command
+ "run"
+ "--rm"
+ "--name" #$name
+ #$@options
+ #$image
+ #$@command)
+ #:user "root"
+ #:group "root"))
+ (stop
+ #~(lambda _
+ (invoke #$docker-command "stop" #$name)))
+ (actions
+ (list
+ (shepherd-action
+ (name 'pull)
+ (documentation
+ (format #f "Pull ~a's image (~a)."
+ name image))
+ (procedure
+ #~(lambda _
+ (invoke #$docker-command "pull" #$image)))))))))
+
+(define (configs->shepherd-services configs)
+ (map oci-container-shepherd-service configs))
+
+(define oci-container-service-type
+ (service-type (name 'oci-container)
+ (extensions (list (service-extension profile-service-type
+ (lambda _ (list docker-cli)))
+ (service-extension shepherd-root-service-type
+ configs->shepherd-services)))
+ (default-value '())
+ (extend append)
+ (compose concatenate)
+ (description
+ "This service allows the management of Docker and OCI
+containers as Shepherd services.")))
base-commit: 8aad7210ea06992ee3f36ca7f57678240949e063
--
2.41.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-10-06 19:09 ` Giacomo Leidi via Guix-patches via
@ 2023-10-14 16:09 ` Ludovic Courtès
2023-10-14 21:29 ` paul via Guix-patches via
0 siblings, 1 reply; 17+ messages in thread
From: Ludovic Courtès @ 2023-10-14 16:09 UTC (permalink / raw)
To: Giacomo Leidi; +Cc: 66160
Hi Giacomo,
Giacomo Leidi <goodoldpaul@autistici.org> skribis:
> * gnu/services/docker.scm (oci-container-configuration): New variable;
> (oci-container-shepherd-service): new variable;
> (oci-container-service-type): new variable.
> * doc/guix.texi: Document it.
We’re almost there! There’s a couple of things I overlooked before (my
apologies), so here we go:
> +@table @asis
> +@item @code{command} (default: @code{()}) (type: list-of-strings)
> +Overwrite the default command (@code{CMD}) of the image.
> +
> +@item @code{entrypoint} (default: @code{""}) (type: string)
> +Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
Apparently this doesn’t match the docstring that’s in
‘define-configuration’.
Could you make sure the docstring is the canonical source? Then you can
use ‘generate-documentation’ to generate the bit that you’ll paste in
guix.texi (info "(guix) Complex Configurations").
> + (entrypoint
> + (string "")
> + "Overwrite the default ENTRYPOINT of the image.")
> + (environment
> + (list '())
> + "Set environment variables."
> + (sanitizer oci-sanitize-environment))
> + (image
> + (string)
> + "The image used to build the container.")
> + (name
> + (string "")
> + "Set a name for the spawned container.")
Please use ‘maybe-string’ in cases where it’s either the Docker default
(default ENTRYPOINT, default CMD, etc.) or some user-provided value.
I find it clearer or at least more conventional than using the empty
string to denote default values.
> +(define oci-container-configuration->options
> + (lambda (config)
> + (let ((entrypoint
> + (oci-container-configuration-entrypoint config))
> + (network
> + (oci-container-configuration-network config)))
> + (apply append
> + (filter (compose not unspecified?)
> + `(,(when (not (string-null? entrypoint))
> + (list "--entrypoint" entrypoint))
> + ,(append-map
> + (lambda (spec)
> + (list "--env" spec))
> + (oci-container-configuration-environment config))
> + ,(when (not (string-null? network))
> + (list "--network" network))
This would thus become:
`(,@(if entrypoint
`("--entrypoint" ,entrypoint)
'())
…)
> + #~(make-forkexec-constructor
> + ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
> + (list #$docker-command
> + "run"
> + "--rm"
> + "--name" #$name
> + #$@(oci-container-configuration->options config)
> + #$(oci-container-configuration-image config)
> + #$@(oci-container-configuration-command config))
> + #:user "root"
> + #:group "root"))
Does ‘docker run’ necessarily need to run as root, or are there cases
where one might want to run it as non-root? (I expect the latter.)
> +(define oci-container-service-type
> + (service-type (name 'oci-container)
> + (extensions (list (service-extension profile-service-type
> + (lambda _ (list docker-cli)))
> + (service-extension shepherd-root-service-type
> + configs->shepherd-services)))
> + (default-value '())
I wonder if it should take a list of configs and be extensible, or
simply take a single config. Users would write:
(service oci-container-service-type
(oci-container-configuration …))
WDYT?
Last thing: there’s no system test (something we normally require), but
since I forgot about it before and I’m already asking for more than I
should :-) I propose to leave it for later.
Thanks!
Ludo’.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-10-14 16:09 ` Ludovic Courtès
@ 2023-10-14 21:29 ` paul via Guix-patches via
2023-10-19 20:13 ` Ludovic Courtès
0 siblings, 1 reply; 17+ messages in thread
From: paul via Guix-patches via @ 2023-10-14 21:29 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 66160
Hi Ludo’ ,
> We’re almost there! There’s a couple of things I overlooked before (my
> apologies), so here we go:
Thank you for your help and the time you spent reviewing this!
>> +@table @asis
>> +@item @code{command} (default: @code{()}) (type: list-of-strings)
>> +Overwrite the default command (@code{CMD}) of the image.
>> +
>> +@item @code{entrypoint} (default: @code{""}) (type: string)
>> +Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
> Apparently this doesn’t match the docstring that’s in
> ‘define-configuration’.
>
> Could you make sure the docstring is the canonical source? Then you can
> use ‘generate-documentation’ to generate the bit that you’ll paste in
> guix.texi (info "(guix) Complex Configurations").
I should have aligned the code and documentation.
>
>> + (entrypoint
>> + (string "")
>> + "Overwrite the default ENTRYPOINT of the image.")
>> + (environment
>> + (list '())
>> + "Set environment variables."
>> + (sanitizer oci-sanitize-environment))
>> + (image
>> + (string)
>> + "The image used to build the container.")
>> + (name
>> + (string "")
>> + "Set a name for the spawned container.")
> Please use ‘maybe-string’ in cases where it’s either the Docker default
> (default ENTRYPOINT, default CMD, etc.) or some user-provided value.
> I find it clearer or at least more conventional than using the empty
> string to denote default values.
I don't know why I didn't do it in the first place, thank you. fixed
>> + #~(make-forkexec-constructor
>> + ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
>> + (list #$docker-command
>> + "run"
>> + "--rm"
>> + "--name" #$name
>> + #$@(oci-container-configuration->options config)
>> + #$(oci-container-configuration-image config)
>> + #$@(oci-container-configuration-command config))
>> + #:user "root"
>> + #:group "root"))
> Does ‘docker run’ necessarily need to run as root, or are there cases
> where one might want to run it as non-root? (I expect the latter.)
yes you are right, it's only required to be in the docker group or in
general have enough permission to operate on the docker daemon socket. I
added a new service extension setting up an oci-container user, that
it's just in the docker group and can not login, that runs oci backed
services. it is also overridable by the user
>
>> +(define oci-container-service-type
>> + (service-type (name 'oci-container)
>> + (extensions (list (service-extension profile-service-type
>> + (lambda _ (list docker-cli)))
>> + (service-extension shepherd-root-service-type
>> + configs->shepherd-services)))
>> + (default-value '())
> I wonder if it should take a list of configs and be extensible, or
> simply take a single config. Users would write:
>
> (service oci-container-service-type
> (oci-container-configuration …))
>
> WDYT?
I get that it's not super consistent with other services but for example
Nextcloud in some case require 3 containers: nextcloud itself, redis and
a cron container. in this case i suppose one would define an
oci-nextcloud-service-type which maybe extends an
oci-redis-service-type. in this case the oci-nextcloud-service-type
would define the 2 shepherd services for nextcloud and the cron process
and the oci-redis-service-type would define one redis service.
Right now we can already do:
(service oci-container-service-type
(list
(oci-container-configuration …)))
and i updated the documentation accordingly. do you have any suggestion
for changing the api of oci-container-configuration to support having
different shepherd oci backed services behind a guix system service?
This way we could remove the (list) call.
>
> Last thing: there’s no system test (something we normally require), but
> since I forgot about it before and I’m already asking for more than I
> should :-) I propose to leave it for later.
I actually looked around but didn't find them, but it was a long time
ago and I certainly didn't look very well. I'm definitely up for testing
this (maybe it's possible to use swineherd? could we use it for
automating integration tests?), could you point me to something similar
i can look to as an example?
Thank you for your time and effort, i'm sending an updated patch
giacomo
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-09-22 20:32 [bug#66160] [PATCH] gnu: Add oci-container-service-type paul via Guix-patches via
` (2 preceding siblings ...)
2023-10-13 22:57 ` Giacomo Leidi via Guix-patches via
@ 2023-10-14 21:36 ` Giacomo Leidi via Guix-patches via
2023-10-14 21:47 ` Giacomo Leidi via Guix-patches via
2023-10-24 20:59 ` [bug#66160] [PATCH v2] " Giacomo Leidi via Guix-patches via
5 siblings, 0 replies; 17+ messages in thread
From: Giacomo Leidi via Guix-patches via @ 2023-10-14 21:36 UTC (permalink / raw)
To: 66160; +Cc: Giacomo Leidi
* gnu/services/docker.scm (oci-container-configuration): New variable;
(oci-container-shepherd-service): new variable;
(oci-container-service-type): new variable.
* doc/guix.texi: Document it.
---
doc/guix.texi | 123 +++++++++++++++++++++
gnu/services/docker.scm | 237 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 359 insertions(+), 1 deletion(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 083504dcb8..54d5074bd7 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -39534,6 +39534,129 @@ Miscellaneous Services
@command{singularity run} and similar commands.
@end defvar
+@cindex OCI-backed, Shepherd services
+@subsubheading OCI backed services
+
+Should you wish to manage your Docker containers with the same consistent
+interface you use for your other Shepherd services,
+@var{oci-container-service-type} is the tool to use: given an
+@acronym{Open Container Initiative, OCI} container image, it will run it in a
+Shepherd service. One example where this is useful: it lets you run services
+that are available as Docker/OCI images but not yet packaged for Guix.
+
+@defvar oci-container-service-type
+
+This is a thin wrapper around Docker's CLI that executes OCI images backed
+processes as Shepherd Services.
+
+@lisp
+(service oci-container-service-type
+ (list
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))))
+
+(service oci-container-service-type
+ (list
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (ports
+ '(("3000" . "3000")))
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana"))))))
+@end lisp
+
+In this example two different Shepherd services are going be added to the
+system. Each @code{oci-container-configuration} record translates to a
+@code{docker run} invocation and its fields directly map to options. You can
+refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run,upstream},
+documentation for the semantics of each value. If the images are not found they
+will be
+@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The
+spawned services are going to be attached to the host network and are supposed
+to behave like other processes.
+
+@end defvar
+
+@c %start of fragment
+
+@deftp {Data Type} oci-container-configuration
+Available @code{oci-container-configuration} fields are:
+
+@table @asis
+@item @code{user} (default: @code{"oci-container"}) (type: string)
+The user under whose authority docker commands will be run.
+
+@item @code{group} (default: @code{"docker"}) (type: string)
+The group under whose authority docker commands will be run.
+
+@item @code{command} (default: @code{()}) (type: list-of-strings)
+Overwrite the default command (@code{CMD}) of the image.
+
+@item @code{entrypoint} (default: @code{""}) (type: string)
+Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
+
+@item @code{environment} (default: @code{()}) (type: list)
+Set environment variables. This can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '("LANGUAGE" . "eo:ca:eu")
+ "JAVA_HOME=/opt/java")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics.
+
+@item @code{image} (type: string)
+The image used to build the container. Images are resolved by the
+Docker Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.
+
+@item @code{name} (default: @code{""}) (type: string)
+Set a name for the spawned container.
+
+@item @code{network} (default: @code{""}) (type: string)
+Set a Docker network for the spawned container.
+
+@item @code{ports} (default: @code{()}) (type: list)
+Set the port or port ranges to expose from the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("8080" . "80")
+ "10443:443")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics.
+
+@item @code{volumes} (default: @code{()}) (type: list)
+Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("/root/data/grafana" . "/var/lib/grafana")
+ "/gnu/store:/gnu/store")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
@cindex Audit
@subsubheading Auditd Service
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
index c2023d618c..2d709bf2ce 100644
--- a/gnu/services/docker.scm
+++ b/gnu/services/docker.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2020 Jesse Dowell <jessedowell@gmail.com>
;;; Copyright © 2021 Brice Waegeneire <brice@waegenei.re>
+;;; Copyright © 2023 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -29,15 +30,34 @@ (define-module (gnu services docker)
#:use-module (gnu services shepherd)
#:use-module (gnu system setuid)
#:use-module (gnu system shadow)
+ #:use-module (gnu packages admin) ;shadow
#:use-module (gnu packages docker)
#:use-module (gnu packages linux) ;singularity
#:use-module (guix records)
+ #:use-module (guix diagnostics)
#:use-module (guix gexp)
+ #:use-module (guix i18n)
#:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
+ #:use-module (ice-9 match)
#:export (docker-configuration
docker-service-type
- singularity-service-type))
+ singularity-service-type
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-name
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-service-type
+ oci-container-shepherd-service))
(define-configuration docker-configuration
(docker
@@ -216,3 +236,218 @@ (define singularity-service-type
(service-extension activation-service-type
(const %singularity-activation))))
(default-value singularity)))
+
+\f
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (match pair
+ (((? valid? key) . (? valid? value))
+ #~(string-append #$key #$delimiter #$value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "pair members must contain only strings, gexps or file-like objects
+but ~a was found")
+ pair)))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (raise
+ (formatted-message
+ (G_ "~a members must be either a string or a pair but ~a was
+found!")
+ name el)))))
+ value))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define-maybe/no-serialization string)
+
+(define-configuration/no-serialization oci-container-configuration
+ (user
+ (string "oci-container")
+ "The user under whose authority docker commands will be run.")
+ (group
+ (string "docker")
+ "The group under whose authority docker commands will be run.")
+ (command
+ (list-of-strings '())
+ "Overwrite the default command (@code{CMD}) of the image.")
+ (entrypoint
+ (maybe-string)
+ "Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.")
+ (environment
+ (list '())
+ "Set environment variables. This can be a list of pairs or strings, even
+mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string)
+ "The image used to build the container. Images are resolved by the Docker
+Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.")
+ (name
+ (maybe-string)
+ "Set a name for the spawned container.")
+ (network
+ (maybe-string)
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container. This can
+be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"8080\" . \"80\")
+ \"10443:443\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"/root/data/grafana\" . \"/var/lib/grafana\")
+ \"/gnu/store:/gnu/store\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-volumes)))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(if (maybe-value-set? entrypoint)
+ `("--entrypoint" ,entrypoint)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(if (maybe-value-set? network)
+ `("--network" ,network)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (maybe-value-set? name)
+ name
+ (string-append "docker-"
+ (basename (car (string-split image #\:))))))
+
+ (let* ((docker-command (file-append docker-cli "/bin/docker"))
+ (user (oci-container-configuration-user config))
+ (group (oci-container-configuration-group config))
+ (command (oci-container-configuration-command config))
+ (config-name (oci-container-configuration-name config))
+ (image (oci-container-configuration-image config))
+ (options (oci-container-configuration->options config))
+ (name (guess-name config-name image)))
+
+ (shepherd-service (provision `(,(string->symbol name)))
+ (requirement '(dockerd user-processes))
+ (respawn? #f)
+ (documentation
+ (string-append
+ "Docker backed Shepherd service for image: " image))
+ (start
+ #~(make-forkexec-constructor
+ ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
+ (list #$docker-command "run" "--rm"
+ "--name" #$name
+ #$@options #$image #$@command)
+ #:user #$user
+ #:group #$group))
+ (stop
+ #~(lambda _
+ (invoke #$docker-command "rm" "-f" #$name)))
+ (actions
+ (list
+ (shepherd-action
+ (name 'pull)
+ (documentation
+ (format #f "Pull ~a's image (~a)."
+ name image))
+ (procedure
+ #~(lambda _
+ (invoke #$docker-command "pull" #$image)))))))))
+
+(define %oci-container-accounts
+ (list (user-account
+ (name "oci-container")
+ (comment "OCI services account")
+ (group "docker")
+ (system? #t)
+ (home-directory "/var/empty")
+ (shell (file-append shadow "/sbin/nologin")))))
+
+(define (configs->shepherd-services configs)
+ (map oci-container-shepherd-service configs))
+
+(define oci-container-service-type
+ (service-type (name 'oci-container)
+ (extensions (list (service-extension profile-service-type
+ (lambda _ (list docker-cli)))
+ (service-extension account-service-type
+ (const %oci-container-accounts))
+ (service-extension shepherd-root-service-type
+ configs->shepherd-services)))
+ (default-value '())
+ (extend append)
+ (compose concatenate)
+ (description
+ "This service allows the management of Docker and OCI
+containers as Shepherd services.")))
base-commit: 8aad7210ea06992ee3f36ca7f57678240949e063
--
2.41.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-09-22 20:32 [bug#66160] [PATCH] gnu: Add oci-container-service-type paul via Guix-patches via
` (3 preceding siblings ...)
2023-10-14 21:36 ` Giacomo Leidi via Guix-patches via
@ 2023-10-14 21:47 ` Giacomo Leidi via Guix-patches via
2023-10-24 20:59 ` [bug#66160] [PATCH v2] " Giacomo Leidi via Guix-patches via
5 siblings, 0 replies; 17+ messages in thread
From: Giacomo Leidi via Guix-patches via @ 2023-10-14 21:47 UTC (permalink / raw)
To: 66160; +Cc: Giacomo Leidi
* gnu/services/docker.scm (oci-container-configuration): New variable;
(oci-container-shepherd-service): new variable;
(oci-container-service-type): new variable.
* doc/guix.texi: Document it.
---
doc/guix.texi | 120 ++++++++++++++++++++
gnu/services/docker.scm | 237 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 356 insertions(+), 1 deletion(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 083504dcb8..6de46a1ebe 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -39534,6 +39534,126 @@ Miscellaneous Services
@command{singularity run} and similar commands.
@end defvar
+@cindex OCI-backed, Shepherd services
+@subsubheading OCI backed services
+
+Should you wish to manage your Docker containers with the same consistent
+interface you use for your other Shepherd services,
+@var{oci-container-service-type} is the tool to use: given an
+@acronym{Open Container Initiative, OCI} container image, it will run it in a
+Shepherd service. One example where this is useful: it lets you run services
+that are available as Docker/OCI images but not yet packaged for Guix.
+
+@defvar oci-container-service-type
+
+This is a thin wrapper around Docker's CLI that executes OCI images backed
+processes as Shepherd Services.
+
+@lisp
+(service oci-container-service-type
+ (list
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (ports
+ '(("3000" . "3000")))
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana")))))
+@end lisp
+
+In this example two different Shepherd services are going be added to the
+system. Each @code{oci-container-configuration} record translates to a
+@code{docker run} invocation and its fields directly map to options. You can
+refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run,upstream},
+documentation for the semantics of each value. If the images are not found they
+will be
+@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The
+spawned services are going to be attached to the host network and are supposed
+to behave like other processes.
+
+@end defvar
+
+@c %start of fragment
+
+@deftp {Data Type} oci-container-configuration
+Available @code{oci-container-configuration} fields are:
+
+@table @asis
+@item @code{user} (default: @code{"oci-container"}) (type: string)
+The user under whose authority docker commands will be run.
+
+@item @code{group} (default: @code{"docker"}) (type: string)
+The group under whose authority docker commands will be run.
+
+@item @code{command} (default: @code{()}) (type: list-of-strings)
+Overwrite the default command (@code{CMD}) of the image.
+
+@item @code{entrypoint} (default: @code{""}) (type: string)
+Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
+
+@item @code{environment} (default: @code{()}) (type: list)
+Set environment variables. This can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '("LANGUAGE" . "eo:ca:eu")
+ "JAVA_HOME=/opt/java")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics.
+
+@item @code{image} (type: string)
+The image used to build the container. Images are resolved by the
+Docker Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.
+
+@item @code{name} (default: @code{""}) (type: string)
+Set a name for the spawned container.
+
+@item @code{network} (default: @code{""}) (type: string)
+Set a Docker network for the spawned container.
+
+@item @code{ports} (default: @code{()}) (type: list)
+Set the port or port ranges to expose from the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("8080" . "80")
+ "10443:443")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics.
+
+@item @code{volumes} (default: @code{()}) (type: list)
+Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("/root/data/grafana" . "/var/lib/grafana")
+ "/gnu/store:/gnu/store")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
@cindex Audit
@subsubheading Auditd Service
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
index c2023d618c..2d709bf2ce 100644
--- a/gnu/services/docker.scm
+++ b/gnu/services/docker.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2020 Jesse Dowell <jessedowell@gmail.com>
;;; Copyright © 2021 Brice Waegeneire <brice@waegenei.re>
+;;; Copyright © 2023 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -29,15 +30,34 @@ (define-module (gnu services docker)
#:use-module (gnu services shepherd)
#:use-module (gnu system setuid)
#:use-module (gnu system shadow)
+ #:use-module (gnu packages admin) ;shadow
#:use-module (gnu packages docker)
#:use-module (gnu packages linux) ;singularity
#:use-module (guix records)
+ #:use-module (guix diagnostics)
#:use-module (guix gexp)
+ #:use-module (guix i18n)
#:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
+ #:use-module (ice-9 match)
#:export (docker-configuration
docker-service-type
- singularity-service-type))
+ singularity-service-type
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-name
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-service-type
+ oci-container-shepherd-service))
(define-configuration docker-configuration
(docker
@@ -216,3 +236,218 @@ (define singularity-service-type
(service-extension activation-service-type
(const %singularity-activation))))
(default-value singularity)))
+
+\f
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (match pair
+ (((? valid? key) . (? valid? value))
+ #~(string-append #$key #$delimiter #$value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "pair members must contain only strings, gexps or file-like objects
+but ~a was found")
+ pair)))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (raise
+ (formatted-message
+ (G_ "~a members must be either a string or a pair but ~a was
+found!")
+ name el)))))
+ value))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define-maybe/no-serialization string)
+
+(define-configuration/no-serialization oci-container-configuration
+ (user
+ (string "oci-container")
+ "The user under whose authority docker commands will be run.")
+ (group
+ (string "docker")
+ "The group under whose authority docker commands will be run.")
+ (command
+ (list-of-strings '())
+ "Overwrite the default command (@code{CMD}) of the image.")
+ (entrypoint
+ (maybe-string)
+ "Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.")
+ (environment
+ (list '())
+ "Set environment variables. This can be a list of pairs or strings, even
+mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string)
+ "The image used to build the container. Images are resolved by the Docker
+Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.")
+ (name
+ (maybe-string)
+ "Set a name for the spawned container.")
+ (network
+ (maybe-string)
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container. This can
+be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"8080\" . \"80\")
+ \"10443:443\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"/root/data/grafana\" . \"/var/lib/grafana\")
+ \"/gnu/store:/gnu/store\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-volumes)))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(if (maybe-value-set? entrypoint)
+ `("--entrypoint" ,entrypoint)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(if (maybe-value-set? network)
+ `("--network" ,network)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (maybe-value-set? name)
+ name
+ (string-append "docker-"
+ (basename (car (string-split image #\:))))))
+
+ (let* ((docker-command (file-append docker-cli "/bin/docker"))
+ (user (oci-container-configuration-user config))
+ (group (oci-container-configuration-group config))
+ (command (oci-container-configuration-command config))
+ (config-name (oci-container-configuration-name config))
+ (image (oci-container-configuration-image config))
+ (options (oci-container-configuration->options config))
+ (name (guess-name config-name image)))
+
+ (shepherd-service (provision `(,(string->symbol name)))
+ (requirement '(dockerd user-processes))
+ (respawn? #f)
+ (documentation
+ (string-append
+ "Docker backed Shepherd service for image: " image))
+ (start
+ #~(make-forkexec-constructor
+ ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
+ (list #$docker-command "run" "--rm"
+ "--name" #$name
+ #$@options #$image #$@command)
+ #:user #$user
+ #:group #$group))
+ (stop
+ #~(lambda _
+ (invoke #$docker-command "rm" "-f" #$name)))
+ (actions
+ (list
+ (shepherd-action
+ (name 'pull)
+ (documentation
+ (format #f "Pull ~a's image (~a)."
+ name image))
+ (procedure
+ #~(lambda _
+ (invoke #$docker-command "pull" #$image)))))))))
+
+(define %oci-container-accounts
+ (list (user-account
+ (name "oci-container")
+ (comment "OCI services account")
+ (group "docker")
+ (system? #t)
+ (home-directory "/var/empty")
+ (shell (file-append shadow "/sbin/nologin")))))
+
+(define (configs->shepherd-services configs)
+ (map oci-container-shepherd-service configs))
+
+(define oci-container-service-type
+ (service-type (name 'oci-container)
+ (extensions (list (service-extension profile-service-type
+ (lambda _ (list docker-cli)))
+ (service-extension account-service-type
+ (const %oci-container-accounts))
+ (service-extension shepherd-root-service-type
+ configs->shepherd-services)))
+ (default-value '())
+ (extend append)
+ (compose concatenate)
+ (description
+ "This service allows the management of Docker and OCI
+containers as Shepherd services.")))
base-commit: 8aad7210ea06992ee3f36ca7f57678240949e063
--
2.41.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-10-14 21:29 ` paul via Guix-patches via
@ 2023-10-19 20:13 ` Ludovic Courtès
2023-10-19 21:16 ` paul via Guix-patches via
0 siblings, 1 reply; 17+ messages in thread
From: Ludovic Courtès @ 2023-10-19 20:13 UTC (permalink / raw)
To: paul; +Cc: 66160
Hello,
paul <goodoldpaul@autistici.org> skribis:
[...]
>> Does ‘docker run’ necessarily need to run as root, or are there cases
>> where one might want to run it as non-root? (I expect the latter.)
>
> yes you are right, it's only required to be in the docker group or in
> general have enough permission to operate on the docker daemon
> socket. I added a new service extension setting up an oci-container
> user, that it's just in the docker group and can not login, that runs
> oci backed services. it is also overridable by the user
In that case, maybe create an “oci-service” account part of the “docker”
group, and run ‘docker run’ as that user instead of running it as root?
Would that be OK or am I overlooking something?
>>> +(define oci-container-service-type
>>> + (service-type (name 'oci-container)
>>> + (extensions (list (service-extension profile-service-type
>>> + (lambda _ (list docker-cli)))
>>> + (service-extension shepherd-root-service-type
>>> + configs->shepherd-services)))
>>> + (default-value '())
>> I wonder if it should take a list of configs and be extensible, or
>> simply take a single config. Users would write:
>>
>> (service oci-container-service-type
>> (oci-container-configuration …))
>>
>> WDYT?
[...]
> Right now we can already do:
>
> (service oci-container-service-type
> (list
> (oci-container-configuration …)))
>
> and i updated the documentation accordingly. do you have any
> suggestion for changing the api of oci-container-configuration to
> support having different shepherd oci backed services behind a guix
> system service? This way we could remove the (list) call.
What I’m suggesting above is that one would build a list of
‘oci-container-service-type’ instances, like:
(list (service oci-container-service-type
(oci-container-configuration …))
(service oci-container-service-type
(oci-container-configuration …))
…)
Each instance above would correspond to exactly one program in a Docker
image.
I feel it’s slightly more natural than having a service type that
implements support for multiple OCI services at once.
> I actually looked around but didn't find them, but it was a long time
> ago and I certainly didn't look very well. I'm definitely up for
> testing this (maybe it's possible to use swineherd? could we use it
> for automating integration tests?), could you point me to something
> similar i can look to as an example?
Check out under gnu/tests/*.scm, in particular (gnu tests docker).
HTH!
Ludo’.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-10-19 20:13 ` Ludovic Courtès
@ 2023-10-19 21:16 ` paul via Guix-patches via
2023-10-24 15:41 ` Ludovic Courtès
0 siblings, 1 reply; 17+ messages in thread
From: paul via Guix-patches via @ 2023-10-19 21:16 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 66160
[-- Attachment #1: Type: text/plain, Size: 3012 bytes --]
Hello Ludo’ ,
On 10/19/23 22:13, Ludovic Courtès wrote:
> Hello,
>
> paul<goodoldpaul@autistici.org> skribis:
>
>
> [...]
>
>>> Does ‘docker run’ necessarily need to run as root, or are there cases
>>> where one might want to run it as non-root? (I expect the latter.)
>> yes you are right, it's only required to be in the docker group or in
>> general have enough permission to operate on the docker daemon
>> socket. I added a new service extension setting up an oci-container
>> user, that it's just in the docker group and can not login, that runs
>> oci backed services. it is also overridable by the user
> In that case, maybe create an “oci-service” account part of the “docker”
> group, and run ‘docker run’ as that user instead of running it as root?
> Would that be OK or am I overlooking something?
I already added such user in the latest version of my patch. I probably
made a mess with patch subjects.
> What I’m suggesting above is that one would build a list of
> ‘oci-container-service-type’ instances, like:
>
> (list (service oci-container-service-type
> (oci-container-configuration …))
> (service oci-container-service-type
> (oci-container-configuration …))
> …)
>
> Each instance above would correspond to exactly one program in a Docker
> image.
>
> I feel it’s slightly more natural than having a service type that
> implements support for multiple OCI services at once.
I agree it's more natural but (list service-a service-b ...) it's the
same interface exposed by the shepherd-root-service-type, I believe for
the same reasons I need the oci-nextcloud-service-type to instantiate 3
shepherd services but only create a single account, activate a single
data dir under /var/lib, something like this:
(defineoci-nextcloud-service-type
(service-type(name'nextcloud)
(extensions(list(service-extensionoci-container-service-type
(lambda (config) (make-nextcloud-container config)
(make-nextcloud-cron-container config)))
(service-extensionaccount-service-type
(const%nextcloud-accounts))
(service-extensionactivation-service-type
%nextcloud-activation)))
(default-value(nextcloud-configuration))
(description
"This service provides the Nextcloud service as an OCI-backed container.")))
The only way where oci-container-service-type could support this use
case by accepting a single configuration is I guess if multiple
(service-extension oci-container-service-type ...) where allowed, am I
understanding correctly? Is it legal in Guix to write somthing like:
(extensions(list(service-extensionoci-container-service-type
make-nextcloud-container)
(service-extensionoci-container-service-typemake-nextcloud-cron-container)
(service-extensionaccount-service-type
(const%nextcloud-accounts))
(service-extensionactivation-service-type
%nextcloud-activation)))
> Check out under gnu/tests/*.scm, in particular (gnu tests docker).
Thank you for the pointer, I'll look into those.
giacomo
[-- Attachment #2: Type: text/html, Size: 8940 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-10-19 21:16 ` paul via Guix-patches via
@ 2023-10-24 15:41 ` Ludovic Courtès
2023-10-24 20:22 ` paul via Guix-patches via
0 siblings, 1 reply; 17+ messages in thread
From: Ludovic Courtès @ 2023-10-24 15:41 UTC (permalink / raw)
To: paul; +Cc: 66160
Hi,
paul <goodoldpaul@autistici.org> skribis:
[...]
>> In that case, maybe create an “oci-service” account part of the “docker”
>> group, and run ‘docker run’ as that user instead of running it as root?
>> Would that be OK or am I overlooking something?
> I already added such user in the latest version of my patch. I
> probably made a mess with patch subjects.
Oh, my bad; perfect then.
>> What I’m suggesting above is that one would build a list of
>> ‘oci-container-service-type’ instances, like:
>>
>> (list (service oci-container-service-type
>> (oci-container-configuration …))
>> (service oci-container-service-type
>> (oci-container-configuration …))
>> …)
>>
>> Each instance above would correspond to exactly one program in a Docker
>> image.
>>
>> I feel it’s slightly more natural than having a service type that
>> implements support for multiple OCI services at once.
> I agree it's more natural but (list service-a service-b ...) it's the
> same interface exposed by the shepherd-root-service-type, I believe
> for the same reasons I need the oci-nextcloud-service-type to
> instantiate 3 shepherd services but only create a single account,
> activate a single data dir under /var/lib, something like this:
>
> (defineoci-nextcloud-service-type
> (service-type(name'nextcloud)
> (extensions(list(service-extensionoci-container-service-type
> (lambda (config) (make-nextcloud-container config)
> (make-nextcloud-cron-container config)))
[...]
> The only way where oci-container-service-type could support this use
> case by accepting a single configuration is I guess if multiple
> (service-extension oci-container-service-type ...) where allowed, am I
> understanding correctly? Is it legal in Guix to write somthing like:
>
> (extensions(list(service-extensionoci-container-service-type
> make-nextcloud-container)
> (service-extensionoci-container-service-typemake-nextcloud-cron-container)
> (service-extensionaccount-service-type
> (const%nextcloud-accounts))
> (service-extensionactivation-service-type
> %nextcloud-activation)))
If you take the route of one ‘oci-container-service-type’ per
daemon/server that you want to run, then <oci-container-configuration>
should probably have a ‘user’ field to specify under which user to run
the container. ‘oci-container-service-type’ would create exactly one
Shepherd service so, likewise, <oci-container-configuration> would need
a ‘provision’ field to specify the Shepherd service name (the
“provisions”). Likewise, perhaps a field to specify the data directory
is needed.
Does that make sense?
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH] gnu: Add oci-container-service-type.
2023-10-24 15:41 ` Ludovic Courtès
@ 2023-10-24 20:22 ` paul via Guix-patches via
0 siblings, 0 replies; 17+ messages in thread
From: paul via Guix-patches via @ 2023-10-24 20:22 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 66160
Hi Ludo’ ,
thank you for your explanation.
On 10/24/23 17:41, Ludovic Courtès wrote:
> If you take the route of one ‘oci-container-service-type’ per
> daemon/server that you want to run,
what you mention below is already implemented in my latest patch [0]:
> then <oci-container-configuration>
> should probably have a ‘user’ field to specify under which user to run
> the container.
this is oci-container-configuration-user
> <oci-container-configuration> would need
> a ‘provision’ field to specify the Shepherd service name (the
> “provisions”).
this is oci-container-configuration-name. I now realize that "name" it's
not the best field name, so I'm sending a patch with this renamed to
oci-container-configuration-provision .
> Likewise, perhaps a field to specify the data directory
> is needed.
I don't think oci-container-configuration should concern about a data
directory since oci containers themselves only have a volume concept
which is covered. what you brought into my mind is that docker supports
-w/--workdir so I implemented it and added it to the patch I'm about to
send.
> Does that make sense?
Yes, thank you :) My doubts come from shepherd-root-service-type
accepting a list of services. What would be the reason to break
consistency with it? I think we would add the friction of having to write
(service nextcloud-cron-oci-service-type)
(service nextcloud-oci-service-type)
instead of simply
(service nextcloud-cron-oci-service-type)
One way out of this if you think is a good solution could be having an
oci-containers-service-type that's supposed to be only extended whose
value would be a list of <oci-container-configuration> and an
oci-single-service-type that could not be extended whose value would be
a single <oci-container-configuration> . The oci-single-service-type
would simply extend the oci-containers-service-type and maintenance
would be free.
What do you think?
Thank you for your help,
giacomo
[0]: https://issues.guix.gnu.org/66160#10-lineno69
^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH v2] gnu: Add oci-container-service-type.
2023-09-22 20:32 [bug#66160] [PATCH] gnu: Add oci-container-service-type paul via Guix-patches via
` (4 preceding siblings ...)
2023-10-14 21:47 ` Giacomo Leidi via Guix-patches via
@ 2023-10-24 20:59 ` Giacomo Leidi via Guix-patches via
2023-11-23 10:02 ` Ludovic Courtès
5 siblings, 1 reply; 17+ messages in thread
From: Giacomo Leidi via Guix-patches via @ 2023-10-24 20:59 UTC (permalink / raw)
To: 66160; +Cc: Giacomo Leidi
* gnu/services/docker.scm (oci-container-configuration): New variable;
(oci-container-shepherd-service): new variable;
(oci-container-service-type): new variable.
* doc/guix.texi: Document it.
---
doc/guix.texi | 131 ++++++++++++++++++++
gnu/services/docker.scm | 260 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 390 insertions(+), 1 deletion(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index b90078be06..f6363a16a3 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -39518,6 +39518,137 @@ Miscellaneous Services
@command{singularity run} and similar commands.
@end defvar
+@cindex OCI-backed, Shepherd services
+@subsubheading OCI backed services
+
+Should you wish to manage your Docker containers with the same consistent
+interface you use for your other Shepherd services,
+@var{oci-container-service-type} is the tool to use: given an
+@acronym{Open Container Initiative, OCI} container image, it will run it in a
+Shepherd service. One example where this is useful: it lets you run services
+that are available as Docker/OCI images but not yet packaged for Guix.
+
+@defvar oci-container-service-type
+
+This is a thin wrapper around Docker's CLI that executes OCI images backed
+processes as Shepherd Services.
+
+@lisp
+(service oci-container-service-type
+ (list
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (ports
+ '(("3000" . "3000")))
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana")))))
+@end lisp
+
+In this example two different Shepherd services are going be added to the
+system. Each @code{oci-container-configuration} record translates to a
+@code{docker run} invocation and its fields directly map to options. You can
+refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run,upstream},
+documentation for the semantics of each value. If the images are not found they
+will be
+@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The
+spawned services are going to be attached to the host network and are supposed
+to behave like other processes.
+
+@end defvar
+
+@c %start of fragment
+
+@deftp {Data Type} oci-container-configuration
+Available @code{oci-container-configuration} fields are:
+
+@table @asis
+@item @code{user} (default: @code{"oci-container"}) (type: string)
+The user under whose authority docker commands will be run.
+
+@item @code{group} (default: @code{"docker"}) (type: string)
+The group under whose authority docker commands will be run.
+
+@item @code{command} (default: @code{()}) (type: list-of-strings)
+Overwrite the default command (@code{CMD}) of the image.
+
+@item @code{entrypoint} (default: @code{""}) (type: string)
+Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
+
+@item @code{environment} (default: @code{()}) (type: list)
+Set environment variables. This can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '("LANGUAGE" . "eo:ca:eu")
+ "JAVA_HOME=/opt/java")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics.
+
+@item @code{image} (type: string)
+The image used to build the container. Images are resolved by the
+Docker Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.
+
+@item @code{provision} (default: @code{""}) (type: string)
+Set the name of the provisioned Shepherd service.
+
+@item @code{network} (default: @code{""}) (type: string)
+Set a Docker network for the spawned container.
+
+@item @code{ports} (default: @code{()}) (type: list)
+Set the port or port ranges to expose from the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("8080" . "80")
+ "10443:443")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics.
+
+@item @code{volumes} (default: @code{()}) (type: list)
+Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '("/root/data/grafana" . "/var/lib/grafana")
+ "/gnu/store:/gnu/store")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@uref{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics.
+
+@item @code{container-user} (default: @code{""}) (type: string)
+Set the current user inside the spawned container. You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#user,upstream}
+documentation for semantics.
+
+@item @code{workdir} (default: @code{""}) (type: string)
+Set the current working for the spawned Shepherd service.
+You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#workdir,upstream}
+documentation for semantics.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
@cindex Audit
@subsubheading Auditd Service
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
index c2023d618c..ebea0a473a 100644
--- a/gnu/services/docker.scm
+++ b/gnu/services/docker.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2020 Jesse Dowell <jessedowell@gmail.com>
;;; Copyright © 2021 Brice Waegeneire <brice@waegenei.re>
+;;; Copyright © 2023 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -29,15 +30,36 @@ (define-module (gnu services docker)
#:use-module (gnu services shepherd)
#:use-module (gnu system setuid)
#:use-module (gnu system shadow)
+ #:use-module (gnu packages admin) ;shadow
#:use-module (gnu packages docker)
#:use-module (gnu packages linux) ;singularity
#:use-module (guix records)
+ #:use-module (guix diagnostics)
#:use-module (guix gexp)
+ #:use-module (guix i18n)
#:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
+ #:use-module (ice-9 match)
#:export (docker-configuration
docker-service-type
- singularity-service-type))
+ singularity-service-type
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-user
+ oci-container-configuration-group
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-provision
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-service-type
+ oci-container-shepherd-service))
(define-configuration docker-configuration
(docker
@@ -216,3 +238,239 @@ (define singularity-service-type
(service-extension activation-service-type
(const %singularity-activation))))
(default-value singularity)))
+
+\f
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (match pair
+ (((? valid? key) . (? valid? value))
+ #~(string-append #$key #$delimiter #$value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "pair members must contain only strings, gexps or file-like objects
+but ~a was found")
+ pair)))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (raise
+ (formatted-message
+ (G_ "~a members must be either a string or a pair but ~a was
+found!")
+ name el)))))
+ value))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define-maybe/no-serialization string)
+
+(define-configuration/no-serialization oci-container-configuration
+ (user
+ (string "oci-container")
+ "The user under whose authority docker commands will be run.")
+ (group
+ (string "docker")
+ "The group under whose authority docker commands will be run.")
+ (command
+ (list-of-strings '())
+ "Overwrite the default command (@code{CMD}) of the image.")
+ (entrypoint
+ (maybe-string)
+ "Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.")
+ (environment
+ (list '())
+ "Set environment variables. This can be a list of pairs or strings, even
+mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string)
+ "The image used to build the container. Images are resolved by the Docker
+Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.")
+ (provision
+ (maybe-string)
+ "Set the name of the provisioned Shepherd service.")
+ (network
+ (maybe-string)
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container. This can
+be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"8080\" . \"80\")
+ \"10443:443\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"/root/data/grafana\" . \"/var/lib/grafana\")
+ \"/gnu/store:/gnu/store\")
+@end lisp
+
+String are passed directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-volumes))
+ (container-user
+ (maybe-string)
+ "Set the current user inside the spawned container. You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#user,upstream}
+documentation for semantics.")
+ (workdir
+ (maybe-string)
+ "Set the current working for the spawned Shepherd service.
+You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#workdir,upstream}
+documentation for semantics."))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config))
+ (user
+ (oci-container-configuration-user config))
+ (workdir
+ (oci-container-configuration-workdir config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(if (maybe-value-set? entrypoint)
+ `("--entrypoint" ,entrypoint)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(if (maybe-value-set? network)
+ `("--network" ,network)
+ '())
+ ,(if (maybe-value-set? user)
+ `("--user" ,user)
+ '())
+ ,(if (maybe-value-set? workdir)
+ `("--workdir" ,workdir)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (maybe-value-set? name)
+ name
+ (string-append "docker-"
+ (basename (car (string-split image #\:))))))
+
+ (let* ((docker-command (file-append docker-cli "/bin/docker"))
+ (user (oci-container-configuration-user config))
+ (group (oci-container-configuration-group config))
+ (command (oci-container-configuration-command config))
+ (provision (oci-container-configuration-provision config))
+ (image (oci-container-configuration-image config))
+ (options (oci-container-configuration->options config))
+ (name (guess-name provision image)))
+
+ (shepherd-service (provision `(,(string->symbol name)))
+ (requirement '(dockerd user-processes))
+ (respawn? #f)
+ (documentation
+ (string-append
+ "Docker backed Shepherd service for image: " image))
+ (start
+ #~(make-forkexec-constructor
+ ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
+ (list #$docker-command "run" "--rm"
+ "--name" #$name
+ #$@options #$image #$@command)
+ #:user #$user
+ #:group #$group))
+ (stop
+ #~(lambda _
+ (invoke #$docker-command "rm" "-f" #$name)))
+ (actions
+ (list
+ (shepherd-action
+ (name 'pull)
+ (documentation
+ (format #f "Pull ~a's image (~a)."
+ name image))
+ (procedure
+ #~(lambda _
+ (invoke #$docker-command "pull" #$image)))))))))
+
+(define %oci-container-accounts
+ (list (user-account
+ (name "oci-container")
+ (comment "OCI services account")
+ (group "docker")
+ (system? #t)
+ (home-directory "/var/empty")
+ (shell (file-append shadow "/sbin/nologin")))))
+
+(define (configs->shepherd-services configs)
+ (map oci-container-shepherd-service configs))
+
+(define oci-container-service-type
+ (service-type (name 'oci-container)
+ (extensions (list (service-extension profile-service-type
+ (lambda _ (list docker-cli)))
+ (service-extension account-service-type
+ (const %oci-container-accounts))
+ (service-extension shepherd-root-service-type
+ configs->shepherd-services)))
+ (default-value '())
+ (extend append)
+ (compose concatenate)
+ (description
+ "This service allows the management of Docker and OCI
+containers as Shepherd services.")))
base-commit: 7e4324575c80cbe3c18c26b0507776b16ba3989e
--
2.41.0
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#66160] [PATCH v2] gnu: Add oci-container-service-type.
2023-10-24 20:59 ` [bug#66160] [PATCH v2] " Giacomo Leidi via Guix-patches via
@ 2023-11-23 10:02 ` Ludovic Courtès
0 siblings, 0 replies; 17+ messages in thread
From: Ludovic Courtès @ 2023-11-23 10:02 UTC (permalink / raw)
To: Giacomo Leidi; +Cc: 66160
Hi,
Giacomo Leidi <goodoldpaul@autistici.org> skribis:
> * gnu/services/docker.scm (oci-container-configuration): New variable;
> (oci-container-shepherd-service): new variable;
> (oci-container-service-type): new variable.
> * doc/guix.texi: Document it.
Finally applied, thanks!
As discussed earlier, please follow up with a patch adding a system test
so that the service doesn’t bitrot.
Ludo’.
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2023-11-23 10:03 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-22 20:32 [bug#66160] [PATCH] gnu: Add oci-container-service-type paul via Guix-patches via
2023-09-22 20:34 ` Giacomo Leidi via Guix-patches via
2023-10-05 14:30 ` Ludovic Courtès
2023-10-05 17:30 ` paul via Guix-patches via
2023-10-13 22:53 ` paul via Guix-patches via
2023-10-06 19:09 ` Giacomo Leidi via Guix-patches via
2023-10-14 16:09 ` Ludovic Courtès
2023-10-14 21:29 ` paul via Guix-patches via
2023-10-19 20:13 ` Ludovic Courtès
2023-10-19 21:16 ` paul via Guix-patches via
2023-10-24 15:41 ` Ludovic Courtès
2023-10-24 20:22 ` paul via Guix-patches via
2023-10-13 22:57 ` Giacomo Leidi via Guix-patches via
2023-10-14 21:36 ` Giacomo Leidi via Guix-patches via
2023-10-14 21:47 ` Giacomo Leidi via Guix-patches via
2023-10-24 20:59 ` [bug#66160] [PATCH v2] " Giacomo Leidi via Guix-patches via
2023-11-23 10:02 ` Ludovic Courtès
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.