[bug#70265] Add docker cli Guix Home service and some docker authentication plugins

[bug#70265] Add docker cli Guix Home service and some docker authentication plugins

[paul via Guix-patches via]

Hello Guixers,


I'm sending a patchset to:


- add two docker cli authentication plugins for libsecret and pass 
respectively

- a Guix Home service to make the docker cli aware of Guix provided 
plugins. this could be used for docker compose v2 in the future ( I 
actually do use it now with a binary package and it works as far as I 
can tell).


Thank you for your work,

giacomo

[bug#70265] [PATCH 1/3] gnu: Add docker-credential-secretservice.

[Giacomo Leidi via Guix-patches via]

* gnu/packages/docker.scm (docker-credential-helpers): New variable.
* gnu/packages/gnome.scm (docker-credential-secretservice): New variable.

Change-Id: I6c46d429fa2842969b0fcde58ded72e5b04ee321
---
 gnu/packages/docker.scm | 70 ++++++++++++++++++++++++++++++++++++++++-
 gnu/packages/gnome.scm  |  6 +++-
 2 files changed, 74 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 0fe1f2c1c7..31501e50b9 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -8,6 +8,7 @@
 ;;; Copyright © 2020 Jesse Dowell <jessedowell@gmail.com>
 ;;; Copyright © 2021, 2022 Oleg Pykhalov <go.wigust@gmail.com>
 ;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -51,7 +52,8 @@ (define-module (gnu packages docker)
   #:use-module (gnu packages python-web)
   #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages version-control)
-  #:use-module (gnu packages virtualization))
+  #:use-module (gnu packages virtualization)
+  #:export (docker-credential-helpers))
 
 ;; Note - when changing Docker versions it is important to update the versions
 ;; of several associated packages (docker-libnetwork and go-sctp).
@@ -670,6 +672,72 @@ (define-public docker-cli
     (home-page "https://www.docker.com/")
     (license license:asl2.0)))
 
+;; Actual users of this procedure are
+;; docker-credentials-secretservice and docker-credential-pass, they live in
+;; different modules to avoid circular imports.
+(define* (docker-credential-helpers plugin-name #:key (inputs '()))
+  (package
+    (name (string-append "docker-credential-" plugin-name))
+    (version "0.8.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/docker/docker-credential-helpers")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "1kric2yrgypdqncqfrmrh7l7904km5zisygi3fg6zlfkyh6rsm23"))))
+    (build-system go-build-system)
+    (arguments
+     (list
+      #:install-source? #f
+      #:go go-1.19
+      #:unpack-path "github.com/docker/docker-credential-helpers"
+      #:import-path
+      (string-append "github.com/docker/docker-credential-helpers/"
+                     plugin-name "/cmd")
+      #:phases
+      #~(modify-phases %standard-phases
+          (replace 'build
+            (lambda* (#:key unpack-path import-path build-flags #:allow-other-keys)
+              (apply invoke "go" "build"
+                     "-v"
+                     "-x"
+                     (string-append "-ldflags=-s -w "
+                                    "-X github.com/docker/docker-credential-helpers"
+                                    "/credentials.Version=" #$version " "
+                                    "-X github.com/docker/docker-credential-helpers"
+                                    "/credentials.Package=" unpack-path " "
+                                    "-X github.com/docker/docker-credential-helpers"
+                                    "/credentials.Name=" #$name)
+                     "-o" (string-append "bin/" #$name)
+                     `(,@build-flags ,import-path))))
+          (replace 'install
+            (lambda _
+              (let* ((bin
+                     (string-append #$output "/bin"))
+                    (lib
+                     (string-append #$output "/libexec/docker/cli-plugins"))
+                    (entrypoint
+                     (string-append lib "/" #$name)))
+                (mkdir-p bin)
+                (mkdir-p lib)
+                (copy-file (string-append "bin/" #$name) entrypoint)
+                (symlink entrypoint
+                         (string-append bin "/" #$name))))))))
+    (native-inputs
+     (list pkg-config))
+    (inputs inputs)
+    (home-page "https://github.com/docker/docker-credential-helpers")
+    (synopsis "Store Docker login credentials in platform keystores")
+    (description
+     (string-append "docker-credential-helpers is a suite of programs to use native stores to keep
+Docker credentials safe.
+
+This package provides the @code{" name "} plugin."))
+    (license license:expat)))
+
 (define-public cqfd
   (package
     (name "cqfd")
diff --git a/gnu/packages/gnome.scm b/gnu/packages/gnome.scm
index 06256066bc..58b53aba22 100644
--- a/gnu/packages/gnome.scm
+++ b/gnu/packages/gnome.scm
@@ -36,7 +36,7 @@
 ;;; Copyright © 2019 Danny Milosavljevic <dannym@scratchpost.org>
 ;;; Copyright © 2019, 2020, 2022 Marius Bakke <marius@gnu.org>
 ;;; Copyright © 2019 Florian Pelz <pelzflorian@pelzflorian.de>
-;;; Copyright © 2019 Giacomo Leidi <goodoldpaul@autistici.org>
+;;; Copyright © 2019, 2024 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;; Copyright © 2019 Jelle Licht <jlicht@fsfe.org>
 ;;; Copyright © 2019 Jonathan Frederickson <jonathan@terracrypt.net>
 ;;; Copyright © 2019, 2020, 2021, 2022, 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
@@ -5229,6 +5229,10 @@ (define-public libsecret
 and other secrets.  It communicates with the \"Secret Service\" using DBus.")
     (license license:lgpl2.1+)))
 
+(define-public docker-credential-secretservice
+  (docker-credential-helpers "secretservice"
+                             #:inputs (list libsecret)))
+
 (define-public five-or-more
   (package
     (name "five-or-more")

base-commit: 69951a61a1d8f1f2135ea2dc836738be282b97bc
-- 
2.41.0

[bug#70265] [PATCH 2/3] gnu: Add docker-credential-pass.

[Giacomo Leidi via Guix-patches via]

* gnu/packages/gnome.scm (docker-credential-pass): New variable.

Change-Id: I442ff509aaed8cc3809de27714710abd24f7e8e0
---
 gnu/packages/gnome.scm | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/gnu/packages/gnome.scm b/gnu/packages/gnome.scm
index 58b53aba22..6f5f28e008 100644
--- a/gnu/packages/gnome.scm
+++ b/gnu/packages/gnome.scm
@@ -5229,6 +5229,10 @@ (define-public libsecret
 and other secrets.  It communicates with the \"Secret Service\" using DBus.")
     (license license:lgpl2.1+)))
 
+(define-public docker-credential-pass
+  (docker-credential-helpers "pass"
+                             #:inputs (list password-store)))
+
 (define-public docker-credential-secretservice
   (docker-credential-helpers "secretservice"
                              #:inputs (list libsecret)))
-- 
2.41.0

[bug#70265] [PATCH 3/3] gnu: Add home-docker-cli-service-type.

[Giacomo Leidi via Guix-patches via]

* gnu/home/services/containers.scm (home-docker-cli-service-type): New
variable;
* doc/guix.texi: document it;
* gnu/local.mk: add it.

Change-Id: I71e7a2805fe8754511c8e02ee4ae667e34e4aaf6
---
 doc/guix.texi                    |  53 +++++++++++
 gnu/home/services/containers.scm | 148 +++++++++++++++++++++++++++++++
 gnu/local.mk                     |   1 +
 3 files changed, 202 insertions(+)
 create mode 100644 gnu/home/services/containers.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 20f007b1c0..61cc904f9d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -46500,6 +46500,59 @@ Miscellaneous Home Services
           (dicod-configuration @dots{})))
 @end lisp
 
+@subsubheading Container Services
+
+@cindex docker cli service, for Home
+The @code{(gnu home services containers)} module provides the following service:
+
+@defvar home-docker-cli-service-type
+This service allows for configuring the Docker command line interface, for
+example to make it aware of Guix provided plugins.
+@end defvar
+
+For example, you can use it like this to make Docker safely store your registry
+credentials with the system
+@uref{https://wiki.gnome.org/Projects/Libsecret, libsecret} compatible Secret service:
+
+@lisp
+(use-modules (gnu packages docker))
+
+(service home-docker-cli-service-type
+         (docker-cli-configuration
+           (creds-store "secretservice")
+           (cli-plugins
+             (list docker-credential-secretservice))
+           (extra-content ", \"auths\": @{\"https://index.docker.io/v1/\": @{@}@}")))
+@end lisp
+
+
+@c %start of fragment
+
+@deftp {Data Type} docker-cli-configuration
+Available @code{docker-cli-configuration} fields are:
+
+@table @asis
+@item @code{docker-cli} (default: @code{docker-cli}) (type: package)
+The Docker cli package installed to the Home profile.
+
+@item @code{creds-store} (type: maybe-string)
+A native secrets store used to store Docker credentials.
+
+@item @code{cli-plugins} (default: @code{()}) (type: list-of-docker-cli-plugins)
+A list of Docker cli plugin package records that will be configured to
+work with Docker's cli.
+
+@item @code{extra-content} (default: @code{""}) (type: string)
+Additional literal content that will be appended to Docker cli
+config.json.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
 @node Invoking guix home
 @section Invoking @command{guix home}
 
diff --git a/gnu/home/services/containers.scm b/gnu/home/services/containers.scm
new file mode 100644
index 0000000000..89d6ad5f39
--- /dev/null
+++ b/gnu/home/services/containers.scm
@@ -0,0 +1,148 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify
+;;; it under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation, either version 3 of the License, or
+;;; (at your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful,
+;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu home services containers)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+  #:use-module (gnu packages docker)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu home services)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 string-fun)
+  #:export (docker-cli-configuration
+            docker-cli-configuration?
+            docker-cli-configuration-fields
+            docker-cli-configuration-docker-cli
+            docker-cli-configuration-creds-store
+            docker-cli-configuration-cli-plugins
+
+            home-docker-cli-service-type
+            home-docker-cli-configuration-file
+            docker-cli-configuration->json-fields))
+
+;; Turn field names, which are Scheme symbols into strings
+;; cli-plugins-extra-dirs -> cliPluginsExtraDirs
+(define (format-name name)
+  (define without-dashes (string-replace-substring (symbol->string name) "-" " "))
+  (define splitted (string-split without-dashes #\space))
+  (string-replace-substring
+   (apply string-append
+          `(,(car splitted)
+            ,@(map string-capitalize (cdr splitted))))
+   " " ""))
+
+(define (serialize-json-value name value)
+  #~(begin
+      (use-modules (ice-9 format))
+      (format #f "\"~a\": ~a" #$(format-name name) #$value)))
+
+(define (serialize-json-list values)
+  #~(string-append "["
+               (string-join
+                (map (lambda (s) (string-append "\"" s "\""))
+                     (list #$@values))
+                ", ")
+               "]"))
+
+(define (serialize-string field-name value)
+  (serialize-json-value field-name (string-append "\"" value "\"")))
+
+(define (serialize-maybe-string field-name value)
+  (if (maybe-value-set? value)
+      (serialize-string field-name value)
+      '()))
+
+(define (serialize-list-of-strings field-name value)
+  (serialize-json-value field-name (serialize-json-list value)))
+
+(define (serialize-list-of-docker-cli-plugins value)
+  (serialize-json-value 'cli-plugins-extra-dirs
+                        (serialize-json-list
+                         (map (lambda (p)
+                                (file-append p "/libexec/docker/cli-plugins"))
+                              value))))
+
+(define list-of-strings?
+  (list-of string?))
+
+(define list-of-docker-cli-plugins?
+  (list-of package?))
+
+(define-maybe string)
+
+(define-configuration/no-serialization docker-cli-configuration
+  (docker-cli
+   (package docker-cli)
+   "The Docker cli package installed to the Home profile.")
+  (creds-store
+   (maybe-string)
+   "A native secrets store used to store Docker credentials.")
+  (cli-plugins
+   (list-of-docker-cli-plugins '())
+   "A list of Docker cli plugin package records that will be configured to work with Docker's cli.")
+  (extra-content
+   (string "")
+   "Additional literal content that will be appended to Docker cli config.json."))
+
+(define docker-cli-configuration->json-fields
+  (lambda (config)
+    (filter (compose not (lambda (f) (or (null? f) (and (string? f) (string-null? f)))))
+            (map (lambda (f)
+                   (let ((field-name (configuration-field-name f))
+                         (type (configuration-field-type f))
+                         (value ((configuration-field-getter f) config)))
+                     (if (not (member field-name '(docker-cli extra-content)))
+                         (match type
+                           ('string
+                            (serialize-string field-name value))
+                           ('maybe-string
+                            (serialize-maybe-string field-name value))
+                           ('list-of-strings
+                            (serialize-list-of-strings field-name value))
+                           ('list-of-docker-cli-plugins
+                            (serialize-list-of-docker-cli-plugins value))
+                           (_
+                            (raise
+                             (formatted-message
+                              (G_ "Unknown docker-cli-configuration field type: ~a")
+                              type))))
+                         '())))
+                 docker-cli-configuration-fields))))
+
+(define (home-docker-cli-configuration-file config)
+  `((".docker/config.json"
+     ,(computed-file "docker-cli-config.json"
+       #~(with-output-to-file #$output
+           (lambda _
+             (display
+              (string-append "{"
+                             (string-join (list #$@(docker-cli-configuration->json-fields config)) ",")
+                             #$(docker-cli-configuration-extra-content config)
+                             "}\n"))))))))
+
+(define home-docker-cli-service-type
+  (service-type (name 'docker-cli)
+                (extensions (list (service-extension home-profile-service-type
+                                                     (lambda (config)
+                                                       `(,(docker-cli-configuration-docker-cli config)
+                                                         ,@(docker-cli-configuration-cli-plugins config))))
+                                  (service-extension home-files-service-type
+                                                     home-docker-cli-configuration-file)))
+                (default-value (docker-cli-configuration))
+                (description
+                 "This service install and configures Docker's command line interface.")))
diff --git a/gnu/local.mk b/gnu/local.mk
index 7f1006010b..4960f32a1a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -95,6 +95,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/compression.scm				\
   %D%/home.scm					\
   %D%/home/services.scm			\
+  %D%/home/services/containers.scm		\
   %D%/home/services/desktop.scm			\
   %D%/home/services/dict.scm			\
   %D%/home/services/dotfiles.scm		\
-- 
2.41.0

[bug#70265] Add docker cli Guix Home service and some docker authentication plugins

[Ludovic Courtès]

Hi,

Giacomo Leidi <goodoldpaul@autistici.org> skribis:

> * gnu/home/services/containers.scm (home-docker-cli-service-type): New
> variable;
> * doc/guix.texi: document it;
> * gnu/local.mk: add it.

Nice.  For the documentation part, the convention is something like:

  * doc/guix.texi (Miscellaneous Services)[Container Services]: New
  heading.

> +@cindex docker cli service, for Home

s/docker cli/Docker command-line interface/

> +The @code{(gnu home services containers)} module provides the following service:
> +
> +@defvar home-docker-cli-service-type
> +This service allows for configuring the Docker command line interface, for
> +example to make it aware of Guix provided plugins.
> +@end defvar
> +
> +For example, you can use it like this to make Docker safely store your registry
> +credentials with the system
> +@uref{https://wiki.gnome.org/Projects/Libsecret, libsecret} compatible Secret service:

Rather:

  @uref{https://wiki.gnome.org/Projects/Libsecret, libsecret-compatible}
  secret service:

(So that the parenthesized URL appears in the right place in Info and
PDF.)

> +@lisp
> +(use-modules (gnu packages docker))
> +
> +(service home-docker-cli-service-type
> +         (docker-cli-configuration
> +           (creds-store "secretservice")
> +           (cli-plugins
> +             (list docker-credential-secretservice))
> +           (extra-content ", \"auths\": @{\"https://index.docker.io/v1/\": @{@}@}")))

Sounds scary: how can I know as a user where that comma is going to be
stuck in the resulting file?  Providing partial JSON strings should
rather be avoided IMO.

Also, if that part is necessary, it should be explained.

> +(define (format-name name)
> +  (define without-dashes (string-replace-substring (symbol->string name) "-" " "))
> +  (define splitted (string-split without-dashes #\space))
> +  (string-replace-substring
> +   (apply string-append
> +          `(,(car splitted)
> +            ,@(map string-capitalize (cdr splitted))))
> +   " " ""))

Rather:

  (match split  ;past participate of “to split”
    ((head . rest)
     (string-concatenate (cons head (map string-capitalize rest)))))

See
<https://guix.gnu.org/manual/devel/en/html_node/Data-Types-and-Pattern-Matching.html>
for the rationale.

> +(define docker-cli-configuration->json-fields
> +  (lambda (config)
> +    (filter (compose not (lambda (f) (or (null? f) (and (string? f) (string-null? f)))))
> +            (map (lambda (f)

To improve readability, I’d make it:

  (filter-map (lambda (f)
                (match f
                  (() #f)
                  ("" #f)
                  (_ …)))
               docker-cli-configuration-fields)

> +(define (home-docker-cli-configuration-file config)
> +  `((".docker/config.json"
> +     ,(computed-file "docker-cli-config.json"
> +       #~(with-output-to-file #$output
> +           (lambda _
> +             (display
> +              (string-append "{"
> +                             (string-join (list #$@(docker-cli-configuration->json-fields config)) ",")
> +                             #$(docker-cli-configuration-extra-content config)

I think the comma should be automatically added when ‘extra-content’ is
non-empty.

A more general question: do you think this particular example (libsecret
plugin) could be solved in another way without involving Home?  (For
example by having a plugin search path in the package.)

Do you have other use cases in mind?

The reason I’m asking is that it feels heavyweight for what looks like
“basic” Docker configuration.  But maybe Docker is like this and a Home
service is what it takes to make it more easily configurable (I’m not
really familiar with Docker), in which case I’m all for this patch
series!

Thanks,
Ludo’.

[bug#70265] Add docker cli Guix Home service and some docker authentication plugins

[paul via Guix-patches via]

Hello Ludo’ ,

I think you are raising a fair point. I was also not sure about whether 
to upstream this service, even if I have been using it for some months.

On 5/4/24 18:36, Ludovic Courtès wrote:
> A more general question: do you think this particular example (libsecret
> plugin) could be solved in another way without involving Home?  (For
> example by having a plugin search path in the package.)
>
> Do you have other use cases in mind?
My main use case for this is currently is docker compose v2. I packaged 
the binary from github in my personal channel [0] and with this service 
[1]in my home-environment I'm able to use docker compose (without dash) 
commands.
> The reason I’m asking is that it feels heavyweight for what looks like
> “basic” Docker configuration.

It definitely is and whether to accept this patch imho depends on the 
ETA for the optimal solution. Assuming someone packages docker compose 
v2 and its dependencies, then as far as I'm aware there are are 3 ways 
to make docker-cli aware of plugins on Guix:

1. this service, i.e. listing plugins in docker's configuration file. 
the advantage of this solution is that it doesn't require changing the 
docker-cli package. the disadvantage is that docker for some reason 
always tries to write the following to the config file, even if it is 
not needed :

"auths": { "https://index.docker.io/v1/": {} }

this is why i added it in the documentation, but as you pointed out it 
is useless.

2. We make a docker-cli-with-bundled-plugins function or similar that 
takes a list of plugin packages and hardcodes their paths into docker's 
source code before compilation. This is the approach I took here [2]. 
This currently requires recompilation every time since docker compose v2 
is not in guix.

3. We patch docker [3] to make it aware of plugins with a search path 
like DOCKER_CLI_PLUGINS . But I don't think this patch should live in 
Guix. It should go into docker mainline to make sure that it is 
supported also in future releases. I never tried contributing to docker, 
this does not seem a complex change but I wonder why it was never done 
until now.

Given all of the above the shortest way to achieve my goal of using 
docker compose v2 seemed #1 but I can see how for the Guix project the 
best would be #3. What do you think (and everyone CCed as well)? Could 
this service be a temporary workaround (after addressing your comments), 
while I try to see whether docker mainline could be interested in a patch?

I will address your comments after we reach consensus on how to proceed, 
so that I don't make everyone lose more time than I already have. Thank 
you all for your work,

giacomo

[0]: 
https://gitlab.com/orang3/small-guix/-/blob/master/small-guix/packages/compose.scm
[1]: 
https://gitlab.com/orang3/guix-deployments/-/blob/main/modules/common/home/fishinthecalculator/home-configuration.scm?ref_type=heads#L71
[2]: 
https://github.com/bonfire-networks/bonfire-app/blob/main/manifest.scm#L57
[3]: 
https://github.com/docker/cli/blob/master/cli-plugins/manager/manager_unix.go