From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([209.51.188.92]:40858) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hFoI9-0007RV-2i for guix-patches@gnu.org; Sun, 14 Apr 2019 19:12:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hFoI6-0006IT-Qh for guix-patches@gnu.org; Sun, 14 Apr 2019 19:12:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:47984) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hFoI6-0006IK-Kk for guix-patches@gnu.org; Sun, 14 Apr 2019 19:12:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hFoI6-0008My-EG for guix-patches@gnu.org; Sun, 14 Apr 2019 19:12:02 -0400 Subject: [bug#35281] [PATCH] gnu: docker: Add a couple go dependencies and enable docker-proxy. Resent-Message-ID: Received: from eggs.gnu.org ([209.51.188.92]:40443) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hFoHk-0006v7-09 for guix-patches@gnu.org; Sun, 14 Apr 2019 19:11:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hFo9L-000855-O1 for guix-patches@gnu.org; Sun, 14 Apr 2019 19:03:01 -0400 Received: from mail-qt1-x832.google.com ([2607:f8b0:4864:20::832]:42840) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hFo9L-00084O-0G for guix-patches@gnu.org; Sun, 14 Apr 2019 19:02:59 -0400 Received: by mail-qt1-x832.google.com with SMTP id p20so17135523qtc.9 for ; Sun, 14 Apr 2019 16:02:58 -0700 (PDT) Received: from kwak (dsl-10-136-224.b2b2c.ca. [72.10.136.224]) by smtp.gmail.com with ESMTPSA id r31sm28172903qtj.17.2019.04.14.16.02.56 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 14 Apr 2019 16:02:56 -0700 (PDT) From: Maxim Cournoyer Date: Sun, 14 Apr 2019 19:02:53 -0400 Message-ID: <87muksnqaq.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: 35281@debbugs.gnu.org --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Hello Guix! Before this change, attempting to start a docker registry such as with the following command[0]: --8<---------------cut here---------------start------------->8--- docker run -d -p 5000:5000 --restart=always --name registry registry:2 --8<---------------cut here---------------end--------------->8--- Would give the following error: --8<---------------cut here---------------start------------->8--- /gnu/store/dzaijl53fcd3jhkpd70vsf4cnvv10ywj-docker-cli-18.09.3/bin/docker: Error response from daemon: driver failed programming external connectivity on endpoint registry (709754084a9e208c32075e47ea9584296a6f274deeef08283d0de9c9a5161112): exec: "docker-proxy": executable file not found in $PATH. --8<---------------cut here---------------end--------------->8--- This series of patches adds docker-libnetwork-cmd-proxy and a few new go packages it required. The docker service is modified to enable (or disable) using a 'docker-proxy'. Thanks! Maxim --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-gnu-Add-go-sctp.patch Content-Transfer-Encoding: quoted-printable From=202e46dd6c449679ecae6a13a7a922eaf6b6947164 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Fri, 5 Apr 2019 01:07:58 -0400 Subject: [PATCH 1/6] gnu: Add go-sctp. * gnu/packages/networking.scm (go-sctp): New variable. =2D-- gnu/packages/networking.scm | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm index 46aed1e549..5163726393 100644 =2D-- a/gnu/packages/networking.scm +++ b/gnu/packages/networking.scm @@ -26,6 +26,7 @@ ;;; Copyright =C2=A9 2018 Marius Bakke ;;; Copyright =C2=A9 2018 Oleg Pykhalov ;;; Copyright =C2=A9 2018 Pierre Neidhardt +;;; Copyright =C2=A9 2019 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -50,6 +51,7 @@ #:use-module (guix build-system cmake) #:use-module (guix build-system glib-or-gtk) #:use-module (guix build-system gnu) + #:use-module (guix build-system go) #:use-module (guix build-system perl) #:use-module (guix build-system python) #:use-module (gnu packages) @@ -730,6 +732,31 @@ manage, and delete Internet resources from Gandi.net s= uch as domain names, virtual machines, and certificates.") (license license:gpl3+))) =20 +(define-public go-sctp + ;; docker-libnetwork-cmd-proxy requires this exact commit. + (let ((commit "07191f837fedd2f13d1ec7b5f885f0f3ec54b1cb") + (revision "1")) + (package + (name "go-sctp") + (version (git-version "0.0.0" revision commit)) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/ishidawataru/sctp.git") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "1mk9ncm10gwi5pn5wcw4skbyf4qg7n5qdf1mim4gf3mrckvi6g6h"))= )) + (build-system go-build-system) + (arguments + `(#:import-path "github.com/ishidawataru/sctp")) + (home-page "https://github.com/ishidawataru/sctp") + (synopsis "SCTP library for the Go programming language") + (description "This library provides methods for using the stream con= trol +transmission protocol (SCTP) in a Go application.") + (license license:asl2.0)))) + (define-public httping (package (name "httping") =2D-=20 2.20.1 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-gnu-Add-go-netns.patch Content-Transfer-Encoding: quoted-printable From=200081c4231f9e25879c287fca54ec9db4929d1711 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Fri, 5 Apr 2019 01:53:00 -0400 Subject: [PATCH 2/6] gnu: Add go-netns. * gnu/packages/networking.scm (go-netns): New variable. =2D-- gnu/packages/networking.scm | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm index 5163726393..1407f6208e 100644 =2D-- a/gnu/packages/networking.scm +++ b/gnu/packages/networking.scm @@ -732,6 +732,31 @@ manage, and delete Internet resources from Gandi.net s= uch as domain names, virtual machines, and certificates.") (license license:gpl3+))) =20 +(define-public go-netns + (let ((commit "13995c7128ccc8e51e9a6bd2b551020a27180abd") + (revision "1")) + (package + (name "go-netns") + (version (git-version "0.0.0" revision commit)) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/vishvananda/netns.git") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "1zk6w8158qi4niva5rijchbv9ixgmijsgqshh54wdaav4xrhjshn"))= )) + (build-system go-build-system) + (arguments + `(#:import-path "github.com/vishvananda/netns" + #:tests? #f)) ;tests require root privileges + (home-page "https://github.com/vishvananda/netns") + (synopsis "Simple network namespace handling for Go") + (description "The netns package provides a simple interface for +handling network namespaces in Go.") + (license license:asl2.0)))) + (define-public go-sctp ;; docker-libnetwork-cmd-proxy requires this exact commit. (let ((commit "07191f837fedd2f13d1ec7b5f885f0f3ec54b1cb") =2D-=20 2.20.1 --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0003-gnu-Add-go-netlink.patch Content-Transfer-Encoding: quoted-printable From=202a8b23da6fe7ef09f0931231c67f25cd4c60f24f Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Fri, 5 Apr 2019 01:57:44 -0400 Subject: [PATCH 3/6] gnu: Add go-netlink. * gnu/packages/linux.scm (go-netlink): New variable. =2D-- gnu/packages/linux.scm | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index e8ee4df4f3..c8adf52ff8 100644 =2D-- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -33,6 +33,7 @@ ;;; Copyright =C2=A9 2018 Pierre Langlois ;;; Copyright =C2=A9 2018 Vasile Dumitrascu ;;; Copyright =C2=A9 2019 Tim Gesthuizen +;;; Copyright =C2=A9 2019 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -77,6 +78,7 @@ #:use-module (gnu packages gcc) #:use-module (gnu packages gettext) #:use-module (gnu packages glib) + #:use-module (gnu packages golang) #:use-module (gnu packages gperf) #:use-module (gnu packages gtk) #:use-module (gnu packages libunwind) @@ -116,6 +118,7 @@ #:use-module (gnu packages swig) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) + #:use-module (guix build-system go) #:use-module (guix build-system python) #:use-module (guix build-system trivial) #:use-module (guix download) @@ -5151,6 +5154,33 @@ nfnetlink_queue, nfnetlink_conntrack) and their resp= ective users and/or management tools in userspace.") (license license:gpl2))) =20 +(define-public go-netlink + (package + (name "go-netlink") + (version "1.0.0") + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/vishvananda/netlink.git") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "0hpzghf1a4cwawzhkiwdzin80h6hd09fskl77d5ppgc084yvj8x0")))) + (build-system go-build-system) + (arguments + `(#:import-path "github.com/vishvananda/netlink")) + (native-inputs + `(("go-golang-org-x-sys-unix" ,go-golang-org-x-sys-unix) + ("go-netns" ,go-netns))) + (home-page "https://github.com/vishvananda/netlink") + (synopsis "Simple netlink library for Go") + (description "The netlink package provides a simple netlink library for +Go. Netlink is the interface a user-space program in Linux uses to +communicate with the kernel. It can be used to add and remove interfaces,= set +IP addresses and routes, and configure IPsec.") + (license license:asl2.0))) + (define-public xfsprogs (package (name "xfsprogs") =2D-=20 2.20.1 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0004-gnu-Add-docker-libnetwork.patch Content-Transfer-Encoding: quoted-printable From=20721d1a93961bf653a02cccdbfaf92e0514a3ae59 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Fri, 5 Apr 2019 02:32:40 -0400 Subject: [PATCH 4/6] gnu: Add docker-libnetwork. * gnu/packages/docker.scm (docker-libnetwork): New private variable. =2D-- gnu/packages/docker.scm | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm index a11ce266d2..df0bbca1bc 100644 =2D-- a/gnu/packages/docker.scm +++ b/gnu/packages/docker.scm @@ -227,6 +227,47 @@ network attachments.") (home-page "http://containerd.io/") (license license:asl2.0))) =20 +;;; Private package that shouldn't be used directly; its purposes is to be= used +;;; as a template for the various packages it contains. +(define docker-libnetwork + ;; There are no recent release for libnetwork, so choose the last commit= of + ;; the branch that Docker uses, as can be seen in the Docker source file + ;; 'hack/dockerfile/install/proxy.installer'. + (let ((commit "4725f2163fb214a6312f3beae5991f838ec36326") + (version "18.09") + (revision "1")) + (package + (name "docker-libnetwork") + (version (git-version version "1" commit)) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/docker/libnetwork.git") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "1zpnxki8qfzha6ljahpwd3vkzmjhsvkmf73w6crm4ilxxw5vnpfb")) + ;; Delete bundled ("vendored") free software source code. + (modules '((guix build utils))) + (snippet '(begin + (delete-file-recursively "vendor") + #t)))) + (build-system go-build-system) + (arguments + `(#:import-path "github.com/docker/libnetwork/" + ;; The tests fail with the error: + ;; src/github.com/docker/libnetwork/network.go:1057: Warnf format= %q + ;; has arg n.configOnly of wrong type bool. + #:tests? #f)) + (home-page "https://github.com/docker/libnetwork/") + (synopsis "Networking for containers") + (description "Libnetwork provides a native Go implementation for +connecting containers. The goal of @code{libnetwork} is to deliver a robu= st +container network model that provides a consistent programming interface a= nd +the required network abstractions for applications.") + (license license:asl2.0)))) + ;; TODO: Patch out modprobes for ip_vs, nf_conntrack, ;; brige, nf_conntrack_netlink, aufs. (define-public docker =2D-=20 2.20.1 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0005-gnu-Add-docker-libnetwork-cmd-proxy.patch Content-Transfer-Encoding: quoted-printable From=20843ecd2ff5aa5f69ea8a83f2da8e0d783be4b36a Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Fri, 5 Apr 2019 02:33:38 -0400 Subject: [PATCH 5/6] gnu: Add docker-libnetwork-cmd-proxy. * gnu/packages/docker.scm (docker-libnetwork-cmd-proxy): New variable. =2D-- gnu/packages/docker.scm | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm index df0bbca1bc..1067555296 100644 =2D-- a/gnu/packages/docker.scm +++ b/gnu/packages/docker.scm @@ -36,6 +36,7 @@ #:use-module (gnu packages glib) #:use-module (gnu packages golang) #:use-module (gnu packages linux) + #:use-module (gnu packages networking) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) #:use-module (gnu packages python-web) @@ -268,6 +269,28 @@ container network model that provides a consistent pro= gramming interface and the required network abstractions for applications.") (license license:asl2.0)))) =20 +(define-public docker-libnetwork-cmd-proxy + (package + (inherit docker-libnetwork) + (name "docker-libnetwork-cmd-proxy") + (arguments + `(#:import-path "github.com/docker/libnetwork/cmd/proxy" + #:unpack-path "github.com/docker/libnetwork" + #:install-source? #f)) + (native-inputs + `(("go-sctp" ,go-sctp) + ;; For tests. + ("logrus" ,go-github-com-sirupsen-logrus) + ("go-netlink" ,go-netlink) + ("go-netns" ,go-netns) + ("go-golang-org-x-crypto-ssh-terminal" + ,go-golang-org-x-crypto-ssh-terminal) + ("go-golang-org-x-sys-unix" ,go-golang-org-x-sys-unix))) + (synopsis "Docker user-space proxy") + (description "A proxy running in the user space. It is used by the +built-in registry server of Docker.") + (license license:asl2.0))) + ;; TODO: Patch out modprobes for ip_vs, nf_conntrack, ;; brige, nf_conntrack_netlink, aufs. (define-public docker =2D-=20 2.20.1 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0006-services-docker-Add-new-fields-to-support-proxy.patch Content-Transfer-Encoding: quoted-printable From=20fd1003dc333ede95a8fa2813b7e8ab2f6cfe82d2 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Fri, 5 Apr 2019 02:34:16 -0400 Subject: [PATCH 6/6] services: docker: Add new fields to support proxy. The Docker proxy enables inter-container and outside-to-container loopback, and is required by the Docker registry server. * gnu/services/docker.scm (docker-configuration)[proxy, enable-proxy?]: Add fields. (docker-shepherd-service): Use them. (serialize-boolean): New function. =2D-- gnu/services/docker.scm | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm index 8b5edf5cb0..94a04c8996 100644 =2D-- a/gnu/services/docker.scm +++ b/gnu/services/docker.scm @@ -31,13 +31,25 @@ #:export (docker-configuration docker-service-type)) =20 +;;; We're not using serialize-configuration, but we must define this becau= se +;;; the define-configuration macro validates it exists. +(define (serialize-boolean field-name val) + "") + (define-configuration docker-configuration (docker (package docker) "Docker daemon package.") (containerd (package containerd) =2D "containerd package.")) + "containerd package.") + (proxy + (package docker-libnetwork-cmd-proxy) + "The proxy package to support inter-container and outside-container +loop-back communications.") + (enable-proxy? + (boolean #t) + "Enable or disable the user-land proxy (enabled by default).")) =20 (define %docker-accounts (list (user-group (name "docker") (system? #t)))) @@ -66,7 +78,9 @@ (stop #~(make-kill-destructor))))) =20 (define (docker-shepherd-service config) =2D (let* ((docker (docker-configuration-docker config))) + (let* ((docker (docker-configuration-docker config)) + (enable-proxy? (docker-configuration-enable-proxy? config)) + (proxy (docker-configuration-proxy config))) (shepherd-service (documentation "Docker daemon.") (provision '(dockerd)) @@ -83,7 +97,10 @@ udev)) (start #~(make-forkexec-constructor (list (string-append #$docker "/bin/dockerd") =2D "-p" "/var/run/docker.pid") + "-p" "/var/run/docker.pid" + (if #$enable-proxy? "--userland-proxy" "") + "--userland-proxy-path" (string-append #$proxy + "/bin/pr= oxy")) #:pid-file "/var/run/docker.pid" #:log-file "/var/log/docker.log")) (stop #~(make-kill-destructor))))) =2D-=20 2.20.1 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEJ9WGpPiQCFQyn/CfEmDkZILmNWIFAlyzvB0ACgkQEmDkZILm NWKluxAAkXmqru8+EbF9ZMugp9aiZd3GlGWnGl/p/Y3qohkry0rvepGGeLP+RXxY MtEEiryKXN/qDi5GMfyKkTigreJ5UVAPLGIM3QRhSfd1r6LDWLMJuatjS8FZ0xdC fh+tg4TdnqMhwWqASlIOpYb9AE9neidU6VjycPshjyVaHjJ/1rqPgcJAPBO8bPGz bkskq0RuINhD1wcVidayEsqssiPJ2IgwYgxwWeArNuARJphdmzfEXinaU0PJqIyQ hNB43VoBP3aGEJ+rmhChzoJAVLpj/yD1Iw+yr0MjD92nj+8iPpyBTMi78xJlV8dp 6kvyjHbX+8QESVfNXTToYofcITWs7HJfFcqYdxDUEYMwfCxaK8VjJhfsImgjqdWd X3m7jT6FZHRfeDH26NnGR/a1asQ4ZNEdog1foK0F+L54cWh2a82moyjQJ1ceDCw+ 4RNGhpJPcKFjl70Kv/0T+46jCewlmdwgZ7tfiq4BWoc3wE57dIq+5gYbZLqboeUU kh2+KeGWBrPVn1CxworWvEM8iz+KnSS442Xl5aoAxM64c2UQEGgHA5A+SuzKshB/ +UgrE7sc9sMjlTsvQRAkSA1sSely0mqWcSLLZ7YGnoK8NdjZqZzR1AzNJjF86nLD gCOk3oBeUjPah5Rvg5kJ+j7U0H2zs0ENsVJh8TTMHmVgL3jzU3o= =vMxW -----END PGP SIGNATURE----- --==-=-=--