From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id IOAQHBZ+aF/XBwAA0tVLHw (envelope-from ) for ; Mon, 21 Sep 2020 10:19:02 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id wIz6FxZ+aF9aSwAAB5/wlQ (envelope-from ) for ; Mon, 21 Sep 2020 10:19:02 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E8B97940650 for ; Mon, 21 Sep 2020 10:18:59 +0000 (UTC) Received: from localhost ([::1]:58482 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kKIuQ-0002V7-NB for larch@yhetil.org; Mon, 21 Sep 2020 06:18:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54028) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kKIuH-0002Up-Ee for guix-devel@gnu.org; Mon, 21 Sep 2020 06:18:49 -0400 Received: from flashner.co.il ([178.62.234.194]:44452) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kKIuE-0000Kz-Pp for guix-devel@gnu.org; Mon, 21 Sep 2020 06:18:49 -0400 Received: from localhost (unknown [31.210.181.177]) by flashner.co.il (Postfix) with ESMTPSA id 09286401FA; Mon, 21 Sep 2020 10:18:44 +0000 (UTC) Date: Mon, 21 Sep 2020 13:18:12 +0300 From: Efraim Flashner To: Jesse Dowell Subject: Re: Adjustments to Docker related packages and service Message-ID: <20200921101812.GA1756@E5400> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="r5Pyd7+fXNt84Ff3" Content-Disposition: inline In-Reply-To: X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Received-SPF: pass client-ip=178.62.234.194; envelope-from=efraim@flashner.co.il; helo=flashner.co.il X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/21 06:18:45 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org, Pjotr Prins Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -2.61 X-TUID: Q8HkX8HQgrgo --r5Pyd7+fXNt84Ff3 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 19, 2020 at 03:03:29PM -0400, Jesse Dowell wrote: > Hello Guix, >=20 > I've been maintaining a set of patches to Docker in Guix that fix/modify > various aspects of the package and service in ways that are helpful to me. >=20 > I'd like to get these changes upstream if possible but I don't know which > of these changes might be qualified as a patch vs a bug fix. Also, it's > possible that I'm unaware of side-effects that these changes might have on > use-cases outside of my own scope of work. I'm glad you want to upstream these, having more eyes on the packages can only make them better. Also sometimes we end up using the packages without really knowing if they're configured and built correctly/optimally. > The patches are attached. I think that each patch might stand independent= ly > on its own but I haven't tested them in that configuration. >=20 > A quick summary of each patch.... I'm going to make my comments inline in the patches. >=20 > 0001-gnu-docker-Synchronize-docker-dependencies-with-the-.patch >=20 > This patch synchronizes docker-libnetwork and go-sctp with the current > version of docker. Without this patch, I've encountered some issues (I > forget the specifics offhand - sorry). Also, I believe this is an > appropriate change based on comments in the guix source code for the > packages and build instructions I've seen elsewhere. >=20 > 0002-gnu-docker-compile-docker-with-seccomp-enabled.patch >=20 > I utilize docker seccomp profiles and the current version of docker on gu= ix > does not support them. This seems safe enough to enable for everyone but > perhaps there's something I'm overlooking? >=20 > 0003-services-docker-shepherd-service-fix-enable-proxy-op.patch >=20 > This is a bug fix. I don't think the current code properly disables the > userland-proxy when the option is set to false. >=20 > 0004-gnu-docker-use-nftables-via-iptables-legacy.patch >=20 > I'm guessing this patch won't fly as-is but it's helpful for systems using > nftables instead of iptables. I could try to adjust it to make it flexible > if there's any interest. >=20 > That's everything. Any suggestions or comments would be much appreciated. >=20 > Best, > Jesse > From 53e80970f69f2e15be8d320a8e1995ffd0bd8cb0 Mon Sep 17 00:00:00 2001 > From: Jesse Dowell > Date: Sat, 19 Sep 2020 12:08:27 -0400 > Subject: [PATCH 1/4] gnu: docker: Synchronize docker dependencies with the > current version of docker. >=20 I added a copyright line for you to these two files and I adjusted the commit message to mention all of the changes. > --- > gnu/packages/docker.scm | 17 ++++++++++------- > gnu/packages/networking.scm | 8 +++++--- > 2 files changed, 15 insertions(+), 10 deletions(-) >=20 > diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm > index bc7e02ec46..69905e59bb 100644 > --- a/gnu/packages/docker.scm > +++ b/gnu/packages/docker.scm > @@ -48,6 +48,8 @@ > #:use-module (gnu packages version-control) > #:use-module (gnu packages virtualization)) > =20 > +;; Note - when changing Docker versions it is important to update the ve= rsions > +;; of several associated packages (docker-libnetwork and go-sctp). > (define %docker-version "19.03.12") > =20 > (define-public python-docker > @@ -246,9 +248,10 @@ network attachments.") > (define docker-libnetwork > ;; There are no recent release for libnetwork, so choose the last comm= it of > ;; the branch that Docker uses, as can be seen in the Docker source fi= le > - ;; 'hack/dockerfile/install/proxy.installer'. > - (let ((commit "4725f2163fb214a6312f3beae5991f838ec36326") > - (version "18.09") > + ;; 'hack/dockerfile/install/proxy.installer'. NOTE - It is important t= hat > + ;; this version is kept in sync with the version of Docker being used. > + (let ((commit "026aabaa659832804b01754aaadd2c0f420c68b6") > + (version "19.03") this I replaced with (version-major+minor %docker-version) I also added to the comment that it's from the 'bump_19.03' branch > (revision "1")) > (package > (name "docker-libnetwork") > @@ -256,12 +259,12 @@ network attachments.") > (source (origin > (method git-fetch) > (uri (git-reference > - (url "https://github.com/docker/libnetwork") > + (url "https://github.com/moby/libnetwork") I added mention of this to the commit message. > (commit commit))) > (file-name (git-file-name name version)) > (sha256 > (base32 > - "1zpnxki8qfzha6ljahpwd3vkzmjhsvkmf73w6crm4ilxxw5vnpfb"= )) > + "0bli21vn5v7bssw3ydym4jfdjsldhb47fld88kng7d138wl70lkw"= )) > ;; Delete bundled ("vendored") free software source code. > (modules '((guix build utils))) > (snippet '(begin > @@ -269,8 +272,8 @@ network attachments.") > #t)))) > (build-system go-build-system) > (arguments > - `(#:import-path "github.com/docker/libnetwork/")) > - (home-page "https://github.com/docker/libnetwork/") > + `(#:import-path "github.com/moby/libnetwork/")) > + (home-page "https://github.com/moby/libnetwork/") And here. > (synopsis "Networking for containers") > (description "Libnetwork provides a native Go implementation for > connecting containers. The goal of @code{libnetwork} is to deliver a ro= bust > diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm > index 1c2e7cf979..964701af1a 100644 > --- a/gnu/packages/networking.scm > +++ b/gnu/packages/networking.scm > @@ -1435,8 +1435,10 @@ handling network namespaces in Go.") > (license license:asl2.0)))) > =20 > (define-public go-sctp > - ;; docker-libnetwork-cmd-proxy requires this exact commit. > - (let ((commit "07191f837fedd2f13d1ec7b5f885f0f3ec54b1cb") > + ;; docker-libnetwork-cmd-proxy requires this exact commit. NOTE - It = is > + ;; important that this version is kept in sync with the current versio= n of > + ;; Docker and its associated dependencies. > + (let ((commit "6e2cb1366111dcf547c13531e3a263a067715847") > (revision "1")) Since we changed the commit I also bumped the revision so it'll show up as a new version. I also disabled the test suite since it was flakey on the machines I tested with. > (package > (name "go-sctp") > @@ -1449,7 +1451,7 @@ handling network namespaces in Go.") > (file-name (git-file-name name version)) > (sha256 > (base32 > - "1mk9ncm10gwi5pn5wcw4skbyf4qg7n5qdf1mim4gf3mrckvi6g6h"= )))) > + "1ba90fmpdwxa1ba4hrsjhi3gfy3pwmz7x8amw1p5dc9p5a7nnqrb"= )))) > (build-system go-build-system) > (arguments > `(#:import-path "github.com/ishidawataru/sctp")) > --=20 > 2.28.0 >=20 > From e5ecc9051ae14172acbab1e8b81e038f51a7bdf0 Mon Sep 17 00:00:00 2001 > From: Jesse Dowell > Date: Sat, 19 Sep 2020 12:40:38 -0400 > Subject: [PATCH 3/4] services: docker-shepherd-service: fix enable-proxy? > option >=20 > The userland proxy option does not properly disable the userland proxy wh= en > set to false. Docker defaults to enabling the userland proxy if the optio= n is > unset on the command line. Also added a copyright line for you here and adjusted the commit message. > --- > gnu/services/docker.scm | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) >=20 > diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm > index a4eccc7928..db7fdd5f4e 100644 > --- a/gnu/services/docker.scm > +++ b/gnu/services/docker.scm > @@ -121,9 +121,9 @@ loop-back communications.") > #$@(if debug? > '("--debug" "--log-level=3Ddebug") > '()) > - (if #$enable-proxy? "--userland-proxy" "") > - "--userland-proxy-path" (string-append #$proxy > - "/bin/= proxy") > + #$@(if enable-proxy? > + '("--userland-proxy=3Dtrue" (string-ap= pend "--userland-proxy-path=3D" proxy "/bin/proxy")) > + '("--userland-proxy=3Dfalse")) Only change here was to wrap the lines so they tried to fit in under 80 characters. > (if #$enable-iptables? > "--iptables" > "--iptables=3Dfalse")) > --=20 > 2.28.0 >=20 > From 163e84b2dc3d13d716fa1db27ab93893a5419303 Mon Sep 17 00:00:00 2001 > From: Jesse Dowell > Date: Sat, 19 Sep 2020 12:34:47 -0400 > Subject: [PATCH 2/4] gnu: docker: compile docker with seccomp enabled >=20 > --- > gnu/packages/docker.scm | 1 + > 1 file changed, 1 insertion(+) >=20 > diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm > index 69905e59bb..825aa1ef0f 100644 > --- a/gnu/packages/docker.scm > +++ b/gnu/packages/docker.scm > @@ -517,6 +517,7 @@ built-in registry server of Docker.") > #t)) > (replace 'configure > (lambda _ > + (setenv "DOCKER_BUILDTAGS" "seccomp") I looked around a bunch for this. It looks like the seccomp option was added around 2016 and it is currently part of the default build profile for the official docker packages. It is also used for Debian. The other two parts from upstream are apparmor and selinux, neither of which Guix uses natively (IIRC). Since the default is 'nothing' and upstream suggests using it this seems like a safe change. Also, it can be overridden in any case by selecting a different profile or using the '--privileged' flag when using 'docker run'. > (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-vers= ion)) > (setenv "VERSION" (string-append ,%docker-version "-ce")) > ;; Automatically use bundled dependencies. > --=20 > 2.28.0 >=20 > From ac3277477bda6741ff3a8af9530c2fd68e2bb062 Mon Sep 17 00:00:00 2001 > From: Jesse Dowell > Date: Sat, 19 Sep 2020 12:45:39 -0400 > Subject: [PATCH 4/4] gnu: docker: use nftables via iptables-legacy >=20 I think you meant nftables via iptables > --- > gnu/packages/docker.scm | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) >=20 > diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm > index 825aa1ef0f..322f00026f 100644 > --- a/gnu/packages/docker.scm > +++ b/gnu/packages/docker.scm > @@ -438,8 +438,8 @@ built-in registry server of Docker.") > ("pvdisplay" "lvm2" "sbin/pvdisplay") > ("blkid" "util-linux" "sbin/blkid") > ("unpigz" "pigz" "bin/unpigz") > - ("iptables" "iptables" "sbin/iptables") > - ("iptables-legacy" "iptables" "sbin/iptables") > + ("iptables" "iptables" "sbin/iptables-nft") > + ("iptables-legacy" "iptables" "sbin/iptables-legacy") I checked the iptables package. 'iptables' and 'iptables-legacy' are both symlinks for 'xtables-legacy-multi'. There is another binary for 'iptables-nft' which is a symlink for 'xtables-nft-multi'. Checking through the source there aren't actually any references to iptables-legacy in the docker tarball (except in the Dockerfile). Guix has services for both iptables and nftables. I'd rather not break existing workflows by switching iptables in docker to nftables. Also I don't know if it is possible to easily support both in the same package. > ("ip" "iproute2" "sbin/ip")) > =20 > (substitute-Command* > --=20 > 2.28.0 >=20 I've pushed the first 3 patches and I'd love to have some other input on the iptables/nftables bit in the 4th patch. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --r5Pyd7+fXNt84Ff3 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl9ofeEACgkQQarn3Mo9 g1Eq7RAAs6AfqN94PKhCu/x02EfcKUHWsMT/UieGOtvUbXelAin+CSs4BiViCdPr EzTm8HHdHzlRxwDR0SGA3F9JYuDYqLoIMPBXPv1nfL4/dMfqTxZdJFs68ctzIAiF 0ZpkkQ+EqwD18Vx66OPO68Nl5yot079pRWszioBcBTeKCqY25FruzmbjoI0q77/Q 1WmMsorVpIgzM05kk95e52IIb1wzc/vn25aMKypduU8KmdmdVfaBzx+VS2FTThiS iFpGoNC5hJDn0yV6Erpj0iQcKxCpiUFr0tarX4PON6h7xF2Q+xk/mPtEk/HNNjKe uj4JQUTikw5mRsnBTYp/K+t5QGejK8htO5cZpcnVH8p1dYLNy3RTBjj9T7WgPfNn /GwNz9BAd6UO2p+1b7Z/GHpbPtOcQAbkdxTg9nLh1AUXto6ap/P6CHfhGBgVnhwe JGZWRouBduNMno+JkC8dQ+KM3yFDvIVjQBYTzRcsxj1VCozCjuV/AOEyhQw2ueVJ d7WoVi3/KwM+/mau8d4/zaMVa9MHl2NUX5TjXM1Dm611i206Vj58mwVJ+WmEDmF5 I5onnCY65uCfR0zgtW99AMO7fgOxZNWKFgCCAgW8+fGPdaZhtQhAus0Kefg2GlWg O5Lzeh8coO+umm8Qnz8T8vJO1Mh0TO4f3UJ5m5bYIYIbkfyRjp8= =QYXs -----END PGP SIGNATURE----- --r5Pyd7+fXNt84Ff3--