all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [bug#35697] [PATCH 0/8] Make 'guix system docker-image' readily usable
@ 2019-05-12 10:30 Ludovic Courtès
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
  2019-05-15 14:37 ` bug#35697: [PATCH 0/8] Make 'guix system docker-image' readily usable Ludovic Courtès
  0 siblings, 2 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:30 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

Hello Guix,

On current master, ‘guix system docker-image’ produces an image without
an entry point, so one has to carefully follow the “GUIX_NEW_SYSTEM hack”
described in the manual.

Furthermore, due to other issues, the resulting image doesn’t properly
boot because it tries to mount file systems that it cannot mount, such as
/dev/shm and /dev/pts.

These patches fix both issues, such that one can just do ‘docker create’
and ‘docker start’ to get Guix System up and running in the container.
I think that’s a nice improvement.  :-)

We discussed this Friday on IRC and people said that it’s quite unusual
to provide a “full OS” (with PID 1) as a Docker image; instead, people
would rather do one image per (micro)service.  But anyway, that’s
the purpose of ‘guix system docker-image’, and I can imagine it has
use cases too.  For example, it’s a simple way to get Guix set up in
a container, for people who want to perform Guix builds in a container.

Thoughts?

Ludo’.

Ludovic Courtès (8):
  system: Export 'operating-system-default-essential-services'.
  linux-container: Improve filtering of unnecessary file systems.
  services: 'gc-root-service-type' now has a default value.
  linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image
    OSes.
  linux-container: Compute essential services for THIS-OPERATING-SYSTEM.
  system: Add 'operating-system-with-gc-roots'.
  docker: 'build-docker-image' accepts an optional #:entry-point.
  vm: 'system-docker-image' provides an entry point.

 doc/guix.texi                  |  18 +++--
 gnu/services.scm               |   5 +-
 gnu/system.scm                 |  18 ++++-
 gnu/system/linux-container.scm |  30 ++++++---
 gnu/system/vm.scm              |  18 ++++-
 gnu/tests/docker.scm           | 118 ++++++++++++++++++++++++++++++++-
 gnu/tests/install.scm          |  11 ---
 guix/docker.scm                |  15 +++--
 8 files changed, 195 insertions(+), 38 deletions(-)

-- 
2.21.0

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services'.
  2019-05-12 10:30 [bug#35697] [PATCH 0/8] Make 'guix system docker-image' readily usable Ludovic Courtès
@ 2019-05-12 10:37 ` Ludovic Courtès
  2019-05-12 10:37   ` [bug#35697] [PATCH 2/8] linux-container: Improve filtering of unnecessary file systems Ludovic Courtès
                     ` (6 more replies)
  2019-05-15 14:37 ` bug#35697: [PATCH 0/8] Make 'guix system docker-image' readily usable Ludovic Courtès
  1 sibling, 7 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:37 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

* gnu/system.scm (essential-services): Rename to...
(operating-system-default-essential-services): ... this.
(<operating-system>)[essential-services]: Adjust accordingly.
---
 gnu/system.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/gnu/system.scm b/gnu/system.scm
index 0489b9720d..2c4ca55ffc 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -72,6 +72,7 @@
             operating-system-bootloader
             operating-system-services
             operating-system-essential-services
+            operating-system-default-essential-services
             operating-system-user-services
             operating-system-packages
             operating-system-host-name
@@ -213,7 +214,8 @@
 
   (essential-services operating-system-essential-services ; list of services
                       (thunked)
-                      (default (essential-services this-operating-system)))
+                      (default (operating-system-default-essential-services
+                                this-operating-system)))
   (services operating-system-user-services        ; list of services
             (default %base-services))
 
@@ -463,7 +465,7 @@ value of the SYSTEM-SERVICE-TYPE service."
                 ("initrd" ,initrd)
                 ("locale" ,locale))))))   ;used by libc
 
-(define* (essential-services os)
+(define (operating-system-default-essential-services os)
   "Return the list of essential services for OS.  These are special services
 that implement part of what's declared in OS are responsible for low-level
 bookkeeping."
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [bug#35697] [PATCH 2/8] linux-container: Improve filtering of unnecessary file systems.
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
@ 2019-05-12 10:37   ` Ludovic Courtès
  2019-05-12 10:37   ` [bug#35697] [PATCH 3/8] services: 'gc-root-service-type' now has a default value Ludovic Courtès
                     ` (5 subsequent siblings)
  6 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:37 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

* gnu/system/linux-container.scm (containerized-operating-system)[user-file-systems]:
Add trailing slash for the "/dev/" and "/sys/" prefixes.
---
 gnu/system/linux-container.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm
index 149c3d08a3..ded5f279fe 100644
--- a/gnu/system/linux-container.scm
+++ b/gnu/system/linux-container.scm
@@ -65,8 +65,8 @@ containerized OS."
                     (string=? target "/")
                     (and (string? source)
                          (string-prefix? "/dev/" source))
-                    (string-prefix? "/dev" target)
-                    (string-prefix? "/sys" target))))
+                    (string-prefix? "/dev/" target)
+                    (string-prefix? "/sys/" target))))
             (operating-system-file-systems os)))
 
   (define (mapping->fs fs)
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [bug#35697] [PATCH 3/8] services: 'gc-root-service-type' now has a default value.
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
  2019-05-12 10:37   ` [bug#35697] [PATCH 2/8] linux-container: Improve filtering of unnecessary file systems Ludovic Courtès
@ 2019-05-12 10:37   ` Ludovic Courtès
  2019-05-12 10:37   ` [bug#35697] [PATCH 4/8] linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image OSes Ludovic Courtès
                     ` (4 subsequent siblings)
  6 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:37 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

* gnu/services.scm (gc-root-service-type)[default-value]: New field.
---
 gnu/services.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/services.scm b/gnu/services.scm
index f151bbaa9d..7de78105ff 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2015, 2016, 2017, 2018 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -649,7 +649,8 @@ as Wifi cards.")))
                 (extend append)
                 (description
                  "Register garbage-collector roots---i.e., store items that
-will not be reclaimed by the garbage collector.")))
+will not be reclaimed by the garbage collector.")
+                (default-value '())))
 
 \f
 ;;;
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [bug#35697] [PATCH 4/8] linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image OSes.
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
  2019-05-12 10:37   ` [bug#35697] [PATCH 2/8] linux-container: Improve filtering of unnecessary file systems Ludovic Courtès
  2019-05-12 10:37   ` [bug#35697] [PATCH 3/8] services: 'gc-root-service-type' now has a default value Ludovic Courtès
@ 2019-05-12 10:37   ` Ludovic Courtès
  2019-05-12 10:37   ` [bug#35697] [PATCH 5/8] linux-container: Compute essential services for THIS-OPERATING-SYSTEM Ludovic Courtès
                     ` (3 subsequent siblings)
  6 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:37 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

Previously, 'guix system docker-image' would end up providing an OS that
would try to mount all of %CONTAINER-FILE-SYSTEMS as well as /gnu/store,
which is bound to fail in unprivileged Docker.

This patch makes it so that 'guix system container' still gets those
file systems, but 'guix system docker-image' doesn't.

* gnu/system/linux-container.scm (containerized-operating-system): Add
 #:extra-file-systems parameter and honor it.  Do not import
 %STORE-MAPPING.
(container-script): Add %STORE-MAPPING to MAPPINGS and pass
 #:extra-file-systems.
---
 gnu/system/linux-container.scm | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm
index ded5f279fe..5adec064f7 100644
--- a/gnu/system/linux-container.scm
+++ b/gnu/system/linux-container.scm
@@ -53,10 +53,12 @@ from OS that are needed on the bare metal and not in a container."
                      (return `(("locale" ,locale))))))
         base))
 
-(define (containerized-operating-system os mappings)
+(define* (containerized-operating-system os mappings
+                                         #:key
+                                         (extra-file-systems '()))
   "Return an operating system based on OS for use in a Linux container
 environment.  MAPPINGS is a list of <file-system-mapping> to realize in the
-containerized OS."
+containerized OS.  EXTRA-FILE-SYSTEMS is a list of file systems to add to OS."
   (define user-file-systems
     (remove (lambda (fs)
               (let ((target (file-system-mount-point fs))
@@ -88,15 +90,17 @@ containerized OS."
                         (memq (service-kind service)
                               useless-services))
                       (operating-system-user-services os)))
-    (file-systems (append (map mapping->fs (cons %store-mapping mappings))
-                          %container-file-systems
+    (file-systems (append (map mapping->fs mappings)
+                          extra-file-systems
                           user-file-systems))))
 
 (define* (container-script os #:key (mappings '()))
   "Return a derivation of a script that runs OS as a Linux container.
 MAPPINGS is a list of <file-system> objects that specify the files/directories
 that will be shared with the host system."
-  (let* ((os           (containerized-operating-system os mappings))
+  (let* ((os           (containerized-operating-system
+                        os (cons %store-mapping mappings)
+                        #:extra-file-systems %container-file-systems))
          (file-systems (filter file-system-needed-for-boot?
                                (operating-system-file-systems os)))
          (specs        (map file-system->spec file-systems)))
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [bug#35697] [PATCH 5/8] linux-container: Compute essential services for THIS-OPERATING-SYSTEM.
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
                     ` (2 preceding siblings ...)
  2019-05-12 10:37   ` [bug#35697] [PATCH 4/8] linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image OSes Ludovic Courtès
@ 2019-05-12 10:37   ` Ludovic Courtès
  2019-05-12 10:38   ` [bug#35697] [PATCH 6/8] system: Add 'operating-system-with-gc-roots' Ludovic Courtès
                     ` (2 subsequent siblings)
  6 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:37 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

Previously, the 'essential-services' would correspond to the initial,
non-containerized OS.  Thus, all the file systems removed in
'container-essential-services' would actually still be there because the
essential services would be computed on the non-containerized OS.

This is a followup to 69cae3d3356a69b7fe69481338f760545995485e.

* gnu/system/linux-container.scm (container-essential-services): Call
'operating-system-default-essential-services' to get the baseline
services.
(containerized-operating-system): Pass THIS-OPERATING-SYSTEM, not OS, to
'container-essential-services'.
Add a dummy root file system to 'file-systems'.
---
 gnu/system/linux-container.scm | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm
index 5adec064f7..fc2e05a5bc 100644
--- a/gnu/system/linux-container.scm
+++ b/gnu/system/linux-container.scm
@@ -45,7 +45,7 @@ from OS that are needed on the bare metal and not in a container."
                     (list (service-kind %linux-bare-metal-service)
                           firmware-service-type
                           system-service-type)))
-            (operating-system-essential-services os)))
+            (operating-system-default-essential-services os)))
 
   (cons (service system-service-type
                  (let ((locale (operating-system-locale-directory os)))
@@ -85,14 +85,20 @@ containerized OS.  EXTRA-FILE-SYSTEMS is a list of file systems to add to OS."
   (operating-system
     (inherit os)
     (swap-devices '()) ; disable swap
-    (essential-services (container-essential-services os))
+    (essential-services (container-essential-services this-operating-system))
     (services (remove (lambda (service)
                         (memq (service-kind service)
                               useless-services))
                       (operating-system-user-services os)))
     (file-systems (append (map mapping->fs mappings)
                           extra-file-systems
-                          user-file-systems))))
+                          user-file-systems
+
+                          ;; Provide a dummy root file system.
+                          (list (file-system
+                                  (mount-point "/")
+                                  (device "none")
+                                  (type "none")))))))
 
 (define* (container-script os #:key (mappings '()))
   "Return a derivation of a script that runs OS as a Linux container.
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [bug#35697] [PATCH 6/8] system: Add 'operating-system-with-gc-roots'.
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
                     ` (3 preceding siblings ...)
  2019-05-12 10:37   ` [bug#35697] [PATCH 5/8] linux-container: Compute essential services for THIS-OPERATING-SYSTEM Ludovic Courtès
@ 2019-05-12 10:38   ` Ludovic Courtès
  2019-05-12 10:38   ` [bug#35697] [PATCH 7/8] docker: 'build-docker-image' accepts an optional #:entry-point Ludovic Courtès
  2019-05-12 10:38   ` [bug#35697] [PATCH 8/8] vm: 'system-docker-image' provides an entry point Ludovic Courtès
  6 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:38 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

* gnu/tests/install.scm (operating-system-with-gc-roots): Move to...
* gnu/system.scm (operating-system-with-gc-roots): ... here.  New
procedure.
---
 gnu/system.scm        | 12 ++++++++++++
 gnu/tests/install.scm | 11 -----------
 2 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/gnu/system.scm b/gnu/system.scm
index 2c4ca55ffc..01be1243fe 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -109,6 +109,7 @@
             operating-system-boot-script
 
             system-linux-image-file-name
+            operating-system-with-gc-roots
 
             boot-parameters
             boot-parameters?
@@ -519,6 +520,17 @@ bookkeeping."
    (append (operating-system-user-services os)
            (operating-system-essential-services os))))
 
+(define (operating-system-with-gc-roots os roots)
+  "Return a variant of OS where ROOTS are registered as GC roots."
+  (operating-system
+    (inherit os)
+
+    ;; We use this procedure for the installation OS, which already defines GC
+    ;; roots.  Add ROOTS to those.
+    (services (cons (simple-service 'extra-root
+                                    gc-root-service-type roots)
+                    (operating-system-user-services os)))))
+
 \f
 ;;;
 ;;; /etc.
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 430a102378..7b5ee18505 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -123,17 +123,6 @@
                                     (inherit config)
                                     (guix (current-guix))))))))
 
-(define (operating-system-with-gc-roots os roots)
-  "Return a variant of OS where ROOTS are registered as GC roots."
-  (operating-system
-    (inherit os)
-
-    ;; We use this procedure for the installation OS, which already defines GC
-    ;; roots.  Add ROOTS to those.
-    (services (cons (simple-service 'extra-root
-                                    gc-root-service-type roots)
-                    (operating-system-user-services os)))))
-
 \f
 (define MiB (expt 2 20))
 
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [bug#35697] [PATCH 7/8] docker: 'build-docker-image' accepts an optional #:entry-point.
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
                     ` (4 preceding siblings ...)
  2019-05-12 10:38   ` [bug#35697] [PATCH 6/8] system: Add 'operating-system-with-gc-roots' Ludovic Courtès
@ 2019-05-12 10:38   ` Ludovic Courtès
  2019-05-12 10:38   ` [bug#35697] [PATCH 8/8] vm: 'system-docker-image' provides an entry point Ludovic Courtès
  6 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:38 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

* guix/docker.scm (config): Add #:entry-point and honor it.
(build-docker-image): Likewise.
---
 guix/docker.scm | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/guix/docker.scm b/guix/docker.scm
index c6e9c6fee5..7fe83d9797 100644
--- a/guix/docker.scm
+++ b/guix/docker.scm
@@ -1,6 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Ricardo Wurmus <rekado@elephly.net>
-;;; Copyright © 2017, 2018 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2017, 2018, 2019 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2018 Chris Marusich <cmmarusich@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -73,7 +73,7 @@
   `((,(generate-tag path) . ((latest . ,id)))))
 
 ;; See https://github.com/opencontainers/image-spec/blob/master/config.md
-(define (config layer time arch)
+(define* (config layer time arch #:key entry-point)
   "Generate a minimal image configuration for the given LAYER file."
   ;; "architecture" must be values matching "platform.arch" in the
   ;; runtime-spec at
@@ -81,7 +81,9 @@
   `((architecture . ,arch)
     (comment . "Generated by GNU Guix")
     (created . ,time)
-    (config . #nil)
+    (config . ,(if entry-point
+                   `((entrypoint . ,entry-point))
+                   #nil))
     (container_config . #nil)
     (os . "linux")
     (rootfs . ((type . "layers")
@@ -110,6 +112,7 @@ return \"a\"."
                              (transformations '())
                              (system (utsname:machine (uname)))
                              database
+                             entry-point
                              compressor
                              (creation-time (current-time time-utc)))
   "Write to IMAGE a Docker image archive containing the given PATHS.  PREFIX
@@ -118,6 +121,9 @@ must be a store path that is a prefix of any store paths in PATHS.
 When DATABASE is true, copy it to /var/guix/db in the image and create
 /var/guix/gcroots and friends.
 
+When ENTRY-POINT is true, it must be a list of strings; it is stored as the
+entry point in the Docker image JSON structure.
+
 SYMLINKS must be a list of (SOURCE -> TARGET) tuples describing symlinks to be
 created in the image, where each TARGET is relative to PREFIX.
 TRANSFORMATIONS must be a list of (OLD -> NEW) tuples describing how to
@@ -227,7 +233,8 @@ SRFI-19 time-utc object, as the creation time in metadata."
       (with-output-to-file "config.json"
         (lambda ()
           (scm->json (config (string-append id "/layer.tar")
-                             time arch))))
+                             time arch
+                             #:entry-point entry-point))))
       (with-output-to-file "manifest.json"
         (lambda ()
           (scm->json (manifest prefix id))))
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [bug#35697] [PATCH 8/8] vm: 'system-docker-image' provides an entry point.
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
                     ` (5 preceding siblings ...)
  2019-05-12 10:38   ` [bug#35697] [PATCH 7/8] docker: 'build-docker-image' accepts an optional #:entry-point Ludovic Courtès
@ 2019-05-12 10:38   ` Ludovic Courtès
  6 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-12 10:38 UTC (permalink / raw)
  To: 35697; +Cc: Chris Marusich

This simplifies use of images created with 'guix system docker-image'.

* gnu/system/vm.scm (system-docker-image)[boot-program]: New variable.
[os]: Add it to the GC roots.
[build]: Pass #:entry-point to 'build-docker-image'.
* gnu/tests/docker.scm (run-docker-system-test): New procedure.
(%test-docker-system): New variable.
* doc/guix.texi (Invoking guix system): Remove GUIX_NEW_SYSTEM hack and
'--entrypoint' from the example.  Mention 'docker create', 'docker
start', and 'docker exec'.
---
 doc/guix.texi        |  18 ++++---
 gnu/system/vm.scm    |  18 ++++++-
 gnu/tests/docker.scm | 118 ++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 145 insertions(+), 9 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index df7208229c..da65fd8a4e 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24497,20 +24497,26 @@ system configuration file.  You can then load the image and launch a
 Docker container using commands like the following:
 
 @example
-image_id="$(docker load < guix-system-docker-image.tar.gz)"
-docker run -e GUIX_NEW_SYSTEM=/var/guix/profiles/system \\
-    --entrypoint /var/guix/profiles/system/profile/bin/guile \\
-    $image_id /var/guix/profiles/system/boot
+image_id="`docker load < guix-system-docker-image.tar.gz`"
+container_id="`docker create $image_id`"
+docker start $container_id
 @end example
 
 This command starts a new Docker container from the specified image.  It
 will boot the Guix system in the usual manner, which means it will
 start any services you have defined in the operating system
-configuration.  Depending on what you run in the Docker container, it
+configuration.  You can get an interactive shell running in the container
+using @command{docker exec}:
+
+@example
+docker exec -ti $container_id /run/current-system/profile/bin/bash --login
+@end example
+
+Depending on what you run in the Docker container, it
 may be necessary to give the container additional permissions.  For
 example, if you intend to build software using Guix inside of the Docker
 container, you may need to pass the @option{--privileged} option to
-@code{docker run}.
+@code{docker create}.
 
 @item container
 Return a script to run the operating system declared in @var{file}
diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 124abd0fc9..f3027cd4ca 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -473,7 +473,7 @@ the image."
 
 (define* (system-docker-image os
                               #:key
-                              (name "guixsd-docker-image")
+                              (name "guix-docker-image")
                               register-closures?)
   "Build a docker image.  OS is the desired <operating-system>.  NAME is the
 base name to use for the output file.  When REGISTER-CLOSURES? is not #f,
@@ -487,7 +487,19 @@ should set REGISTER-CLOSURES? to #f."
          (local-file (search-path %load-path
                                   "guix/store/schema.sql"))))
 
-  (let ((os    (containerized-operating-system os '()))
+  (define boot-program
+    ;; Program that runs the boot script of OS, which in turn starts shepherd.
+    (program-file "boot-program"
+                  #~(let ((system (cadr (command-line))))
+                      (setenv "GUIX_NEW_SYSTEM" system)
+                      (execl #$(file-append guile-2.2 "/bin/guile")
+                             "guile" "--no-auto-compile"
+                             (string-append system "/boot")))))
+
+
+  (let ((os    (operating-system-with-gc-roots
+                (containerized-operating-system os '())
+                (list boot-program)))
         (name  (string-append name ".tar.gz"))
         (graph "system-graph"))
     (define build
@@ -538,9 +550,11 @@ should set REGISTER-CLOSURES? to #f."
                                  (string-append "/xchg/" #$graph)
                                read-reference-graph)))
                  #$os
+                 #:entry-point '(#$boot-program #$os)
                  #:compressor '(#+(file-append gzip "/bin/gzip") "-9n")
                  #:creation-time (make-time time-utc 0 1)
                  #:transformations `((,root-directory -> ""))))))))
+
     (expression->derivation-in-linux-vm
      name build
      #:make-disk-image? #f
diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm
index 25e172efae..3cd3a27884 100644
--- a/gnu/tests/docker.scm
+++ b/gnu/tests/docker.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2019 Danny Milosavljevic <dannym@scratchpost.org>
+;;; Copyright © 2019 Ludovic Courtès <ludo@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -28,6 +29,7 @@
   #:use-module (gnu services desktop)
   #:use-module (gnu packages bootstrap) ; %bootstrap-guile
   #:use-module (gnu packages docker)
+  #:use-module (gnu packages guile)
   #:use-module (guix gexp)
   #:use-module (guix grafts)
   #:use-module (guix monads)
@@ -38,7 +40,8 @@
   #:use-module (guix tests)
   #:use-module (guix build-system trivial)
   #:use-module ((guix licenses) #:prefix license:)
-  #:export (%test-docker))
+  #:export (%test-docker
+            %test-docker-system))
 
 (define %docker-os
   (simple-operating-system
@@ -166,3 +169,116 @@ standard output device and then enters a new line.")
    (name "docker")
    (description "Test Docker container of Guix.")
    (value (build-tarball&run-docker-test))))
+
+\f
+(define (run-docker-system-test tarball)
+  "Load DOCKER-TARBALL as Docker image and run it in a Docker container,
+inside %DOCKER-OS."
+  (define os
+    (marionette-operating-system
+     %docker-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     ;; FIXME: Because we're using the volatile-root setup where the root file
+     ;; system is a tmpfs overlaid over a small root file system, 'docker
+     ;; load' must be able to store the whole image into memory, hence the
+     ;; huge memory requirements.  We should avoid the volatile-root setup
+     ;; instead.
+     (memory-size 3000)
+     (port-forwardings '())))
+
+  (define test
+    (with-imported-modules '((gnu build marionette)
+                             (guix build utils))
+      #~(begin
+          (use-modules (srfi srfi-11) (srfi srfi-64)
+                       (gnu build marionette)
+                       (guix build utils))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "docker")
+
+          (test-assert "service running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (match (start-service 'dockerd)
+                  (#f #f)
+                  (('service response-parts ...)
+                   (match (assq-ref response-parts 'running)
+                     ((pid) (number? pid))))))
+             marionette))
+
+          (test-assert "load system image and run it"
+            (marionette-eval
+             `(begin
+                (define (slurp command . args)
+                  ;; Return the output from COMMAND.
+                  (let* ((port (apply open-pipe* OPEN_READ command args))
+                         (output (read-line port))
+                         (status (close-pipe port)))
+                    output))
+
+                (define (docker-cli command . args)
+                  ;; Run the given Docker COMMAND.
+                  (apply invoke #$(file-append docker-cli "/bin/docker")
+                         command args))
+
+                (define (wait-for-container-file container file)
+                  ;; Wait for FILE to show up in CONTAINER.
+                  (docker-cli "exec" container
+                              #$(file-append guile-2.2 "/bin/guile")
+                              "-c"
+                              (object->string
+                               `(let loop ((n 15))
+                                  (when (zero? n)
+                                    (error "file didn't show up" ,file))
+                                  (unless (file-exists? ,file)
+                                    (sleep 1)
+                                    (loop (- n 1)))))))
+
+                (let* ((line (slurp #$(file-append docker-cli "/bin/docker")
+                                    "load" "-i" #$tarball))
+                       (repository&tag (string-drop line
+                                                    (string-length
+                                                     "Loaded image: ")))
+                       (container (slurp
+                                   #$(file-append docker-cli "/bin/docker")
+                                   "create" repository&tag)))
+                  (docker-cli "start" container)
+
+                  ;; Wait for shepherd to be ready.
+                  (wait-for-container-file container
+                                           "/var/run/shepherd/socket")
+
+                  (docker-cli "exec" container
+                              "/run/current-system/profile/bin/herd"
+                              "status")
+                  (slurp #$(file-append docker-cli "/bin/docker")
+                         "exec" container
+                         "/run/current-system/profile/bin/herd"
+                         "status" "guix-daemon")))
+             marionette))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "docker-system-test" test))
+
+(define %test-docker-system
+  (system-test
+   (name "docker-system")
+   (description "Run a system image as produced by @command{guix system
+docker-image} inside Docker.")
+   (value (with-monad %store-monad
+            (>>= (system-docker-image (simple-operating-system))
+                 run-docker-system-test)))))
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* bug#35697: [PATCH 0/8] Make 'guix system docker-image' readily usable
  2019-05-12 10:30 [bug#35697] [PATCH 0/8] Make 'guix system docker-image' readily usable Ludovic Courtès
  2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
@ 2019-05-15 14:37 ` Ludovic Courtès
  1 sibling, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2019-05-15 14:37 UTC (permalink / raw)
  To: 35697-done; +Cc: Chris Marusich

Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

> Ludovic Courtès (8):
>   system: Export 'operating-system-default-essential-services'.
>   linux-container: Improve filtering of unnecessary file systems.
>   services: 'gc-root-service-type' now has a default value.
>   linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image
>     OSes.
>   linux-container: Compute essential services for THIS-OPERATING-SYSTEM.
>   system: Add 'operating-system-with-gc-roots'.
>   docker: 'build-docker-image' accepts an optional #:entry-point.
>   vm: 'system-docker-image' provides an entry point.

Pushed!

I had to rebase and adjust to changes made over the last couple of days
in that area of the code.

Feedback welcome!

Ludo’.

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2019-05-15 14:39 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-12 10:30 [bug#35697] [PATCH 0/8] Make 'guix system docker-image' readily usable Ludovic Courtès
2019-05-12 10:37 ` [bug#35697] [PATCH 1/8] system: Export 'operating-system-default-essential-services' Ludovic Courtès
2019-05-12 10:37   ` [bug#35697] [PATCH 2/8] linux-container: Improve filtering of unnecessary file systems Ludovic Courtès
2019-05-12 10:37   ` [bug#35697] [PATCH 3/8] services: 'gc-root-service-type' now has a default value Ludovic Courtès
2019-05-12 10:37   ` [bug#35697] [PATCH 4/8] linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image OSes Ludovic Courtès
2019-05-12 10:37   ` [bug#35697] [PATCH 5/8] linux-container: Compute essential services for THIS-OPERATING-SYSTEM Ludovic Courtès
2019-05-12 10:38   ` [bug#35697] [PATCH 6/8] system: Add 'operating-system-with-gc-roots' Ludovic Courtès
2019-05-12 10:38   ` [bug#35697] [PATCH 7/8] docker: 'build-docker-image' accepts an optional #:entry-point Ludovic Courtès
2019-05-12 10:38   ` [bug#35697] [PATCH 8/8] vm: 'system-docker-image' provides an entry point Ludovic Courtès
2019-05-15 14:37 ` bug#35697: [PATCH 0/8] Make 'guix system docker-image' readily usable Ludovic Courtès

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.