From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id aKxvMIUzNV8ISAAA0tVLHw (envelope-from ) for ; Thu, 13 Aug 2020 12:35:17 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id qOEwLIUzNV9gYQAAbx9fmQ (envelope-from ) for ; Thu, 13 Aug 2020 12:35:17 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 58ABF94005D for ; Thu, 13 Aug 2020 12:35:17 +0000 (UTC) Received: from localhost ([::1]:34522 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6CRw-0007BC-BG for larch@yhetil.org; Thu, 13 Aug 2020 08:35:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47220) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k6CRj-00076J-RZ for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:35816) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1k6CRj-00046P-Hh for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1k6CRj-0000zT-F8 for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42849] [PATCH 3/3] installer: Run the installation inside a container. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 13 Aug 2020 12:35:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42849 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42849@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 42849-submit@debbugs.gnu.org id=B42849.15973220833761 (code B ref 42849); Thu, 13 Aug 2020 12:35:03 +0000 Received: (at 42849) by debbugs.gnu.org; 13 Aug 2020 12:34:43 +0000 Received: from localhost ([127.0.0.1]:47360 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k6CRP-0000yW-1Y for submit@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:43 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37078) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k6CRK-0000xu-OL for 42849@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:39 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53340) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6CRF-00044F-H7 for 42849@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:33 -0400 Received: from [2a01:e0a:19b:d9a0:3107:b202:556:bd51] (port=48336 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k6CRD-0001L2-KH; Thu, 13 Aug 2020 08:34:32 -0400 From: Mathieu Othacehe Date: Thu, 13 Aug 2020 14:34:19 +0200 Message-Id: <20200813123419.263639-3-othacehe@gnu.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200813123419.263639-1-othacehe@gnu.org> References: <20200813123419.263639-1-othacehe@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: 3.99 X-TUID: 7z8+KcSo3eJT When the store overlay is mounted, other processes such as kmscon, udev and guix-daemon may open files from the store, preventing the underlying install support from being umounted. See: https://lists.gnu.org/archive/html/guix-devel/2018-12/msg00161.html. To avoid this situation, mount the store overlay inside a container, and run the installation from within that container. * gnu/services/base.scm (guix-shepherd-service): Support an optional PID argument passed to the "start" method. If that argument is passed, ensure that guix-daemon enters the given PID MNT namespace. * gnu/installer/final.scm (umount-cow-store): Remove it, (install-system): run the installation from within a container. * gnu/installer/newt/final.scm (run-install-shell): Remove the display hack. --- gnu/installer/final.scm | 125 +++++++++++++++++------------------ gnu/installer/newt/final.scm | 7 -- gnu/services/base.scm | 60 ++++++++++------- 3 files changed, 99 insertions(+), 93 deletions(-) diff --git a/gnu/installer/final.scm b/gnu/installer/final.scm index 685aa81d89..a19018dc85 100644 --- a/gnu/installer/final.scm +++ b/gnu/installer/final.scm @@ -26,6 +26,8 @@ #:use-module (guix build syscalls) #:use-module (guix build utils) #:use-module (gnu build accounts) + #:use-module (gnu build install) + #:use-module (gnu build linux-container) #:use-module ((gnu system shadow) #:prefix sys:) #:use-module (rnrs io ports) #:use-module (srfi srfi-1) @@ -133,49 +135,18 @@ USERS." (_ #f)))))) pids))) -(define (umount-cow-store) - "Remove the store overlay and the bind-mount on /tmp created by the -cow-store service. This procedure is very fragile and a better approach would -be much appreciated." - (catch #t - (lambda () - (let ((tmp-dir "/remove")) - (syslog "Unmounting cow-store.~%") - - (mkdir-p tmp-dir) - (mount (%store-directory) tmp-dir "" MS_MOVE) - - ;; The guix-daemon has possibly opened files from the cow-store, - ;; restart it. - (restart-service 'guix-daemon) - - (syslog "Killing cow users.") - - ;; Kill all processes started while the cow-store was active (logins - ;; on other TTYs for instance). - (kill-cow-users tmp-dir) - - ;; Try to umount the store overlay. Some process such as udevd - ;; workers might still be active, so do some retries. - (let loop ((try 5)) - (syslog "Umount try ~a~%" (- 5 try)) - (sleep 1) - (let ((umounted? (false-if-exception (umount tmp-dir)))) - (if (and (not umounted?) (> try 0)) - (loop (- try 1)) - (if umounted? - (syslog "Umounted ~a successfully.~%" tmp-dir) - (syslog "Failed to umount ~a.~%" tmp-dir))))) - - (umount "/tmp"))) - (lambda args - (syslog "~a~%" args)))) - (define* (install-system locale #:key (users '())) "Create /etc/shadow and /etc/passwd on the installation target for USERS. Start COW-STORE service on target directory and launch guix install command in a subshell. LOCALE must be the locale name under which that command will run, or #f. Return #t on success and #f on failure." + (define backing-directory + ;; Sub-directory used as the backing store for copy-on-write. + "/tmp/guix-inst") + + (define (assert-exit x) + (primitive-exit (if x 0 1))) + (let* ((options (catch 'system-error (lambda () ;; If this file exists, it can provide @@ -188,7 +159,11 @@ or #f. Return #t on success and #f on failure." "--fallback") options (list (%installer-configuration-file) - (%installer-target-dir))))) + (%installer-target-dir)))) + (database-dir "/var/guix/db") + (database-file (string-append database-dir "/db.sqlite")) + (saved-database (string-append database-dir "/db.save")) + (ret #f)) (mkdir-p (%installer-target-dir)) ;; We want to initialize user passwords but we don't want to store them in @@ -198,27 +173,51 @@ or #f. Return #t on success and #f on failure." ;; passwords that we've put in there. (create-user-database users (%installer-target-dir)) - (dynamic-wind - (lambda () - (start-service 'cow-store (list (%installer-target-dir)))) - (lambda () - ;; If there are any connected clients, assume that we are running - ;; installation tests. In that case, dump the standard and error - ;; outputs to syslog. - (if (not (null? (current-clients))) - (with-output-to-file "/dev/console" - (lambda () - (with-error-to-file "/dev/console" - (lambda () - (setvbuf (current-output-port) 'none) - (setvbuf (current-error-port) 'none) - (run-command install-command #:locale locale))))) - (run-command install-command #:locale locale))) - (lambda () - (stop-service 'cow-store) - ;; Remove the store overlay created at cow-store service start. - ;; Failing to do that will result in further umount calls to fail - ;; because the target device is seen as busy. See: - ;; https://lists.gnu.org/archive/html/guix-devel/2018-12/msg00161.html. - (umount-cow-store) - #f)))) + ;; When the store overlay is mounted, other processes such as kmscon, udev + ;; and guix-daemon may open files from the store, preventing the + ;; underlying install support from being umounted. See: + ;; https://lists.gnu.org/archive/html/guix-devel/2018-12/msg00161.html. + ;; + ;; To avoid this situation, mount the store overlay inside a container, + ;; and run the installation from within that container. + (zero? + (call-with-container '() + (lambda () + (dynamic-wind + (lambda () + ;; Save the database, so that it can be restored once the + ;; cow-store is umounted. + (copy-file database-file saved-database) + (mount-cow-store (%installer-target-dir) backing-directory)) + (lambda () + ;; We need to drag the guix-daemon to the container MNT + ;; namespace, so that it can operate on the cow-store. + (stop-service 'guix-daemon) + (start-service 'guix-daemon (list (number->string (getpid)))) + + (setvbuf (current-output-port) 'none) + (setvbuf (current-error-port) 'none) + + ;; If there are any connected clients, assume that we are running + ;; installation tests. In that case, dump the standard and error + ;; outputs to syslog. + (set! ret + (if (not (null? (current-clients))) + (with-output-to-file "/dev/console" + (lambda () + (with-error-to-file "/dev/console" + (lambda () + (run-command install-command + #:locale locale))))) + (run-command install-command #:locale locale)))) + (lambda () + ;; Restart guix-daemon so that it does no keep the MNT namespace + ;; alive. + (restart-service 'guix-daemon) + (copy-file saved-database database-file) + + ;; Finally umount the cow-store and exit the container. + (umount-cow-store (%installer-target-dir) backing-directory) + (assert-exit ret)))) + #:namespaces '(mnt) + #:jail? #f)))) diff --git a/gnu/installer/newt/final.scm b/gnu/installer/newt/final.scm index fa8d6fea71..89684c4d8a 100644 --- a/gnu/installer/newt/final.scm +++ b/gnu/installer/newt/final.scm @@ -102,13 +102,6 @@ a specific step, or restart the installer.")) #:key (users '())) (clear-screen) (newt-suspend) - ;; XXX: Force loading 'bold' font files before mouting the - ;; cow-store. Otherwise, if the file is loaded by kmscon after the cow-store - ;; in mounted, it will be necessary to kill kmscon to umount to cow-store. - (display - (colorize-string - (format #f (G_ "Installing Guix System ...~%")) - (color BOLD))) (let ((install-ok? (install-system locale #:users users))) (newt-resume) install-ok?)) diff --git a/gnu/services/base.scm b/gnu/services/base.scm index 491f35702a..f62fd861ca 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -1558,36 +1558,50 @@ proxy of 'guix-daemon'...~%") (provision '(guix-daemon)) (requirement '(user-processes)) (actions (list shepherd-set-http-proxy-action)) - (modules '((srfi srfi-1))) + (modules '((srfi srfi-1) + (ice-9 match))) (start - #~(lambda _ + #~(lambda args (define proxy ;; HTTP/HTTPS proxy. The 'http_proxy' variable is set by ;; the 'set-http-proxy' action. (or (getenv "http_proxy") #$http-proxy)) (fork+exec-command - (cons* #$(file-append guix "/bin/guix-daemon") - "--build-users-group" #$build-group - "--max-silent-time" #$(number->string max-silent-time) - "--timeout" #$(number->string timeout) - "--log-compression" #$(symbol->string log-compression) - #$@(if use-substitutes? - '() - '("--no-substitutes")) - "--substitute-urls" #$(string-join substitute-urls) - #$@extra-options - - ;; Add CHROOT-DIRECTORIES and all their dependencies - ;; (if these are store items) to the chroot. - (append-map (lambda (file) - (append-map (lambda (directory) - (list "--chroot-directory" - directory)) - (call-with-input-file file - read))) - '#$(map references-file - chroot-directories))) + ;; When running the installer, we need guix-daemon to operate + ;; from within the same MNT namespace as the installation + ;; container. In that case only, enter the namespace of the + ;; process PID passed as start argument. + (append + (match args + ((pid) + (list #$(file-append util-linux "/bin/nsenter") + "-t" pid "-m")) + (else '())) + (cons* #$(file-append guix "/bin/guix-daemon") + "--build-users-group" #$build-group + "--max-silent-time" + #$(number->string max-silent-time) + "--timeout" #$(number->string timeout) + "--log-compression" + #$(symbol->string log-compression) + #$@(if use-substitutes? + '() + '("--no-substitutes")) + "--substitute-urls" #$(string-join substitute-urls) + #$@extra-options + + ;; Add CHROOT-DIRECTORIES and all their dependencies + ;; (if these are store items) to the chroot. + (append-map + (lambda (file) + (append-map (lambda (directory) + (list "--chroot-directory" + directory)) + (call-with-input-file file + read))) + '#$(map references-file + chroot-directories)))) #:environment-variables (append (list #$@(if tmpdir -- 2.28.0