From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id aJ+NGnkCD2N96QAAbAwnHQ (envelope-from ) for ; Wed, 31 Aug 2022 08:40:57 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id aJKkGnkCD2NfFgEA9RJhRA (envelope-from ) for ; Wed, 31 Aug 2022 08:40:57 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D12001E523 for ; Wed, 31 Aug 2022 08:40:56 +0200 (CEST) Received: from localhost ([::1]:48560 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oTHPD-0000m0-Rh for larch@yhetil.org; Wed, 31 Aug 2022 02:40:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34488) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oTHOF-0000li-3D for help-guix@gnu.org; Wed, 31 Aug 2022 02:39:55 -0400 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21189) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oTHOC-0006RT-Sy; Wed, 31 Aug 2022 02:39:54 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1661927978; cv=none; d=zohomail.com; s=zohoarc; b=Rt6dNF0v8tKzrqCFvOjDASzW41c83fmVezEgoo9Ih2108MyY3QYxN0EsVRTI3iOp8xxam3wA/pXAb2vo9lJHnZAzg+P8Of6veYZZD16cx90QY1GCZ7Sl1e1eGbf2IHZseu2+eA7sm4zgfSaeLZNWxrVI5YLXpR/mc3F6jEnCGds= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661927978; h=Content-Type:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=fKuzz3FTkSUa54IPlN93s/PQT8UtD2HTj1MraXGxrjA=; b=Liu4mLGW5BKy3TQkeh4q1OhSzWHPvkcGjVskeDbxgXA+y/IEiwX4fWpUr88cjITpuTrk7lShNLdcHn+A8QthjUE/2fJgVGum9JKs28tv73uuykqrldTKtV0IpajE4iXfJW69aCSJoY9ha9Uk9dfD1dav22FiXJrJmE3N8edH3iA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1661927978; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:In-reply-to:Message-ID:MIME-Version:Content-Type:Message-Id:Reply-To; bh=fKuzz3FTkSUa54IPlN93s/PQT8UtD2HTj1MraXGxrjA=; b=CYckJnZiWAEnY3fjWtAC3i4u+5xB7/jOUiueHGz0AtBiNKrfvAwPGLDh2FdF5mco tpRNYUhhqzBJXdTqp6gXQ8HFOIw6QG+vb/FAsnw136DWZIHGx4CEKt2ybZTKptDkZRJ 7hYzKfhJqqSYgpLJCKJBP8MiEJaQ+sJONGmNnQMM= Received: from localhost (i5E862DD2.versanet.de [94.134.45.210]) by mx.zohomail.com with SMTPS id 16619279742626.567199343086372; Tue, 30 Aug 2022 23:39:34 -0700 (PDT) References: <8735dzqhge.fsf@beadling.co.uk> <47774701-8E8E-4185-9DB9-7E5D7F91ACD2@yasuaki.com> <87lerbxxfs.fsf@elephly.net> User-agent: mu4e 1.8.7; emacs 28.1 From: Ricardo Wurmus To: "Thompson, David" Cc: Yasuaki Kudo , Phil , Ludovic =?utf-8?Q?Court=C3=A8s?= , Benjamin Slade , Olivier Dion , help-guix Subject: Re: Enterprise Guix Hosting? Date: Wed, 31 Aug 2022 08:33:25 +0200 In-reply-to: X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Message-ID: <878rn4syql.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1661928057; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=fKuzz3FTkSUa54IPlN93s/PQT8UtD2HTj1MraXGxrjA=; b=CUqVD3kPYCdhZCdpUMdLlTzEdSiBqSxK3bAlFpw8yWumeE7oA7puAzBojlGGmGLxNyL/Dp jtF4YD6LtAjSjG/0z39eTTWrbv3iUNk6+bAQDQg++N5AYMu69jzbDjaUDUyyDVYTd97X0C RE77YE5c/GW9FA3aC423mper2OJAzkRP5YY4X/heVmavwNH2wo4YxUSXDl0rWctKEX/oFB MQyBuTb3S+tiUUCwoQHvmUYyL2FpwKwle89U6Wgzbmoil5N0TpTTnRhavjG+HZlhhXYMJS W742pSSoIcK13AIeLUsOwqLIAa7h5zNC/U5ul3Mgdl1yV6P0TD/3HJNZ0KcRcw== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1661928057; a=rsa-sha256; cv=pass; b=PSIcO9KUS4Oq2sNRvQhfWQHIiz1GONLaRE7RC3PZbwUYuLLe1UbexIGtFYhwQ5wqBqIUuc qPG2xI61b436atvJOV42D3NfC+A8BIXnL4/Yl2XVEkkIu3h/tdMHHQZrRamyvtwYnIR4PX kuOWvBgTHFddZHc5y9av/wOlpLsw9zzMyR1IgcWZfYQK3bZTqm3rhDDxotCRALrk6IcRmO C/2h0lwDq+cBP7JqgvOMadkmF0nZLJSZ+5fJaeqGn55sVR7MWS86C3QMrInli7HOgLyzdQ gzFe3T0cEISbMDZHOAiQ8vq+0zyLl6R2xJlU3rBlMlBiLx2ZVV13rI70l/wnWg== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=CYckJnZi; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.79 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=CYckJnZi; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: D12001E523 X-Spam-Score: -1.79 X-Migadu-Scanner: scn1.migadu.com X-TUID: 0akTW1cG0jhj "Thompson, David" writes: >> Using a shared /gnu/store as a big cache for all containers can be a >> real asset. We can learn lessons from the HPC experience here. > > What might have a positive impact is if Guix had an answer to 'docker > compose'. Most of the pieces are there already. Such a tool could be > combined with 'guix shell' so you could get all the tools needed for > local development *and* automatically start any necessary daemons, > like database servers, in isolated containers. Yes, this would be useful. Another thing that seems to be missing is a way to supervise and manage running containers. I use a shepherd instance for this with container-specific actions like this: (define %ip (let ((loc #f)) (lambda () "Return the absolute file name of the ip command." (or loc (let ((value (which "ip"))) (set! loc value) value))))) (define (launch container-id script) (herd "root" "eval" (format #false "~s" `(begin (use-modules (srfi srfi-2) (ice-9 popen) (ice-9 rdelim) (ice-9 match)) (define (guix-container id script) (make #:provides (list (string->symbol (string-append "guix-container-" id))) #:docstring "Run a Guix System container" #:start ;; TODO: Using /bin/sh -c is ugly, but ;; without it the container would be stuck in the early boot process. (make-forkexec-constructor `("/bin/sh" "-c" (string-join (list "exec" script ,@args) #\space))) #:stop (make-kill-destructor) #:actions (make-actions (pid "Show the PID of the system container." (lambda (running) (let ((pid (call-with-input-file (format #false "/proc/~a/task/~a/children" running running) read))) (display (match pid ((? eof-object?) "") (_ pid)))))) (ip "Show the IP address of the system container." (lambda (running) (let* ((pid (number->string (call-with-input-file (format #false "/proc/~a/task/~a/children" running running) read))) (ns (format #false "guix-~a" pid)) (ip ,(%ip)) (address (catch #true (lambda () (let* ((pipe (open-pipe* OPEN_READ ip "netns" "exec" ns "ip" "-o" "-4" "-family" "inet" "addr" "show" "dev" (format #false "ceth-~a" pid))) (output (read-line pipe))) (match (string-tokenize output) ((number if "inet" ip . rest) ip) (_ "")))) (lambda _ "")))) (display address)))) (up "Connect network for the system container." (lambda (running) (let* ((pid (number->string (call-with-input-file (format #false "/proc/~a/task/~a/children" running running) read))) (ns (format #false "guix-~a" pid)) (host (format #false "veth-~a" pid)) (client (format #false "ceth-~a" pid)) (ip ,(%ip)) (sys (lambda args (or (zero? (apply system* args)) (error args))))) ;; Make existing network namespace available to ip netns (sys ip "netns" "attach" ns pid) ;; Create veth pair and move the client side into the container. (sys ip "link" "add" host "type" "veth" "peer" "name" client) (sys ip "link" "set" host "up") (sys ip "link" "set" client "netns" ns) ;; Attach host side to host bridge (sys ip "link" "set" host "master" "br0") ;; Bring up interface in container (sys ip "netns" "exec" ns "ip" "link" "set" "lo" "up") (sys ip "netns" "exec" ns "ip" "link" "set" client "up")))) (down "Disconnect network for the system container." (lambda (running) (let* ((pid (number->string (call-with-input-file (format #false "/proc/~a/task/~a/children" running running) read))) (ns (format #false "guix-~a" pid)) (host (format #false "veth-~a" pid)) (ip ,(%ip)) (sys (lambda args (or (zero? (apply system* args)) (error args))))) (sys ip "netns" "delete" ns) (sys ip "link" "delete" host)))) (netstat (lambda (running) (and-let* ((pid (number->string (call-with-input-file (format #false "/proc/~a/task/~a/children" running running) read))) (template (lambda (what) (format #false "/sys/class/net/veth-~a/statistics/~a_bytes" pid what))) (rx (call-with-input-file (template "rx") read)) (tx (call-with-input-file (template "tx") read))) (format #true "receive:~a transmit:~a" rx tx))))))) (let ((service (guix-container ,vm-id ,script))) (register-services service) (start service)))))) -- Ricardo