From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 2MlSCw+/vWEl9AAAgWs5BA (envelope-from ) for ; Sat, 18 Dec 2021 11:59:27 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id KCAVBw+/vWFXLgAAB5/wlQ (envelope-from ) for ; Sat, 18 Dec 2021 10:59:27 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E82A237B60 for ; Sat, 18 Dec 2021 11:59:26 +0100 (CET) Received: from localhost ([::1]:53148 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1myXR0-0005sW-2b for larch@yhetil.org; Sat, 18 Dec 2021 05:59:26 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59376) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1myXQs-0005sN-3A for guix-devel@gnu.org; Sat, 18 Dec 2021 05:59:18 -0500 Received: from mout01.posteo.de ([185.67.36.65]:45913) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1myXQo-0003Mh-KX for guix-devel@gnu.org; Sat, 18 Dec 2021 05:59:17 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 58585240029 for ; Sat, 18 Dec 2021 11:59:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1639825151; bh=HYNYHpe9i0jVaW/ZIXsnib3mp0L5v/V7izDOUjOazZQ=; h=From:To:Cc:Subject:Date:From; b=nm0nzVXRikMxbsRXsm7IC50qpH6DxDk+AVo8zKaFHkCiD5CeGhpbhq29oE93gH58N q1VilX8PDNp2vtgnIXunf2FFTWIuV5QsRaelGZ/MTF+aQdLhnTw3qSJpnQvdjkzTwR ke3YwQ2467gLpxYv4kuNJQ9P0iub683eclKVaYrjkU0iilScv65BSuflBlevsWYXsb pCShc9ZxWIZRc6E3DNq2gqhoF6SSuSTJ8ogp15yV9lAztLG7aQs7RXpM5p1SizkHrJ IOR5lVmfbzPaCzoKPKc/V88YWs/AYxK0SZKMA4aUmAZQ1Mk2NTNqM7FBGwqYu046r9 BnZLrtYR4U1tw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JGNC22Qwkz9rxL; Sat, 18 Dec 2021 11:59:10 +0100 (CET) References: <87v8zorien.fsf@inria.fr> From: pukkamustard To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: Solstice infrastructure hackathon Date: Sat, 18 Dec 2021 10:57:59 +0000 In-reply-to: <87v8zorien.fsf@inria.fr> Message-ID: <86fsqqkwlg.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.67.36.65; envelope-from=pukkamustard@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org, guix-sysadmin Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1639825167; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=yrXJNy6KOxEOv8mOoPYeWUJBDkYBOOP6AWlun0uvpN0=; b=oiWZ+jlKrXYj7r1teSUbt3OXyk40LpEq5GqvZfJbi6A82bX7bJo1M6MoD0SAu75OMKgQrx iDR9jPEn7xE0IOHJtkX5/pQIEc9i1lHmKrqsrhqYYg4UTbluDJmNXgCFJvS3Aijbty4rzO Edkkw28daOyqenQmOXmbWO/riowK/vy8XhsgdsqQE+TM9Smz1NwOzuhng8gR49p+aOT1Cc k836oxlBz7bR93kpXBasZOfyAMcGf16MtlH9ork5haRgefvIAoGZwgNsgFfFkXQHwO9Pfw hi621czXuMX3SwtQwYgsn4CJ2aFQJSbw7qIKZtj8ZukTZGTFkMMqpnDuA7+muA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1639825167; a=rsa-sha256; cv=none; b=D/vghSdhFQwXnqQOtyznTHuTek1dWMUd84NUqWoEjjm5jWSXeJcsFSWN8bS3xkNrR8l+8J KJ1AWCDozHt6PovkgO2nFADrL/+fbCUk0CZaiT/HzNX9iEGJQ1xXMZUqIuyhLJvRZwN/1J Wx4Xfvciwhk509hQywxnyGWSpEK0H+E7p7ZYJlCeqQDfgXVChyRXwTEVljIobl9CgMulgj EClnMeH5TN2XFfugYNhxGPYUk1F7ZmHFQUBlqLgrFkzDPswO93HkROiztrPQ4l0tE9G4nL 8+00UHW7zernuvzdg1ZRLwa11kd3gylMTfsPLePKMCKcss6PtfxtfSyzYrstug== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=nm0nzVXR; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.40 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=nm0nzVXR; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: E82A237B60 X-Spam-Score: -2.40 X-Migadu-Scanner: scn0.migadu.com X-TUID: 8xTYzsLqLq1n I'm in! Ludovic Court=C3=A8s writes: > Hello Guix! > > This week, the node behind {ci,issues,disarchive}.guix.gnu.org and > guix.gnu.org was down twice for a few hours=E2=80=94nothing terrible in t= he end, > but it reminded us that, even though Guix doesn=E2=80=99t rely on any par= ticular > machine, we can definitely feel the inconvenience when it=E2=80=99s down. > > We were unlucky enough that it happened days after the other build farm, > bordeaux.guix.gnu.org, ran out of disk space and had its CI stopped, > right before the big merge=E2=80=94so it doesn=E2=80=99t have substitutes= for current > master. > > While discussing this on IRC the other day, we thought that perhaps it > was time to have an infrastructure hackathon. How about Tuesday, > Dec. 21st? (Probably with a followup in January.) > > Here are tasks that were brought up: > > =E2=80=A2 Set up a backup server for berlin.guix.gnu.org, the head node= of the > ci.guix.gnu.org, possibly moving some services such as the web site > there. > > =E2=80=A2 Add DNS redundancy for guix.gnu.org so it can point to one of= two > hosts (need to figure out certbot challenges so both machines can > update their certificates). > > =E2=80=A2 Set up status.guix.gnu.org with sysadmin status updates (poss= ibly > using Prometheus?). > > =E2=80=A2 Come up with a plan to add disks to the RAID array on bayfron= t, the > head node of bordeaux.guix.gnu.org. > > =E2=80=A2 Work on a plan to back up the Disarchive database currently on > berlin.guix. > > =E2=80=A2 Work on a plan to mirror nars from ci.guix and bordeaux.guix,= using > plain rsync or . > > =E2=80=A2 Have a documented procedure to set up substitute mirrors, suc= h as > the one in .cn (I can=E2=80=99t find the URL), ideally with plain rsy= nc > access. > > Am I forgetting something? > > Some of these tasks require root or physical access for the final steps, > but most of them are about (1) coming up with a plan, and (2) adjusting > the system configuration at > . > Anyone with sysadmin experience to share and Guix System knowledge can > join! We=E2=80=99ll communicate over #guix on irc.libera.chat. > > Who=E2=80=99s in? :-) > > Cheers, > Ludo=E2=80=99.