From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id QLyNNJkfE2KSpQAAgWs5BA (envelope-from ) for ; Mon, 21 Feb 2022 06:14:01 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id iJDOMZkfE2J5HQAA9RJhRA (envelope-from ) for ; Mon, 21 Feb 2022 06:14:01 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5A5BA1CDCA for ; Mon, 21 Feb 2022 06:14:01 +0100 (CET) Received: from localhost ([::1]:52720 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nM11M-0002AA-2R for larch@yhetil.org; Mon, 21 Feb 2022 00:14:00 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nM117-00028l-Uz; Mon, 21 Feb 2022 00:13:45 -0500 Received: from [2607:f8b0:4864:20::732] (port=40527 helo=mail-qk1-x732.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nM116-0003Bz-9T; Mon, 21 Feb 2022 00:13:45 -0500 Received: by mail-qk1-x732.google.com with SMTP id c7so13002821qka.7; Sun, 20 Feb 2022 21:13:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=B/2B65fkLNKD23Kv7ewPY/VEKFfSQhLpl0QkmEyMiBw=; b=Fc7Vi8crZupTS8ImTaf88FxEsvyNFSausd1zxO7QM0RQ6RJkBKvUriJzWMtPQipqoc qmORvEKlcVOAYJfrEeG14tmbg4XaFHPvZcW81dEjQyygjHYi3ZSlzOEAQKUGehJ+hOWH DSozXdd2uG2uoQTSIsfI01LnnfF15cX6nauI3UoPBaoCOAWiIgkO/WGcatacXtKLY+Xy NgW7GPupjvz9PKPu3zV9zQLa5hAw0UGgNcNz+EuZ/JQEhhVaKX2UVoMcFVq9P8Q7gBy7 dx/Oo+q2iL3XiuCa5zIXQN6Zf2cByt8Libt+E1W8eesWMsbeH/YxsQ6WTJM7joMQ5K3y UyMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=B/2B65fkLNKD23Kv7ewPY/VEKFfSQhLpl0QkmEyMiBw=; b=7yE4oMlbYphHXduDdhe+Vu5t2LGfhERna6hQCj4lGtE0GZg54OABY1KrZNKDcvnH9S 08dZO0rlReRCsJV5r3d1JgfosCCsOuv39bIv9cAF9CmMqZDvOsOtqv3jGVYX3A7eAynD P1d33rYiek9CYXcZddGhk9h6sDn7dDwrKS9346Dk8gB9oecT0k/MNkyXdq9QBArjW+oC F3SbqsQgx5OLxJHpQNFOkjrtn3FvRbUH1BsAfXef8uoiXDkHN1tDx5Cz5aZbnlDo5Btc 1G9xhjSsZvohVMrDyhh1F4/jA0V/iney6pWPNZA6M8IFWUduS//I5Tj8CXfu/ajCOS+q w0gw== X-Gm-Message-State: AOAM5324/Cbpz1qpipV/iewFa09uxxB6wJjPZumvQToeAA1r1Eahm2Vy PU+ezakIoLWHiX7eJOc2KfD9nLzbBnI= X-Google-Smtp-Source: ABdhPJx8oCxE+ZorzmWgTo53UCoW8NVp/mvJTDLi0PsSW5engYTTqvOX2vG3s4+Vr3NFjqPxyl6NxA== X-Received: by 2002:a05:620a:1a1d:b0:648:af1f:90d7 with SMTP id bk29-20020a05620a1a1d00b00648af1f90d7mr4977426qkb.340.1645420421611; Sun, 20 Feb 2022 21:13:41 -0800 (PST) Received: from hurd (dsl-153-17.b2b2c.ca. [66.158.153.17]) by smtp.gmail.com with ESMTPSA id k5sm1139657qkk.82.2022.02.20.21.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 21:13:41 -0800 (PST) From: Maxim Cournoyer To: Mathieu Othacehe Subject: Re: Dropping gzip-compressed substitutes References: <878rum1pph.fsf@gmail.com> <87h791s5fr.fsf@gnu.org> <87r184l3sg.fsf@gnu.org> Date: Mon, 21 Feb 2022 00:13:40 -0500 In-Reply-To: <87r184l3sg.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 15 Feb 2022 13:20:31 +0100") Message-ID: <878ru4pzsr.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::732 (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::732; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qk1-x732.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel , guix-sysadmin Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1645420441; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=B/2B65fkLNKD23Kv7ewPY/VEKFfSQhLpl0QkmEyMiBw=; b=PhHChqcCuKaJAXln6O8jpAgnezA7GR1PROQhnRoWMCPhJvpYuHwWG+Jo5T6t02ifs9V7br nQRnQL8KlkrbrIV9ZAiAmXyBKevYIiVcROrRriq5qlAQXh20VVm5xgTbxicFu78MvkV8wC Mqni/gYHHu6epwboNjuV/foqL5xygxezn6DIhNaesV98Yah8h/Rq5EGm+n1VKMKqcagAud pRfXs7ag+81fdFLHVk1IBwPwluaCgYc+shWmtxKPk3dnBQ18jmMsRjWYP/Bru8F3CJmycx WI0L5L6BHl4JkO1lj4YrevZL1rXGLExPId9M57Fve2g7oK4pG5wlHNqwxJ+0ag== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1645420441; a=rsa-sha256; cv=none; b=AKrjWzwIITphlNPyV6mOUtDhyoEYjIxajQJUx5kwHeK/kQZ69zRZnAhHC6JI6HLx4/4glf OjgBM/7ipiDnkpW150FiA1mvLGlEdOXZAefeSkq1DKkzqJMBPvRIIKhm80Gdn6vorBwYax UOTJidW7xk1BxnxHviSQrRJNKRMNlxCwnLf8Lv8+/oTt5zfaA7H7gJbvJgToaX5En3Dj3C ddDQVeIYjAV1+CoPT2jvlHQW4iZe7JPmLEEmBlJNQ+vJgG+nBB5cnI4/nR89eOneEwV/Z1 575wAgFM6Wr6ZV5rauVHrXV7U5q98iA14SMfq7SB/m/aoHQRoQDoZbqgvrqqNQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Fc7Vi8cr; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.03 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Fc7Vi8cr; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 5A5BA1CDCA X-Spam-Score: -2.03 X-Migadu-Scanner: scn1.migadu.com X-TUID: 2tzrMHijHUkb Hi everyone, Mathieu Othacehe writes: > Hey Ludo, > >> As discussed on IRC, I=E2=80=99m skeptical about this because: >> >> 1. It requires the development and testing of a custom tool that=E2=80= =99s >> easy to get wrong=E2=80=94e.g., it removes a gzipped nar for someth= ing that >> had nothing but gzip available, etc. >> >> 2. That code would have to run with privileges that give it access to >> the signing key on berlin. >> >> 3. Those 6.5=C2=A0TB are an initial constant factor; growth of the sto= rage >> requirements going forward probably matters more and >> will give us more flexibility >> on that. > > While those are valid points, we need to keep in mind that it is > important that we manage to move the store to the new SSD array quite > quickly to start GCing again. I turns out that the store contains about 14 TiB of probably mostly unnecessary things that just weren't garbage collected (due to the prohibitive cost of doing so on the HDD hard drive). Currently the new array still has 5 TiB so I think we can try to migrate today and then work trimming the gzip substitutes to have some extra head room. With Ricardo, we've attempted to reboot Berlin onto a freshly 'guix system init'ed system using the new array last Thursday. Unfortunately we hit some issue where mounting the root file system was apparently occurring before all the 6 drives had the time to be visible. This week, I'd like to try the following to see if we could get past this: 1. Do the experiment again, now a 'rootdelay=3D20' kernel parameter was added to Berlin's config. This may well be enough. 2. In case mounting the RAID 10 Btrfs root partition still fails with missing drive errors, try the following workaround suggested in the #btrfs channel, which forces a 'btrfs device scan' on each device of the array, with the following mount option: "device=3D/dev/sda3,/dev/sdb3,/dev/sdc3,/dev/sdd3,/dev/sde3,/dev/sdf3" To make it more convenient to experiment with different values for the rootdelay or add the device option above, I'm planning to 'guix system init' with the following patch applied: https://issues.guix.gnu.org/40998, which allows providing 'rootflags' directly from the kernel command line (thus by editing the GRUB config at boot). I'll try to synchronize with Ricardo in the channel and hope they weren't too frightened by our last experiment to not shy away from trying again :-). That's it for the update. Until this is sorted out, I'd ask to not reboot the machine, and not even reconfigure it. I'll give you all an update as soon as it is sorted out. Thanks! Maxim