From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id mNYJD9UbwmGGCgEAgWs5BA (envelope-from ) for ; Tue, 21 Dec 2021 19:24:21 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 8Oq6CtUbwmEAHwAAB5/wlQ (envelope-from ) for ; Tue, 21 Dec 2021 18:24:21 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 832443B7F for ; Tue, 21 Dec 2021 19:24:18 +0100 (CET) Received: from localhost ([::1]:35092 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mzjo9-0003W8-3x for larch@yhetil.org; Tue, 21 Dec 2021 13:24:17 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49332) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mzjnu-0003Vj-Gz for bug-guix@gnu.org; Tue, 21 Dec 2021 13:24:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:44076) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mzjnt-0003Rf-Qc for bug-guix@gnu.org; Tue, 21 Dec 2021 13:24:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mzjnt-0000to-No for bug-guix@gnu.org; Tue, 21 Dec 2021 13:24:01 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#51787: Disk performance on ci.guix.gnu.org Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 21 Dec 2021 18:24:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51787 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ricardo Wurmus Received: via spool by 51787-submit@debbugs.gnu.org id=B51787.16401110193425 (code B ref 51787); Tue, 21 Dec 2021 18:24:01 +0000 Received: (at 51787) by debbugs.gnu.org; 21 Dec 2021 18:23:39 +0000 Received: from localhost ([127.0.0.1]:55622 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mzjnX-0000tA-7z for submit@debbugs.gnu.org; Tue, 21 Dec 2021 13:23:39 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48536) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mzjnV-0000sy-4e for 51787@debbugs.gnu.org; Tue, 21 Dec 2021 13:23:37 -0500 Received: from [2001:470:142:3::e] (port=60754 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mzjnP-0003PP-J4; Tue, 21 Dec 2021 13:23:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=C3dpeCCzYZiMUrYrYhZD0PYoQTvAt6/iG4SGGStFGsg=; b=IidPS+yIsd+MnOw2IzaI AUn+CyJG/DbN48PYDbxRHi/2L0B4clI1BQ/+g66VrIo65fabUsnp79AucanNriMn+d5b0jKOrPqLI okCWnTDDm5Je9hVRgPqIRUS/K/LYL39bbZJIa4mUiNQ5IVr4lAPGjmZ4XwQKpOMLlE3NRyZqCGPsy y6MN7BUNWUqBLSkQdglVRq+seqGFR08mv8vJm2x4qzcQXFfVRmtdZnhhYuCfS87NGVS2LkYe2mgWD DcJA9cH4QgOl6fWNtl2O/EOzz11xcXFKHXPvKXrF9jJfgKHBuf1qcvDjhjXkZcUW5MeqJUJ9E6qjx P/lI4D8M/7hMIA==; Received: from [2a01:cb18:832e:5f00:3563:417e:2a38:86d8] (port=49210 helo=meije) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mzjnP-0005LD-GZ; Tue, 21 Dec 2021 13:23:31 -0500 From: Mathieu Othacehe References: <875yrjpi1y.fsf@elephly.net> <87o85bjjpm.fsf@gnu.org> <871r27p5jq.fsf@elephly.net> <87a6gv3pue.fsf@gnu.org> <87v8zhn9m1.fsf@elephly.net> Date: Tue, 21 Dec 2021 19:23:28 +0100 In-Reply-To: <87v8zhn9m1.fsf@elephly.net> (Ricardo Wurmus's message of "Tue, 21 Dec 2021 18:26:03 +0100") Message-ID: <874k71g6lb.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 51787@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1640111058; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=C3dpeCCzYZiMUrYrYhZD0PYoQTvAt6/iG4SGGStFGsg=; b=YtrZqfClH0YDnS5x2fwBVkuXGJncdb8iTOUOQxSM0Zh+sxcQlYXAmCo0Jv10V2y4A679bT c+MKaIhSHaXyxpzBq99Dh80BciEiNfoXl8RhlW3TTCAPnqPyCchvrkP3Be9/cDOQiisWqK f7ikIqlPe48BBFMM2FGZyaVepVSGaDSN/EAZfTQ+KvV2zpvEr21H4rkZ8kqIvVu0gX1Y3Z OgFOGtFWBIh6I6QT0tYzZPyTvqQXedt/EjioDdl72/Nfo+LyJbgSE811Lky0QGcRJUbhe8 GtP9cuHVJZDQ7QqbOpoET2hPHE4SGinMeC9uEf2kMQPR/l7XwjRYwQIvmZsfTg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1640111058; a=rsa-sha256; cv=none; b=MxpwHC6ccFAJ1WairAaWCaYVfYtSjVFVjiDFXtPbcyuRtWHh86E1cykEtGCm/ubKhKGe82 SDfXrEuz2g0ryt1KAZlNG/eHBtMbKHjs5AJJfGbZbFX9xby0qak1uGNGP9LqUjOIluSEba iqzq/b8UvmGv9rgLgcVu+Xi2PlkXJTmyV9sYCWKTPPHoxz5Xaw+ESEIjIcblRAg0Zz+HTd xR0M2qUFwJnbBULCb0T3hejm4y2oWUf/CgPPd7ESVENxWJultlha5OLhxl0proATulq8A4 YM0k5/c1zEHlHrryAal7en/E7QKhCZE8UT2EEv1xa+AoS5QvKOqMrUb/T7RVYw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=IidPS+yI; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.03 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=IidPS+yI; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 832443B7F X-Spam-Score: -4.03 X-Migadu-Scanner: scn0.migadu.com X-TUID: jX7hh604mjXG Hey, > Today we discovered a few more things and discussed them on IRC. Here=E2= =80=99s > a summary. Nice summary :) > We could take this opportunity to reformat /gnu with btrfs, which > performs quite a bit more poorly than ext4 but would be immune to > defragmentation. It=E2=80=99s not clear that defragmentation matters her= e. It > could just be that the problem is exclusively caused by having these > incredibly large, flat /gnu/store, /gnu/store/.links, and > /gnu/store/trash directories. > > A possible alternative for this file system might also be XFS, which > performs well when presented with unreasonably large directories. > > It may be a good idea to come up with realistic test scenarios that we > could test with each of these three file systems at scale. We could compare xfs, btrfs and ext4 performances on a store subset, 1TiB for instance that we would create on the SAN. Realistic test scenario could be: - Time the copy of new items to the test store. - Time the removal of randomly picked items from the test store. - Time the creation of nar archives from the test store. That will allow us to choose the file-system that has the best performances for our use-case, regardless of fragmentation. Now fragmentation may or may not be a problem as you mentioned. What we could do is repeat the same tests but on a test store that is created and removed N times, to simulate file-system aging. This is more or less what is done in this article[1] by "git pulling" N times a repository and testing read performances. For them btrfs > xfs > ext4 in term of performances, but we might draw different conclusions for our specific use case. Do you think it is realistic? If so, we can start working on some test scripts. Thanks, Mathieu [1]: https://www.usenix.org/system/files/hotstorage19-paper-conway.pdf