From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id GJc1IE2uLmfGGAEA62LTzQ:P1 (envelope-from ) for ; Sat, 09 Nov 2024 00:35:25 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id GJc1IE2uLmfGGAEA62LTzQ (envelope-from ) for ; Sat, 09 Nov 2024 01:35:25 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=pQcfQg8G; dkim=fail ("body hash did not verify") header.d=dokmelody.org header.s=mail header.b="lSZt/GQ+"; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=dokmelody.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1731112525; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=fJWNhK8EBp5LXPbkK6O3SPYwhpE12OIg3Z2U8WWcegU=; b=kzoeom1OJ6oBnSZCkluMra5VK/IFyl7vCtjcmL63CjzoMpnzKMA+jkl5PjatP8o3KSKPU8 TXRMan7vK/DTPH3qOAfTfof5IwauD6MY/zhqYaUVTCG5ZQZXw3BHd08OIe03C7/naDoRTJ zNKLdFxEEqGEqDwtog/QONLN4FI5OB++HFpTFihvSOKHg/IKZGyCHDn029u52iID4gLnaQ PT8CauylW3tK5Csa8xEB5T2ZqvhGrppqYmDogy0NX9FLSqoBmSdpLQykADp/FGF4jYdQxV VTKNUT5Sl4AKeFy6t1GvlvbtXpGeG07/qIMR9LiTG4yPvAa1kSHIwQ8tDWaAXQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=pQcfQg8G; dkim=fail ("body hash did not verify") header.d=dokmelody.org header.s=mail header.b="lSZt/GQ+"; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=dokmelody.org (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1731112525; a=rsa-sha256; cv=none; b=W3BpwFbEwZuJxwHUrpNm54aLhvAebuOGZpLcObo1noxg6yI8Nwm7dKNH3Ao0SmLrfU80Ta lS41x0wLQy3O4k9XAseEWYrVkLMYlTtOLoTkryGVzp4Ww69TQTtU/sAfTvBeNYT5Ci1VWz 0ia9Y+Pl8YuAqTBiwIKxs5xEWYq2oeqXukTsPBEH98wFaTfRxDPebBGUczn6gF9wS1xo+q MYHCo3p6Q8iS3+DChhJE7t7nnlU4kJ3qtlI2We7xvLSJqoaxkgqqmNFmn78s9TVhHQSg/9 v+tYdPoYIBO1Q4NYXNl/A4cN0R0lTBY3CVwjv7XHnQEKuKXbtjwyJlBWFyooCw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B97128EFC2 for ; Sat, 09 Nov 2024 01:35:24 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t9ZRY-0002jn-ND; Fri, 08 Nov 2024 19:35:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9ZRW-0002iw-N8 for guix-patches@gnu.org; Fri, 08 Nov 2024 19:35:10 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t9ZRW-0007cw-DO; Fri, 08 Nov 2024 19:35:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:From:To:Subject; bh=8D0oHQZ7lu02GmInNeEyMWnh/ELa6111/moiCw82ftU=; b=pQcfQg8GxfYak5gX29YT2U4oCJnCiZNmqk81mu7nA6sHkN94+HzXh5RXyV8oLMY6nGjKb4cw6oWK2tACwlhhflnxgumKUiK0q6KntAqznZ3hzSjp1MB0o3U5UllOpv0PtL4MskhgG+cEQ1ynVLEMWx/9caHOgu5HXogeyhFJeVKVm2oljHCmpnLBks8RlYCXj2qwXRurnAhKRTOBfXKIP+D2nfppwo3H152oXDd8yhIHif8VGPLjia2K41epuCGzQYjKoNVk/k+FMqj23/TUkDcwm5YvQgYjAvsB2KoAjrjN/FJgwF6Xc9G7nOk5B95uD0j99yAAc16h2t0C6/a6HQ==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1t9ZRO-0003ME-4b; Fri, 08 Nov 2024 19:35:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#74273] [PATCH] Support for bcachefs-like multi-device file-systems. Resent-From: Massimo Zaniboni Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Sat, 09 Nov 2024 00:35:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 74273 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 74273@debbugs.gnu.org Cc: Massimo Zaniboni , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-To: guix-patches@gnu.org X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by submit@debbugs.gnu.org id=B.173111247412853 (code B ref -1); Sat, 09 Nov 2024 00:35:01 +0000 Received: (at submit) by debbugs.gnu.org; 9 Nov 2024 00:34:34 +0000 Received: from localhost ([127.0.0.1]:52655 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t9ZQv-0003LE-Lp for submit@debbugs.gnu.org; Fri, 08 Nov 2024 19:34:34 -0500 Received: from lists.gnu.org ([209.51.188.17]:33444) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t9ZQs-0003L5-3J for submit@debbugs.gnu.org; Fri, 08 Nov 2024 19:34:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9ZQr-0002c2-MB for guix-patches@gnu.org; Fri, 08 Nov 2024 19:34:29 -0500 Received: from mail.asterisell.com ([193.30.121.134]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9ZQn-0007OW-MY for guix-patches@gnu.org; Fri, 08 Nov 2024 19:34:28 -0500 From: Massimo Zaniboni DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokmelody.org; s=mail; t=1731112431; bh=8D0oHQZ7lu02GmInNeEyMWnh/ELa6111/moiCw82ftU=; h=From:To:Cc:Subject:Date; b=lSZt/GQ+A9UF5UeNV7OSozHA1H2dfJVRYiX3Kj1B3h+FXHxZRY7P71tgirXkm/VUi kau+nA58SqwoSnGEJDS0QJWx9isRLe3CjeoSQhBLUFzvXZeBJ4atIqwOT4PXzbFte9 yhP0dyTeniiXwZWvWu2ES1NA548sGoMc5LR7fAaQ= Date: Sat, 9 Nov 2024 01:23:43 +0100 Message-ID: <2142f04036761f24a045a176098b1d0f958ce3bf.1731111823.git.mzan@dokmelody.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=193.30.121.134; envelope-from=mzan@dokmelody.org; helo=mail.asterisell.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam: Yes X-Migadu-Spam: Yes X-Migadu-Spam-Score: 13.57 X-Spam-Score: 13.57 X-Migadu-Queue-Id: B97128EFC2 X-Migadu-Scanner: mx11.migadu.com X-TUID: CNPn1AJfsBmu Support multi-device like "/dev/sda:/dev/sdb". Change-Id: Iddd9c31f8c083a55e7a1fb193e7bbfb396e2def6 --- I'm using this patch on my system. This is the first patch that I send using Stacked Git (`stg`). I hope that the email format is correct. gnu/build/file-systems.scm | 49 ++++++++++++++++++++++++++++--------- gnu/machine/ssh.scm | 23 ++++++++++++++++- gnu/system/file-systems.scm | 15 ++++++++++++ guix/scripts/system.scm | 25 ++++++++++++++++++- 4 files changed, 98 insertions(+), 14 deletions(-) diff --git a/gnu/build/file-systems.scm b/gnu/build/file-systems.scm index 41e1c9e..7dba7e0 100644 --- a/gnu/build/file-systems.scm +++ b/gnu/build/file-systems.scm @@ -9,6 +9,7 @@ ;;; Copyright © 2022 Oleg Pykhalov ;;; Copyright © 2024 Nicolas Graves ;;; Copyright © 2024 Richard Sent +;;; Copyright © 2024 Massimo Zaniboni ;;; ;;; This file is part of GNU Guix. ;;; @@ -1138,9 +1139,9 @@ (define find-partition-by-luks-uuid (define (canonicalize-device-spec spec) - "Return the device name corresponding to SPEC, which can be a , a -, the string 'none' or another string (typically a /dev -file name or an nfs-root containing ':/')." + "Return the device name corresponding to SPEC, which can be a , +a , the string 'none' or another string like a device, +a multi-device, file name, nfs-root." (define max-trials ;; Number of times we retry partition label resolution, 1 second per ;; trial. Note: somebody reported a delay of 16 seconds (!) before their @@ -1162,20 +1163,44 @@ (define (canonicalize-device-spec spec) (sleep 1) (loop (+ 1 count)))))))) + (define (resolve-multi-device find-partition multi-device) + (let ((specs (string-split multi-device #\:))) + (let loop + ((count 0)) + (let ((nfp (find (lambda (d) (not (find-partition d))) specs))) + (if nfp + ;; Some devices take a bit of time to appear, most notably USB + ;; storage devices. Thus, wait for the device to appear. + (if (> count max-trials) + (error "failed to resolve partition" nfp) + (begin + (format #t "waiting for partition '~a' to appear...~%" nfp) + (sleep 1) + (loop (+ 1 count)))) + multi-device))))) + (match spec ((? string?) - (if (or (string-contains spec ":/") ;nfs - (and (>= (string-length spec) 2) - (equal? (string-take spec 2) "//")) ;cifs - (string=? spec "none")) - spec ; do not resolve NFS / CIFS / tmpfs devices - ;; Nothing to do, but wait until SPEC shows up. - (resolve identity spec identity))) + (cond + ((multi-device-spec? spec) + (resolve-multi-device identity spec)) + ((string-contains spec ":/") + ;NFS, something like 'server:/some/path' + spec) + ((and (>= (string-length spec) 2) + (equal? (string-take spec 2) "//")) + ;CIFS + spec) + ((string=? spec "none") + ;tmpfs + spec) + (else + ;; Nothing to do, but wait until SPEC shows up. + (resolve identity spec identity)))) ((? file-system-label?) ;; Resolve the label. (resolve find-partition-by-label - (file-system-label->string spec) - identity)) + (file-system-label->string spec) identity)) ((? uuid?) (resolve find-partition-by-uuid (uuid-bytevector spec) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm index 3e10d98..0054adf 100644 --- a/gnu/machine/ssh.scm +++ b/gnu/machine/ssh.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2019 Jakob L. Kreuze ;;; Copyright © 2020-2023 Ludovic Courtès ;;; Copyright © 2024 Ricardo +;;; Copyright © 2024 Massimo Zaniboni ;;; ;;; This file is part of GNU Guix. ;;; @@ -241,6 +242,22 @@ (define (machine-check-file-system-availability machine) (file-system-device fs) (strerror errno)))))) + (define (check-multi-device-file-system fs) + (define multi-device (file-system-device fs)) + (define devices (string-split multi-device #\:)) + (define (check-device device) + (remote-let ((errno #~(catch 'system-error + (lambda () + (stat #$device) + #t) + (lambda args + (system-error-errno args))))) + (when (number? errno) + (raise (formatted-message (G_ "device '~a' not found: ~a") + device + (strerror errno)))))) + (map check-device devices)) + (define (check-labeled-file-system fs) (define remote-exp (with-imported-modules (source-module-closure @@ -278,8 +295,12 @@ (define (machine-check-file-system-availability machine) (machine-configuration machine)) (append (map check-literal-file-system (filter (lambda (fs) - (string? (file-system-device fs))) + (single-device-spec? (file-system-device fs))) file-systems)) + (append-map check-multi-device-file-system + (filter (lambda (fs) + (multi-device-spec? (file-system-device fs))) + file-systems)) (map check-labeled-file-system (filter (lambda (fs) (file-system-label? (file-system-device fs))) diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm index 4ea8237..9f91bd7 100644 --- a/gnu/system/file-systems.scm +++ b/gnu/system/file-systems.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2020, 2021 Maxim Cournoyer ;;; Copyright © 2021 Tobias Geerinckx-Rice ;;; Copyright © 2022 Oleg Pykhalov +;;; Copyright © 2024 Massimo Zaniboni ;;; ;;; This file is part of GNU Guix. ;;; @@ -73,6 +74,9 @@ (define-module (gnu system file-systems) spec->file-system specification->file-system-mapping + multi-device-spec? + single-device-spec? + %pseudo-file-system-types %fuse-control-file-system %binary-format-file-system @@ -309,6 +313,17 @@ (define (file-system-needed-for-boot? fs) (and (file-prefix? (file-system-mount-point fs) (%store-prefix)) (not (memq 'bind-mount (file-system-flags fs)))))) +(define (multi-device-spec? spec) + "Return #t if the specification is like '/dev/sda:/dev/sdb'." + (and (string? spec) + (string-contains spec ":/") + (string-prefix? "/dev/" spec))) + +(define (single-device-spec? spec) + "Return #t if the specification is a string, but not a multi-device." + (and (string? spec) + (not (multi-device-spec? spec)))) + (define (file-system->spec fs) "Return a list corresponding to file-system FS that can be passed to the initrd code." diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 99c58f3..3459891 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -11,6 +11,7 @@ ;;; Copyright © 2021 Brice Waegeneire ;;; Copyright © 2021 Simon Tournier ;;; Copyright © 2022 Tobias Geerinckx-Rice +;;; Copyright © 2024 Massimo Zaniboni ;;; ;;; This file is part of GNU Guix. ;;; @@ -601,9 +602,16 @@ (define (check-file-system-availability file-systems) (file-system-label? (file-system-device fs))) relevant)) + (define multi-device + (filter (lambda (fs) + (and (string? (file-system-device fs)) + (multi-device-spec? (file-system-device fs)))) + relevant)) + (define literal (filter (lambda (fs) - (string? (file-system-device fs))) + (and (string? (file-system-device fs)) + (single-device-spec? (file-system-device fs)))) relevant)) (define uuid @@ -637,6 +645,21 @@ (define (check-file-system-availability file-systems) label, write @code{(file-system-label ~s)} in your @code{device} field.") device device)))))) literal) + (for-each + (lambda (fs) + (let* ((devices-str (file-system-device fs)) + (devices (string-split devices-str #\:))) + (for-each + (lambda (device) + (catch 'system-error + (lambda () (stat device)) + (lambda args + (let ((errno (system-error-errno args))) + (error (file-system-location* fs) + (G_ " #8605 device '~a' not found in multi-device '~a': ~a~%") + device devices-str (strerror errno)))))) + devices))) + multi-device) (for-each (lambda (fs) (let ((label (file-system-label->string (file-system-device fs)))) base-commit: 2a6d96425eea57dc6dd48a2bec16743046e32e06 prerequisite-patch-id: 25d78fbfbd3268c16c93cd5d222386a7f421979b prerequisite-patch-id: 30bc9aa990c70c6c1c45c951a58cf9a532b388fb prerequisite-patch-id: 0000000000000000000000000000000000000000 prerequisite-patch-id: e22870a8d4b3ab67b12e05b6242b7f1bf5ac193b -- 2.46.0