From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id gOfUEwRO015FEwAA0tVLHw (envelope-from ) for ; Sun, 31 May 2020 06:26:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id yKmhDwRO014eCAAAbx9fmQ (envelope-from ) for ; Sun, 31 May 2020 06:26:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A34A894051F for ; Sun, 31 May 2020 06:26:11 +0000 (UTC) Received: from localhost ([::1]:56108 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfHQ8-0006yT-T6 for larch@yhetil.org; Sun, 31 May 2020 02:26:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33692) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfHQ2-0006y9-2G for guix-patches@gnu.org; Sun, 31 May 2020 02:26:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:48075) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jfHQ1-0001T9-On for guix-patches@gnu.org; Sun, 31 May 2020 02:26:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jfHQ1-0007Z8-Ks for guix-patches@gnu.org; Sun, 31 May 2020 02:26:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#41501] [PATCH] Add mergerfs/mergerfs-tools Resent-From: Lars-Dominik Braun Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 31 May 2020 06:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41501 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Marius Bakke Cc: 41501@debbugs.gnu.org Received: via spool by 41501-submit@debbugs.gnu.org id=B41501.159090634329057 (code B ref 41501); Sun, 31 May 2020 06:26:01 +0000 Received: (at 41501) by debbugs.gnu.org; 31 May 2020 06:25:43 +0000 Received: from localhost ([127.0.0.1]:59621 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfHPi-0007Ya-HE for submit@debbugs.gnu.org; Sun, 31 May 2020 02:25:42 -0400 Received: from mout-u-204.mailbox.org ([91.198.250.253]:13250) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jfHPf-0007YK-K5 for 41501@debbugs.gnu.org; Sun, 31 May 2020 02:25:41 -0400 Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:105:465:1:2:0]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by mout-u-204.mailbox.org (Postfix) with ESMTPS id 49ZSwY2fylzQlG9; Sun, 31 May 2020 08:25:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6xq.net; s=MBO0001; t=1590906331; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7EXlCNCiUcKjTfBsRSx9en/EzVOTuRZ+B2xZsUQ7OW8=; b=D3yxT5nNXGiB9dc9tY14SJCypQXe8E9snLHAMz8ZQHpEUeRvLQ8aIg93FnwUrnudYCc3/9 pkKeGlWVMrABoA0nJqTia3pFB69lmyAIBMdApi0ewjZJVA71/kTFX+orMayDNotP7Gs6Vu X30EPM+vCo8/eo9VMSpMo+cw7uLcdS9wrmEJ3TG6Zc/BNQt99te8I7Y29NQ4BGJLUy0alI 9FbP2r//8KCMb3GzNhfBEHN8f5zivibF0pqHBWfjNog1FGK9zMxPav9gpTxqJ1Ui6PPG8l KptA3M9C7Wkw/JkmQ9OO56aPsvM3lLrhv2Oxt4lJp0u+XVjfYcHq9jXLRfzu0A== Received: from smtp2.mailbox.org ([80.241.60.241]) by spamfilter03.heinlein-hosting.de (spamfilter03.heinlein-hosting.de [80.241.56.117]) (amavisd-new, port 10030) with ESMTP id DOsIlmMrSYLb; Sun, 31 May 2020 08:25:29 +0200 (CEST) Date: Sun, 31 May 2020 08:25:27 +0200 From: Lars-Dominik Braun Message-ID: <20200530171259.GB186839@noor.fritz.box> References: <20200524085448.GA1363@noor.fritz.box> <87v9kd1t9j.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="0F1p//8PRICkK4MW" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <87v9kd1t9j.fsf@gnu.org> X-Rspamd-Queue-Id: CD7EB175B X-Rspamd-Score: -5.11 / 15.00 / 15.00 X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.7 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=6xq.net header.s=MBO0001 header.b=D3yxT5nN; dmarc=fail reason="SPF not aligned (relaxed)" header.from=6xq.net (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: 625U3wXsMQDv --0F1p//8PRICkK4MW Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Hi, > > (file-system > > (device "/storage/disk*") > > (mount-point "/storage/pool") > > (type "mergerfs") > > (flags '('allow_other, 'use_ino, "moveonenospc=true", "category.create=mfs")) > > (check? #f)) > Oh, fun. I suppose we'll have to add support for mergerfs in the system > configuration to deal with it. maybe a generic (device (literal "/storage/disk*")) would do? I’m sure there are more FUSE filesystems out there which trigger these checks. And I don’t think adding an exception like the one for NFS is a good solution long-term. > Can you send updated patches? TIA! All done, see attachment. I hope I did not miss anything. Cheers, Lars --0F1p//8PRICkK4MW Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-gnu-Add-mergerfs-tools.patch" >From 7aa69a86f8933c3d833ae3beb53840ded9115978 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Sat, 30 May 2020 19:10:55 +0200 Subject: [PATCH 2/2] gnu: Add mergerfs-tools. * gnu/packages/storage.scm (mergerfs-tools): New variable. --- gnu/packages/file-systems.scm | 53 +++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm index 6d605c4922..1b0473d6ec 100644 --- a/gnu/packages/file-systems.scm +++ b/gnu/packages/file-systems.scm @@ -27,6 +27,7 @@ #:use-module (guix download) #:use-module (guix git-download) #:use-module (guix build-system cmake) + #:use-module (guix build-system copy) #:use-module (guix build-system gnu) #:use-module (guix build-system linux-module) #:use-module (guix build-system trivial) @@ -53,7 +54,9 @@ #:use-module (gnu packages photo) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) + #:use-module (gnu packages python-xyz) #:use-module (gnu packages readline) + #:use-module (gnu packages rsync) #:use-module (gnu packages sqlite) #:use-module (gnu packages tls) #:use-module (gnu packages valgrind) @@ -693,3 +696,53 @@ is similar to mhddfs, unionfs, and aufs.") license:isc ; mergerfs license:gpl2 license:lgpl2.0 ; Imported libfuse code. )))) + +(define-public mergerfs-tools + (let ((commit "c926779d87458d103f3b674603bf97801ae2486d") + (revision "1")) + (package + (name "mergerfs-tools") + ;; No released version exists. + (version (git-version "0.0" revision commit)) + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/trapexit/mergerfs-tools.git") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "04hhwcib0xv4cf1mkj8zrp2aqpxkncml9iqg4m1mz6a5zhzsk0vm")))) + (build-system copy-build-system) + (inputs + `(("python" ,python) + ("python-xattr" ,python-xattr) + ("rsync" ,rsync))) + (arguments + '(#:install-plan + '(("src/" "bin/")) + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'patch-paths + (lambda* (#:key inputs #:allow-other-keys) + (substitute* (find-files "src" "^mergerfs\\.") + (("'rsync'") + (string-append "'" (assoc-ref inputs "rsync") "/bin/rsync'")) + (("'rm'") + (string-append "'" (assoc-ref inputs "coreutils") "/bin/rm'"))) + (substitute* "src/mergerfs.mktrash" + (("xattr") + (string-append (assoc-ref inputs "python-xattr") "/bin/xattr")) + (("mkdir") + (string-append (assoc-ref inputs "coreutils") "/bin/mkdir"))) + #t))))) + (synopsis "Tools to help manage data in a mergerfs pool") + (description "mergerfs-tools is a suite of programs that can audit +permissions and ownership of files and directories on a mergerfs volume, +duplicates files and directories across branches in its pool, find and remove +duplicate files, balance pool drives, consolidate files in a single mergerfs +directory onto a single drive and create FreeDesktop.org Trash specification +compatible directories.") + (home-page "https://github.com/trapexit/mergerfs-tools") + (license license:isc)))) -- 2.26.2 --0F1p//8PRICkK4MW Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-gnu-Add-mergerfs.patch" >From 4b2500e04f956df0c038ba4b71d91f01b2919d1e Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Sat, 30 May 2020 19:10:30 +0200 Subject: [PATCH 1/2] gnu: Add mergerfs. * gnu/packages/storage.scm (mergerfs): New variable. --- gnu/packages/file-systems.scm | 47 +++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm index 293322c76b..6d605c4922 100644 --- a/gnu/packages/file-systems.scm +++ b/gnu/packages/file-systems.scm @@ -646,3 +646,50 @@ APFS.") originally developed for Solaris and is now maintained by the OpenZFS community.") (license license:cddl1.0))) + +(define-public mergerfs + (package + (name "mergerfs") + (version "2.29.0") + (source + (origin + (method url-fetch) + (uri (string-append "https://github.com/trapexit/mergerfs/releases/download/" + version "/mergerfs-" version ".tar.gz")) + (sha256 + (base32 + "17gizw4vgbqqjd2ykkfpp276942jb5qclp0lkiwkmq1yjgyjqfmk")))) + (build-system gnu-build-system) + (arguments + `(#:tests? #f ; No tests exist. + #:phases + (modify-phases %standard-phases + (delete 'configure) + (add-after 'unpack 'fix-paths + (lambda* (#:key inputs outputs #:allow-other-keys) + (setenv "CC" "gcc") + ;; These were copied from the package libfuse. + (substitute* '("libfuse/lib/mount_util.c" "libfuse/util/mount_util.c") + (("/bin/(u?)mount" _ maybe-u) + (string-append (assoc-ref inputs "util-linux") + "/bin/" maybe-u "mount"))) + (substitute* '("libfuse/util/mount.mergerfs.c") + (("/bin/sh") + (which "sh"))) + ;; The Makefile does not allow overriding PREFIX via make variables. + (substitute* '("Makefile" "libfuse/Makefile") + (("= /usr/local") (string-append "= " (assoc-ref outputs "out"))) + ;; cannot chown as build user + (("chown root:root") "true")) + #t))))) + ;; mergerfs bundles a heavily modified copy of libfuse. + (inputs `(("util-linux" ,util-linux))) + (home-page "https://github.com/trapexit/mergerfs") + (synopsis "Featureful union filesystem") + (description "mergerfs is a union filesystem geared towards simplifying +storage and management of files across numerous commodity storage devices. It +is similar to mhddfs, unionfs, and aufs.") + (license (list + license:isc ; mergerfs + license:gpl2 license:lgpl2.0 ; Imported libfuse code. + )))) -- 2.26.2 --0F1p//8PRICkK4MW--