From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id GHDxIZRb0l7rbgAA0tVLHw (envelope-from ) for ; Sat, 30 May 2020 13:11:48 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id KI+3HZRb0l43RAAAbx9fmQ (envelope-from ) for ; Sat, 30 May 2020 13:11:48 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1E0509404DF for ; Sat, 30 May 2020 13:11:47 +0000 (UTC) Received: from localhost ([::1]:40670 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jf1H5-0005h1-Rz for larch@yhetil.org; Sat, 30 May 2020 09:11:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jf0x4-0007wg-In for guix-patches@gnu.org; Sat, 30 May 2020 08:51:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:45573) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jf0x4-000772-7g for guix-patches@gnu.org; Sat, 30 May 2020 08:51:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jf0x4-0000Rq-60 for guix-patches@gnu.org; Sat, 30 May 2020 08:51:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#41501] [PATCH] Add mergerfs/mergerfs-tools Resent-From: Marius Bakke Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 30 May 2020 12:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41501 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Lars-Dominik Braun , 41501@debbugs.gnu.org Received: via spool by 41501-submit@debbugs.gnu.org id=B41501.15908430551709 (code B ref 41501); Sat, 30 May 2020 12:51:02 +0000 Received: (at 41501) by debbugs.gnu.org; 30 May 2020 12:50:55 +0000 Received: from localhost ([127.0.0.1]:57119 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jf0wl-0000RJ-MG for submit@debbugs.gnu.org; Sat, 30 May 2020 08:50:54 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:58043) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jf0wi-0000R4-3G for 41501@debbugs.gnu.org; Sat, 30 May 2020 08:50:42 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id B665A5C0095; Sat, 30 May 2020 08:50:34 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Sat, 30 May 2020 08:50:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= from:to:subject:in-reply-to:references:date:message-id :mime-version:content-type; s=fm3; bh=G1ZJFdBlvTrRJHz6qL/a9g5t7h toPAXPmhoZx1/8UaE=; b=DPvFRQbFlAl14XKnqHqZyy8L97owokJkpI/CXf3nAn j3kjmfdZYq4Rkn/6P0/QJxPU+5xpQNwkM2WZ22dYm8K12HEZyTmOUwleUjuyrYc+ u3q1OThoMhsI2XAhVqlkutM7aHbggsMmQaHbvos0XZPn2CeO7vRFi5T8JdlzbCpR MSS2vlFQGwNInaY4vJipZud0tJKnNnM5NVvT56Prw6SV6oV5mqLS1t1InWiUIpIX BOexFfp4rcTEeup8iOKOX4IM9m1rj7JqF7JRg8VqjftT0Dbidq/mwdzPTUObUrBr MAYYzhFugGYBwkUSAcaw777WjOt1EN22Rx+Qhw+3ofhw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=G1ZJFd BlvTrRJHz6qL/a9g5t7htoPAXPmhoZx1/8UaE=; b=GzlGn/6hML7fOe1tCYtC2V ef5PZKcwygDZcoLEg9fuoGoMbmV3j47PhPXDa3OfqpXEp7VuW9Jpz6f7Kr3XV6GX BbpBCCHbuLfhLdF37rlJ2iNkGW5okT9Ds/U4zCfxE9fb0qe2q4eXfZtdjwI/trXI JmkfESIfb5EOLe41Zs4j3FU/yuhZmScu/VcwThK+euT3XwDHtNjgATiE7/dvhTYo iqYRiu+24gbWDRU6UrYs6SPq4k6zPgTFvIPsCyRJARSWrBU+n3+1MuTPgNQLQ43y uRdmthBTSLkPulmM3XrgYtZIsqWHmHjsdTxqCtYxR6FA6YXh3No6OsJyHQmMGnpw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudeftddgheeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufgjfhffkfggtgesghdtreertddtjeenucfhrhhomhepofgrrhhiuhhs uceurghkkhgvuceomhgsrghkkhgvsehfrghsthhmrghilhdrtghomheqnecuggftrfgrth htvghrnhepteekveduhfeifeelveetjedvledugeejuefhhedvteejvefhudeftefftdfg hffgnecuffhomhgrihhnpehgihhthhhusgdrtghomhdpfhhrvggvuggvshhkthhophdroh hrghenucfkphepkeegrddvtddvrdeikedrjeehnecuvehluhhsthgvrhfuihiivgeptden ucfrrghrrghmpehmrghilhhfrhhomhepmhgsrghkkhgvsehfrghsthhmrghilhdrtghomh X-ME-Proxy: Received: from localhost (ti0006q161-2604.bb.online.no [84.202.68.75]) by mail.messagingengine.com (Postfix) with ESMTPA id 30FF53280064; Sat, 30 May 2020 08:50:34 -0400 (EDT) From: Marius Bakke In-Reply-To: <20200524085448.GA1363@noor.fritz.box> References: <20200524085448.GA1363@noor.fritz.box> Date: Sat, 30 May 2020 14:50:32 +0200 Message-ID: <87v9kd1t9j.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-Mailman-Approved-At: Sat, 30 May 2020 09:11:35 -0400 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=fastmail.com header.s=fm3 header.b=DPvFRQbF; dkim=fail (rsa verify failed) header.d=messagingengine.com header.s=fm2 header.b=GzlGn/6h; dmarc=fail reason="SPF not aligned (relaxed)" header.from=fastmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -2.01 X-TUID: Kp7jTzJyK1ki --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Lars-Dominik Braun writes: > Hi, > > the attached patch series adds mergerfs, an union file system, and > associated tools mergerfs-tools. I=E2=80=99ve been running it for a while= now > and it seems to be stable. Unfortunately I=E2=80=99m not able to mount it= in the > system configuration via > > (file-system > (device "/storage/disk*") > (mount-point "/storage/pool") > (type "mergerfs") > (flags '('allow_other, 'use_ino, "moveonenospc=3Dtrue", "category.create= =3Dmfs")) > (check? #f)) > > because device is a pattern and not an actual file system path. Oh, fun. I suppose we'll have to add support for mergerfs in the system configuration to deal with it. > From ac0ff2afbbcc63d9b6b7b448877f54b58e975668 Mon Sep 17 00:00:00 2001 > From: Lars-Dominik Braun > Date: Sun, 24 May 2020 10:48:02 +0200 > Subject: [PATCH 2/2] gnu: Add mergerfs-tools. > > * gnu/packages/storage.scm (mergerfs-tools): New variable. I think mergerfs is better suited in file-systems.scm. Can you rebase these patches accordingly? > diff --git a/gnu/packages/storage.scm b/gnu/packages/storage.scm > index b8090c7eaa..ee5967aff6 100644 > --- a/gnu/packages/storage.scm > +++ b/gnu/packages/storage.scm > @@ -24,6 +24,8 @@ > #:use-module (guix utils) > #:use-module (guix build-system cmake) > #:use-module (guix build-system gnu) > + #:use-module (guix build-system copy) > + #:use-module (guix git-download) > #:use-module (gnu packages) > #:use-module (gnu packages admin) > #:use-module (gnu packages assembly) > @@ -46,6 +48,7 @@ > #:use-module (gnu packages pkg-config) > #:use-module (gnu packages python) > #:use-module (gnu packages python-xyz) > + #:use-module (gnu packages rsync) > #:use-module (gnu packages sphinx) > #:use-module (gnu packages tls) > #:use-module (gnu packages web) > @@ -299,3 +302,53 @@ storage protocols (S3, NFS, and others) through the = RADOS gateway.") > license:isc > ;; imported libfuse code > license:gpl2 license:lgpl2.0)))) > + > +(define-public mergerfs-tools > + (let ((commit "c926779d87458d103f3b674603bf97801ae2486d") > + (revision "1")) > + (package > + (name "mergerfs-tools") > + ;; unreleased, no version Please use full sentences in code comments, i.e. capitalizations and full stops. > + (version (git-version "0" revision commit)) The convention is to use "0.0" for situations like these, mainly because it looks funnier. > + (source > + (origin > + (method git-fetch) > + (uri (git-reference > + (url "https://github.com/trapexit/mergerfs-tools.git") > + (commit commit))) > + (file-name (git-file-name name version)) > + (sha256 > + (base32 > + "04hhwcib0xv4cf1mkj8zrp2aqpxkncml9iqg4m1mz6a5zhzsk0vm")))) > + (build-system copy-build-system) > + (inputs > + `(("python" ,python) > + ("python-xattr" ,python-xattr) > + ("rsync" ,rsync))) > + (arguments > + '(#:install-plan > + '(("src/" "bin/")) > + #:phases > + (modify-phases %standard-phases > + (add-after 'unpack 'patch-paths > + (lambda* (#:key inputs #:allow-other-keys) > + (substitute* (find-files "src" "^mergerfs\\.") > + (("'rsync'") > + (string-append "'" (assoc-ref inputs "rsync") "/bin/rs= ync'")) > + (("'rm'") > + (string-append "'" (assoc-ref inputs "coreutils") "/bi= n/rm'"))) > + (substitute* "src/mergerfs.mktrash" > + (("xattr") > + (string-append (assoc-ref inputs "python-xattr") "/bin= /xattr")) > + (("mkdir") > + (string-append (assoc-ref inputs "coreutils") "/bin/mk= dir"))) > + #t))))) > + (synopsis "Optional tools to help manage data in a mergerfs pool") I think we can drop 'optional' from here. > + (description > + "Audit permissions and ownership of files and directories, duplic= ates > + files & directories across branches in a pool, find and remove > + duplicate files, balance pool drives, consolidate files in a sin= gle > + mergerfs directory onto a single drive and create FreeDesktop.or= g Trash > + specification compatible directories.") These lines should not be indented apart from the first one. Also, the description reads somewhat unnatural to me. Taken literally, the description makes it sound like this package can do all that without special support from anything? It would be good to start along the lines of "mergerfs-tools is a suite of programs that can ..." and throw in that it needs a mergerfs to actually work. > From 529a3aa70ab1a3079f2c5ab20fb776e92e2ba1cf Mon Sep 17 00:00:00 2001 > From: Lars-Dominik Braun > Date: Sun, 24 May 2020 09:53:30 +0200 > Subject: [PATCH 1/2] gnu: Add mergerfs. > > * gnu/packages/storage.scm (mergerfs): New variable. Can you move this too to file-systems.scm? [...] > +(define-public mergerfs > + (package > + (name "mergerfs") > + (version "2.29.0") > + ;; mergerfs bundles a heavily modified copy of libfuse Full sentences please. :-) Maybe even an "XXX" in this case. > + (source > + (origin > + (method url-fetch) > + (uri (string-append "https://github.com/trapexit/mergerfs/release= s/download/" > + version "/mergerfs-" version ".tar.gz")) > + (sha256 > + (base32 > + "17gizw4vgbqqjd2ykkfpp276942jb5qclp0lkiwkmq1yjgyjqfmk")))) > + (build-system gnu-build-system) > + (arguments > + `(#:tests? #f ; no tests exist > + #:phases > + (modify-phases %standard-phases > + (delete 'configure) > + (add-after 'unpack 'fix-paths > + (lambda* (#:key inputs outputs #:allow-other-keys) > + (setenv "CC" "gcc") > + ;; These were copied from the package libfuse > + (substitute* '("libfuse/lib/mount_util.c" "libfuse/util/mou= nt_util.c") > + (("/bin/(u?)mount" _ maybe-u) > + (string-append (assoc-ref inputs "util-linux") > + "/bin/" maybe-u "mount"))) > + (substitute* '("libfuse/util/mount.mergerfs.c") > + (("/bin/sh") > + (which "sh"))) > + ;; The Makefile does not allow overriding PREFIX via make v= ariables > + (substitute* '("Makefile" "libfuse/Makefile") > + (("=3D /usr/local") (string-append "=3D " (assoc-ref outp= uts "out"))) > + ;; cannot chown as build user > + (("chown root:root") "true")) > + #t))))) > + (inputs `(("util-linux" ,util-linux))) > + (home-page "https://github.com/trapexit/mergerfs") > + (synopsis > + "Featureful union filesystem") This line break is unnecessary. > + (description > + "mergerfs is a union filesystem geared towards simplifying storage = and > + management of files across numerous commodity storage devices.= It is > + similar to mhddfs, unionfs, and aufs.") No indentation here. > + (license (list > + ;; mergerfs > + license:isc > + ;; imported libfuse code > + license:gpl2 license:lgpl2.0)))) These would do well as margin comments, i.e.: license:isc ;mergerfs license:gpl2 license:lgpl2.0 ;imported libfuse code Can you send updated patches? TIA! --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAl7SVpgACgkQoqBt8qM6 VPqsqQgAuio5SB0ion5qvAdhRWov3aowazgEcnN1PC8oDQBdki1xTfTNUP8DBksV pE0/0DXhFKgxrdZTuk6ip7LD4/Gum5jVoJPk5V4eXHJVpiSn+0fBt9nWoXm0x3hA cbrydraldsVrF+bt/cXBKPE3TnaPjvQ2CXWdiRyRTd1UQndbKJ+0JULOpl3gwUUM r+KAh8C4MCupyCyDOT5Wzwv5Yx5rV7H1D2yIx27lINeS4ZSQQQeREgefpVjpNGGJ CRBKrZAhTf5M9DZWv8aP0ePwdOWRBPsF/IbULkwGZtCtzwm8O6fI2JY2YB8SVeH6 optj6GB4HaYY9jJTXqw27reyzlC4ew== =996e -----END PGP SIGNATURE----- --=-=-=--