From: Pierre Neidhardt <mail@ambrevar.xyz>
To: 60406@debbugs.gnu.org
Subject: [bug#60406] [PATCH 2/2] gnu: Add dwarfs.
Date: Thu, 29 Dec 2022 15:55:37 +0100 [thread overview]
Message-ID: <87v8lup8d2.fsf@ambrevar.xyz> (raw)
In-Reply-To: <874jteqn67.fsf@ambrevar.xyz>
[-- Attachment #1: Type: text/plain, Size: 5927 bytes --]
* gnu/packages/file-systems.scm (dwarfs): New variable.
---
gnu/packages/file-systems.scm | 100 ++++++++++++++++++++++++++++++++++
1 file changed, 100 insertions(+)
diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm
index 3d45634169..e268c52f06 100644
--- a/gnu/packages/file-systems.scm
+++ b/gnu/packages/file-systems.scm
@@ -46,26 +46,36 @@ (define-module (gnu packages file-systems)
#:use-module (gnu packages admin)
#:use-module (gnu packages attr)
#:use-module (gnu packages autotools)
+ #:use-module (gnu packages backup)
#:use-module (gnu packages base)
#:use-module (gnu packages bash)
#:use-module (gnu packages bison)
+ #:use-module (gnu packages boost)
#:use-module (gnu packages check)
#:use-module (gnu packages compression)
+ #:use-module (gnu packages cpp)
#:use-module (gnu packages crypto)
#:use-module (gnu packages curl)
#:use-module (gnu packages cyrus-sasl)
#:use-module (gnu packages datastructures)
+ #:use-module (gnu packages digest)
#:use-module (gnu packages documentation)
#:use-module (gnu packages docbook)
+ #:use-module (gnu packages elf)
#:use-module (gnu packages flex)
#:use-module (gnu packages gawk)
#:use-module (gnu packages glib)
#:use-module (gnu packages gnupg)
#:use-module (gnu packages golang)
#:use-module (gnu packages guile)
+ #:use-module (gnu packages jemalloc)
#:use-module (gnu packages kerberos)
+ #:use-module (gnu packages libevent)
#:use-module (gnu packages libffi)
+ #:use-module (gnu packages libunwind)
#:use-module (gnu packages linux)
+ #:use-module (gnu packages maths)
+ #:use-module (gnu packages man)
#:use-module (gnu packages nfs)
#:use-module (gnu packages onc-rpc)
#:use-module (gnu packages openldap)
@@ -73,6 +83,8 @@ (define-module (gnu packages file-systems)
#:use-module (gnu packages perl)
#:use-module (gnu packages photo)
#:use-module (gnu packages pkg-config)
+ #:use-module (gnu packages popt)
+ #:use-module (gnu packages pretty-print)
#:use-module (gnu packages python)
#:use-module (gnu packages python-crypto)
#:use-module (gnu packages python-web)
@@ -1790,3 +1802,91 @@ (define-public fuse-overlayfs
(description "This package provides an implementation of overlay+shiftfs
in FUSE for rootless containers.")
(license license:gpl3)))
+
+(define-public dwarfs
+ (package
+ (name "dwarfs")
+ (version "0.6.2")
+ (source (origin
+ (method url-fetch)
+ ;; The release archive is needed so that version.h is included.
+ (uri (string-append "https://github.com/mhx/dwarfs/releases/download/v"
+ version "/dwarfs-" version ".tar.xz"))
+ (sha256
+ (base32
+ "1kncxf85gsj3anck8ccjmxn2azp5ifqbgkiky2kharmvphkbmfcv"))
+ (snippet
+ #~(begin
+ (use-modules (guix build utils))
+ ;; Prefer system libraries instead of submodules.
+ ;; TODO: Package fbthrift.
+ ;; TODO: Can we use Guix own folly? There is no CMake option for it.
+ ;; TODO: Package parallel-hashmap.
+ (for-each delete-file-recursively
+ '(;; "fbthrift"
+ ;; "folly"
+ ;; "parallel-hashmap"
+ "xxHash"
+ "zstd"))))))
+ (build-system cmake-build-system)
+ (arguments
+ '(#:tests? #f ; TODO: 1 test fails because 'modprobe fuse' needs privileged access.
+ #:configure-flags
+ (list "-DPREFER_SYSTEM_ZSTD=ON"
+ "-DPREFER_SYSTEM_XXHASH=ON"
+ "-DPREFER_SYSTEM_GTEST=ON"
+ "-DWITH_TESTS=ON"
+ ;; Disable man pages since ronn fails to run without hpricot.
+ "-DWITH_MAN_PAGES=OFF")))
+ (native-inputs
+ (list
+ ;; FIXME: Building with ronn fail because hpricot is missing.
+ ;; ronn
+ folly googletest libdwarf libevent pkg-config))
+ (inputs
+ (list
+ boost
+ double-conversion
+ fmt
+ fuse-3
+ gflags
+ jemalloc
+ libarchive
+ libunwind
+ lz4
+ openssl
+ xxhash
+ xz
+ `(,zstd "lib")))
+ (home-page "https://github.com/mhx/dwarfs")
+ (synopsis "Fast high compression read-only file system")
+ (description "DwarFS is a read-only file system with a focus on achieving
+very high compression ratios in particular for very redundant data.
+
+DwarFS also doesn't compromise on speed and for some cases it is on par with
+or performs better than SquashFS. For the primary use case, DwarFS
+compression is an order of magnitude better than SquashFS compression, it's 6
+times faster to build the file system, it's typically faster to access files
+on DwarFS and it uses less CPU resources.
+
+Distinct features of DwarFS are:
+
+@itemize
+
+@item Clustering of files by similarity using a similarity hash function. This
+makes it easier to exploit the redundancy across file boundaries.
+
+@item Segmentation analysis across file system blocks in order to reduce the
+size of the uncompressed file system. This saves memory when using the
+compressed file system and thus potentially allows for higher cache hit rates
+as more data can be kept in the cache.
+
+@item Highly multi-threaded implementation. Both the file system creation tool
+as well as the FUSE driver are able to make good use of the many cores of your
+system.
+
+@item Optional experimental Python scripting support to provide custom
+filtering and ordering functionality.
+
+@end itemize\n")
+ (license license:gpl3)))
--
2.38.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]
next prev parent reply other threads:[~2022-12-29 14:56 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-29 14:50 [bug#60406] [PATCH 0/2] Add DwarFS Pierre Neidhardt
2022-12-29 14:55 ` [bug#60406] [PATCH 1/2] gnu: Add libdwarf Pierre Neidhardt
2022-12-29 14:55 ` Pierre Neidhardt [this message]
2023-01-12 13:58 ` [bug#60406] [PATCH 0/2] Add DwarFS Simon Tournier
2023-01-13 12:09 ` Pierre Neidhardt
2023-01-23 13:20 ` Pierre Neidhardt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87v8lup8d2.fsf@ambrevar.xyz \
--to=mail@ambrevar.xyz \
--cc=60406@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).