[bug#60250] [PATCH] gnu: Add bees.

[bug#60250] [PATCH] gnu: Add bees.

[Hilton Chain via Guix-patches via]

* gnu/packages/file-systems.scm (bees): New variable.
---
 gnu/packages/file-systems.scm | 39 +++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm
index 57a25a0d90..cce406a01d 100644
--- a/gnu/packages/file-systems.scm
+++ b/gnu/packages/file-systems.scm
@@ -1766,3 +1766,42 @@ (define-public udftools
 and other optical media.  It supports read-only media (DVD/CD-R)
 and rewritable media that wears out (DVD/CD-RW).")
     (license license:gpl2+)))
+
+(define-public bees
+  (package
+    (name "bees")
+    (version "0.8")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/Zygo/bees")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "1kxpz1p9k5ir385kpvmfjawki5vg22hlx768k7835w6n5z5a65y4"))))
+    (build-system gnu-build-system)
+    (arguments
+     (list #:test-target "test"
+           #:make-flags
+           #~(list (string-append "CC=" #$(cc-for-target))
+                   (string-append "DESTDIR=" #$output)
+                   (string-append "BEES_VERSION=" #$version)
+                   "PREFIX=''"
+                   "LIBEXEC_PREFIX=/lib/bees")
+           #:phases
+           #~(modify-phases %standard-phases
+               (delete 'configure)
+               (add-after 'install 'fix-path
+                 (lambda _
+                   (substitute* (string-append #$output "/sbin/beesd")
+                     (("/lib/bees/bees" all)
+                      (string-append #$output all))))))))
+    (home-page "https://github.com/Zygo/bees")
+    (synopsis "Best-Effort Extent-Same, a btrfs dedupe agent")
+    (description
+     "@code{bees} is a block-oriented userspace deduplication agent designed
+for large btrfs filesystems.  It is an offline dedupe combined with an
+incremental data scan capability to minimize time data spends on disk from
+write to dedupe.")
+    (license license:gpl3+)))

base-commit: 0744540d09ddef8dbf25cc5d65da9d029dab338c
-- 
2.38.1

[bug#60250] [PATCH] gnu: Add bees.

[Adam Faiz via Guix-patches via]

Hi,

> #~(list (string-append "CC=" #$(cc-for-target))
> +                   (string-append "DESTDIR=" #$output)
> +                   (string-append "BEES_VERSION=" #$version)
> +                   "PREFIX=''"
> +                   "LIBEXEC_PREFIX=/lib/bees"
The LIBEXEC_PREFIX shouldn't be set to this, this is actually a bug with 
beesd.in and also the sed invocation(or $TEMPLATE_COMPILER which is what 
it's called in the Makefile).

> #~(modify-phases %standard-phases
> +               (delete 'configure)
> +               (add-after 'install 'fix-path
> +                 (lambda _
> +                   (substitute* (string-append #$output "/sbin/beesd")
> +                     (("/lib/bees/bees" all)
> +                      (string-append #$output all))))))))
The 'fix-path phase isn't necessary, and the LIBEXEC_PREFIX doesn't need 
to be set. The snippet below fixes the bug in bees:

>               (modules '((guix build utils)))
>               (snippet
>                #~(begin
>                    (substitute* "Defines.mk"
>                      (("^sed.*" all)
>                       (string-append all
>                                      "\t\t-e's#@DESTDIR@#$(DESTDIR)#' \\\n")))
>                    (substitute* "scripts/beesd.in"
>                      (("@LIBEXEC_PREFIX@") "@DESTDIR@/@LIBEXEC_PREFIX@"))))))

The files lib/city.cc and include/crucible/city.h need to be unbundled 
and the cityhash included in guix used instead. The cityhash package 
needs to be in the inputs, and these files need to be deleted as part of 
the snippet above:>                    (for-each delete-file
>                              '("lib/city.cc" "include/crucible/city.h"))

It appears that bees hasn't been built on a non-systemd distribution, 
because an optional feature hard fails. This fixes it, the systemd unit 
files are installed conditionally anyways:
>                    (substitute* "Makefile"
>                      (("pkg-config systemd --variable=systemdsystemunitdir" all)
>                       (string-append all " || true")))

>                    (substitute* "lib/Makefile"
>                      (("city.o.*") ""))
>                    (substitute* "src/bees-hash.cc"
>                      (("#include .crucible/city.h.") "#include <city.h>"))
These substitutions remove references to the bundled cityhash(which is 
11 years old!)

Native inputs and inputs for bees:
>     (native-inputs
>      (list pkg-config markdown))
>     (inputs
>      (list cityhash))

All the modifications made as part of the snippet should definitely be 
upstreamed, could you do that for me?

[bug#60250] [PATCH] gnu: Add bees.

[Adam Faiz via Guix-patches via]

 From the docs/install.md, it looks like btrfs-progs and 
util-linux(needed for the blkid program) are used at runtime. References 
to those programs in the source should probably just be substituted in a 
phase.

[bug#60250] [PATCH] gnu: Add bees.

[Hilton Chain via Guix-patches via]

Hi Adam,

On Fri, 23 Dec 2022 02:52:38 +0800,
Adam Faiz wrote:
>
> Hi,
>
> > #~(list (string-append "CC=" #$(cc-for-target))
> > +                   (string-append "DESTDIR=" #$output)
> > +                   (string-append "BEES_VERSION=" #$version)
> > +                   "PREFIX=''"
> > +                   "LIBEXEC_PREFIX=/lib/bees"
> The LIBEXEC_PREFIX shouldn't be set to this, this is actually a bug with beesd.in and also the sed
> invocation(or $TEMPLATE_COMPILER which is what it's called in the Makefile).
>
> > #~(modify-phases %standard-phases
> > +               (delete 'configure)
> > +               (add-after 'install 'fix-path
> > +                 (lambda _
> > +                   (substitute* (string-append #$output "/sbin/beesd")
> > +                     (("/lib/bees/bees" all)
> > +                      (string-append #$output all))))))))
> The 'fix-path phase isn't necessary, and the LIBEXEC_PREFIX doesn't need to be set. The snippet
> below fixes the bug in bees:
>
> >               (modules '((guix build utils)))
> >               (snippet
> >                #~(begin
> >                    (substitute* "Defines.mk"
> >                      (("^sed.*" all)
> >                       (string-append all
> >                                      "\t\t-e's#@DESTDIR@#$(DESTDIR)#' \\\n")))
> >                    (substitute* "scripts/beesd.in"
> >                      (("@LIBEXEC_PREFIX@") "@DESTDIR@/@LIBEXEC_PREFIX@"))))))
>
> The files lib/city.cc and include/crucible/city.h need to be unbundled and the cityhash included in
> guix used instead. The cityhash package needs to be in the inputs, and these files need to be
> deleted as part of the snippet above:>                    (for-each delete-file
> >                              '("lib/city.cc" "include/crucible/city.h"))
>
> It appears that bees hasn't been built on a non-systemd distribution, because an optional feature
> hard fails. This fixes it, the systemd unit files are installed conditionally anyways:
> >                    (substitute* "Makefile"
> >                      (("pkg-config systemd --variable=systemdsystemunitdir" all)
> >                       (string-append all " || true")))
>
> >                    (substitute* "lib/Makefile"
> >                      (("city.o.*") ""))
> >                    (substitute* "src/bees-hash.cc"
> >                      (("#include .crucible/city.h.") "#include <city.h>"))
> These substitutions remove references to the bundled cityhash(which is 11 years old!)
>
> Native inputs and inputs for bees:
> >     (native-inputs
> >      (list pkg-config markdown))
> >     (inputs
> >      (list cityhash))
>
> All the modifications made as part of the snippet should definitely be upstreamed, could you do that
> for me?

Thank you for the snippet and explanation!

I've made a pull request (of the beesd part) to the upstream:
<https://github.com/Zygo/bees/pull/241>

(And it's merged before I send the mail XDDD)

However I'm not sure about the systemd part, could you explain more about which optional feature
hard fails?  And I get same binaries whether-ever this part is used.

As pkg-config is only used for systemd, and markdown for documentations that won't be installed,
I'd rather remove them from native-inputs, WDYT?

[bug#60250] [PATCH v2] gnu: Add bees.

[Hilton Chain via Guix-patches via]

* gnu/packages/patches/bees-beesd-honor-destdir-on-installation.patch: New
file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/file-systems.scm (bees): New variable.
---
v1 -> v2:
1. Unbundle cityhash.
2. Patch beesd.in for DESTDIR handling.
3. Modify beesd.in to use absolute paths of commands.

 gnu/local.mk                                  |  1 +
 gnu/packages/file-systems.scm                 | 73 +++++++++++++++++++
 ...-beesd-honor-destdir-on-installation.patch | 40 ++++++++++
 3 files changed, 114 insertions(+)
 create mode 100644 gnu/packages/patches/bees-beesd-honor-destdir-on-installation.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 45d05de02d..a733d296bd 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -935,6 +935,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/bsd-games-prevent-name-collisions.patch	\
   %D%/packages/patches/bsd-games-stdio.h.patch			\
   %D%/packages/patches/beancount-disable-googleapis-fonts.patch	\
+  %D%/packages/patches/bees-beesd-honor-destdir-on-installation.patch	\
   %D%/packages/patches/beignet-correct-file-names.patch		\
   %D%/packages/patches/bidiv-update-fribidi.patch		\
   %D%/packages/patches/binutils-2.37-file-descriptor-leak.patch	\
diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm
index 57a25a0d90..b7c681d1fb 100644
--- a/gnu/packages/file-systems.scm
+++ b/gnu/packages/file-systems.scm
@@ -81,6 +81,7 @@ (define-module (gnu packages file-systems)
   #:use-module (gnu packages rsync)
   #:use-module (gnu packages sssd)
   #:use-module (gnu packages sqlite)
+  #:use-module (gnu packages textutils)
   #:use-module (gnu packages time)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages valgrind)
@@ -1766,3 +1767,75 @@ (define-public udftools
 and other optical media.  It supports read-only media (DVD/CD-R)
 and rewritable media that wears out (DVD/CD-RW).")
     (license license:gpl2+)))
+
+(define-public bees
+  (package
+    (name "bees")
+    (version "0.8")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/Zygo/bees")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
+              (modules '((guix build utils)))
+              (snippet
+               ;; Unbundle cityhash.
+               #~(begin
+                   (for-each delete-file
+                             '("lib/city.cc" "include/crucible/city.h"))
+                   (substitute* "lib/Makefile"
+                     (("city.o.*") ""))
+                   (substitute* "src/bees-hash.cc"
+                     (("#include .crucible/city.h.") "#include <city.h>"))))
+              (patches
+               (search-patches
+                ;; XXX: Cherry-picked from upstream, remove the patch when
+                ;; bumping version.
+                "bees-beesd-honor-destdir-on-installation.patch"))
+              (sha256
+               (base32
+                "1kxpz1p9k5ir385kpvmfjawki5vg22hlx768k7835w6n5z5a65y4"))))
+    (build-system gnu-build-system)
+    (arguments
+     (list #:test-target "test"
+           #:make-flags
+           #~(list (string-append "CC=" #$(cc-for-target))
+                   (string-append "DESTDIR=" #$output)
+                   (string-append "BEES_VERSION=" #$version)
+                   "PREFIX=''")
+           #:phases
+           #~(modify-phases %standard-phases
+               (delete 'configure)
+               (add-after 'unpack 'fixpath
+                 (lambda* (#:key inputs #:allow-other-keys)
+                   (substitute* "scripts/beesd.in"
+                     (((string-append "\\<(" (string-join (list "realpath"
+                                                                "uuidparse"
+                                                                "grep"
+                                                                "false"
+                                                                "sed"
+                                                                "true"
+                                                                "head"
+                                                                "mkdir"
+                                                                "mount"
+                                                                "touch"
+                                                                "du"
+                                                                "cut"
+                                                                "rm"
+                                                                "truncate"
+                                                                "chmod")
+                                                          "|") ")\\>") command)
+                      (search-input-file inputs (string-append "/bin/" command)))
+
+                     (("btrfs sub")
+                      (string-append (search-input-file inputs "/bin/btrfs") " sub"))))))))
+    (inputs (list btrfs-progs cityhash util-linux))
+    (home-page "https://github.com/Zygo/bees")
+    (synopsis "Best-Effort Extent-Same, a btrfs dedupe agent")
+    (description
+     "@code{bees} is a block-oriented userspace deduplication agent designed
+for large btrfs filesystems.  It is an offline dedupe combined with an
+incremental data scan capability to minimize time data spends on disk from
+write to dedupe.")
+    (license license:gpl3+)))
diff --git a/gnu/packages/patches/bees-beesd-honor-destdir-on-installation.patch b/gnu/packages/patches/bees-beesd-honor-destdir-on-installation.patch
new file mode 100644
index 0000000000..4800bfc618
--- /dev/null
+++ b/gnu/packages/patches/bees-beesd-honor-destdir-on-installation.patch
@@ -0,0 +1,40 @@
+From 66b00f8a972ebb4da68f7aa0d0656f43ce2a2c3a Mon Sep 17 00:00:00 2001
+From: Hilton Chain <hako@ultrarare.space>
+Date: Fri, 23 Dec 2022 11:04:46 +0800
+Subject: [PATCH] beesd: Honor DESTDIR on installation.
+
+Co-authored-by: Adam Faiz <adam.faiz@disroot.org>
+Signed-off-by: Hilton Chain <hako@ultrarare.space>
+---
+ Defines.mk       | 1 +
+ scripts/beesd.in | 2 +-
+ 2 files changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/Defines.mk b/Defines.mk
+index 9e8df40..e5394ba 100644
+--- a/Defines.mk
++++ b/Defines.mk
+@@ -2,6 +2,7 @@ MAKE += PREFIX=$(PREFIX) LIBEXEC_PREFIX=$(LIBEXEC_PREFIX) ETC_PREFIX=$(ETC_PREFI
+
+ define TEMPLATE_COMPILER =
+ sed $< >$@ \
++		-e's#@DESTDIR@#$(DESTDIR)#' \
+ 		-e's#@PREFIX@#$(PREFIX)#' \
+ 		-e's#@ETC_PREFIX@#$(ETC_PREFIX)#' \
+ 		-e's#@LIBEXEC_PREFIX@#$(LIBEXEC_PREFIX)#'
+diff --git a/scripts/beesd.in b/scripts/beesd.in
+index 174bb6c..35d04aa 100755
+--- a/scripts/beesd.in
++++ b/scripts/beesd.in
+@@ -15,7 +15,7 @@ readonly AL128K="$((128*1024))"
+ readonly AL16M="$((16*1024*1024))"
+ readonly CONFIG_DIR=@ETC_PREFIX@/bees/
+
+-readonly bees_bin=$(realpath @LIBEXEC_PREFIX@/bees)
++readonly bees_bin=$(realpath @DESTDIR@/@LIBEXEC_PREFIX@/bees)
+
+ command -v "$bees_bin" &> /dev/null || ERRO "Missing 'bees' agent"
+
+--
+2.38.1
+

base-commit: 1b29fccff2334fb5d6e979a290233a452969e39b
--
2.38.1

[bug#60250] [PATCH] gnu: Add bees.

[Adam Faiz via Guix-patches via]

On 12/23/22 14:25, Hilton Chain wrote:
> Hi Adam,
> 
> On Fri, 23 Dec 2022 02:52:38 +0800,
> Adam Faiz wrote:
>>
>> Hi,
>>
>> It appears that bees hasn't been built on a non-systemd distribution, because an optional feature
>> hard fails. This fixes it, the systemd unit files are installed conditionally anyways:
>>>                     (substitute* "Makefile"
>>>                       (("pkg-config systemd --variable=systemdsystemunitdir" all)
>>>                        (string-append all " || true")))
> Thank you for the snippet and explanation!
> 
> I've made a pull request (of the beesd part) to the upstream:
> <https://github.com/Zygo/bees/pull/241>
> 
> (And it's merged before I send the mail XDDD)
> 
> However I'm not sure about the systemd part, could you explain more about which optional feature
> hard fails?  And I get same binaries whether-ever this part is used.
I had a problem earlier, but I can't reproduce it now. You can remove 
this one.

> As pkg-config is only used for systemd, and markdown for documentations that won't be installed,
> I'd rather remove them from native-inputs, WDYT?
I agree that it should be done, but it's a shame that there's no 
install_docs rule in the Makefile. That should be a feature request/patch.

I already have a bees-remove-bundled-cityhash.patch and 
bees-add-cityhash-to-install.md.patch to send upstream. Should I email 
them to you?

[bug#60250] [PATCH] gnu: Add bees.

[Hilton Chain via Guix-patches via]

On Fri, 23 Dec 2022 15:57:53 +0800,
Adam Faiz wrote:
> > As pkg-config is only used for systemd, and markdown for documentations that won't be installed,
> > I'd rather remove them from native-inputs, WDYT?
> I agree that it should be done, but it's a shame that there's no install_docs rule in the
> Makefile. That should be a feature request/patch.
>
> I already have a bees-remove-bundled-cityhash.patch and bees-add-cityhash-to-install.md.patch to
> send upstream. Should I email them to you?

As v2 has unbundled cityhash, there's no need to send the patches to me.

One thing to mention is that, according to Repology[1], cityhash is not widely packaged, as a
result, it might be better to implement the unbundle as an option.

[1]: <https://repology.org/project/cityhash/versions>

[bug#60250] [PATCH] gnu: Add bees.

[Adam Faiz via Guix-patches via]

On 12/24/22 19:34, Hilton Chain wrote:
> On Fri, 23 Dec 2022 15:57:53 +0800,
> Adam Faiz wrote:
>>> As pkg-config is only used for systemd, and markdown for documentations that won't be installed,
>>> I'd rather remove them from native-inputs, WDYT?
>> I agree that it should be done, but it's a shame that there's no install_docs rule in the
>> Makefile. That should be a feature request/patch.
>>
>> I already have a bees-remove-bundled-cityhash.patch and bees-add-cityhash-to-install.md.patch to
>> send upstream. Should I email them to you?
> 
> As v2 has unbundled cityhash, there's no need to send the patches to me.
> 
> One thing to mention is that, according to Repology[1], cityhash is not widely packaged, as a
> result, it might be better to implement the unbundle as an option.
> 
> [1]: <https://repology.org/project/cityhash/versions>
I don't think that should be done. It's all the more reason these 
distributions need cityhash packaged and people to help them.

If they need a reference, they can always look at the cityhash package 
in guix or other distributions and how they do it.

bug#60250: [PATCH] gnu: Add bees.

[Ludovic Courtès]

Hi,

Hilton Chain <hako@ultrarare.space> skribis:

> * gnu/packages/patches/bees-beesd-honor-destdir-on-installation.patch: New
> file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/file-systems.scm (bees): New variable.
> ---
> v1 -> v2:
> 1. Unbundle cityhash.
> 2. Patch beesd.in for DESTDIR handling.
> 3. Modify beesd.in to use absolute paths of commands.

Applied, thanks!

Ludo’.