[bug#29714] Lzip variations

[bug#29714] Lzip variations

[Tobias Geerinckx-Rice]

Guix,

This adds some more lzip implementations (“imps”) by the same author.

Kind regards,

T G-R

[bug#29714] [PATCH 1/4] gnu: Add lunzip.

[Tobias Geerinckx-Rice]

* gnu/packages/compression.scm (lunzip): New public variable.
---
 gnu/packages/compression.scm | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 6b8b28b9f..6bd0e257e 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -1937,3 +1937,30 @@ type by using either Perl modules, or command-line tools on your system.")
     (description "Tukaani-xz is an implementation of xz compression/decompression
 algorithms in Java.")
     (license license:public-domain)))
+
+(define-public lunzip
+  (package
+    (name "lunzip")
+    (version "1.9")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "http://download.savannah.gnu.org/releases/lzip/"
+                           name "/" name "-" version ".tar.gz"))
+       (sha256
+        (base32 "1ax3d9cp66z1qb9q7lfzg5bpx9630xrxgq9a5sw569wm0qqgpg2q"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:configure-flags
+       (list "CC=gcc")))
+    (home-page "http://www.nongnu.org/lzip/lunzip.html")
+    (synopsis "Small, stand-alone lzip decompressor")
+    (description
+     "Lunzip is a decompressor for files in the lzip compression format (.lz),
+written as a single small C tool with no dependencies.  This makes it
+well-suited to embedded and other systems without a C++ compiler, or for use in
+applications such as software installers that need only to decompress files,
+not compress them.
+Lunzip is intended to be fully compatible with the regular lzip package.")
+    (license (list license:bsd-2        ; carg_parser.[ch]
+                   license:gpl2+))))    ; everything else
-- 
2.15.0

[bug#29714] [PATCH 2/4] gnu: Add clzip.

[Tobias Geerinckx-Rice]

* gnu/packages/compression.scm (clzip): New public variable.
---
 gnu/packages/compression.scm | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 6bd0e257e..f4c90c25c 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -1964,3 +1964,28 @@ not compress them.
 Lunzip is intended to be fully compatible with the regular lzip package.")
     (license (list license:bsd-2        ; carg_parser.[ch]
                    license:gpl2+))))    ; everything else
+
+(define-public clzip
+  (package
+    (name "clzip")
+    (version "1.9")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "http://download.savannah.gnu.org/releases/lzip/"
+                           name "/" name "-" version ".tar.gz"))
+       (sha256
+        (base32 "1brvsnpihzj81cf4wk2x5bnr2qldlq0wncpdbzxmzvxapm1cq2yc"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:configure-flags
+       (list "CC=gcc")))
+    (home-page "http://www.nongnu.org/lzip/clzip.html")
+    (synopsis "Small, stand-alone lzip compressor and decompressor")
+    (description
+     "Lunzip is a compressor and decompressor for files in the lzip compression
+format (.lz), written as a single small C tool with no dependencies.  This makes
+it well-suited to embedded and other systems without a C++ compiler, or for use
+in other applications like package managers.
+Clzip is intended to be fully compatible with the regular lzip package.")
+    (license license:gpl2+)))
-- 
2.15.0

[bug#29714] [PATCH 2/4] gnu: Add clzip.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 678 bytes --]

On Thu, Dec 14, 2017 at 10:37:22PM +0100, Tobias Geerinckx-Rice wrote:
> * gnu/packages/compression.scm (clzip): New public variable.

> +    (synopsis "Small, stand-alone lzip compressor and decompressor")
> +    (description
> +     "Lunzip is a compressor and decompressor for files in the lzip compression
> +format (.lz), written as a single small C tool with no dependencies.  This makes
> +it well-suited to embedded and other systems without a C++ compiler, or for use
> +in other applications like package managers.
> +Clzip is intended to be fully compatible with the regular lzip package.")

This description is mistakenly copy-pasted from the lunzip package,
right?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#29714] [PATCH 2/4] gnu: Add clzip.

[Tobias Geerinckx-Rice]

[-- Attachment #1.1: Type: text/plain, Size: 1098 bytes --]

Leo,

Leo Famulari wrote on 15/12/17 at 20:14:
> On Thu, Dec 14, 2017 at 10:37:22PM +0100, Tobias Geerinckx-Rice wrote:
>> * gnu/packages/compression.scm (clzip): New public variable.
> 
>> +    (synopsis "Small, stand-alone lzip compressor and decompressor")
>> +    (description
>> +     "Lunzip is a compressor and decompressor for files in the lzip compression
>> +format (.lz), written as a single small C tool with no dependencies.  This makes
>> +it well-suited to embedded and other systems without a C++ compiler, or for use
>> +in other applications like package managers.
>> +Clzip is intended to be fully compatible with the regular lzip package.")
> 
> This description is mistakenly copy-pasted from the lunzip package,
> right?

Partly. The left-over ‘Lunzip’ is indeed a mistake, but the rest is
intentional. The upstream home pages are equally similar, up to the
example of package managers (here without the ‘that only need to
decompress’ caveat).

They're certainly & confusingly similar; thanks for noticing the error!

Kind regards,

T G-R


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

[bug#29714] [PATCH 3/4] gnu: Add lzlib.

[Tobias Geerinckx-Rice]

* gnu/packages/compression.scm (lzlib): New public variable.
---
 gnu/packages/compression.scm | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index f4c90c25c..42af4cd51 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -1989,3 +1989,31 @@ it well-suited to embedded and other systems without a C++ compiler, or for use
 in other applications like package managers.
 Clzip is intended to be fully compatible with the regular lzip package.")
     (license license:gpl2+)))
+
+(define-public lzlib
+  (package
+    (name "lzlib")
+    (version "1.9")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "http://download.savannah.gnu.org/releases/lzip/"
+                           name "/" name "-" version ".tar.gz"))
+       (sha256
+        (base32 "13mssf3hrcnmd4ijbqnxfk0zgj1q5lvpxxkm1hmrbl1h73czhwi4"))))
+    (build-system gnu-build-system)
+    ;; The included minilzip binary is only ~16 smaller than the ‘real’ lzip.
+    ;; It's used during the test suite, but don't be tempted to install it.
+    (arguments
+     `(#:configure-flags
+       (list "CC=gcc"
+             "--enable-shared")))       ; only static (.a) is built by default
+    (home-page "http://www.nongnu.org/lzip/lzlib.html")
+    (synopsis "Lzip data compression C library")
+    (description
+     "Lzlib is a C library for in-memory LZMA compression and decompression in
+the lzip format.  It supports integrity checking of the decompressed data, and
+all functions are thread-safe.  The library should never crash, even in case of
+corrupted input.")
+    (license (list license:bsd-2        ; the library itself
+                   license:gpl2+))))    ; main.c (i.e. minilzip, used by tests)
-- 
2.15.0

[bug#29714] [PATCH 4/4] gnu: Add plzip.

[Tobias Geerinckx-Rice]

* gnu/packages/compression.scm (plzip): New public variable.
---
 gnu/packages/compression.scm | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 42af4cd51..c3a6345b1 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -2017,3 +2017,33 @@ all functions are thread-safe.  The library should never crash, even in case of
 corrupted input.")
     (license (list license:bsd-2        ; the library itself
                    license:gpl2+))))    ; main.c (i.e. minilzip, used by tests)
+
+(define-public plzip
+  (package
+    (name "plzip")
+    (version "1.6")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "http://download.savannah.gnu.org/releases/lzip/"
+                           name "/" name "-" version ".tar.gz"))
+       (sha256
+        (base32 "0z2cs6vn4xl65wakd013xl3sdfpg8dr0cvcjwc2slh8y9bz7j7ax"))))
+    (build-system gnu-build-system)
+    (inputs
+     `(("lzlib" ,lzlib)))
+    (home-page "http://www.nongnu.org/lzip/plzip.html")
+    (synopsis "Parallel lossless data compressor for the lzip format")
+    (description
+     "Plzip is a massively parallel (multi-threaded) lossless data compressor
+and decompressor that uses the lzip file format (.lz).  Files produced by plzip
+are fully compatible with lzip and can be rescued with lziprecover.
+On multiprocessor machines, plzip can compress and decompress large files much
+faster than lzip, at the cost of a slightly reduced compression ratio (0.4% to
+2%).  The number of usable threads is limited by file size: on files of only a
+few MiB, plzip is no faster than lzip.
+Files that were compressed with regular lzip will also not be decompressed
+faster by plzip, unless the @code{-b} option was used: lzip usually produces
+single-member files which can't be decompressed in parallel.")
+    (license (list license:bsd-2        ; arg_parser.{cc,h}
+                   license:gpl2+))))    ; everything else
-- 
2.15.0

[bug#29714] [PATCH 4/4] gnu: Add plzip.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

On Thu, Dec 14, 2017 at 10:37:24PM +0100, Tobias Geerinckx-Rice wrote:
> * gnu/packages/compression.scm (plzip): New public variable.

> +    (synopsis "Parallel lossless data compressor for the lzip format")
> +    (description
> +     "Plzip is a massively parallel (multi-threaded) lossless data compressor
> +and decompressor that uses the lzip file format (.lz).  Files produced by plzip
> +are fully compatible with lzip and can be rescued with lziprecover.
> +On multiprocessor machines, plzip can compress and decompress large files much
> +faster than lzip, at the cost of a slightly reduced compression ratio (0.4% to
> +2%).  The number of usable threads is limited by file size: on files of only a
> +few MiB, plzip is no faster than lzip.
> +Files that were compressed with regular lzip will also not be decompressed
> +faster by plzip, unless the @code{-b} option was used: lzip usually produces
> +single-member files which can't be decompressed in parallel.")

Maybe this would be useful in ((guix packages) patch-and-repack), (guix
utils) and (guix scripts pack). In those modules, we compress XZ
archives in parallel to speed it up.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#29714] [PATCH 4/4] gnu: Add plzip.

[Tobias Geerinckx-Rice]

[-- Attachment #1.1: Type: text/plain, Size: 429 bytes --]

Leo Famulari wrote on 15/12/17 at 20:22:
> On Thu, Dec 14, 2017 at 10:37:24PM +0100, Tobias Geerinckx-Rice wrote:
>> * gnu/packages/compression.scm (plzip): New public variable.
> 
> Maybe this would be useful in ((guix packages) patch-and-repack), (guix
> utils) and (guix scripts pack). In those modules, we compress XZ
> archives in parallel to speed it up.

Stop exposing my secret plan.

Kind regards,

T G-R


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

[bug#29714] [PATCH 1/4] gnu: Add lunzip.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 284 bytes --]

On Thu, Dec 14, 2017 at 10:37:21PM +0100, Tobias Geerinckx-Rice wrote:
> * gnu/packages/compression.scm (lunzip): New public variable.

If the closure of this package is smaller than that of lzip, perhaps we
could use it instead as the default LZ decompressor when building
packages.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#29714] [PATCH 1/4] gnu: Add lunzip.

[Tobias Geerinckx-Rice]

[-- Attachment #1.1: Type: text/plain, Size: 569 bytes --]

Leo,

Leo Famulari wrote on 15/12/17 at 20:17:
> On Thu, Dec 14, 2017 at 10:37:21PM +0100, Tobias Geerinckx-Rice wrote:
>> * gnu/packages/compression.scm (lunzip): New public variable.
> 
> If the closure of this package is smaller than that of lzip, perhaps we
> could use it instead as the default LZ decompressor when building
> packages.

Indeed! I was thinking the same thing :-)

A performance comparison of all these implementations was on my to-do
list for the weekend, but might be delayed by mandatory holiday cheer.

Kind regards,

T G-R


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

[bug#29714] [PATCH 1/4] gnu: Add lunzip.

[Tobias Geerinckx-Rice]

[-- Attachment #1.1: Type: text/plain, Size: 1823 bytes --]

Tobias Geerinckx-Rice wrote on 15/12/17 at 20:27:
>> * gnu/packages/compression.scm (lunzip): New public variable.
>
> A performance comparison of all these implementations was on my to-do
> list for the weekend, but might be delayed by mandatory holiday cheer.

...or, apparently, dental surgery.

It's about as much fun, but you're not allowed to drink.

Leo Famulari wrote on 15/12/17 at 20:17:
> If the closure of this package is smaller than that of lzip, perhaps we
> could use it instead as the default LZ decompressor when building
> packages.

If I were disappointed, it'd be by a surprisingly lean lzip:

  $ du -hs /gnu/store/wbww96afpdsfpnwhq4kxd6q70nz478jn-lunzip-1.9
  72K      ... # the lunzip binary is 48K
  $ guix size /gnu/store/wbww96afpdsfpnwhq4kxd6q70nz478jn-lunzip-1.9
  store item                          total    self
  /gnu/store/...-glibc-2.25              38.5    37.1  54.0%
  /gnu/store/...-gcc-5.4.0-lib           68.6    30.1  43.8%
  /gnu/store/...-bash-static-4.4.12       1.4     1.4   2.1%
  /gnu/store/...-lunzip-1.9              68.7     0.0   0.1%
  total: 68.7 MiB

  $ du -hs /gnu/store/8abzs1c39klm0d2sh8bwv6xn7924blam-lzip-1.18
  172K     ... # the lzip binary is 124K
  $ guix size /gnu/store/8abzs1c39klm0d2sh8bwv6xn7924blam-lzip-1.18
  store item                          total    self
  /gnu/store/...-glibc-2.25              38.5    37.1  54.0%
  /gnu/store/...-gcc-5.4.0-lib           68.6    30.1  43.8%
  /gnu/store/...-bash-static-4.4.12       1.4     1.4   2.1%
  /gnu/store/...-lzip-1.18               68.8     0.1   0.2%
  total: 68.8 MiB

Lzip: too cool to fit in 100%.

Both variants are equally fast. Switching to lunzip saves 100K.
I'm tempted to try it, but that might be the opiates talking.

Kind regards,

T G-R


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

bug#29714: [PATCH 1/4] gnu: Add lunzip.

[Tobias Geerinckx-Rice]

Pushed with minor fixes as 34e8cf225cc380c2f248eecd43151f6751eeda97 et al.