Re: Compressing nars with lzip or similar

[Pierre Neidhardt <mail@ambrevar.xyz>]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

I've done some quick research over the various options.

- lzip: better than gz and bzip2 for sure, possibly better than xz (at
least according to the author).

- plzip: for "parallel lzip".  With 4 threads I was able to compress
icecat 2.5x faster.  It used 5x more memory though.  The compression
ratio is 1-2% worse.

- lrzip: it would crash whenever I would change the compression level.
  Seems less stable.  It's as fast as plzip, while being 1-2% less
  compressed.  I don't think it's worth using.

All in all, lzip is a definite win over most options.  The main question
is: lzip or plzip?

In my opinion it makes more sense to use plzip, gotta put those cores to
some use (on the user side at least)!  Not sure what the build farm
would think about this though.

Finally, plzip can be quite memory-intensive.  When passed "--threads=1"
however, it gets closer to lzip (bit faster, bit more memory).  So maybe
the ideal would be to support plzip with a user-settable "threads" option.

Thoughts?

--
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]