Re: Parallel (de-)compression

[Efraim Flashner <efraim@flashner.co.il>]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

On Wed, 2 Dec 2015 19:52:17 +0100
Andreas Enge <andreas@enge.fr> wrote:

> How about this:
>    http://anthon.home.xs4all.nl/rants/2013/parallel_xz/
> ?
> 
> Andreas
> 
> 

From what I've read, parallel bzip breaks everything into chunks and then
reassembles everything into a .tar.bz2 file. On one of my mail folders
containing ~3000 pieces of mail ~1GiB, my 2 core laptop took 10 minutes with
bzip and 7 minutes with pbzip to compress the mail, with the
resulting .tar.bz2 files being ~100KiB difference in size. I didn't get a
chance to test decompression because my battery died :). One of the big
plusses to pbzip is that the compressed files are fully compatable with bzip,
and I believe all the normally used flags are also usable. It can also be
dropped into tar with --use-compress-prog=pbzip2.

Parallel xz I know much less about.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]