From mboxrd@z Thu Jan 1 00:00:00 1970 From: Efraim Flashner Subject: Re: Parallel (de-)compression Date: Wed, 2 Dec 2015 21:18:32 +0200 Message-ID: <20151202211832.0ce44d64@debian-netbook> References: <20151202184550.GA5741@debian> <20151202185217.GA5808@debian> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/=e67Ga22y0JciPP_5OQB/xH"; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:43421) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a4Cvh-000261-0I for guix-devel@gnu.org; Wed, 02 Dec 2015 14:19:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a4Cvc-0008Ow-Rt for guix-devel@gnu.org; Wed, 02 Dec 2015 14:19:04 -0500 Received: from flashner.co.il ([178.62.234.194]:46215) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a4Cvc-0008Ng-Kw for guix-devel@gnu.org; Wed, 02 Dec 2015 14:19:00 -0500 In-Reply-To: <20151202185217.GA5808@debian> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Andreas Enge Cc: guix-devel@gnu.org --Sig_/=e67Ga22y0JciPP_5OQB/xH Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, 2 Dec 2015 19:52:17 +0100 Andreas Enge wrote: > How about this: > http://anthon.home.xs4all.nl/rants/2013/parallel_xz/ > ? >=20 > Andreas >=20 >=20 =46rom what I've read, parallel bzip breaks everything into chunks and then reassembles everything into a .tar.bz2 file. On one of my mail folders containing ~3000 pieces of mail ~1GiB, my 2 core laptop took 10 minutes with bzip and 7 minutes with pbzip to compress the mail, with the resulting .tar.bz2 files being ~100KiB difference in size. I didn't get a chance to test decompression because my battery died :). One of the big plusses to pbzip is that the compressed files are fully compatable with bzi= p, and I believe all the normally used flags are also usable. It can also be dropped into tar with --use-compress-prog=3Dpbzip2. Parallel xz I know much less about. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --Sig_/=e67Ga22y0JciPP_5OQB/xH Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCgAGBQJWX0QIAAoJEPTB05F+rO6T2EMP+gNfqwbzMIMCgnSPsg0TWCSn Pkd5JPZ5g0/KGKR2rwgHT1zWqEPTFCDS8+kqoIBhElwFgdn7PLX34XGyTzENKpsk BMixU7BtDpcP1rcIv9GU+mSsMN1Q1CPU1+is4/Z+tl9dwIsPdHnwJgY0MZ3gS1Vh t9USXTvkF7r1wzw5SrJJoINovLvjP6Q8Fw1pGoG76L+H1fWQF+J7o2bXNvt7syND 9uzrYLxwcLhAPoaEwo+jkDLVnoFTP1IFrcUp3oIbHpGerv1GsF24g+9rLsYUSghI l6OJHmBXvnf9XmCUZsHVNLUmwAeJaII/0Ms9a7A/y8Asa/KChnmRGMYExnQTalvT x+L0cQtG4dLpcp41YqjtZhBA5cmHTYU3+bU65RIE5+nEo75OJNPW2tPaMX1weor5 mpjHzYZLLNJVkSJ0hx/mWx+Wh1bMBBmyAsmjRQ7PqRovvzMb1lrQhUuO5P4etdzi sXKpFejTl8qmBUbZOhm29s8MpduhsJNsB20lxr/TOTkuovwxEZAKmc/Ot5V1Xy64 8jYuVMzAtcLQVlhA6eRYTPCVRBsA0QzQWxY1/nRx3bTuo81YGNhMzqy1nJzAZ1BC IuUCWtdv+ngZ8qSC2dAbZMQo1VSA//nMe8elos+wu/tDOn/GO0u+NzinGfwOzC7J F3UE0B+8n3gUPPaTGPGz =u5Sz -----END PGP SIGNATURE----- --Sig_/=e67Ga22y0JciPP_5OQB/xH--