From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54285) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gYEWu-000281-JF for guix-patches@gnu.org; Sat, 15 Dec 2018 13:19:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gYELA-0001Mz-C7 for guix-patches@gnu.org; Sat, 15 Dec 2018 13:07:06 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:45333) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gYEL8-0001L6-1T for guix-patches@gnu.org; Sat, 15 Dec 2018 13:07:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gYEL7-0005xq-Rj for guix-patches@gnu.org; Sat, 15 Dec 2018 13:07:01 -0500 Subject: [bug#33600] Compressing nars with lzip or similar Resent-Message-ID: From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20181203154335.10366-1-ludo@gnu.org> <87tvju6145.fsf@gnu.org> <87ftv7l6gy.fsf@gmail.com> <87pnua244k.fsf@gnu.org> <65fa6be4-3ddc-703b-708d-e223c2474d0b@crazy-compilers.com> <87tvjgeb12.fsf@ambrevar.xyz> <874lbg2mh3.fsf_-_@gnu.org> <878t0sdthe.fsf@ambrevar.xyz> <87k1kbrnlq.fsf@ambrevar.xyz> Date: Sat, 15 Dec 2018 19:06:21 +0100 In-Reply-To: <87k1kbrnlq.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Sat, 15 Dec 2018 13:17:05 +0100") Message-ID: <87o99my89u.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Pierre Neidhardt Cc: guix-devel@gnu.org, Hartmut Goebel , 33600@debbugs.gnu.org Pierre Neidhardt skribis: > I've done some quick research over the various options. > > - lzip: better than gz and bzip2 for sure, possibly better than xz (at > least according to the author). > > - plzip: for "parallel lzip". With 4 threads I was able to compress > icecat 2.5x faster. It used 5x more memory though. The compression > ratio is 1-2% worse. > > - lrzip: it would crash whenever I would change the compression level. > Seems less stable. It's as fast as plzip, while being 1-2% less > compressed. I don't think it's worth using. > > All in all, lzip is a definite win over most options. The main question > is: lzip or plzip? =E2=80=98guix publish=E2=80=99 has its own worker pool and handles parallel= ism internally, so in that context plain sequential lzip would be more appropriate IMO. Ludo=E2=80=99.