From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: Re: Compressing nars with lzip or similar Date: Sat, 15 Dec 2018 19:06:21 +0100 Message-ID: <87o99my89u.fsf@gnu.org> References: <20181203154335.10366-1-ludo@gnu.org> <87tvju6145.fsf@gnu.org> <87ftv7l6gy.fsf@gmail.com> <87pnua244k.fsf@gnu.org> <65fa6be4-3ddc-703b-708d-e223c2474d0b@crazy-compilers.com> <87tvjgeb12.fsf@ambrevar.xyz> <874lbg2mh3.fsf_-_@gnu.org> <878t0sdthe.fsf@ambrevar.xyz> <87k1kbrnlq.fsf@ambrevar.xyz> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:54338) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gYEWv-00029H-RN for guix-devel@gnu.org; Sat, 15 Dec 2018 13:19:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gYEKi-0000x7-4c for guix-devel@gnu.org; Sat, 15 Dec 2018 13:06:36 -0500 In-Reply-To: <87k1kbrnlq.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Sat, 15 Dec 2018 13:17:05 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Pierre Neidhardt Cc: guix-devel@gnu.org, 33600@debbugs.gnu.org Pierre Neidhardt skribis: > I've done some quick research over the various options. > > - lzip: better than gz and bzip2 for sure, possibly better than xz (at > least according to the author). > > - plzip: for "parallel lzip". With 4 threads I was able to compress > icecat 2.5x faster. It used 5x more memory though. The compression > ratio is 1-2% worse. > > - lrzip: it would crash whenever I would change the compression level. > Seems less stable. It's as fast as plzip, while being 1-2% less > compressed. I don't think it's worth using. > > All in all, lzip is a definite win over most options. The main question > is: lzip or plzip? =E2=80=98guix publish=E2=80=99 has its own worker pool and handles parallel= ism internally, so in that context plain sequential lzip would be more appropriate IMO. Ludo=E2=80=99.