From: "Ludovic Courtès" <ludo@gnu.org>
To: Pierre Neidhardt <mail@ambrevar.xyz>
Cc: guix-devel@gnu.org, 33600@debbugs.gnu.org
Subject: Re: Compressing nars with lzip or similar
Date: Sat, 15 Dec 2018 19:06:21 +0100 [thread overview]
Message-ID: <87o99my89u.fsf@gnu.org> (raw)
In-Reply-To: <87k1kbrnlq.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Sat, 15 Dec 2018 13:17:05 +0100")
Pierre Neidhardt <mail@ambrevar.xyz> skribis:
> I've done some quick research over the various options.
>
> - lzip: better than gz and bzip2 for sure, possibly better than xz (at
> least according to the author).
>
> - plzip: for "parallel lzip". With 4 threads I was able to compress
> icecat 2.5x faster. It used 5x more memory though. The compression
> ratio is 1-2% worse.
>
> - lrzip: it would crash whenever I would change the compression level.
> Seems less stable. It's as fast as plzip, while being 1-2% less
> compressed. I don't think it's worth using.
>
> All in all, lzip is a definite win over most options. The main question
> is: lzip or plzip?
‘guix publish’ has its own worker pool and handles parallelism
internally, so in that context plain sequential lzip would be more
appropriate IMO.
Ludo’.
WARNING: multiple messages have this Message-ID (diff)
From: "Ludovic Courtès" <ludo@gnu.org>
To: Pierre Neidhardt <mail@ambrevar.xyz>
Cc: guix-devel@gnu.org, Hartmut Goebel <h.goebel@crazy-compilers.com>,
33600@debbugs.gnu.org
Subject: [bug#33600] Compressing nars with lzip or similar
Date: Sat, 15 Dec 2018 19:06:21 +0100 [thread overview]
Message-ID: <87o99my89u.fsf@gnu.org> (raw)
In-Reply-To: <87k1kbrnlq.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Sat, 15 Dec 2018 13:17:05 +0100")
Pierre Neidhardt <mail@ambrevar.xyz> skribis:
> I've done some quick research over the various options.
>
> - lzip: better than gz and bzip2 for sure, possibly better than xz (at
> least according to the author).
>
> - plzip: for "parallel lzip". With 4 threads I was able to compress
> icecat 2.5x faster. It used 5x more memory though. The compression
> ratio is 1-2% worse.
>
> - lrzip: it would crash whenever I would change the compression level.
> Seems less stable. It's as fast as plzip, while being 1-2% less
> compressed. I don't think it's worth using.
>
> All in all, lzip is a definite win over most options. The main question
> is: lzip or plzip?
‘guix publish’ has its own worker pool and handles parallelism
internally, so in that context plain sequential lzip would be more
appropriate IMO.
Ludo’.
next prev parent reply other threads:[~2018-12-15 18:19 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-03 15:43 [PATCH 0/3] Defaulting to ci.guix.info (aka. berlin.guixsd.org) Ludovic Courtès
2018-12-03 16:12 ` Using a CDN or some other mirror? Ludovic Courtès
2018-12-03 20:47 ` Ricardo Wurmus
2018-12-04 10:40 ` Hartmut Goebel
2018-12-04 14:05 ` Ludovic Courtès
2018-12-04 17:03 ` Pjotr Prins
2018-12-04 17:58 ` Thompson, David
2018-12-05 2:32 ` Meiyo Peng
2018-12-05 5:38 ` Leo Famulari
2018-12-05 10:59 ` Pierre Neidhardt
2018-12-05 11:46 ` Hartmut Goebel
2018-12-07 14:05 ` Ludovic Courtès
2018-12-09 9:44 ` Hartmut Goebel
2018-12-04 21:15 ` ng0
2018-12-04 21:50 ` Thompson, David
2018-12-05 9:28 ` ng0
2018-12-09 3:33 ` Chris Marusich
2018-12-09 12:12 ` Hartmut Goebel
2018-12-09 13:58 ` Ludovic Courtès
2018-12-11 16:38 ` Giovanni Biscuolo
2018-12-11 16:38 ` [bug#33600] " Giovanni Biscuolo
2018-12-14 8:35 ` Hartmut Goebel
2018-12-14 8:35 ` [bug#33600] " Hartmut Goebel
2018-12-14 9:02 ` Pierre Neidhardt
2018-12-14 14:48 ` Compressing nars with lzip or similar Ludovic Courtès
2018-12-14 14:48 ` [bug#33600] " Ludovic Courtès
2018-12-14 15:21 ` Pierre Neidhardt
2018-12-15 12:17 ` Pierre Neidhardt
2018-12-15 18:06 ` Ludovic Courtès [this message]
2018-12-15 18:06 ` [bug#33600] " Ludovic Courtès
2019-03-05 11:36 ` Pierre Neidhardt
2018-12-15 18:04 ` Ludovic Courtès
2018-12-14 14:45 ` Using a CDN or some other mirror? Ludovic Courtès
2018-12-09 15:59 ` CDN performance Ludovic Courtès
2018-12-11 5:17 ` Meiyo Peng
[not found] ` <CAAYZrgbOZYyKhaHzziWfKz-nHVcUWS6WCo4TAq8bbDn9=YMTZA@mail.gmail.com>
2018-12-11 5:59 ` Meiyo Peng
[not found] ` <CAAYZrgb431xW1RD0Hf0d15T3AiW5yZWLL6oqHsyanv1qSf8Zuw@mail.gmail.com>
2018-12-11 6:14 ` Meiyo Peng
2018-12-13 7:11 ` Chris Marusich
2018-12-17 6:48 ` Meiyo Peng
2018-12-17 6:48 ` [bug#33600] " Meiyo Peng
2018-12-21 10:22 ` Chris Marusich
2018-12-21 16:04 ` Meiyo Peng
2018-12-21 16:04 ` [bug#33600] " Meiyo Peng
2018-12-13 8:05 ` Chris Marusich
2018-12-13 10:41 ` Giovanni Biscuolo
2018-12-15 1:40 ` Mark H Weaver
2018-12-19 12:40 ` Giovanni Biscuolo
2018-12-21 0:23 ` Trustworthiness of build farms (was Re: CDN performance) Mark H Weaver
2018-12-21 20:47 ` CDN performance Marius Bakke
2018-12-21 20:47 ` [bug#33600] " Marius Bakke
2018-12-24 14:47 ` Ricardo Wurmus
2018-12-14 10:26 ` guix.gnu.org sub-domain Ludovic Courtès
2018-12-15 23:20 ` Chris Marusich
2018-12-15 23:20 ` [bug#33600] " Chris Marusich
2019-01-25 4:54 ` Amin Bandali
2018-12-14 10:35 ` CDN performance Ludovic Courtès
2018-12-13 9:21 ` Using a CDN or some other mirror? Giovanni Biscuolo
2018-12-14 12:17 ` Chris Marusich
2018-12-03 18:20 ` [PATCH 0/3] Defaulting to ci.guix.info (aka. berlin.guixsd.org) Amin Bandali
2018-12-04 14:11 ` Ludovic Courtès
2018-12-11 5:41 ` Amin Bandali
2018-12-03 23:44 ` Mark H Weaver
2018-12-04 5:55 ` Ricardo Wurmus
2018-12-04 5:55 ` [bug#33600] " Ricardo Wurmus
2018-12-04 9:03 ` Ludovic Courtès
2018-12-04 10:08 ` Andreas Enge
2018-12-04 8:59 ` Andreas Enge
2018-12-04 10:28 ` Ludovic Courtès
2018-12-04 10:46 ` Andreas Enge
2018-12-04 14:12 ` Ludovic Courtès
2018-12-04 3:40 ` Meiyo Peng
2018-12-04 14:13 ` Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87o99my89u.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=33600@debbugs.gnu.org \
--cc=guix-devel@gnu.org \
--cc=mail@ambrevar.xyz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.