unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: "Ludovic Courtès" <ludo@gnu.org>
To: Pierre Neidhardt <mail@ambrevar.xyz>
Cc: guix-devel <guix-devel@gnu.org>,
	"Nicolò Balzarotti" <anothersms@gmail.com>
Subject: Re: When substitute download + decompression is CPU-bound
Date: Tue, 15 Dec 2020 12:42:10 +0100	[thread overview]
Message-ID: <87blevmh2l.fsf@gnu.org> (raw)
In-Reply-To: <871rfrld59.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Tue, 15 Dec 2020 08:52:18 +0100")

Hi,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Another option is plzip (parallel Lzip, an official part of Lzip).
>
>> decompression of ungoogled-chromium from the LAN completes in 2.4s for
>> gzip vs. 7.1s for lzip.  On a low-end ARMv7 device, also on the LAN, I
>> get 32s (gzip) vs. 53s (lzip).
>
> With four cores, plzip would beat gzip in the first case.
> With only 2 cores, plzip would beat gzip in the second case.
>
> What's left to do to implement plzip support?  That's the good news:
> almost nothing!
>
> - On the Lzip binding side, we need to add support for multi pages.
>   It's a bit of work but not that much.
> - On the Guix side, there is nothing to do.

Well, ‘guix publish’ would first need to create multi-member archives,
right?

Also, lzlib (which is what we use) does not implement parallel
decompression, AIUI.

Even if it did, would we be able to take advantage of it?  Currently
‘restore-file’ expects to read an archive stream sequentially.

Even if I’m wrong :-), decompression speed would at best be doubled on
multi-core machines (wouldn’t help much on low-end ARM devices), and
that’s very little compared to the decompression speed achieved by zstd.

Ludo’.


  parent reply	other threads:[~2020-12-15 11:42 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-14 22:20 When substitute download + decompression is CPU-bound Ludovic Courtès
2020-12-14 22:29 ` Julien Lepiller
2020-12-14 22:59 ` Nicolò Balzarotti
2020-12-15  7:52   ` Pierre Neidhardt
2020-12-15  9:45     ` Nicolò Balzarotti
2020-12-15  9:54       ` Pierre Neidhardt
2020-12-15 10:03         ` Nicolò Balzarotti
2020-12-15 10:13           ` Pierre Neidhardt
2020-12-15 10:14             ` Pierre Neidhardt
2020-12-15 11:42     ` Ludovic Courtès [this message]
2020-12-15 12:31       ` Pierre Neidhardt
2020-12-18 14:59         ` Ludovic Courtès
2020-12-18 15:33           ` Pierre Neidhardt
2020-12-15 11:36   ` Ludovic Courtès
2020-12-15 11:45     ` Nicolò Balzarotti
2020-12-15 10:40 ` Jonathan Brielmaier
2020-12-15 19:43   ` Joshua Branson
2021-01-07 10:45     ` Guillaume Le Vaillant
2021-01-07 11:00       ` Pierre Neidhardt
2021-01-07 11:33         ` Guillaume Le Vaillant
2021-01-14 21:51       ` Ludovic Courtès
2021-01-14 22:08         ` Nicolò Balzarotti
2021-01-28 17:53           ` Are gzip-compressed substitutes still used? Ludovic Courtès
2021-03-17 17:12             ` Ludovic Courtès
2021-03-17 17:33               ` Léo Le Bouter
2021-03-17 18:08                 ` Vagrant Cascadian
2021-03-18  0:03                   ` zimoun
2021-03-18 16:00                     ` Vagrant Cascadian
2021-03-18 18:53                       ` Leo Famulari
2021-03-20 11:23                   ` Ludovic Courtès
2021-03-17 18:06               ` zimoun
2021-03-17 18:20               ` Jonathan Brielmaier
2021-03-18 17:25               ` Pierre Neidhardt
2021-01-15  8:10         ` When substitute download + decompression is CPU-bound Pierre Neidhardt
2021-01-28 17:58           ` Ludovic Courtès
2021-01-29  9:45             ` Pierre Neidhardt
2021-01-29 11:23               ` Guillaume Le Vaillant
2021-01-29 11:55                 ` Nicolò Balzarotti
2021-01-29 12:13                   ` Pierre Neidhardt
2021-01-29 13:06                     ` Guillaume Le Vaillant
2021-01-29 14:55                     ` Nicolò Balzarotti
2021-02-01 22:18                 ` Ludovic Courtès
2021-01-29 13:33             ` zimoun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87blevmh2l.fsf@gnu.org \
    --to=ludo@gnu.org \
    --cc=anothersms@gmail.com \
    --cc=guix-devel@gnu.org \
    --cc=mail@ambrevar.xyz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).