Speeding up guix?

Speeding up guix?

[yarnton--- via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]


Hi all,

I'm periodically testdriving GuixSD with the intention to migrate several workstations and servers in my organization. It's very exciting to see Guix is quickly catching up with Nix and NixOS despite being much younger.

A slightly annoying aspect of Guix that IMHO should be improved is download speed. I haven't inspected any code, but it seems to me that Guix is reopening a connection for downloading every item? If so, that's a bit wasteful, and makes things unnecessarily slow even with a good connection.

Thanks.


[-- Attachment #2: Type: text/html, Size: 783 bytes --]

Re: Speeding up guix?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 457 bytes --]

Hi!

There has been discussion about this in the past:

https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00694.html

Browse the thread, pipelining and reuse of connections are discussed.

Julien did work on a patch regarding downloads (if I recall correctly it
was to enable parallel downloads), I believe it's still hanging
somewhere in the bug tracker.


Tests are welcome!

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

Re: Speeding up guix?

[Ludovic Courtès]

Hi,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> There has been discussion about this in the past:
>
> https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00694.html
>
> Browse the thread, pipelining and reuse of connections are discussed.
>
> Julien did work on a patch regarding downloads (if I recall correctly it
> was to enable parallel downloads), I believe it's still hanging
> somewhere in the bug tracker.

Yes, it’s one of the many patches waiting for feedback:  :-)

  https://issues.guix.gnu.org/39728

I contemplated another way to improve on download speeds, by spawning a
single ‘guix substitute’ process and thus potentially reusing
connections, but it’s trickier than it seems.

Ludo’.

PS to yarnton: Note that Nix would also have to catch up on some
  features: grafts, authenticated checkouts, reduced binary seeds, etc.
  ;-)

Re: Speeding up guix?

[yarnton--- via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 755 bytes --]



>  https://issues.guix.gnu.org/39728
>
> I contemplated another way to improve on download speeds, by spawning a
> single ‘guix substitute’ process and thus potentially reusing
> connections, but it’s trickier than it seems.
>
Can I help testing this patch?

> PS to yarnton: Note that Nix would also have to catch up on some
>  features: grafts, authenticated checkouts, reduced binary seeds, etc.
>  ;-)
>
I know! That's why I'm trying to migrate to GuixSD. And I say this as a maintainer of a dozen  packages in NixPkgs.

Another big motivation is that Guix packages are cleaner and well tested. Nix has quite a few things that are half broken. I've fixed some that I use myself, but it's too much work to get it all sorted.


[-- Attachment #2: Type: text/html, Size: 1296 bytes --]

Re: Speeding up guix?

[Leo Famulari]

On Tue, Nov 03, 2020 at 05:51:49PM +0100, yarnton--- via Development of GNU Guix and the GNU System distribution. wrote:
> >  https://issues.guix.gnu.org/39728
> >
> Can I help testing this patch?

Yes, the steps required for this are outlined in the manual chapter
Contributing — specifically the sections Building From Git and Running
Guix Before It Is Installed.

Basically, you'll build Guix from Git and then use the `./pre-inst-env`
script to use Guix based on the Git repo, with the patch applied.

If you get stuck, the quickest way to get help is in our IRC channel
#guix on freenode, and you can also get help here on the mailing list.

Re: Speeding up guix?

[zimoun]

Hi,

On Tue, 3 Nov 2020 at 18:39, Leo Famulari <leo@famulari.name> wrote:
>
> On Tue, Nov 03, 2020 at 05:51:49PM +0100, yarnton--- via Development of GNU Guix and the GNU System distribution. wrote:
> > >  https://issues.guix.gnu.org/39728
> > >
> > Can I help testing this patch?

More all the Leo's word, the daemon should be stopped and you should
start the patched one.  Do not hesitate to ask if you get stuck.

All the best,
simon

Re: Speeding up guix?

[Leo Famulari]

On Tue, Nov 03, 2020 at 01:59:56AM +0100, yarnton--- via Development of GNU Guix and the GNU System distribution. wrote:
> A slightly annoying aspect of Guix that IMHO should be improved is download speed. I haven't inspected any code, but it seems to me that Guix is reopening a connection for downloading every item? If so, that's a bit wasteful, and makes things unnecessarily slow even with a good connection.

This is indeed a big source of slowness while using Guix, and it has
been discussed previously:

https://lists.gnu.org/archive/html/guix-devel/2019-11/msg00226.html

It seems like we discuss it every November!

We just need someone to lead the work to improve it :)