Guix / Nix Benchmarks

Guix / Nix Benchmarks

[Nicolas Graves via Development of GNU Guix and the GNU System distribution.]

One of the criticism that can be read online about Guix (compared to
Nix) is its speed. I have never tried Nix and probably won't in a near
future, but I was wondering if some work has been made to compare them
on basic tasks (package installation, removal...) (the reason why I ask
is to be able to give an honest answer to someone hesitating between
both).

-- 
Best regards,
Nicolas Graves

Re: Guix / Nix Benchmarks

[kiasoc5]

On 6/19/23 08:54, Nicolas Graves via Development of GNU Guix and the GNU 
System distribution. wrote:
> 
> One of the criticism that can be read online about Guix (compared to
> Nix) is its speed. I have never tried Nix and probably won't in a near
> future, but I was wondering if some work has been made to compare them
> on basic tasks (package installation, removal...) (the reason why I ask
> is to be able to give an honest answer to someone hesitating between
> both).
> 

At least package installation is faster with Nix. One reason is that 
Hydra (their "substitute server") has faster download speeds compared to 
CI/Bordeaux, likely due to multiple geographical locations/mirrors.

Re: Guix / Nix Benchmarks

[Csepp]

kiasoc5 <kiasoc5@disroot.org> writes:

> On 6/19/23 08:54, Nicolas Graves via Development of GNU Guix and the
> GNU System distribution. wrote:
>> One of the criticism that can be read online about Guix (compared to
>> Nix) is its speed. I have never tried Nix and probably won't in a near
>> future, but I was wondering if some work has been made to compare them
>> on basic tasks (package installation, removal...) (the reason why I ask
>> is to be able to give an honest answer to someone hesitating between
>> both).
>> 
>
> At least package installation is faster with Nix. One reason is that
> Hydra (their "substitute server") has faster download speeds compared
> to CI/Bordeaux, likely due to multiple geographical locations/mirrors.

Conversely, Nix itself is likely to be slower than Guile.  I remember
search being so memory intensive with Nix that it got OOM killed even
with 4 GB of RAM, although they may have fixed that since.

Re: Guix / Nix Benchmarks

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]

On Mon, Jun 19, 2023 at 02:55:21PM -0400, kiasoc5 wrote:
> On 6/19/23 08:54, Nicolas Graves via Development of GNU Guix and the GNU
> System distribution. wrote:
> > 
> > One of the criticism that can be read online about Guix (compared to
> > Nix) is its speed. I have never tried Nix and probably won't in a near
> > future, but I was wondering if some work has been made to compare them
> > on basic tasks (package installation, removal...) (the reason why I ask
> > is to be able to give an honest answer to someone hesitating between
> > both).
> > 
> 
> At least package installation is faster with Nix. One reason is that Hydra
> (their "substitute server") has faster download speeds compared to
> CI/Bordeaux, likely due to multiple geographical locations/mirrors.

I believe on the Nix side that includes a ~$9000/month bill for AWS.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Guix / Nix Benchmarks

[W. T. Meyer]

Hi,

Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:

> One of the criticism that can be read online about Guix (compared to
> Nix) is its speed.

I am using guix as well as nix, and depending on the operation you're
performing there's not much of a noticeable difference (I usually tend
to build packages from source, so for most of the build process it
doesn't really matter what, as in guix or nix, calls the corresponding
build-system; as that's not too relevant during the build process in
terms of time relevance).

However, the download speed from the official substitute servers is
noticeable slower; but that's to be expected considering the resources^1
NixOS is allocating towards their binary/build caching efforts.

> I have never tried Nix and probably won't in a near
> future, but I was wondering if some work has been made to compare them
> on basic tasks (package installation, removal...) (the reason why I ask
> is to be able to give an honest answer to someone hesitating between
> both).

I'd personally advise against over-priotizing raw performance of package
management tasks as a feasible metric to choose a package manager by; as
having a "good enough" (which is highly subjective) performance should
be enough. In this context it'd be probably more helpful to have a look
at nix vs. guix cli tooling and nix as a language vs. guile basing your
decision upon that.

Regards,

Wilko Meyer

[1]: https://discourse.nixos.org/t/the-nixos-foundations-call-to-action-s3-costs-require-community-support/28672

Re: Guix / Nix Benchmarks

[Nicolas Graves via Development of GNU Guix and the GNU System distribution.]

On 2023-06-20 12:45, W. T. Meyer wrote:

> Hi,
>
> Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:
>
>> One of the criticism that can be read online about Guix (compared to
>> Nix) is its speed.
>
> I am using guix as well as nix, and depending on the operation you're
> performing there's not much of a noticeable difference (I usually tend
> to build packages from source, so for most of the build process it
> doesn't really matter what, as in guix or nix, calls the corresponding
> build-system; as that's not too relevant during the build process in
> terms of time relevance).
>
> However, the download speed from the official substitute servers is
> noticeable slower; but that's to be expected considering the resources^1
> NixOS is allocating towards their binary/build caching efforts.
>
>> I have never tried Nix and probably won't in a near
>> future, but I was wondering if some work has been made to compare them
>> on basic tasks (package installation, removal...) (the reason why I ask
>> is to be able to give an honest answer to someone hesitating between
>> both).
>
> I'd personally advise against over-priotizing raw performance of package
> management tasks as a feasible metric to choose a package manager by; as
> having a "good enough" (which is highly subjective) performance should
> be enough. In this context it'd be probably more helpful to have a look
> at nix vs. guix cli tooling and nix as a language vs. guile basing your
> decision upon that.

I fully agree with this statement, over-priotizing raw performance was
not my point, and I won't change myself. I agree with the fact that
having a fair comparison would first require comparing tooling, although
I also think that keeping an eye on performance is not a waste either.

Then the "best" actual comparison might be the one done by Andrew Tropin
a few years ago, right here :
https://gist.github.com/abcdw/e54807b0a25e61fe2cf1bf8991410f83

Maybe we could get in touch with some nix folks to try to update, enrich
and maintain this kind of comparison up-to-date?

(I've seen that some nice reviewing has been done on NixOS by linux
"influencers" during the last week, and would like to also see Guix
benefit from that. Although both are not in competition, a fair and
shared comparison might help Guix grow in the space of Linux
distributions.)

>
> Regards,
>
> Wilko Meyer
>
> [1]: https://discourse.nixos.org/t/the-nixos-foundations-call-to-action-s3-costs-require-community-support/28672

-- 
Best regards,
Nicolas Graves

Re: Guix / Nix Benchmarks

[Ludovic Courtès]

Hi,

Nicolas Graves via "Development of GNU Guix and the GNU System
distribution." <guix-devel@gnu.org> skribis:

> Then the "best" actual comparison might be the one done by Andrew Tropin
> a few years ago, right here :
> https://gist.github.com/abcdw/e54807b0a25e61fe2cf1bf8991410f83
>
> Maybe we could get in touch with some nix folks to try to update, enrich
> and maintain this kind of comparison up-to-date?

It’d be useful to maintain such a comparison because it always comes up
in on-line discussions.

A comparison should included non-functional facets such as packaging
policies (free software, unbundling, building from source, etc.) as well
as social aspects (governance, economics, contribution workflow, etc.).

Ludo’.

Re: Guix / Nix Benchmarks

[Ricardo Wurmus]

"W. T. Meyer" <w@wmeyer.eu> writes:

> However, the download speed from the official substitute servers is
> noticeable slower; but that's to be expected considering the resources^1
> NixOS is allocating towards their binary/build caching efforts.

I’ve found download speeds to vary wildly dependent on where you are.
From an AWS EC2 instance in Frankfurt we get about 50MiB/s download
speeds from ci.guix.gnu.org, whereas some in France get sub 1MiB/s.

-- 
Ricardo

Re: Guix / Nix Benchmarks

[Wilko Meyer]

Ricardo Wurmus <rekado@elephly.net> writes:
> I’ve found download speeds to vary wildly dependent on where you are.
> From an AWS EC2 instance in Frankfurt we get about 50MiB/s download
> speeds from ci.guix.gnu.org, whereas some in France get sub 1MiB/s.

So, this could be an ISP related issue, e.g. maybe an issue regarding
peering capacities? As far as I know there have been historical issues
in this direction between commerical ISPs (namely DTAG being the one I'm
aware of) and educational networks such as X-WiN, so (without having any
deeper knowledge on Guixes situation on this) this could apply here as
well?

As ci.guix.gnu.org is hosted at AS210482/the Max-Delbrueck-Centrum, I
would guess that there could be something similar going on causing this,
e.g. commerical ISP trying to enforce their double payed traffic
strategies and the educational networks where ci.guix.gnu.org is hosted
(rightfully) not participating in that/not having direct peering with
these ISPs ASNs?

Regards,

Wilko Meyer

Re: Guix / Nix Benchmarks

[Ricardo Wurmus]

Wilko Meyer <w@wmeyer.eu> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>> I’ve found download speeds to vary wildly dependent on where you are.
>> From an AWS EC2 instance in Frankfurt we get about 50MiB/s download
>> speeds from ci.guix.gnu.org, whereas some in France get sub 1MiB/s.
>
> So, this could be an ISP related issue, e.g. maybe an issue regarding
> peering capacities? As far as I know there have been historical issues
> in this direction between commerical ISPs (namely DTAG being the one I'm
> aware of) and educational networks such as X-WiN, so (without having any
> deeper knowledge on Guixes situation on this) this could apply here as
> well?
>
> As ci.guix.gnu.org is hosted at AS210482/the Max-Delbrueck-Centrum, I
> would guess that there could be something similar going on causing this,
> e.g. commerical ISP trying to enforce their double payed traffic
> strategies and the educational networks where ci.guix.gnu.org is hosted
> (rightfully) not participating in that/not having direct peering with
> these ISPs ASNs?

Yes, this is our guess.  The head node and ~24 nodes of the Berlin build
farm are hosted in the MDC data center, which is hooked up to the
internet via the DFN network IIUC.

-- 
Ricardo

Re: Guix / Nix Benchmarks

[Simon Tournier]

Hi,

On Mon, 19 Jun 2023 at 14:54, Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> wrote:

> One of the criticism that can be read online about Guix (compared to
> Nix) is its speed. I have never tried Nix and probably won't in a near
> future, but I was wondering if some work has been made to compare them
> on basic tasks (package installation, removal...) (the reason why I ask
> is to be able to give an honest answer to someone hesitating between
> both).

I would find that very helpful.  For some command, I find Guix very
slow, especially when compared with Apt or Aptitude from Debian.  But
that is comparing Apple to Orange. :-)

For sure, “guix search” is very slow [1].

1: https://issues.guix.gnu.org/39258


Cheers,
simon