From: Simon Tournier <zimon.toutoune@gmail.com>
To: Emmanuel Beffara <manu@beffara.org>, help-guix@gnu.org
Subject: Re: guix pull performance
Date: Fri, 01 Dec 2023 18:13:24 +0100 [thread overview]
Message-ID: <87il5h977v.fsf@gmail.com> (raw)
In-Reply-To: <20231128220725.GC7864@beffara.org>
Hi,
If I read correctly, your machine is:
a reasonably powerful machine (Core i7, 12 cores, 16G RAM, 1 TB SSD drive
with plenty of available space)
Correct?
On Tue, 28 Nov 2023 at 22:07, Emmanuel Beffara <manu@beffara.org> wrote:
> Now, even if I didn't expect a significant improvement, I am still badly
> surprised by the behaviour of Guix: one month ago, any `guix pull` took 10
> minutes to complete, and these days it takes 40 minutes (the same command, on
> the same machine in the same conditions). When running a second `guix pull`
> while the channels are unchanged, it takes "only" 6 minutes of computation
> before telling me that there is nothing to do.
>
> How is it possible that things are four times slower over just one month ?
> Did the package collection increase that much ?
> Were there structural changes that caused a loss of efficiency ?
Using my desktop machine (i7, 16 cores, 64G RAM), I get:
+ plain “guix pull”: ~10min
+ redo “guix pull” with the same revision: ~2min
--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 29 Oct 25 2023 14:42:15 (current)
guix aabfddb
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: aabfddbe8bc16fa82067e88e7d79c8c1bc802414
$ time guix pull -p /tmp/new
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Authenticating channel 'guix', commits 9edb3f6 to 842a11f (82 new commits)...
Building from this channel:
guix https://git.savannah.gnu.org/git/guix.git 842a11f
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
[...]
config.scm 507B 174KiB/s 00:00 ▕██████████████████▏ 100.0%
git.scm 105B 191KiB/s 00:00 ▕██████████████████▏ 100.0%
hash.scm 130B 105KiB/s 00:00 ▕██████████████████▏ 100.0%
module-import 2KiB 335KiB/s 00:00 ▕██████████████████▏ 100.0%
module-import 2KiB 504KiB/s 00:00 ▕██████████████████▏ 100.0%
module-import-compiled 1.2MiB 2.4MiB/s 00:01 ▕██████████████████▏ 100.0%
module-import-compiled 1.2MiB 3.2MiB/s 00:00 ▕██████████████████▏ 100.0%
building /gnu/store/g83w55vbr1r4p1jbxdxlrd00hsziqyvq-compute-guix-derivation.drv...
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
git-minimal-2.41.0 4.6MiB 4.9MiB/s 00:01 ▕██████████████████▏ 100.0%
guix-daemon-1.4.0-15.e0885fc 258KiB 702.4MiB/s 00:00 ▕████▍ ▏ 24.8%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
The following derivations will be built:
/gnu/store/02czx493wwvilvc8yc38insc94s0l5h7-profile.drv
/gnu/store/j2l9k04mp2wdnkvin6xfqxm9j45i1myd-guix-842a11f1c.drv
/gnu/store/1jf0d5ch5rpjgbbj7hbh7xwmzbvkp6cm-guix-misc.drv
/gnu/store/7q0mghfzrpd1kb0ckcc9hg2skigq2d4i-guix-command.drv
/gnu/store/lvnm196rxcn6d2smz76amj0xj3w73pnk-guix-module-union.drv
/gnu/store/yc9w06iifsb5l3xikkg7idc4jk9ldsw4-guix-842a11f1c-modules.drv
/gnu/store/2qqzddfp48inn38zsq2bi9if69aia1dk-guix-home-modules.drv
/gnu/store/fcrdfvhnm36x3vlh5j87mvl9frwwxxcj-guix-home.drv
/gnu/store/k2mardcbpcsivc9mygzdgx9xyvvxvq0p-guix-packages-base.drv
/gnu/store/vfbxyq50j7brlqyai7dr8npbgnivjlnb-guix-system.drv
/gnu/store/z9acy6mp2zbfvn9jzrqs70f4y7x1im3d-guix-packages.drv
/gnu/store/crwap92xzmvk3crd2nm3mxv2wfy0b6gg-guix-packages-modules.drv
/gnu/store/fnpq63yknym9wfg13wxfqfnc0mdd2ynh-guix-extra-modules.drv
/gnu/store/hbz4z9bq05z8kcmp2j686v76n45l2kfl-guix-system-modules.drv
/gnu/store/ivfpj6mbx12bizh8v4li6as2xd3sxw0l-guix-packages-base-modules.drv
/gnu/store/qs9bax2627scfrv9k5q2k8qa9qfri138-guix-cli-core-modules.drv
/gnu/store/6hb81r74zij9vswbh2zqyy9csifv53cm-guix-cli-core.drv
/gnu/store/qvf2wkgs2jlqkk983jy7qa7wv9mf4mc2-guix-cli-modules.drv
/gnu/store/mn5kjnwym7hikdgqkxz5v6q5icr2knp5-guix-cli.drv
/gnu/store/wjmzwngblydz65ylrh7kkgh0mq62lz0m-guix-config-modules.drv
/gnu/store/4d1ql296bygb0ph375316ny8i6d5m4va-guix-config-source.drv
/gnu/store/0lcmrv5mx312w0h80lrbjzbp25xhl8a3-config.scm.drv
/gnu/store/zvh0ixd433rmganyrcb1prcx4c9q5axb-guix-system-tests-modules.drv
/gnu/store/cbywx3457flfbf135kzn884ik802q2dz-guix-system-tests.drv
/gnu/store/qzhy5lyr56qppryvldghl22db5j66qp6-guix-manual.drv
/gnu/store/wg49p4988b3acgq8mcdrmc7p9gcd7i3y-guix-daemon.drv
/gnu/store/8jxl2mxyj7cqaqlq7rr4qxy0nrhj1mra-inferior-script.scm.drv
/gnu/store/zqpnys3yd7gnljv6nan936hzg7j81ygw-profile.drv
4.0 MB will be downloaded
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
guix-locale-guix 3.0MiB/s 00:00 | 486KiB transferred
git-minimal-2.41.0 4.6MiB 2.8MiB/s 00:02 ▕██████████████████▏ 100.0%
guix-daemon-1.4.0-15.e0885fc 258KiB 3.3MiB/s 00:00 ▕██████████████████▏ 100.0%
guix-config 4KiB 395KiB/s 00:00 ▕██████████████████▏ 100.0%
guix-daemon.cil 3KiB 4.9MiB/s 00:00 ▕██████████████████▏ 100.0%
guix-extra 2.8MiB 6.1MiB/s 00:00 ▕██████████████████▏ 100.0%
guix-translated-texinfo 1.6MiB 2.4MiB/s 00:01 ▕██████████████████▏ 100.0%
building /gnu/store/0lcmrv5mx312w0h80lrbjzbp25xhl8a3-config.scm.drv...
applying 9 grafts for gd-2.3.2 ...
applying 8 grafts for guile-avahi-0.4.1 ...
applying 19 grafts for graphviz-minimal-7.0.1 ...
applying 7 grafts for guile-ssh-0.16.3 ...
building /gnu/store/4d1ql296bygb0ph375316ny8i6d5m4va-guix-config-source.drv...
building /gnu/store/fnpq63yknym9wfg13wxfqfnc0mdd2ynh-guix-extra-modules.drv...
building /gnu/store/wjmzwngblydz65ylrh7kkgh0mq62lz0m-guix-config-modules.drv...
building /gnu/store/qzhy5lyr56qppryvldghl22db5j66qp6-guix-manual.drv...
building /gnu/store/1jf0d5ch5rpjgbbj7hbh7xwmzbvkp6cm-guix-misc.drv...
building /gnu/store/k2mardcbpcsivc9mygzdgx9xyvvxvq0p-guix-packages-base.drv...
building /gnu/store/6hb81r74zij9vswbh2zqyy9csifv53cm-guix-cli-core.drv...
building /gnu/store/ivfpj6mbx12bizh8v4li6as2xd3sxw0l-guix-packages-base-modules.drv...
building /gnu/store/qs9bax2627scfrv9k5q2k8qa9qfri138-guix-cli-core-modules.drv...
building /gnu/store/z9acy6mp2zbfvn9jzrqs70f4y7x1im3d-guix-packages.drv...
building /gnu/store/crwap92xzmvk3crd2nm3mxv2wfy0b6gg-guix-packages-modules.drv...
building /gnu/store/vfbxyq50j7brlqyai7dr8npbgnivjlnb-guix-system.drv...
building /gnu/store/fcrdfvhnm36x3vlh5j87mvl9frwwxxcj-guix-home.drv...
building /gnu/store/hbz4z9bq05z8kcmp2j686v76n45l2kfl-guix-system-modules.drv...
building /gnu/store/mn5kjnwym7hikdgqkxz5v6q5icr2knp5-guix-cli.drv...
building /gnu/store/2qqzddfp48inn38zsq2bi9if69aia1dk-guix-home-modules.drv...
building /gnu/store/qvf2wkgs2jlqkk983jy7qa7wv9mf4mc2-guix-cli-modules.drv...
building /gnu/store/cbywx3457flfbf135kzn884ik802q2dz-guix-system-tests.drv...
building /gnu/store/zvh0ixd433rmganyrcb1prcx4c9q5axb-guix-system-tests-modules.drv...
building /gnu/store/yc9w06iifsb5l3xikkg7idc4jk9ldsw4-guix-842a11f1c-modules.drv...
building /gnu/store/lvnm196rxcn6d2smz76amj0xj3w73pnk-guix-module-union.drv...
building /gnu/store/7q0mghfzrpd1kb0ckcc9hg2skigq2d4i-guix-command.drv...
building /gnu/store/wg49p4988b3acgq8mcdrmc7p9gcd7i3y-guix-daemon.drv...
building /gnu/store/j2l9k04mp2wdnkvin6xfqxm9j45i1myd-guix-842a11f1c.drv...
building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
building profile with 1 package...
building /gnu/store/8jxl2mxyj7cqaqlq7rr4qxy0nrhj1mra-inferior-script.scm.drv...
building package cache...
building profile with 1 package...
hint: Consider setting the necessary environment variables by running:
GUIX_PROFILE="/tmp/new"
. "$GUIX_PROFILE/etc/profile"
Alternately, see `guix package --search-paths -p "/tmp/new"'.
hint: After setting `PATH', run `hash guix' to make sure your shell refers to `/tmp/new/bin/guix'.
real 10m51.145s
user 2m57.646s
sys 0m1.010s
$ /tmp/new/bin/guix describe
Generation 1 Dec 01 2023 16:33:07 (current)
guix 842a11f
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 842a11f1caa1bb929c427722ad9d7b7c1ff65727
$ time /tmp/new/bin/guix pull --commit=842a11f1caa1bb929c427722ad9d7b7c1ff65727 -p /tmp/again
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Building from this channel:
guix https://git.savannah.gnu.org/git/guix.git 842a11f
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
[...]
building /gnu/store/k99k4dqavfbpk04bahnv9z36374mnjj1-module-import.drv...
module-import 2KiB 134KiB/s 00:00 ▕██████████████████▏ 100.0%
module-import-compiled 1.2MiB 4.3MiB/s 00:00 ▕██████████████████▏ 100.0%
module-import-compiled 1.2MiB 3.3MiB/s 00:00 ▕██████████████████▏ 100.0%
building /gnu/store/y8g46qfpg7yakglc40avby0grgxc2jb6-compute-guix-derivation.drv...
Computing Guix derivation for 'x86_64-linux'... /
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
The following derivations will be built:
/gnu/store/shrj2p15crrrsj15yd0yancxq0pwi9qi-profile.drv
/gnu/store/8gwgjd9rq23hpi9hzg05p4fd8lcnyd5y-guix-842a11f1c.drv
/gnu/store/7n5jbjagkqi38gmvf0sayfxvsw6r4imz-guix-command.drv
/gnu/store/82f2w8qgliw8db8aaal9hw4v7mgmzgqk-guix-module-union.drv
/gnu/store/f3zl4nnjdbfj9zl04q1cn7hcrpdgdqvz-guix-842a11f1c-modules.drv
/gnu/store/q16d608sr9nn2l4vdhkbr0b5fwf88238-guix-config-modules.drv
/gnu/store/dlk4i3q2x64p4xqi4w628pbx9xir0jha-guix-config.drv
/gnu/store/hnlw8r17x70wdph7lkmj3jqba9cqn9cg-guix-config-source.drv
/gnu/store/lcw7jwv1sav314b3pys380p4vyy9vr8c-config.scm.drv
/gnu/store/m2pba333pg2c1s0rszvzgrfvvqqzbghi-guix-daemon.drv
/gnu/store/906nmg79jw2jxqdb1sbw77wkm21ddmbd-profile.drv
/gnu/store/dmakvrb0nvpvf6g959q6zggygc88nj6c-inferior-script.scm.drv
building /gnu/store/lcw7jwv1sav314b3pys380p4vyy9vr8c-config.scm.drv...
building /gnu/store/hnlw8r17x70wdph7lkmj3jqba9cqn9cg-guix-config-source.drv...
building /gnu/store/dlk4i3q2x64p4xqi4w628pbx9xir0jha-guix-config.drv...
building /gnu/store/q16d608sr9nn2l4vdhkbr0b5fwf88238-guix-config-modules.drv...
building /gnu/store/f3zl4nnjdbfj9zl04q1cn7hcrpdgdqvz-guix-842a11f1c-modules.drv...
building /gnu/store/82f2w8qgliw8db8aaal9hw4v7mgmzgqk-guix-module-union.drv...
building /gnu/store/7n5jbjagkqi38gmvf0sayfxvsw6r4imz-guix-command.drv...
building /gnu/store/m2pba333pg2c1s0rszvzgrfvvqqzbghi-guix-daemon.drv...
building /gnu/store/8gwgjd9rq23hpi9hzg05p4fd8lcnyd5y-guix-842a11f1c.drv...
building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
building profile with 1 package...
building /gnu/store/dmakvrb0nvpvf6g959q6zggygc88nj6c-inferior-script.scm.drv...
building package cache...
building profile with 1 package...
hint: Consider setting the necessary environment variables by running:
GUIX_PROFILE="/tmp/again"
. "$GUIX_PROFILE/etc/profile"
Alternately, see `guix package --search-paths -p "/tmp/again"'.
hint: After setting `PATH', run `hash guix' to make sure your shell refers to `/tmp/again/bin/guix'.
real 2m5.273s
user 3m0.652s
sys 0m1.315s
$ time /tmp/new/bin/guix pull --commit=842a11f1caa1bb929c427722ad9d7b7c1ff65727 -p /tmp/again
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Building from this channel:
guix https://git.savannah.gnu.org/git/guix.git 842a11f
Computing Guix derivation for 'x86_64-linux'... \
nothing to be done
hint: After setting `PATH', run `hash guix' to make sure your shell refers to `/tmp/again/bin/guix'.
real 1m39.950s
user 2m57.294s
sys 0m1.195s
--8<---------------cut here---------------end--------------->8---
> Or might there actually be someting wrong with my setup, and in this case how
> could I diagnose it ?
Well, ~40min seems something wrong with your setup. Considering “guix
pull”, what is the part (from a quick look) that appears to you the
longest?
Cheers,
simon
prev parent reply other threads:[~2023-12-01 17:17 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-12 11:03 guix pull performance Emmanuel Beffara
2023-10-12 11:24 ` Ignas Lapėnas
2023-10-12 13:51 ` Emmanuel Beffara
2023-10-12 17:02 ` Simon Tournier
2023-11-28 21:07 ` Emmanuel Beffara
2023-12-01 17:13 ` Simon Tournier [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87il5h977v.fsf@gmail.com \
--to=zimon.toutoune@gmail.com \
--cc=help-guix@gnu.org \
--cc=manu@beffara.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).