unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: 44053@debbugs.gnu.org
Subject: bug#44053: Poor profile generation performance on spinning disks
Date: Sat, 17 Oct 2020 23:35:45 -0400	[thread overview]
Message-ID: <87zh4k435a.fsf@gmail.com> (raw)

Hello!

I've noticed on multiple occasions that using Guix on traditional
spinning drives can be quite slow.

On my home machine, will is still relying on 2 x 1 TB spinning drives in
RAID1, rebuilding my user profile, which contains 182 entries, takes on
average about 20 minutes, even when there are no packages to be built:

--8<---------------cut here---------------start------------->8--- $ time
guix package -i perl --max-jobs=1 The following package will be
upgraded: perl (dependencies or package changed)

The following derivation will be built:
   /gnu/store/lhywla1z2zcz16df4hbvvvngr9zmswr7-profile.drv

building CA certificate bundle...
building fonts directory...
generating GLib schema cache...
creating GTK+ icon theme cache...
building cache files for GTK+ input methods...
building directory of Info manuals...
building database for manual pages...
building XDG desktop file cache...
building XDG MIME database...
building profile with 182 packages...

real    19m0.126s
user    0m5.648s
sys     0m0.333s
--8<---------------cut here---------------end--------------->8---

Most of the time remains spent after the message 'building profile with
182 package...'.  That part seems IO-bound, with the spinning disks
grinding heavily and the CPU mostly idling.  The rest of the time (3
minutes), was used by the profile hooks.

The same operation on a second, more modern machine equipped with M2
SSDs does much better and takes about 1 minute to accomplish the same,
so it seems the bad performance can be mostly attributed to the much
slower disk seek times of the spinning disks.

On the older machine, two profile hooks are also sticking out w.r.t. the
time they take (they take more than one minute opposed to a few
seconds):

--8<---------------cut here---------------start------------->8---
The following profile hook will be built:
   /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
building database for manual pages...
Creating manual page database...
[2139/2139] building list of man-db entries...
175322 entries processed in 95.1 s
successfully built /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
successfully built /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
/gnu/store/wzp4mk2r7r4ysciw74gqbfkyai0zmrcc-manual-database

real    1m36.378s
user    0m1.674s
sys     0m0.108s

The following profile hook will be built:
   /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
building XDG MIME database...
successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
/gnu/store/j0bznlj2ibnhirijhnwpkkxzz4qfk8wb-xdg-mime-database

real    1m7.344s
user    0m1.331s
sys     0m0.053s
--8<---------------cut here---------------end--------------->8---

So we should profile what's going on while generating the profile (no
pun intended) and try to improve this at first since this is where most
of the time is spent on spinning drives (17 minutes out of the 20 in the
above example).

After that we could look into the two above profile hooks.

Thanks,

Maxim




             reply	other threads:[~2020-10-18  3:37 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-18  3:35 Maxim Cournoyer [this message]
2020-10-18 23:05 ` bug#44053: Poor profile generation performance on spinning disks Luis Felipe via Bug reports for GNU Guix
2020-10-19  8:18 ` zimoun
2020-10-19 18:18   ` Maxim Cournoyer
2022-03-23 12:38 ` zimoun
2022-03-23 16:17   ` Maxim Cournoyer
2022-03-23 16:54     ` zimoun
2023-08-26 22:11 ` jbranso--- via Bug reports for GNU Guix
2023-08-27 17:59   ` Luis Felipe via Bug reports for GNU Guix
2023-08-29  9:19     ` Ludovic Courtès
2023-08-29 21:51       ` Luis Felipe via Bug reports for GNU Guix
2023-09-09 11:02         ` Ludovic Courtès

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87zh4k435a.fsf@gmail.com \
    --to=maxim.cournoyer@gmail.com \
    --cc=44053@debbugs.gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).