From: "Ludovic Courtès" <ludo@gnu.org>
To: Sarah Morgensen <iskarian@mgsn.dev>
Cc: Ricardo Wurmus <rekado@elephly.net>, 49439@debbugs.gnu.org
Subject: bug#49439: grafts cause “guix environment” to get killed with OOM
Date: Thu, 29 Jul 2021 00:03:46 +0200 [thread overview]
Message-ID: <87wnpa14fx.fsf@gnu.org> (raw)
In-Reply-To: <87k0la3ghk.fsf@gnu.org> ("Ludovic Courtès"'s message of "Wed, 28 Jul 2021 12:00:39 +0200")
Hi,
Ludovic Courtès <ludo@gnu.org> skribis:
> Thinking about it, the grafts code depends on what’s in the store: when
> nothing is in the store, it bounces to the “build handler”, which
> accumulates the list of missing store items, until it starts building
> them.
So I can reproduce the problem Ricardo and you initially reported by
running:
./pre-inst-env guix environment pigx-scrnaseq --search-paths
after removing some of the ungrafted store items with:
guix gc -D $(guix build r-rlang --no-grafts) \
$(guix gc --references $(guix build pigx-scrnaseq --no-grafts))
The seemingly endless CPU usage and unbound memory use comes from the
‘build-accumulator’ build handler. Here, the graph of ‘pigx-scrnaseq’
has many nodes, and many of them depend on, say, ‘r-rlang’. Thus, when
‘r-rlang’ is not in the store, the grafting code keeps asking for it by
calling ‘build-derivations’, which aborts to the build handler; the
build handler saves the .drv file name and the continuation and keeps
going. But since the next package also depends on ‘r-langr’, we abort
again to the build handler, and so on.
The end result is a very long list of <unresolved> nodes, probably of
this order in this case:
$ guix graph -t reverse-package r-rlang |grep 'label = "'|wc -l
594
Presumably, the captured continuations occupy quite a bit of memory,
hence the quick growth.
I suppose one solution is to fire suspended builds when the build
handler sees a build request for a given derivation for the second time.
It needs more thought and testing…
Ludo’.
PS: Did you know ‘pigx-scrnaseq’ has twice as many nodes as
‘libreoffice’?
$ guix graph -t bag pigx-scrnaseq |grep 'label = "'|wc -l
1359
$ guix graph -t bag libreoffice |grep 'label = "'|wc -l
699
That makes it a great example to study and fix scalability issues!
next prev parent reply other threads:[~2021-07-28 22:04 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-06 14:38 bug#49439: grafts cause “guix environment” to get killed with OOM Ricardo Wurmus
2021-07-23 4:59 ` Sarah Morgensen
2021-07-27 14:52 ` Julien Lepiller
2021-07-27 16:28 ` Ludovic Courtès
2021-07-27 23:35 ` Sarah Morgensen
2021-07-28 10:00 ` Ludovic Courtès
2021-07-28 22:03 ` Ludovic Courtès [this message]
2021-07-29 3:20 ` Sarah Morgensen
2021-08-09 21:26 ` Ludovic Courtès
2021-08-10 15:36 ` Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87wnpa14fx.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=49439@debbugs.gnu.org \
--cc=iskarian@mgsn.dev \
--cc=rekado@elephly.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.