From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@gnu.org>
Subject: bug#38167: guix pull takes over 8 GiB of memory to finish if there
 are no substitutes
Date: Sat, 28 Dec 2019 15:52:40 +0100
Message-ID: <877e2gbiyv.fsf@gnu.org>
References: <20191111080635.14f7b39a@scratchpost.org>
 <87eeyec18l.fsf@cbaines.net> <878somno0f.fsf@gnu.org>
 <20191226204555.1abe4d21@scratchpost.org>
 <20191226204828.239470dd@scratchpost.org> <871rsphc5f.fsf@gnu.org>
 <20191228120128.010d72ee@scratchpost.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Return-path: <bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org>
Received: from eggs.gnu.org ([2001:470:142:3::10]:39883)
 by lists.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1ilDSh-0006Qs-J6
 for bug-guix@gnu.org; Sat, 28 Dec 2019 09:53:04 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1ilDSg-0004h9-G3
 for bug-guix@gnu.org; Sat, 28 Dec 2019 09:53:03 -0500
Received: from debbugs.gnu.org ([209.51.188.43]:52088)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <Debian-debbugs@debbugs.gnu.org>)
 id 1ilDSf-0004gj-U7
 for bug-guix@gnu.org; Sat, 28 Dec 2019 09:53:02 -0500
Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2)
 (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1ilDSf-00020I-T9
 for bug-guix@gnu.org; Sat, 28 Dec 2019 09:53:01 -0500
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
Resent-Message-ID: <handler.38167.B38167.15775447697680@debbugs.gnu.org>
In-Reply-To: <20191228120128.010d72ee@scratchpost.org> (Danny Milosavljevic's
 message of "Sat, 28 Dec 2019 12:01:28 +0100")
List-Id: Bug reports for GNU Guix <bug-guix.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/bug-guix>,
 <mailto:bug-guix-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/bug-guix>
List-Post: <mailto:bug-guix@gnu.org>
List-Help: <mailto:bug-guix-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/bug-guix>,
 <mailto:bug-guix-request@gnu.org?subject=subscribe>
Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org
Sender: "bug-Guix" <bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org>
To: Danny Milosavljevic <dannym@scratchpost.org>
Cc: 38167@debbugs.gnu.org

Hi,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Fri, 27 Dec 2019 19:11:08 +0100
> Ludovic Court=C3=A8s <ludo@gnu.org> wrote:
>
>> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>>=20
>> > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMA=
ND
>> > dannym   19221 20.8 87.4 9404812 6884184 pts/0 Tl   20:34   2:40 /gnu/=
store/sc7z07gim1iq5zvfz1amdwf2irxrzifg-guile-2.2.6/bin/guile --no-auto-comp=
ile /home/dannym/.config/guix/current/bin/guix pull=20=20
>>=20
>> Oh, that=E2=80=99s an RSS of 6=C2=A0GiB for =E2=80=98guix pull=E2=80=99 =
itself?  Weird, I don=E2=80=99t see
>> how that can happen.
>>=20
>> Could it be that ~/.cache/guix/checkouts/THE-THING is too big, which in
>> turn causes libgit2 to consume too much memory somehow?
>
> $ du -hs ~/.cache/guix/checkouts/
> 395M    /home/dannym/.cache/guix/checkouts/

OK, nothing bad.

>> What happens if you attach strace to this process at the moment where
>> it=E2=80=99s consuming a lot of memory?  Is it traversing Git pack files=
 or
>> something like that?
>
> Right now the process is in a paused state and I think when I attach stra=
ce
> it will continue.  Should I still do it?  I don't want to destroy our deb=
ugging
> opportunity.

I think you should resume it (SIGCONT), attach strace, and grab a few
seconds of log so we have an idea of what it=E2=80=99s doing.  You can also
check /proc/PID/fd and keep a copy of /proc/PID/maps while we=E2=80=99re at=
 it.

Thanks!

Ludo=E2=80=99.