From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Subject: bug#27284: Memory leak in 'guix pull' or 'make' in guix source Date: Mon, 13 Nov 2017 10:28:50 +0100 Message-ID: <87vaiebk8d.fsf@gnu.org> References: <20170608083935.izw747zaetkaxv4o@abyayala> <87poamv2i7.fsf@gnu.org> <87375jchbu.fsf@gnu.org> <87shdituzk.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:33466) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEB3B-0007S3-9w for bug-guix@gnu.org; Mon, 13 Nov 2017 04:29:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEB38-00026P-4d for bug-guix@gnu.org; Mon, 13 Nov 2017 04:29:05 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:57243) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eEB38-00026H-0E for bug-guix@gnu.org; Mon, 13 Nov 2017 04:29:02 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87shdituzk.fsf@elephly.net> (Ricardo Wurmus's message of "Mon, 13 Nov 2017 09:59:11 +0100") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Ricardo Wurmus Cc: 27284@debbugs.gnu.org Hi! Ricardo Wurmus skribis: >> In the meantime, our best workaround to reduce memory consumption is=E2= =80=A6 to >> split large files into smaller ones. Per M-x guix-locations, the >> candidates are: >> >> gnu/packages/python.scm 986 >> gnu/packages/perl.scm 401 >> gnu/packages/haskell.scm 348 > > [=E2=80=A6] > >> I think we could focus on the first two or three files. FTR, compiling >> bioinformatics.scm peaks at ~500 MiB resident on x86_64. >> >> Ricardo, WDYT? > > I was hoping we could avoid this, but whatever: let=E2=80=99s do this :) Yeah, me too. The problem we have is that Guix is hardly releasable in its current state because on systems with 1=C2=A0GiB of memory you can=E2= =80=99t upgrade, and I think that=E2=80=99s unacceptable. So what are the options? If we get a bug-fix for Guile=E2=80=99s compiler today, does it help? If we graft it then we can deliver it without having to wait for a Guile release, which helps a bit? I think it=E2=80=99s all about time: we could wait (and hack!) some more, a= nd solve the root problem. This is the best long-term course of action, but at the same time it delays the Guix release. >> If we do this, do we split arbitrarily? Like the second half of >> python.scm goes to python-cont.scm (provided there are no cross >> top-level references)? Or do you have a better idea? > > Ultimately, I=E2=80=99d like to move packages to locations where they cou= ld > permanently live, but that would probably take longer. > > Would it make sense to move all the python2-* packages to a new module? I=E2=80=99m not sure we can do this, because that may lead to top-level references across these two modules, which is not OK. > This would make the split rather simple and users wouldn=E2=80=99t have to > remember which of their packages ended up in which half of the modules. > It also means that we probably won=E2=80=99t have to mess with the copyri= ght > headers. > > For perl.scm I have no good ideas. Let=E2=80=99s split it up at an arbit= rary > point. > > For haskell.scm I=E2=80=99d begin by moving the following packages away: > > - check.scm: ghc-tasty*, ghc-quickcheck*, ghc-test*, ghc-hunit*, hspec*, > ghc-hspec*, =E2=80=A6 > > - web.scm: ghc-tagsoup, ghc-cookie, ghc-http*, ghc-wai*, ghc-json, > ghc-warp*, ghc-multipart, ghc-aeson* > > - crypto.scm: ghc-tf-random, ghc-digest, ghc-cryptonite, ghc-x509*, > ghc-asn1*, ghc-pem, ghc-cryptohash*, ghc-entropy, ghc-crypto-api*, > ghc-puremd5 > > - tls.scm: ghc-tls > > Maybe that=E2=80=99s enough already. > > Does that seem like a good idea? It does. Actually, we could do similarly for Perl and Python: python-web, python-check, python-crypto, etc. WDYT? Thanks, Ludo=E2=80=99.