From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id OM33KsVYymCzAAAAgWs5BA (envelope-from ) for ; Wed, 16 Jun 2021 22:02:13 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id oHLBJsVYymBiWQAA1q6Kng (envelope-from ) for ; Wed, 16 Jun 2021 20:02:13 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0F0042793E for ; Wed, 16 Jun 2021 22:02:13 +0200 (CEST) Received: from localhost ([::1]:34284 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ltbjm-0001cq-D5 for larch@yhetil.org; Wed, 16 Jun 2021 16:02:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39098) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ltbje-0001bP-Op for bug-guix@gnu.org; Wed, 16 Jun 2021 16:02:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:41845) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ltbje-0006ah-5p for bug-guix@gnu.org; Wed, 16 Jun 2021 16:02:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ltbje-00034d-30 for bug-guix@gnu.org; Wed, 16 Jun 2021 16:02:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#48963: "guix build guix" halts after exhausting memory Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 16 Jun 2021 20:02:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48963 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Julien Lepiller Received: via spool by 48963-submit@debbugs.gnu.org id=B48963.162387369111777 (code B ref 48963); Wed, 16 Jun 2021 20:02:02 +0000 Received: (at 48963) by debbugs.gnu.org; 16 Jun 2021 20:01:31 +0000 Received: from localhost ([127.0.0.1]:53391 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltbj9-00033s-7h for submit@debbugs.gnu.org; Wed, 16 Jun 2021 16:01:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:41074) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltbj8-00033c-2J for 48963@debbugs.gnu.org; Wed, 16 Jun 2021 16:01:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60172) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ltbj1-0006Bd-Gk; Wed, 16 Jun 2021 16:01:23 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=35826 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ltbj0-0004Lw-5v; Wed, 16 Jun 2021 16:01:23 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> Date: Wed, 16 Jun 2021 22:01:20 +0200 In-Reply-To: <20210612002330.64cd06c1@tachikoma.lepiller.eu> (Julien Lepiller's message of "Sat, 12 Jun 2021 00:23:30 +0200") Message-ID: <87im2dpoj3.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 48963@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1623873733; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post; bh=3ecP8NUm37gU75CecgQAxeyVm8kYpQ/c/YICPESa098=; b=d1BJ2kLk/PuA+Jmp3/jAwFabXtNYZDqHt3rs7rzi1NFKVKi8TjGxcgXg6C04ZRi9q2GwrU MIfjddwCLLlD/HVn/35oFC8phSrRgq52aFYSYcsEzsjuXRmHuXUJLfaA3qGXAMP+y6k55U 9x6qrlDkkn5wByf690iDMpvzrZOJl/0ZyEEViQak1XwjRDBwoSRzXd+AkF1/pOadoP9XR+ sb7auF+WQt9Y+hXnY0qt2FTfygSXkWlUGBGrj2rotwSTW1hBlp6jRsucDYF9VsPvNOSsPA LFPIOWY/uXo/1rAX4raH4C6GRjENVOWxuZivtOsOyEu3EsMx2p4ARqwSYAql3g== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1623873733; a=rsa-sha256; cv=none; b=YucsywzkqKGaL1XPnS9IU7+rN9PYRE538LZywnkl9+OdJEdM8mLY4BfvPpdDX2oX3AQZlL sn3/Eh2RuzkwD8NLs9Be7aeg5PcIRNxPJZ88P2prrdwkHjIKW8HIht3dLzYPypZlg1dSdH EKzQ6kxyC5RhQA5jwtpzU1HF80Y/Q0wNx+QzLUCadzLyL0SocN/f6y+e7aEoA9AoR07+wo xPSDZt7Z5PCwRVed4SCsZRBGJcqAdw3668layyuz/O0ntLAxJJ411PlY2qNbqDr0w6dOJ7 AT6xixB6GqHhcVLb5GRMHMvdbBbM6SbYP226wsBnhGbpfilNhRAdinOJpJsjbg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -2.92 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 0F0042793E X-Spam-Score: -2.92 X-Migadu-Scanner: scn1.migadu.com X-TUID: yuIFzb33giZu --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Julien Lepiller skribis: > I tried updating my system on my armhf board (2GB of RAM), but during > "guix system reconfigure", guix tries to build itself (the guix package > from (gnu packages package-management)). This package uses too much > memory to build, and I start getting GC warnings like so: > > GC Warning: Out of memory - trying to allocate requested amount (552 > bytes)... > GC Warning: Header allocation failed: dropping block > GC Warning: Out of Memory! Heap size: 2571 MiB. Returning NULL! > Warning: Unwind-only out of memory exception; skipping pre-unwind > handler. > > after some more progress and a lot more warnings, the build stops (CPU > is at 2%), but the memory is not freed. The build stayed stuck for a > long time until I decided to cancel it. This is ridiculous. :-/ > Can we instead break the build (at the Makefile level) into multiple > smaller chunks, that require less memory, in the same way (guix self) > works? Yes, that=E2=80=99s a good idea. Could you check the extent to which the attached patch helps on this machine? It doesn=E2=80=99t split as much as (guix self) does, only in three pieces,= but hopefully that helps a bit. A side effect is that progress report is now off, but we can fix it later. Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/Makefile.am b/Makefile.am index aa21b5383b..758d8b9b8a 100644 --- a/Makefile.am +++ b/Makefile.am @@ -663,7 +663,11 @@ CLEANFILES = \ # the whole thing. Likewise, set 'XDG_CACHE_HOME' to avoid loading possibly # stale files from ~/.cache/guile/ccache. %.go: make-go ; @: -make-go: $(MODULES) guix/config.scm $(dist_noinst_DATA) +make-go: make-core-go make-packages-go make-system-go + +define guile-compilation-rule + +$(1): $(2) $(AM_V_at)echo "Compiling Scheme modules..." ; \ unset GUILE_LOAD_COMPILED_PATH ; \ XDG_CACHE_HOME=/nowhere \ @@ -671,7 +675,19 @@ make-go: $(MODULES) guix/config.scm $(dist_noinst_DATA) $(top_builddir)/pre-inst-env \ $(GUILE) -L "$(top_builddir)" -L "$(top_srcdir)" \ --no-auto-compile \ - -s "$(top_srcdir)"/build-aux/compile-all.scm $^ + -s "$(top_srcdir)"/build-aux/compile-all.scm $$(filter %.scm,$$^) + +.PHONY: $(1) + +endef + +MODULES_CORE = $(filter guix/%,$(MODULES)) +MODULES_PACKAGES = $(filter gnu/packages/%,$(MODULES)) +MODULES_SYSTEM = $(filter-out gnu/packages/%,$(filter gnu/%,$(MODULES))) + +$(eval $(call guile-compilation-rule,make-core-go,$(MODULES_CORE) guix/config.scm $(dist_noinst_DATA))) +$(eval $(call guile-compilation-rule,make-packages-go,$(MODULES_PACKAGES) make-core-go)) +$(eval $(call guile-compilation-rule,make-system-go,$(MODULES_SYSTEM) make-packages-go make-core-go)) SUFFIXES = .go --=-=-=--