From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id EIPjJAFQZGfOPAEA62LTzQ:P1 (envelope-from ) for ; Thu, 19 Dec 2024 16:55:29 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id EIPjJAFQZGfOPAEA62LTzQ (envelope-from ) for ; Thu, 19 Dec 2024 17:55:29 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=FCRmNoDh; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=C9v9OpKm; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (strict), DKIM not aligned (strict)" header.from=posteo.net (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1734627329; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=CYYyAkcv04nK93uOATzvoNPk7bwkgrAw3Hwt5UKu5U0=; b=pimVYlpeZ8ROboRCWTIj/fxuurek25ZV5bRfg/WjjwYkpwHU5uNs1QjNfzwLTvcUa0ryP7 b0NsaWg+BE3jpjh8NLxlmtiANCMkTXs8HauC3Ez+C2bymCsC+scYv6IeqPwvlf/PHsqk8k Ze3YJ/MnoOBx863vaiYnC1ZcrjR2fbc8eWnkHHrahHTD9R9GfREjD6UaHCojw1R2FcjQPI pG4B1q6bi61TcL/dp7pm7hyXsORNoTcpCmu3Q3PrtYcp/H15l4g6jF8PL3dMwqY47LSGPv BPHj8/GjULJ1m6ZpDnYQ2WCOl6zaDECMotqeRQtVImOr4Xu3LXwf12DDTzqKQw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=FCRmNoDh; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=C9v9OpKm; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (strict), DKIM not aligned (strict)" header.from=posteo.net (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1734627329; a=rsa-sha256; cv=none; b=R5oR+eyzI66eHPctG2Cm8KG5fpuDwRjp95WjJWVJ4+oxovcHySQJ7mTtUCUxzNIFKdBSrm P1L07tnwHuAUJIpSOkKgLfv38rgcgmmIA+R8lnjgzxysjx1K0C9k7ShngUnBHNnKBN1O8e +WWD67HqpGRIkfQduucyhMReWSTVjbuiH7t6FSaSvBHpCurLcBDEAxPDAxXPsKtGIWGG+A DX9mGUzRh9lfra5eq2G/gKYNjpRYOQf8RE8wAO0Z/cZUvyTN1UwSy11penvrN9UaiUAdqb XbYVfetewHHgfTRynsWWKYgYkHtcbyfF5Z+MNWsDPrmPrwWeT3PkEpU4mkgT3Q== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 37B8E36E2 for ; Thu, 19 Dec 2024 17:55:29 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tOJnl-00061Q-An; Thu, 19 Dec 2024 11:55:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tOJnj-0005zD-4K for bug-guix@gnu.org; Thu, 19 Dec 2024 11:55:03 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tOJni-00022k-MY for bug-guix@gnu.org; Thu, 19 Dec 2024 11:55:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=From:MIME-Version:Date:To:In-Reply-To:References:Subject; bh=CYYyAkcv04nK93uOATzvoNPk7bwkgrAw3Hwt5UKu5U0=; b=FCRmNoDhCMnEmQp4HAMvcAhGPps2XTtujHLDGshkz4qCYYG2WsHmyFbA3UFfc8eXuOTj/943jgkSIAn/iGEbeWb9xloJVAaUMBl2Q/SqA/VPbc0NqXBnvbbyM020+lA2MRX/Uduz8bI4KzF+CfEbHqvOfxs8i82holn6e2HjpvVmaQs7jeoCDymay2cV4M+bPAbc1R99kqDk3Qjd6jwHLDMra9tzBO34/lf9u2wnivg7Xg5EdHGqz+Q/75XdjO3lK75w6HSS1+PYA6eeFPk9J02zOYW81hAsiMAgRoUBxaYds8jVepNnQrLry/i+ZYqqlC4Lyo0OGW0qRiflTgQKtA==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tOJni-0001ON-8B for bug-guix@gnu.org; Thu, 19 Dec 2024 11:55:02 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#74381: High memory usage during guix pull (i686-linux, guile jit) References: <3faad437-c87c-44d0-ad77-9e42418bf2dd@posteo.net> In-Reply-To: <3faad437-c87c-44d0-ad77-9e42418bf2dd@posteo.net> Resent-From: Dariqq Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 19 Dec 2024 16:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 74381 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 74381@debbugs.gnu.org Received: via spool by 74381-submit@debbugs.gnu.org id=B74381.17346272635291 (code B ref 74381); Thu, 19 Dec 2024 16:55:02 +0000 Received: (at 74381) by debbugs.gnu.org; 19 Dec 2024 16:54:23 +0000 Received: from localhost ([127.0.0.1]:39864 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tOJn4-0001NH-RX for submit@debbugs.gnu.org; Thu, 19 Dec 2024 11:54:23 -0500 Received: from mout01.posteo.de ([185.67.36.65]:51087) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tOJmz-0001Mw-Qk for 74381@debbugs.gnu.org; Thu, 19 Dec 2024 11:54:21 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id A5C63240027 for <74381@debbugs.gnu.org>; Thu, 19 Dec 2024 17:54:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1734627251; bh=dYoQ9O5VT60PtWyZm1bW9q6rE83vqFxWAXTCjUnmkGU=; h=Message-ID:Date:MIME-Version:To:Subject:From:Content-Type: Content-Transfer-Encoding:From; b=C9v9OpKmARpR6PaoIKKMyCakRtYO0kKJ85HEoWOlgpL0gVe0sa+N7hH/itpVeL+Bm d5m9CCfQPO57K97PMRvxBZ4kkI2U+XECbmcMRricJ2Z1oOurckMX5mpc4YwrS9lMu9 HcTzq+398OX2lf6l9QH0z2Yp+oyZpJaCMVAgJwRhUAp7wrAmopXK5C5c0an+w/IKYG xF1IN+pxGxo2hikNvVFTTw1xQygBy8uOXVUbxOnj4ZjNN+lphEVdp39weizeOrat89 CD87Tg4U4WW3PNFQa4wlgwaZe2n676/SmQAeouFdLMpsopYo0fUTqaRA//mxrsGuYL EFcP4glt3S/ww== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4YDc8L4PvPz9rxV for <74381@debbugs.gnu.org>; Thu, 19 Dec 2024 17:54:10 +0100 (CET) Message-ID: Date: Thu, 19 Dec 2024 16:54:04 +0000 MIME-Version: 1.0 Content-Language: en-US From: Dariqq Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -4.88 X-Spam-Score: -4.88 X-Migadu-Queue-Id: 37B8E36E2 X-TUID: s/6YIiwxaFXh It turns out disabling guile jit does not work reliably and even without it guix pull may oom building guix-packages-base using 3GB+ of RAM. My workaround for now is to run 'guix pull -s i686-linux -p /tmp/whatever' on an x86_64 machine (to have more RAM available) with guile jit disabled multiple times until it succeeds (not exactly sure why that works, is something getting cached somewhere?) and use that machine to serve substitutes. After mentioning this on irc today janneke mentioned this commit https://git.savannah.gnu.org/cgit/guix.git/commit/?id=15c5f1a2c20b21de0f19f42db1ccab4c42117ebb which reduces the number of files per chunk from 25 to 10 to get guix pull working on the 32 bit hurd. I tried reducing that even further to 5 in my local checkout and the max RAM usage went to (a little bit more reasonable) 1.5-2 GB (both on i686 natively and when emulating with -s i686 with and without guile jit). This would be still not enough for my little machine but might be enough for the substitute servers to build it reliably. I also wanted to try this in a i586-gnu childhurd but guix pull immediately segfaulted.