From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id ANJOAh3ILmaY3gAAe85BDQ:P1 (envelope-from ) for ; Mon, 29 Apr 2024 00:05:17 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id ANJOAh3ILmaY3gAAe85BDQ (envelope-from ) for ; Mon, 29 Apr 2024 00:05:17 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=AQHkT6xL; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1714341916; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=AB0aSjCjRLKl/piNrs9wyI4nJqdpC3fBD02dxcPerBY=; b=uX2eokI6urrSQity06CE2slxQsz9n7S2u701PwJNlNowp9o3BgtzCInBisuw1RvKG72hVB ngBb8D8j6r7EFoDQ51ZTBqNVVL+LjnfwiExs/kDyXmER8qMBoqw5QSxtRn1REsVb7SU8+I 8psZaxdE7O2XoLXi5rR3VLwI5nlseMudnB56KYRy0KLfefO/MQq+uLwNX7Fh6IwlMtU95i xGrSkj7HLRZuutSZmc+vx2ymhtL54aWnHtnWq31C/+kjawNNtyvRHMuuAlxIJZcq4ZBJ3I Wi15tszktmpHaxIsbf+qTSmCwLuTbcfW/BaV9afG7yP+r5BzDjzNWndX59ERJg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=AQHkT6xL; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Seal: i=1; s=key1; d=yhetil.org; t=1714341916; a=rsa-sha256; cv=none; b=ewA9h+ARE5rj24RUhbX5r5m9L3e8a4/iK2/Ir3Nk6VcUQs7DLolj88ChM7cnfieTGXfQiI FK6QWKcroStsshXLpjRGXo0Pw09rz6hmdSioLcYe2uVQz7SAHo7Lwj2bx75hxtXKq+tVRe sXiwTu1Oq3w2XZi18sIGPo62gouSm4YSo4hyDwL2u3PXy2OY2MdKAs48HaGNFx94KxgJMe mRXOapae5T1HybIiPb3VJtEDbGCJqphIOPmpmQ+8pxHBNhZgEWS1XG/AKi2rxMveKjuOR9 lGLzYFLCeMQ3JYS4jZOvTwXHuIMIRbSNSJWLhZKePrXV5kWVp6YcpxxoGlZsjw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5D7DE6BC68 for ; Mon, 29 Apr 2024 00:05:15 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s1CdX-0003cg-Cp; Sun, 28 Apr 2024 18:04:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s1CdW-0003cW-E7 for bug-guix@gnu.org; Sun, 28 Apr 2024 18:04:42 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s1CdW-0003Sk-55 for bug-guix@gnu.org; Sun, 28 Apr 2024 18:04:42 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1s1Cdp-0002hy-Lt for bug-guix@gnu.org; Sun, 28 Apr 2024 18:05:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#69596: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?publish=E2=80=99?= memory leak Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 28 Apr 2024 22:05:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 69596 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 69596@debbugs.gnu.org Received: via spool by 69596-submit@debbugs.gnu.org id=B69596.171434186810383 (code B ref 69596); Sun, 28 Apr 2024 22:05:01 +0000 Received: (at 69596) by debbugs.gnu.org; 28 Apr 2024 22:04:28 +0000 Received: from localhost ([127.0.0.1]:53129 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s1CdH-0002hO-L2 for submit@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42008) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s1CdF-0002hG-Ng for 69596@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s1Ccq-0003O6-Qj for 69596@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=AB0aSjCjRLKl/piNrs9wyI4nJqdpC3fBD02dxcPerBY=; b=AQHkT6xLgNar6O+8HlxH 655exyDUmvhqyDuXL+VgQgF1lpKAIsy18GSl0x25iUysOjW4rkuGZQaH/eRImsBk0WMze/M+osNOd vNYXPEyPFAlEysPqvDlC4NfcpEocq8m6bQ/GmZ+UJifPY0Nl/oNafis9cejIghTvgumv7lxS5HypT q9DtEGHTYOFxgFIkcemso34ThWC6Hv4LHSNKHXniEZG5yW2s3dseidpvDCE1UPhdspVxs1gN5Ajf5 2vx8hj9Yf2ZD0hwVWZ9nTajkZGNP9qmwn9J1QYg+NfcBURthLj1+/2W1XlbuptshK0/WQMKcngXaZ /g6TapgFPbwhoA==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: <87zfvb6nwi.fsf@inria.fr> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Wed, 06 Mar 2024 22:39:09 +0100") References: <87zfvb6nwi.fsf@inria.fr> Date: Mon, 29 Apr 2024 00:03:48 +0200 Message-ID: <87le4xgn5n.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam-Score: -7.09 X-Migadu-Queue-Id: 5D7DE6BC68 X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -7.09 X-TUID: j/IKOuoT8jIV Hi! Ludovic Court=C3=A8s skribis: > It seems that =E2=80=98guix publish=E2=80=99 has been leaking memory noti= ceably I=E2=80=99d say > since the beginning of the year on berlin. After roughly 10 days, it > has several GiB resident and needs to be restarted or it becomes too > unresponsive. > > I wonder what could be causing this because we used to let it run for > months without problems I believe. > > This is what=E2=80=99s currently running on berlin: > > /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libex= ec/guix/guile \ > /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/g= uix \ > publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=3Dnar \ > --listen=3Dlocalhost --workers=3D8 --ttl=3D15552000s \ > --cache=3D/var/cache/guix/publish --cache-bypass-threshold=3D157286400 > > =E2=80=A6 coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8. It turned out to be a guile-lzlib leak that had always been present: https://notabug.org/guile-lzlib/guile-lzlib/commit/74bd35b690801a10ed775d= 486fffc7372b1b341c The reason we were seeing it more on berlin is probably because we increased the cache-bypass-threshold, which goes through the =E2=80=98make-lzip-output-port=E2=80=99 code path (as opposed to =E2=80=98call-with-lzip-output-port=E2=80=99). The bug could be reproduced with: guix publish -p 8124 =E2=80=A6 & while true ; do wget -q -O/dev/full http://localhost:8124/nar/lzip/=E2=80= =A6-coreutils-9.1 ; done (Replace the ellipses with the actual store file name of coreutils.) Fixed by commit 7cef6b7ba555a9dfaf6d09cb7e112b0df77d5141, which updates guile-lzlib. Ludo=E2=80=99.