From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: MPS: out-of-memory Date: Mon, 08 Jul 2024 14:57:02 +0300 Message-ID: <865xtg14hd.fsf@gnu.org> References: <-plQctKgNkvp-LJ9ov2QAiXQKxd9V-hI0yz_opRGxQtbknubCjH4rH2-ymgbw_Qr1ZhB1rtlmiEW8XtuIVNr7nR_Yj20AH6WkH6kUGp68g0=@protonmail.com> <_mNcR6ailVKpYHLxgfo_tJlYGeR0AQIzQWluspYYp5_g5pIIKkHLNfFkklQQgOKNiVW8jn8NS3i2dJ7_B2Qyx9v-Dq3MQ9mP8HNL30UWsqY=@protonmail.com> <878qyf4sgm.fsf@gmail.com> <878qye3l81.fsf@gmail.com> <86a5iu4tiy.fsf@gnu.org> <87msmu1uy5.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7371"; mail-complaints-to="usenet@ciao.gmane.io" Cc: acorallo@gnu.org, eller.helmut@gmail.com, pipcet@protonmail.com, emacs-devel@gnu.org To: Gerd =?utf-8?Q?M=C3=B6llmann?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Jul 08 13:58:25 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sQn0j-0001b4-4D for ged-emacs-devel@m.gmane-mx.org; Mon, 08 Jul 2024 13:58:25 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sQmzw-0000i6-Ts; Mon, 08 Jul 2024 07:57:37 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sQmzp-0000JI-EH for emacs-devel@gnu.org; Mon, 08 Jul 2024 07:57:30 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sQmzo-0002ar-A6; Mon, 08 Jul 2024 07:57:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=Kul9ljZtde69R1jMSPI7TVLG/evCrYCpfuZ3In+J+OM=; b=NZSGBayjWWaWYkKtM0Jh EWCsgLeu08plkZzn41PAp82ByGu6+BGkTleQs2E535dtP7Ihri2E5tUV0rRrKF1dhLd5cZ0Pp4rRI TWsxr+GMb4RQCZKV2ADoWmreZUt8J0aVgXr/J2hEFrR4gd2BUY+NVm6efjFhHB16EkRWfuwN52aGN PVllpKMs6xfMFcGwPwrpzYfvcZ0WdR8BWxWdu9+bqpmLxn187apl30sjIi9nHCkRnR4/Z66xeZ9pV US35xr2gnSP4w4VoAwHx5vHLqsEs9wpe5tt8YuCNHEvsar6F3y6kz++33gsAqYC/lgAY0SyuFd45O ZprUauK/ykRElg==; In-Reply-To: (message from Gerd =?utf-8?Q?M=C3=B6llmann?= on Mon, 08 Jul 2024 11:24:25 +0200) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:321538 Archived-At: [I've changed the Subject, since this ceased to be about weak hash tables long ago.] > From: Gerd Möllmann > Cc: Helmut Eller , Eli Zaretskii , > pipcet@protonmail.com, emacs-devel@gnu.org > Date: Mon, 08 Jul 2024 11:24:25 +0200 > > Andrea Corallo writes: > > >> ./src/emacs -Q -batch -l \ > >> ~/.emacs.d/elpa/elisp-benchmarks-1.14/elisp-benchmarks.el --eval \ > >> '(elisp-benchmarks-run "pidigits" nil 1)' > >> > >> finishes and requires 41.82 seconds compared to the 12.26 of the non-MPS > >> version. And it uses something like 2.8 GB compared to 350 MB of the > >> non-MPS version (estimated by looking at the RES column in top). > > > > Such a difference in memory footprint is because MPS can't GC at the > > speed the main thread is allocating or for some other reason? > > Yes, that's it basically. The client is allocating at a high rate, > and not much is dying until the end of that phase. Both together mean > that MPS is scanning like wild to make some room, without having a > chance to find something it can discard. Btw, the current GC in Emacs has a feature for when Emacs runs out of memory: it sets some small amount of memory aside, and uses it when "memory-full" condition happens, because some memory is needed to show the memory-full error message, and then shut down in an orderly fashion: void memory_full (size_t nbytes) { if (!initialized) fatal ("memory exhausted"); /* Do not go into hysterics merely because a large request failed. */ bool enough_free_memory = false; if (SPARE_MEMORY < nbytes) { void *p; MALLOC_BLOCK_INPUT; p = malloc (SPARE_MEMORY); if (p) { free (p); enough_free_memory = true; } MALLOC_UNBLOCK_INPUT; } if (! enough_free_memory) { Vmemory_full = Qt; consing_until_gc = min (consing_until_gc, memory_full_cons_threshold); /* The first time we get here, free the spare memory. */ for (int i = 0; i < ARRAYELTS (spare_memory); i++) if (spare_memory[i]) { if (i == 0) free (spare_memory[i]); else if (i >= 1 && i <= 4) lisp_align_free (spare_memory[i]); else lisp_free (spare_memory[i]); spare_memory[i] = 0; } } /* This used to call error, but if we've run out of memory, we could get infinite recursion trying to build the string. */ xsignal (Qnil, Vmemory_signal_data); } See also refill_memory_reserve. Do we perhaps need something like that with MPS?