From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Opportunistic GC Date: Wed, 10 Mar 2021 22:41:16 +0200 Message-ID: <83a6raoi6r.fsf@gnu.org> References: <83a6rdso07.fsf@gnu.org> <83a6rcqy4v.fsf@gnu.org> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31156"; mail-complaints-to="usenet@ciao.gmane.io" Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Pip Cet Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Mar 10 21:50:09 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lK5mT-0007zi-ED for ged-emacs-devel@m.gmane-mx.org; Wed, 10 Mar 2021 21:50:09 +0100 Original-Received: from localhost ([::1]:44342 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lK5mS-0007sg-Fw for ged-emacs-devel@m.gmane-mx.org; Wed, 10 Mar 2021 15:50:08 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:38786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lK5e0-0006Rg-CH for emacs-devel@gnu.org; Wed, 10 Mar 2021 15:41:24 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:43741) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lK5dz-0004XK-Et; Wed, 10 Mar 2021 15:41:23 -0500 Original-Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2414 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lK5dv-0001Vy-HK; Wed, 10 Mar 2021 15:41:21 -0500 In-Reply-To: (message from Pip Cet on Wed, 10 Mar 2021 20:21:16 +0000) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:266305 Archived-At: > From: Pip Cet > Date: Wed, 10 Mar 2021 20:21:16 +0000 > Cc: Stefan Monnier , emacs-devel@gnu.org > > > > > What happens when, on a 8GB machine, we have an Emacs session with a > > > > 5GB memory footprint, and you fork and start marking (which triggers > > > > copy-on-write)? > > > > > > That is, indeed, one of the cases in which we have to use the > > > synchronous GC or suffer the consequences. > > > > How do we know if/when we are in that case? Because if we err, we > > have a meeting with the OOM killer and a swift death. > > Yes, but there's a good chance we would have had that anyway, since > it's "only" a factor of two, at most. I don't see how that helps to get out of the conundrum. The synchronous GC doesn't enlarge the memory footprint of the Emacs process, so if we had enough memory when we started GC, there are good chances we will have it throughout the GC. By contrast, when you fork, you enlarge the memory pressure in the system, and the amount of the increment is unknown in advance (because it's determined by the degree of nesting of the Lisp data you have in the session at that moment). You could enlarge the memory enough to cause OOM killer to kick in. So it's important to know when not to attempt the fork. What is the algorithm for that? If you want to use the fraction of the memory the Emacs process takes, and set the threshold at 50%, then we now need to discuss how to know that fraction reliably, what with some of the memory coming from sbrk and some from mmap -- this is why we all but eliminated memory-limit as not being very useful, and you now want to use the same technique to base a critical decision. > If you're arguing that there is a strong case that synchronous GC > should remain the default, I'm inclined to agree No, I'm not arguing that yet, not until I see the implementation. I'm just thinking aloud about some pesky problems we'd need to solve on the way. > > > > Also, I quite frequently need to run Emacs on a system where I'm > > > > forbidden to run more than 2 processes simultaneously ("make -j3" > > > > aborts with an error message), and you propose to take those two > > > > processes with 2 copies of Emacs? > > > > > > That is, indeed, one of the cases in which we have to use synchronous GC. > > > > Again, how do we know we are in that case? > > I assume you don't have a reliable getrlimit that lets you know? I don't know -- does getrlimit always report this stuff, with all the virtualization that goes on nowadays?