From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Opportunistic GC Date: Thu, 11 Mar 2021 11:03:48 +0200 Message-ID: <83wnuem58r.fsf@gnu.org> References: <83a6rdso07.fsf@gnu.org> <83a6rcqy4v.fsf@gnu.org> <83a6raoi6r.fsf@gnu.org> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="23461"; mail-complaints-to="usenet@ciao.gmane.io" Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Pip Cet Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Mar 11 10:04:34 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lKHFC-0005tI-91 for ged-emacs-devel@m.gmane-mx.org; Thu, 11 Mar 2021 10:04:34 +0100 Original-Received: from localhost ([::1]:43712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lKHFB-0006wQ-5W for ged-emacs-devel@m.gmane-mx.org; Thu, 11 Mar 2021 04:04:33 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:49474) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lKHEV-0006Iy-Ih for emacs-devel@gnu.org; Thu, 11 Mar 2021 04:03:51 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:58072) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lKHEU-0007gh-VE; Thu, 11 Mar 2021 04:03:50 -0500 Original-Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:4599 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lKHER-0006xt-Nt; Thu, 11 Mar 2021 04:03:48 -0500 In-Reply-To: (message from Pip Cet on Thu, 11 Mar 2021 08:06:50 +0000) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:266320 Archived-At: > From: Pip Cet > Date: Thu, 11 Mar 2021 08:06:50 +0000 > Cc: Stefan Monnier , emacs-devel@gnu.org > > > I don't see how that helps to get out of the conundrum. The > > synchronous GC doesn't enlarge the memory footprint of the Emacs > > process, so if we had enough memory when we started GC, there are good > > chances we will have it throughout the GC. By contrast, when you > > fork, you enlarge the memory pressure in the system, and the amount of > > the increment is unknown in advance (because it's determined by the > > degree of nesting of the Lisp data you have in the session at that > > moment). You could enlarge the memory enough to cause OOM killer to > > kick in. > > All of these problems apply to all forks, not just GC ones. You mean, when we start a sub-process? In that case the memory of the Emacs process is not cloned, because we immediately call 'exec'. But in the case of forking and running GC, some part of the Emacs memory will have to be cloned. Which part is that is hard to predict in advance; it could be a large part. > > So it's important to know when not to attempt the fork. > > I don't think "normal" GNU/Linux systems should ever be in that > situation, since they have virtual memory. I don't think I follow. Virtual memory can be exhausted as well, right? That's the danger I'm talking about. > And, of course, it's possible even on GNU/Linux system for the fork() > not to succeed, which we need to handle. The problem is with the scenario where the trouble begins long after the 'fork' call succeeded. Or are you saying that the OS will fail the 'fork' call right when we call it, and if it doesn't, there's no chance the forked child will come close to exhausting VM? > > What is the > > algorithm for that? If you want to use the fraction of the memory the > > Emacs process takes, and set the threshold at 50%, then we now need to > > discuss how to know that fraction reliably, what with some of the > > memory coming from sbrk and some from mmap -- this is why we all but > > eliminated memory-limit as not being very useful, and you now want to > > use the same technique to base a critical decision. > > Ideally, we'd use the same (broken) technique as the (broken) OS uses > to determine whether to kill us. Is that possible? Does the OS expose enough APIs to glean this info? If so, perhaps we could use that to resurrect the usefulness of memory-limit?