From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Garbage collector: is 800kb a good default? Date: Thu, 09 Apr 2020 09:47:38 -0400 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="63945"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: emacs-devel To: Dmitrii Korobeinikov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Apr 09 15:48:26 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jMXXe-000GZM-3c for ged-emacs-devel@m.gmane-mx.org; Thu, 09 Apr 2020 15:48:26 +0200 Original-Received: from localhost ([::1]:49634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jMXXd-0008Hf-5D for ged-emacs-devel@m.gmane-mx.org; Thu, 09 Apr 2020 09:48:25 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:60814) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jMXWx-0007eZ-S0 for emacs-devel@gnu.org; Thu, 09 Apr 2020 09:47:44 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jMXWw-0001Qu-Dc for emacs-devel@gnu.org; Thu, 09 Apr 2020 09:47:43 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:2060) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jMXWw-0001Qn-7O for emacs-devel@gnu.org; Thu, 09 Apr 2020 09:47:42 -0400 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 73F6E44FED6; Thu, 9 Apr 2020 09:47:41 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id B49D444FED0; Thu, 9 Apr 2020 09:47:39 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1586440059; bh=XigjHO3MYUl+cGNxbgHMNNmC+VOrMu4NaMIe91wLVjA=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=Hb/B6Is6nwSbwvo4yH0xGBq6wOmp1wiqqe/+hqFBRZvDxJOsvvZjjyJR1Yl1zfah7 1JnbTHfrkNr2S77eIItfjmMzSrv8PA5seZcZwq+318Yb73B9mTha994gjzpECnnFHT 0CJoVnBNYUeTa9qdvYMhVzDWCAwQlkNVR+iJUkMGxGy2turfjRC1ZpGs3kK9oAszTD ikaf/TnYQDV7GbSaWGEyDgmsaGmxn4WqIiIzwYEHSmn+qjanPTAh/LSnbyDV9eatHw KsW2LUUOYfOTB1msZnqlJ4zG2JIT3BL6HLb3GsURj5YXNySKiRmYngQUV91GRObl6o ERtGSnf80NYPw== Original-Received: from alfajor (unknown [104.247.241.114]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 4020312089E; Thu, 9 Apr 2020 09:47:39 -0400 (EDT) In-Reply-To: (Dmitrii Korobeinikov's message of "Thu, 9 Apr 2020 17:59:04 +0600") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 132.204.25.50 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:246710 Archived-At: > Of course, raising the threshold significantly higher on its own is > not a very good idea. But if paired with an idle timer like suggested > here [2], then it all starts looking like a decent combination: I agree that it's worth investigating improvements based on dynamically changing the GC threshold. What you're pointing out is that from the user's point of view, a GC during idle time is free (so it can occur frequently) and also that a GC during non-idle time can delay redisplay, so (ignoring all other impacts of GC) we should refrain from running GC while non-idle and trigger GC everything we're idle. Running GC frequently is done largely to try and keep total heap size and memory fragmentation under control. So completely refraining from running GC while non-idle is probably not a great solution, but I do agree that adding a "GC when idle" would not only be great in itself but would also let us increase the default thresholds somewhat. Tho I don't think we can increase the threshold as much as you suggest without having significant detrimental effects. There 2 additional ways to attack the problem, BTW: 1- replace our GC with a concurrent GC. This would let us move those 38% GC overheard to one of the other CPU cores sitting idle while the only CPU core running Emacs is frantically trying to scroll through your buffer (it would also slow down both the GC and the main thread a bit, but in the current context of plentiful CPU cores it would still be very worthwhile). [ Along the same lines, making our GC parallel could cut those 38% down to some extent, as would a generational GC. ] 2- 38% is pretty high. Usually this indicates that the GC is not efficient: it works hard yet reclaims very little memory likely because the application is allocating a lot of objects which are *not* temporary. This is also the typical situation during Emacs's startup where we're loading packages and initializing big data-structures, so after allocating our GC-threshold of data the GC is run but doesn't collect much garbage because all that data is there to stay. We can't really know beforehand if a GC will reclaim a lot of memory, but we could "use the past to predict the future": we could set the threshold higher when the last GC reclaimed too little garbage. I had sent a tentative patch for the "GC when idle" feature but it had some rough edges, and some details of the GC code have changed since. I'd welcome a patch to do that. It shouldn't require many changes. Similarly, point (2) above should be fairly simple to add. Stefan