From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Dmitrii Korobeinikov Newsgroups: gmane.emacs.devel Subject: Re: Garbage collector: is 800kb a good default? Date: Thu, 9 Apr 2020 20:20:20 +0600 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="82065"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Apr 09 16:21:12 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jMY3L-000LAg-Pq for ged-emacs-devel@m.gmane-mx.org; Thu, 09 Apr 2020 16:21:11 +0200 Original-Received: from localhost ([::1]:50678 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jMY3K-0007rX-Qz for ged-emacs-devel@m.gmane-mx.org; Thu, 09 Apr 2020 10:21:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:39228) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jMY2l-0007E3-C1 for emacs-devel@gnu.org; Thu, 09 Apr 2020 10:20:36 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jMY2j-0002GA-UP for emacs-devel@gnu.org; Thu, 09 Apr 2020 10:20:35 -0400 Original-Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:35346) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1jMY2j-0002Es-M3 for emacs-devel@gnu.org; Thu, 09 Apr 2020 10:20:33 -0400 Original-Received: by mail-wr1-x434.google.com with SMTP id g3so12166607wrx.2 for ; Thu, 09 Apr 2020 07:20:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=l7AT+Z2fEi4cb+zOPHb7us/6ky/cCP+lLS78NtI5gns=; b=cxXATnZMSkJhiJcEMBFWrGYk8S5YjwQsldBble8l4nKBSzx97Ubz9ffaftQGKo765W 62AGKpS+KD9FNMZe2neZ6qtUOQ/l6e/ehG8Y0E5TmjBYPWqsTqfA4IwV1DIFT2BOieAy 8MAjD4/PwQ05DTX9GU/KUBbm/WopM7Z7rPmhw+rYThrqakX4h3hpWkxIlG5VBPHNnuyA WmPWROohqUtdf9P9ug4F9CXxD+vi+QOTqeyfS7RmbVWMuozSh7Kqrug6dNLjcqQNH4WC 2+CHQdEQuWyQY8ZaC04MkbMDwnXGJnQUlmdksE1u0aA3A49k3tk/5xLU7/74dqZoiZKd VTNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=l7AT+Z2fEi4cb+zOPHb7us/6ky/cCP+lLS78NtI5gns=; b=ewDDagoTKTdoU4EI6mfVaDgtC+9SHlGe6iD+rYyDYP61i9NuY/P7Qv8LbZNYVaGDJr 4NeA38gKtMrfQtBMj1cNK1OwEo362aTH24BYa+gN4CIoVRVcAYu180GhIZaUZpSrNFgI 3BfWGui9I01byZ4UW4Zz+9FlLTK7/Q+WSPOP5EkRdoXCGAi3dFo9pl9r6qlIGXEJM9uR T/H9SIZfmcAfBUotlIgcf0tgCYSb6u4bg9dCDjPNUlMyVVA1qudlIM0DGcBazfziAf8F 7MA2egOtF+m43X6h2LaseKjbuOZim2iNuQda6tYmJ4UNJ/qCsw0X1mhwgovqNLxyyis2 BXlQ== X-Gm-Message-State: AGi0PuZ1VrlSY9qsQZeDavS1cv80MnviQn7dcdSdxfTOLpRDAytuGxBC RTo15GQLeqYd0HXj1cgGftSqmCb/gZ7/a6Nu9Oi5ggb+ X-Google-Smtp-Source: APiQypLTmZ7qJfp7yX7CDU8UTWqx/DmXJ2ysk4bESKXRwVPOXml/tcnNXgzqbE/usJOtEjYaevWS22RcK7Mwt4pxb5s= X-Received: by 2002:adf:e986:: with SMTP id h6mr3559955wrm.256.1586442031998; Thu, 09 Apr 2020 07:20:31 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::434 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:246713 Archived-At: > Running GC frequently is done largely to try and keep total heap size > and memory fragmentation under control. So completely refraining from > running GC while non-idle is probably not a great solution, but I do > agree that adding a "GC when idle" would not only be great in itself but > would also let us increase the default thresholds somewhat. Tho I don't > think we can increase the threshold as much as you suggest without > having significant detrimental effects. So, are there any stats on how much worse the fragmentation gets w/ higher thresholds and how much it effects emacs as a result? (By the way, I am not really suggesting 80MB, just testing it out at the moment : ) ) > 2- 38% is pretty high Just for the reference, I've seen the value of 25 in one of the threads nearby [1], not sure why mine is so high. Best, DK [1] https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg00406.html =D1=87=D1=82, 9 =D0=B0=D0=BF=D1=80. 2020 =D0=B3. =D0=B2 19:47, Stefan Monni= er : > > > Of course, raising the threshold significantly higher on its own is > > not a very good idea. But if paired with an idle timer like suggested > > here [2], then it all starts looking like a decent combination: > > I agree that it's worth investigating improvements based on dynamically > changing the GC threshold. What you're pointing out is that from the > user's point of view, a GC during idle time is free (so it can occur > frequently) and also that a GC during non-idle time can delay redisplay, > so (ignoring all other impacts of GC) we should refrain from running GC > while non-idle and trigger GC everything we're idle. > > Running GC frequently is done largely to try and keep total heap size > and memory fragmentation under control. So completely refraining from > running GC while non-idle is probably not a great solution, but I do > agree that adding a "GC when idle" would not only be great in itself but > would also let us increase the default thresholds somewhat. Tho I don't > think we can increase the threshold as much as you suggest without > having significant detrimental effects. > > There 2 additional ways to attack the problem, BTW: > > 1- replace our GC with a concurrent GC. This would let us move those > 38% GC overheard to one of the other CPU cores sitting idle while the > only CPU core running Emacs is frantically trying to scroll through > your buffer (it would also slow down both the GC and the main thread > a bit, but in the current context of plentiful CPU cores it would > still be very worthwhile). > [ Along the same lines, making our GC parallel could cut those 38% > down to some extent, as would a generational GC. ] > > 2- 38% is pretty high. Usually this indicates that the GC is not > efficient: it works hard yet reclaims very little memory likely > because the application is allocating a lot of objects which are > *not* temporary. This is also the typical situation during Emacs's > startup where we're loading packages and initializing big > data-structures, so after allocating our GC-threshold of data the GC > is run but doesn't collect much garbage because all that data is > there to stay. > We can't really know beforehand if a GC will reclaim a lot of memory, > but we could "use the past to predict the future": we could set the > threshold higher when the last GC reclaimed too little garbage. > > I had sent a tentative patch for the "GC when idle" feature but it had > some rough edges, and some details of the GC code have changed since. > I'd welcome a patch to do that. It shouldn't require many changes. > Similarly, point (2) above should be fairly simple to add. > > > Stefan >