From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: raman@google.com (T.V Raman) Newsgroups: gmane.emacs.devel Subject: Re: Increase gc-cons-threshold during package-init? Date: Mon, 19 Feb 2018 16:49:07 -0800 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1519087663 5106 195.159.176.226 (20 Feb 2018 00:47:43 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 20 Feb 2018 00:47:43 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Feb 20 01:47:39 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1enw5Y-0008IU-66 for ged-emacs-devel@m.gmane.org; Tue, 20 Feb 2018 01:47:20 +0100 Original-Received: from localhost ([::1]:52978 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1enw7a-0005mJ-IA for ged-emacs-devel@m.gmane.org; Mon, 19 Feb 2018 19:49:26 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48751) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1enw7U-0005m3-Dl for emacs-devel@gnu.org; Mon, 19 Feb 2018 19:49:21 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1enw7Q-00007T-Dc for emacs-devel@gnu.org; Mon, 19 Feb 2018 19:49:20 -0500 Original-Received: from mail-pl0-x22a.google.com ([2607:f8b0:400e:c01::22a]:40415) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1enw7Q-000073-6z for emacs-devel@gnu.org; Mon, 19 Feb 2018 19:49:16 -0500 Original-Received: by mail-pl0-x22a.google.com with SMTP id g18so6532522plo.7 for ; Mon, 19 Feb 2018 16:49:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=Km3me7upBi3/mOThEun/phmG1tPnA+sHuT1CiAk3eBE=; b=IomZkV8CfHCUHsAKUkc5uWKNjeDVkcH4D1D8rCUNiXuCS/XpP0mI5697ktxiNLNWcX wIDYkaAzBTmnp+xe0ww/ICI9zkVP6DTltmmcRWBadsygoqy/rbBTMEokuN4mYmctJuZE Fuk+sPcr206GUsqBb0FnPSB8QKjS5djUi4aOBP/BJbWF2Yxid8oGCAJxY6o6dA+o+9po Qcz4pPIRDm3nR9EppBQ6oxRX+DcOwuHrtFI572EEdLVaD40Iv6sRq/L1XjXR8aR+MY0+ Ujw6U4mTc1WdyFYp6XYhYTo60LufQ0Syk/5RyI2YCWu9Lgyh0L6nzrLMLVP9xgqw3EZ0 eXwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=Km3me7upBi3/mOThEun/phmG1tPnA+sHuT1CiAk3eBE=; b=XuSKhhkDiCjcYDQ0IlQe1uPIf+A9n3ncrvhJHY17Wf+KlJhwedKj2Ha9Q207KOEf3x O8oZdDwdBsYDC3LTQgHjq+WRAT1OcH4mZFEAvJvGja3gcSJag6/JP9s53cDek4ZtWVNT balzfxSSNcmjxyHpPlKWMqn5bOEK1Yq7x+Gj0+NrdBow7k8/NjqStstoFktlLlMalF2c Ln5H/iFB2Rs8DHHgU+Ql8EgD0zqoWbBrOJNq4f9QM8gzI/iGpYPN4dzN8yxzH/XRUpP8 w47LPIOG8DKyvCVPfzUUd3Zq6hgOvVYBrdJ8AvghPux+eRTFKFtPZRCuKY8XjRnrk/md yY8Q== X-Gm-Message-State: APf1xPCX9ukSxdfEo/WpduAUzL3j7TDBHcu1LiDG2alvCdx8HO8TyoCG EnZKPHsIOTptoreQkZs1+4nWSlp+Kkk= X-Google-Smtp-Source: AH8x227QM8xlXsObQ85EMzji/VOwRSu+sIF1rdo6793Te4z1K4BsWWKVETX9ReI8l8h+nnn/jlpcgg== X-Received: by 2002:a17:902:5814:: with SMTP id m20-v6mr15767884pli.389.1519087752308; Mon, 19 Feb 2018 16:49:12 -0800 (PST) Original-Received: from raman-glaptop (c-24-4-174-65.hsd1.ca.comcast.net. [24.4.174.65]) by smtp.gmail.com with ESMTPSA id r28sm7106158pfl.9.2018.02.19.16.49.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 19 Feb 2018 16:49:07 -0800 (PST) In-Reply-To: (Stefan Monnier's message of "Mon, 19 Feb 2018 16:15:44 -0500") X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::22a X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:223484 Stefan Monnier writes: By Now I meant "earlier that package-initialize was happening in my startup file where the numbers were tweaked". Agree to the rest of what you say -- the right thing might be to tweak those during startup as a whole.>> With Emacs now calling package-initialize on its own, > > Not sure what you mean by "now": it's been the case since Emacs-24.1 > (recent changes on `master` have only moved the time at which it takes > place). > >> that emacs startup was now entailing 3 times as many gc calls during >> startup as before. One guess is that my init file was lexically >> binding gc-cons-threshold to 64MB --- > > Lexically-binding it would have no effect on the GC, and since that var > is declared as dynamically scoped, it was probably dynamically bound. > > Have you checked how much extra time is spent in the GC because of those > extra calls? > >> the default gc-cons-threshold of 800K is likely from the past and >> could be increased. > > This is a misunderstanding: the real gc-cons-threshold is computed > dynamically based on the heap size (see gc-cons-percentage). > > Maybe those values need to be tweaked because tradeoffs have shifted > somewhat, but it doesn't seem very likely. > > The problem with the above GC calls is much more fundamental and > unrelated to the actual values we choose: during periods of > initialization, programs typically allocate a lot of memory without > generating much garbage. So during those times running a GC tends to be > a waste of effort (we scan the whole heap only to find a few crumbs here > and there). > > People often look at situations like yours and conclude that > gc-cons-threshold is too small. Yes, it's too small for those cases, > but bumping it once and forall makes things worse for the "usual" case, > leading to increased heap size, fragmentation, and longer pauses during > normal operation (tho this "usual case" is much more difficult to > measure). > > There's no sadly way the GC can magically know that the last N megabytes > of allocation generated very little garbage, tho. > > Maybe should bump those gc-knobs around the startup.el code (or maybe > more generally we should bump those knobs while loading a file). > >> Even if we leave the default as is, it might make sense to temporarily >> increase gc-cons-threshold within package-initialize > > Right. Tho doing this will likely cause a GC to happen right after > package-initialize, i.e. just before loading your ~/.emacs. > Better would be to bump it around both, I guess. > > > Stefan > > --