From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Larger GC thresholds for non-interactive Emacs Date: Fri, 17 Jun 2022 09:49:41 -0400 Message-ID: References: <875yl9e7zm.fsf@gmail.com> <83czfh12kp.fsf@gnu.org> <87pmjhghu2.fsf@localhost> <835yl910gp.fsf@gnu.org> <87wndndbhq.fsf@gmail.com> <83bkuzznws.fsf@gnu.org> <877d5mqmkh.fsf@localhost> <83y1y2utnd.fsf@gnu.org> <87r13up587.fsf@localhost> <83o7yyur0l.fsf@gnu.org> <87leu2p3nu.fsf@localhost> <83leu2uewn.fsf@gnu.org> <87r13qv701.fsf@localhost> <83bkuursya.fsf@gnu.org> <87h74l9jk8.fsf@localhost> <83bkutqb3z.fsf@gnu.org> <9778F176-E724-4E61-B0FB-327BCDD316C0@acm.org> <87sfo4epeo.fsf@localhost> <87bkurrc5e.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18025"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: Mattias =?windows-1252?Q?Engdeg=E5rd?= , Eli Zaretskii , Tim Cross , rms@gnu.org, Alan Mackenzie , emacs-devel To: Ihor Radchenko Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Jun 17 15:52:12 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o2COR-0004Sz-Lo for ged-emacs-devel@m.gmane-mx.org; Fri, 17 Jun 2022 15:52:11 +0200 Original-Received: from localhost ([::1]:53826 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2COQ-00009Q-5r for ged-emacs-devel@m.gmane-mx.org; Fri, 17 Jun 2022 09:52:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:48396) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2CMB-00068U-0m for emacs-devel@gnu.org; Fri, 17 Jun 2022 09:49:51 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:7380) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2CM7-0004bf-7i; Fri, 17 Jun 2022 09:49:49 -0400 Original-Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 25D941003F2; Fri, 17 Jun 2022 09:49:44 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 3974E100138; Fri, 17 Jun 2022 09:49:42 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1655473782; bh=qUykH7bPls2BLhXYMX23OONuNAKTkFmQJatM33mvv5E=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=YHb4cSVNrz4lCVKftOlmojGCiFceDAdQNsL5CtKTxj4lwAQcsn9IwVZMO/oicstEc 4IrRlGWNfbDjMe/AVwKhRzoBPoloRAXNQMS58uc4WRzdpclkmOforyPvgiX/Ls3uNq 5LoMKur6f+Oa+bmf5MLldGaiH77dRUedlJG5OZIUOK8o4ReLFaGuZPWPFWrKcRv3PL qEmvXvehwMn31fSz5OBeDUETnFsWKKhVLklNms1QOrYNkdoHZcIhNPT3ICoSZBAwSZ tiXN0roCD4t/rON+hwH1RKOCO51ywFOGXInzlJc3e7VyvzHB181bFfL+nNOwzmsmbp KGVNhD48LucVQ== Original-Received: from pastel (unknown [45.72.221.51]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E5A86120161; Fri, 17 Jun 2022 09:49:41 -0400 (EDT) In-Reply-To: <87bkurrc5e.fsf@localhost> (Ihor Radchenko's message of "Fri, 17 Jun 2022 15:18:21 +0800") Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:291293 Archived-At: >> Maybe it's time for some benchmarks with various values for these knobs >> to see if the values we're using now are "good enough" or might need to >> be changed. > > Note that I did not initially ask to change the GC defaults for normal > Emacs session. You're absolutely right, but we've had various reports of late that indicate that we're probably spending too much time in the GC. So I think we're due for some re-evaluation. > What I am trying to suggest is: > 1. Have some conservative GC settings in interactive Emacs session (as > we do now; I do _not_ suggest changing it) > 2. Increase GC thresholds in non-interactive --batch sessions, which > tend to not run for a long time. Agreed. Then again, we occasionally also use long-running commands interactively so it might be worthwhile to tune the settings for interactive use as well. > I am a bit afraid to touch the topic of changing the interactive GC > defaults. People tend to have contradicting opinions on this. :-) The issue is that it's easy to see "oh god, we're spending so much time in the GC, let's disable it" but much harder to see the "long term" impact of increasing the thresholds (which tend to increase the fragmentation and the total heap size both of which tend to slow down execution but the second additionally makes every GC cycle take longer, again increasing the pressure from naive onlookers to disable the GC). > I'd like to ask one thing from the very beginning here: Please spin a > separate thread if you have objections on actual GC defaults applicable > to normal Emacs sessions. No, I think focusing on batch use is a good idea. Also because it makes it much easier to actually measure the impact. > A possible useful thing Emacs could help with would be a macro dedicated > to let-binds like the above. Something like: > > (with-reduced-gc > ;; Do staff) Sounds about right, tho I think the name of the macro should be related to what the code does rather than to what the author thinks the GC should do about it. Something like `this-code-does-not-generate-garbage`. > with-reduced-gc could take care about determining the inner specifics > on what alternative gc-cons-threshold value should be used (possibly > depending on the system memory information). Sounds good. >> It's hard to know beforehand whether a GC will be useful or not, tho. >> But maybe we can find good heuristics. E.g. have something like >> a `gc-cons-percentage` which depends on how much garbage we collected in >> the last GC: if a GC doesn't collect any garbage (or very little of it), >> it's a sign that we're in a phase where running the GC is not very >> useful so we should wait a bit more until the next GC, whereas if the GC >> collected a fair bit of garbage, it's a sign that we're in a phase where >> running the GC is beneficial and we can run it a bit more often. > > FYI, I have played with this approach making use of > https://gitlab.com/koral/gcmh But this one focuses on interactive use. The kind of heuristic I'm proposing above would only make sense within a single (long-running) command, or in batch mode, I think. Stefan