From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: [PATCH v2 3/3] Start opportunistic GC timer at startup Date: Fri, 04 Dec 2020 18:19:27 -0500 Message-ID: References: <87h7pij7xo.fsf@catern.com> <20201122050829.9097-3-sbaugh@catern.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="34714"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: Eli Zaretskii , emacs-devel@gnu.org To: Spencer Baugh Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Dec 05 00:20:36 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1klKNQ-0008w8-PM for ged-emacs-devel@m.gmane-mx.org; Sat, 05 Dec 2020 00:20:36 +0100 Original-Received: from localhost ([::1]:45786 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1klKNP-0000fN-Rn for ged-emacs-devel@m.gmane-mx.org; Fri, 04 Dec 2020 18:20:35 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:54136) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1klKMS-0000Bw-Q4 for emacs-devel@gnu.org; Fri, 04 Dec 2020 18:19:37 -0500 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:43637) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1klKMO-0008GY-Sn; Fri, 04 Dec 2020 18:19:36 -0500 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id A93D5440A0A; Fri, 4 Dec 2020 18:19:30 -0500 (EST) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id C19BC44095E; Fri, 4 Dec 2020 18:19:28 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1607123968; bh=q9O49UwyYD3vgs5GJxc7SfqoToJXkQHbEBoCC09UFo8=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=RYsHH72rhJ9cjJPrz5vpJuAcbIMaFLH7y80oQENk9RDFcW3Z/Iv/CBIpujAaan1JK ek++TmFX4X/IBToVJ5izVwCHB+ctgZkT0PhXNHRWcOlnMvVeUKR9AfqIRDTJEUX5pU Qw9JuKIs/Dj0NMlMPxcryYhqP9DJIn3GfHdEiRnupiBIW92iXvndv2VzMsn4OmT1qh tfDAmEhssjWrMV/jgoG+0PtvZejPfP7b5LFePDTAXsPbr/CKVXM/B9Kq/nGOXqerx9 Ioe/LQ7iX19mPK6PzOxAdERNGMNknxoyAsSGh5r2634d371r/HIJT6tbV5s5mEvMkB LRG/0tJRDvTTw== Original-Received: from alfajor (69-165-136-52.dsl.teksavvy.com [69.165.136.52]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 813F51200BF; Fri, 4 Dec 2020 18:19:28 -0500 (EST) In-Reply-To: <20201122050829.9097-3-sbaugh@catern.com> (Spencer Baugh's message of "Sun, 22 Nov 2020 00:08:29 -0500") Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:260333 Archived-At: > +(defun gc-start-opportunistic () > + "Start an opportunistic GC which may run at some point in the future. > + > +This function may perform a single GC at some point while Emacs > +is idle, with the goal of improving interactive performance by > +avoiding GC while the user is actively interacting with Emacs. > + > +The higher `gc-opportunistic-eager-factor' is, the more likely > +this function is to actually perform a GC. Note that increasing > +this variable can worsen performance by performing excessive GCs. > +If `gc-opportunistic-eager-factor' is nil, this function will do > +nothing. > + > +If `gc-opportunistic-eager-factor' is non-nil at Emacs startup, > +this function will be run by an idle timer. Such a timer can > +also be started after Emacs startup with `run-with-idle-timer'." > + (when gc-opportunistic-eager-factor > + (when gc-next-opportunistic-timer > + (cancel-timer gc-next-opportunistic-timer)) > + (setq gc-next-opportunistic-timer > + (run-with-idle-timer > + gc-estimated-time nil > + (lambda () > + (if (garbage-collect-maybe gc-opportunistic-eager-factor) > + (setq gc-opportunistic-performed > + (1+ gc-opportunistic-performed)))) > + )))) > + > (defun command-line () > "A subroutine of `normal-top-level'. > Amongst another things, it parses the command-line arguments." > @@ -1420,6 +1467,15 @@ please check its value") > (eq face-ignored-fonts old-face-ignored-fonts)) > (clear-face-cache))) > > + ;; Start opportunistic GC (after loading the init file, so we obey > + ;; its settings). This is desirable for two reason: > + ;; - It reduces the number of times we have to GC in the middle of > + ;; an operation. > + ;; - It means we GC when the C stack is short, reducing the risk of false > + ;; positives from the conservative stack scanning. > + (when gc-opportunistic-eager-factor > + (run-with-idle-timer 1 t #'gc-start-opportunistic)) This looks a bit complicated: why use 2 timers? I currently use the following instead: (defvar gc--opportunistic-counter 0) (defun gc--opportunistic () "Run the GC during idle time." ;; This is good for two reason: ;; - It reduces the number of times we have to GC in the middle of ;; an operation. ;; - It means we GC when the C stack is short, reducing the risk of false ;; positives from the conservative stack scanning. (when (garbage-collect-maybe 3) (setq gc--opportunistic-counter (1+ gc--opportunistic-counter)) ;; Recalibrate the timer. (cancel-function-timers #'gc--opportunistic) (run-with-idle-timer ;; FIXME: Magic formula! (+ 1 (* 10 (/ gc-elapsed gcs-done))) t #'gc--opportunistic))) (run-with-idle-timer 1 t #'gc--opportunistic) Also notice that I use a different idle time (I think we should impose a minimum of at the very least 0.2s, I used 1s), and I think we may want to play with that as well. When I looked at `gc--opportunistic-performed` I wasn't sure what to think of it, when comparing it to `gcs-done`: I don't want this to be too high (because that means we're doing way more GCs) and I don't want it to be too low either (because it likely means it's ineffective). Maybe what I want to compare it against is the number of commands during which the GC is run exactly once? The idea is to filter out those commands which are compute-intensive and hence run the GC internally (for which an opportunistic GC is of no use anyway). So I added: (defvar gc--opportunistic-single-gc-cmds 0) (let ((last-gcs nil)) (add-hook 'pre-command-hook (lambda () (setq last-gcs gcs-done))) (add-hook 'post-command-hook (lambda () (if (eq (1- gcs-done) last-gcs) (setq gc--opportunistic-single-gc-cmds (1+ gc--opportunistic-single-gc-cmds)))))) So far, this seems to indicate that it's working: OT1H `gc--opportunistic-single-gc-cmds` stays much lower than `gc--opportunistic-counter`, indicating that we seem to be able to avoid GC in many cases, and OTOH `gc--opportunistic-counter` is much lower than `gcs-done` indicating that we're not excessively increasing the number of GCS. Stefan