From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: [ELPA] New package: emacs-gc-stats Date: Sat, 10 Jun 2023 12:33:43 -0400 Message-ID: References: <87ttxil7k3.fsf@localhost> <87bkhpuheb.fsf@localhost> <83r0ql0wji.fsf@gnu.org> <87sfaztyk9.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28347"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , Philip Kaludercic , emacs-devel@gnu.org To: Ihor Radchenko Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jun 10 18:34:55 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q81YE-00075z-1B for ged-emacs-devel@m.gmane-mx.org; Sat, 10 Jun 2023 18:34:54 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q81XM-0003ns-8G; Sat, 10 Jun 2023 12:34:00 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q81XI-0003nf-HO for emacs-devel@gnu.org; Sat, 10 Jun 2023 12:33:56 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q81XG-0000nZ-7F; Sat, 10 Jun 2023 12:33:56 -0400 Original-Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id C0A2F801D6; Sat, 10 Jun 2023 12:33:50 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 9FDEF803B1; Sat, 10 Jun 2023 12:33:44 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1686414824; bh=RlviMZKXVjAvRKQhUIqBd0rZFIk61KuSS9Fhajmmiz0=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=YohC0NjuTdDQg2amMG1n0k66SzWhSj/AFrA0A8YbnIGAA4Y6nUnam6sfOls/+yH4+ rEfsBkKbYDjS6cM6WS3cSUybrWWlHsu4MQ9Rf90YnXT0PadIX1xspQncWi82xF9V57 PuaGq1Pen5Ud8WSUSaofoPMDZb8VPGSaxgIvmkjChXyQorlmaw/rsY412yfZDFOyT8 w2P3h+pTvBnzbO30V6lnk49jDpFc7zt6VidBcOBZCpcRuVFmH/eN7lPVXkESStwUBV fuMnYqaMNH4LSdZNgghfFhBBDV2dNZNdQ/quXT9bq+Ez6KDf9y7CNTfIHtu6jra9sW 4cVfk9eeRqhxA== Original-Received: from pastel (76-10-180-239.dsl.teksavvy.com [76.10.180.239]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 651E912046B; Sat, 10 Jun 2023 12:33:44 -0400 (EDT) In-Reply-To: <87sfaztyk9.fsf@localhost> (Ihor Radchenko's message of "Sat, 10 Jun 2023 10:13:26 +0000") Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:306723 Archived-At: >> Let me know if you need help adding the package (I couldn't find a URL >> for the code, so I have no opinion about it (yet :-)) > Oops... https://git.sr.ht/~yantar92/emacs-gc-stats Thanks, added. >> PS: In recent Emacsen, `gc-cons-percentage` is set to 1.0 when running in >> batch. In non-batch I currently use 0.5 together with a timer that >> tries to opportunistically run the GC during idle time. > This is yet another thing we can ask users to test, if we want to. > My idea is have users set GC settings randomly in their init file, > selecting from pre-configured set of alternatives we are yet to discuss. FWIW, see below the code I'm using. On my main Emacs instance, `gc--opportunistic-score` tells me: ((jit 17 0 0) (cmd 22 3 7) (earlier-gcs . 76) (opportunistic-gcs . 1478) (noncmd 194 6 52) (commands . 109401)) IOW the majority of GCs happen "opportunistically" (i.e. during idle time). The `cmd/noncmd` distinction is supposed to distinguish those GCs that happen while running timers and process filters, but I'm not sure my code is careful enough to keep track of those correctly. Stefan ;;;; Opportunistic GC (defvar gc--opportunistic-last-gcs gcs-done) (defvar gc--opportunistic-state 'noncmd) (defvar gc--opportunistic-counters nil) (defun gc--check () (let ((gcs-counted (+ (alist-get 'multi-gcs gc--opportunistic-counters 0) (alist-get 'earlier-gcs gc--opportunistic-counters 0) (alist-get 'single-gc-cmds gc--opportunistic-counters 0) (alist-get 'noncmds-gcs gc--opportunistic-counters 0) (alist-get 'opportunistic-gcs gc--opportunistic-counters 0) (or (car (alist-get 'error-gcs gc--opportunistic-counters)) 0)))) (unless (= gcs-done gcs-counted) (push (+ (- gcs-done gcs-counted) (or (car (alist-get 'error-gcs gc--opportunistic-counters)) 0)) (alist-get 'error-gcs gc--opportunistic-counters))))) (defun gc--opportunistic-record (nextstate) (let ((counts (alist-get gc--opportunistic-state gc--opportunistic-counters))) (unless counts (setf (alist-get gc--opportunistic-state gc--opportunistic-counters) (setq counts (list 0 0 0)))) (pcase (prog1 (- gcs-done gc--opportunistic-last-gcs) (setq gc--opportunistic-last-gcs gcs-done)) ((pred (>= 0)) nil) (1 (cl-incf (nth 0 counts))) (gcs (cl-incf (nth 1 counts)) (cl-incf (nth 2 counts) gcs)))) (setq gc--opportunistic-state nextstate)) (defun gc--opportunistic-postch () (cl-incf (alist-get 'commands gc--opportunistic-counters 0)) (gc--opportunistic-record 'noncmd)) (defun gc--opportunistic-prech () (cl-callf identity (alist-get 'earlier-gcs gc--opportunistic-counters gcs-done)) (gc--opportunistic-record 'cmd) ;; (gc--check) ) (defun gc--opportunistic-jitlock (orig-fun start) (if (eq gc--opportunistic-state 'cmd) ;; Count jit-lock execution which happens during a command as ;; being part of command execution rather than as part of jit-lock! (funcall orig-fun start) (let ((gc--opportunistic-state gc--opportunistic-state)) (gc--opportunistic-record 'jit) (unwind-protect (funcall orig-fun start) (gc--opportunistic-record 'postjit))))) (add-hook 'pre-command-hook #'gc--opportunistic-prech) (add-hook 'post-command-hook #'gc--opportunistic-postch) (advice-add 'jit-lock-function :around #'gc--opportunistic-jitlock) (defun gc--opportunistic () "Run the GC during idle time." ;; This is good for two reasons: ;; - It reduces the number of times we have to GC in the middle of ;; an operation. ;; - It means we GC when the C stack is short, reducing the risk of false ;; positives from the conservative stack scanning. (unless (> gc-cons-percentage 0.1) (setq gc-cons-percentage 0.5)) (when (garbage-collect-maybe 10) (cl-incf (alist-get 'opportunistic-gcs gc--opportunistic-counters 0)) ;; Don't double count this GC in other categories. (cl-incf gc--opportunistic-last-gcs) ;; Recalibrate the timer. (cancel-function-timers #'gc--opportunistic) (run-with-idle-timer ;; FIXME: Magic formula! (+ 1 (* 10 (/ gc-elapsed gcs-done))) t #'gc--opportunistic))) (defun gc--opportunistic-score () "Show the current counters's that keep track of GC behavior." (interactive) (message "%S" gc--opportunistic-counters))