From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) Date: Fri, 01 Jul 2022 09:18:08 +0300 Message-ID: <83wncxe4pr.fsf@gnu.org> References: <83czfh12kp.fsf@gnu.org> <87pmjhghu2.fsf@localhost> <835yl910gp.fsf@gnu.org> <87wndndbhq.fsf@gmail.com> <83bkuzznws.fsf@gnu.org> <877d5mqmkh.fsf@localhost> <83y1y2utnd.fsf@gnu.org> <87r13up587.fsf@localhost> <83o7yyur0l.fsf@gnu.org> <87leu2p3nu.fsf@localhost> <83leu2uewn.fsf@gnu.org> <87r13qv701.fsf@localhost> <83bkuursya.fsf@gnu.org> <87h74l9jk8.fsf@localhost> <83bkutqb3z.fsf@gnu.org> <9778F176-E724-4E61-B0FB-327BCDD316C0@acm.org> <87sfo4epeo.fsf@localhost> <87bkurrc5e.fsf@localhost> <87v8shk1c5.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="14500"; mail-complaints-to="usenet@ciao.gmane.io" Cc: monnier@iro.umontreal.ca, mattiase@acm.org, theophilusx@gmail.com, rms@gnu.org, acm@muc.de, emacs-devel@gnu.org To: Ihor Radchenko Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Jul 01 08:20:27 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o7A0x-0003fR-N2 for ged-emacs-devel@m.gmane-mx.org; Fri, 01 Jul 2022 08:20:27 +0200 Original-Received: from localhost ([::1]:41354 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o7A0w-0001RL-8i for ged-emacs-devel@m.gmane-mx.org; Fri, 01 Jul 2022 02:20:26 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41244) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o79yf-0008GS-QN for emacs-devel@gnu.org; Fri, 01 Jul 2022 02:18:05 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:43136) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o79ye-0004hQ-7l; Fri, 01 Jul 2022 02:18:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=L40d/KsiimenlJ511n+uTh/Rt3p0WgwAiK49HuF5ZKs=; b=a5n1p20IdoNAhhpUlwUZ +aWH/+spnIuYR3QvKhoBe9GyXkw3jwuGfPONNQShI8egE/PnYamZOCc7Yvh2oRxEj166n7NT/seSN I0Bky4NLrPzI6gOG7jnaIrNnpKNROW1sqZzqL9RQPUzB4Zd4AQxTC/zY0cWu3nD8H4qMOVXjgQBeb CRHXRWTf/PJ7TcG8IprpV0p1HA7KmNJPkvPEwe0ORg0SgA4KmRXjFCAM9XoxDL1zDWHJ3Vw1qAcjZ jwafP9klFp4PN8FvEZq6ITIrmxGsZelUOP3FmZC9Zoy+itM7rrZqdkMRKt6vKyQ0Q83Fpd2uPuaVZ 6jRcPnv3H5Auww==; Original-Received: from [87.69.77.57] (port=4465 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o79yY-0004uc-SB; Fri, 01 Jul 2022 02:17:59 -0400 In-Reply-To: <87v8shk1c5.fsf@localhost> (message from Ihor Radchenko on Fri, 01 Jul 2022 10:34:34 +0800) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:291764 Archived-At: > From: Ihor Radchenko > Cc: Mattias Engdegård , Eli Zaretskii > , > Tim Cross , rms@gnu.org, Alan Mackenzie > , emacs-devel > Date: Fri, 01 Jul 2022 10:34:34 +0800 > > (defvar lst-data) > (benchmark-progn > (let (result) > (dotimes (i 1000000) > (push i result)) > (setq lst-data result) > nil)) > > This code does not really generate any garbage. Yet GC is triggered > frequently: > > Elapsed time: 0.619852s (0.426893s in 11 GCs) > > If I instead do > > (defvar lst-data) > (benchmark-progn > (let ((gc-cons-threshold most-positive-fixnum)) ;; the value is just for demo purposes > (let (result) > (dotimes (i 1000000) > (push i result)) > (setq lst-data result) > nil))) > > Elapsed time: 0.216380s (0.031766s in 1 GCs) > > The difference is obvious. Please don't forget that GC doesn't only collects unused Lisp objects, it also does other useful memory-management related tasks. It compacts buffer text and strings, and it frees unused slots in various caches (font cache, image cache, etc.). You can find in the archives discussions where innocently-looking code could cause Emacs run out of memory because it used too many fonts without flushing the font cache (any program that works on the list of fonts returned by the likes of x-list-fonts is in danger of bumping into that). > More generally, any code that generates/returns large data structures is > going to trigger frequent GCs regardless whether such code generates any > garbage. > > On the other hand, we cannot, in general terms, predict if real-life > code, which allocates large permanent data structures, also produces a > lot of actual valid garbage that should be collected. Yes, that's the conundrum. > Having some way to prevent excessive garbage collection that is also > smarter than simply setting gc-cons-threshold to large value would be > useful. > > As one idea, a lisp program may mark some of the variables to be skipped > by GC and to not contribute to GC threshold checks (that is, allocating > memory into the marked variables will not increase the memory counter > used by GC). > > WDYT? I'm not sure I understand how this idea can be implemented. The counting of how much Lisp data since last GC was produced is done _before_ variables are bound to the produced data as values. So by the time we know the data is bound to such "special" variables, it's already too late, and the only way to do what you suggest would be to increase consing_until_gc back after we realize this fact. Which would mean computing how much consing was done for the value of these variables, and that would probably slow down the generation of Lisp data, wouldn't it? Or what am I missing?