From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.devel Subject: Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) Date: Fri, 01 Jul 2022 10:34:34 +0800 Message-ID: <87v8shk1c5.fsf@localhost> References: <83czfh12kp.fsf@gnu.org> <87pmjhghu2.fsf@localhost> <835yl910gp.fsf@gnu.org> <87wndndbhq.fsf@gmail.com> <83bkuzznws.fsf@gnu.org> <877d5mqmkh.fsf@localhost> <83y1y2utnd.fsf@gnu.org> <87r13up587.fsf@localhost> <83o7yyur0l.fsf@gnu.org> <87leu2p3nu.fsf@localhost> <83leu2uewn.fsf@gnu.org> <87r13qv701.fsf@localhost> <83bkuursya.fsf@gnu.org> <87h74l9jk8.fsf@localhost> <83bkutqb3z.fsf@gnu.org> <9778F176-E724-4E61-B0FB-327BCDD316C0@acm.org> <87sfo4epeo.fsf@localhost> <87bkurrc5e.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10765"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= , Eli Zaretskii , Tim Cross , rms@gnu.org, Alan Mackenzie , emacs-devel To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Jul 01 04:34:42 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o76UU-0002YX-Kn for ged-emacs-devel@m.gmane-mx.org; Fri, 01 Jul 2022 04:34:42 +0200 Original-Received: from localhost ([::1]:35322 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o76UT-0001HK-3v for ged-emacs-devel@m.gmane-mx.org; Thu, 30 Jun 2022 22:34:41 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:33648) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o76TK-0000aI-Rf for emacs-devel@gnu.org; Thu, 30 Jun 2022 22:33:30 -0400 Original-Received: from mail-oa1-x2d.google.com ([2001:4860:4864:20::2d]:33601) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o76TJ-0003mc-6n; Thu, 30 Jun 2022 22:33:30 -0400 Original-Received: by mail-oa1-x2d.google.com with SMTP id 586e51a60fabf-101d2e81bceso1886705fac.0; Thu, 30 Jun 2022 19:33:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=YT9fCnzniYowWs8r0k7ENcaEhaGQYloAzx/BcrvPzdU=; b=UAdzpA47VvBVVVbcSDdqAiPL1Iqh4L8Ch8qV4SIQIG543Ev+EPTdx1ZwrZsMI+qd8J FxHkWkJK0zKiAcwcUtfojN7K2i45kgHxgnCRvj9cpe7ftohepvRy9clY0c1F+20aaaXG GgoNMfxnWLWdDUfcMzaKO8hkZxXMHEY+fMy2my5X0AlE1kc+IywP0b3d2QqVNgYbvWaQ HP/faJ+CmsaSY3slltDHMX8Dv3397jXBkaeOAVLwNrRlH4T1wCJxoLiiCE4mbrEyKmFk awfa75W8j2UyNi0h1m/g7sX8FsZ22Vr/7wRjlwFocLjfL8FU4VgafqxzvdzBcPhsO5YG 9ilw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=YT9fCnzniYowWs8r0k7ENcaEhaGQYloAzx/BcrvPzdU=; b=O7bUajU9I3TJ71YDQvpcl+0h0gTsDErIEM0XHinchbrk8/69LH210zTpYGW05guDWv jdqjzj9iUlDjNqSy31L0GeaFoEgTswvrHa9hE6YN5uKkigoN6U6XdSMfzblh7kcMc1Bl ocghKpH+zo0wyODPfbLIRflN119H6hg97XhuG5veDmYXfbkI66jOV120CrJJco04xReW Wx0GlVHw7TMnz/XezoLXJSGVvGysXF8/HW7wwOESuFM4/cIVl+AyUfeEUNXF8h1NgCiD /TtembQfmNGCybwSXL3ZmrNCg1w3Q+mhPVjot7ErXhQ7Po4mEFMJ3MPBDh0VFCzoLR5o VHOg== X-Gm-Message-State: AJIora+5fA9yW10f4yBVqHVUSA8CVTpicsf34O5rcaDdKfQPDPbabT0D 5clkEoPtsA2NKlOKxZplXZo= X-Google-Smtp-Source: AGRyM1uRevJ873Zlgz/5MJnaD3Urf+HOMGkoxwVUW6D1OJYHKYdClNZDulNqsJRVSiSPWVqidR3wEw== X-Received: by 2002:a05:6870:239c:b0:10b:8a3c:f9e7 with SMTP id e28-20020a056870239c00b0010b8a3cf9e7mr6575889oap.161.1656642807179; Thu, 30 Jun 2022 19:33:27 -0700 (PDT) Original-Received: from localhost ([207.126.88.10]) by smtp.gmail.com with ESMTPSA id x14-20020a4a800e000000b0035eb4e5a6b3sm11647746oof.9.2022.06.30.19.33.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jun 2022 19:33:26 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2001:4860:4864:20::2d; envelope-from=yantar92@gmail.com; helo=mail-oa1-x2d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:291761 Archived-At: Stefan Monnier writes: >> A possible useful thing Emacs could help with would be a macro dedicated >> to let-binds like the above. Something like: >> >> (with-reduced-gc >> ;; Do staff) > > Sounds about right, tho I think the name of the macro should be related > to what the code does rather than to what the author thinks the GC should > do about it. Something like `this-code-does-not-generate-garbage`. > >> with-reduced-gc could take care about determining the inner specifics >> on what alternative gc-cons-threshold value should be used (possibly >> depending on the system memory information). > > Sounds good. This part of the discussion appears to be missed from the following replies. So, I would like to bring it up again. To clarify a bit on the usefulness of the proposed macro idea, consider the following code: (defvar lst-data) (benchmark-progn (let (result) (dotimes (i 1000000) (push i result)) (setq lst-data result) nil)) This code does not really generate any garbage. Yet GC is triggered frequently: Elapsed time: 0.619852s (0.426893s in 11 GCs) If I instead do (defvar lst-data) (benchmark-progn (let ((gc-cons-threshold most-positive-fixnum)) ;; the value is just for demo purposes (let (result) (dotimes (i 1000000) (push i result)) (setq lst-data result) nil))) Elapsed time: 0.216380s (0.031766s in 1 GCs) The difference is obvious. More generally, any code that generates/returns large data structures is going to trigger frequent GCs regardless whether such code generates any garbage. On the other hand, we cannot, in general terms, predict if real-life code, which allocates large permanent data structures, also produces a lot of actual valid garbage that should be collected. Having some way to prevent excessive garbage collection that is also smarter than simply setting gc-cons-threshold to large value would be useful. As one idea, a lisp program may mark some of the variables to be skipped by GC and to not contribute to GC threshold checks (that is, allocating memory into the marked variables will not increase the memory counter used by GC). WDYT? Best, Ihor