From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Larger GC thresholds for non-interactive Emacs Date: Fri, 17 Jun 2022 22:32:30 -0400 Message-ID: References: <87wndndbhq.fsf@gmail.com> <83bkuzznws.fsf@gnu.org> <877d5mqmkh.fsf@localhost> <83y1y2utnd.fsf@gnu.org> <87r13up587.fsf@localhost> <83o7yyur0l.fsf@gnu.org> <87leu2p3nu.fsf@localhost> <83leu2uewn.fsf@gnu.org> <87r13qv701.fsf@localhost> <83bkuursya.fsf@gnu.org> <87h74l9jk8.fsf@localhost> <83bkutqb3z.fsf@gnu.org> <9778F176-E724-4E61-B0FB-327BCDD316C0@acm.org> <87sfo4epeo.fsf@localhost> <87bkurrc5e.fsf@localhost> <87bkur72b7.fsf@gnus.org> <874k0j40e7.fsf@gnus.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5536"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: Ihor Radchenko , Mattias =?windows-1252?Q?Engdeg?= =?windows-1252?Q?=E5rd?= , Eli Zaretskii , Tim Cross , rms@gnu.org, Alan Mackenzie , emacs-devel To: Lars Ingebrigtsen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jun 18 04:33:24 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o2OH5-0001An-FZ for ged-emacs-devel@m.gmane-mx.org; Sat, 18 Jun 2022 04:33:23 +0200 Original-Received: from localhost ([::1]:40896 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2OH3-0002w8-OB for ged-emacs-devel@m.gmane-mx.org; Fri, 17 Jun 2022 22:33:21 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:53826) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2OGM-0002Fw-UG for emacs-devel@gnu.org; Fri, 17 Jun 2022 22:32:39 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:37355) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2OGJ-0003A8-Rc; Fri, 17 Jun 2022 22:32:37 -0400 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 27D87442103; Fri, 17 Jun 2022 22:32:33 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 901E0441C47; Fri, 17 Jun 2022 22:32:31 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1655519551; bh=XG1y+F/Wx8sN2/auuRcf4165ShQXdZ/Q+6ztigC6XeY=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=E8FXSouO2H8yw6C5JiGgzGxinM3dEwDmPFOYsr/SokgP0qUpf8gAnS6M+WqsRUtr1 C8vG6JXp4Q7NB1y2LESBuxqkWwPWCnLyqDudmmsx4sX8RSWMhLYn6hNrM+04zNgOmC pZUBxn+0OXm4t3nMnASPwd95Aby4yzrt8w/PsGRjjGxeErntHnXt+WbEp8kto01oqp ccqI2h3L4vtYDVk59dP0YNyAW0LmGE0+68sXrbBn3Kkk4K9eoU6Y2mH5rxqOeKKoO9 43x521t0wFNRH1ftjXrPqT9zabTTd4YD6c5E6GAXK/Wk2tVeFVqaVrCMNT24Wlbs/E Vv7dvxASCXvCA== Original-Received: from alfajor (unknown [45.72.221.51]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 37E131203D5; Fri, 17 Jun 2022 22:32:31 -0400 (EDT) In-Reply-To: <874k0j40e7.fsf@gnus.org> (Lars Ingebrigtsen's message of "Fri, 17 Jun 2022 20:20:48 +0200") Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:291326 Archived-At: > Yup. But do you mean in general? I.e., -batch would set that variable > to 2.0? Would there be any likely major repercussions -- i.e., jobs > that used to run fine would run out of memory? I've looked a bit further at some of the repercussions. The most obvious immediate one is that a program that needs 10MB of live data will now use up 30MB of heap (10MB of live data at the last GC plus upto 20MB of data allocated since the last GC), so it will increase the process sizes in a non-trivial way. I'd lean towards 1.0 instead of 2.0 for that kind of reason. I also took a look at related data. E.g. comparing p=1.0 to the old p-0.1 on the process that performs the highest number of GCs (188 cycles with p=1.0 and 650 cycles with p=0.1) during an Emacs build. Here are the corresponding last few GC cycles: % grep GC-26227 ./+make-0.1.log | tail -n 10 GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.6M free=2.0M thresold=1.9M GC-26227 p=0.1 total=18.7M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.7M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.8M free=1.8M thresold=1.9M GC-26227 p=0.1 total=145.7M free=32.6M thresold=14.6M GC-26227 p=0.1 total=132.8M free=60.7M thresold=13.3M % grep GC-898 ./+make-1.0.log | tail -n 10 GC-898 p=1.0 total=18.5M free=7.5M thresold=18.5M GC-898 p=1.0 total=18.6M free=7.4M thresold=18.6M GC-898 p=1.0 total=18.6M free=7.4M thresold=18.6M GC-898 p=1.0 total=18.7M free=7.4M thresold=18.7M GC-898 p=1.0 total=18.7M free=7.3M thresold=18.7M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M GC-898 p=1.0 total=145.7M free=32.6M thresold=145.7M % Obviously this process ends up with a very large one-step allocation which is arguably interesting in itself (I suspect there's some GC-inhibition going on there), but the more interesting point is that with p=0.1 we get an amount of free space after GC (i.e. blocks we can't release, because of fragmentation) that's about as large as the next threshold, i.e. about 10%, whereas with p=1.0 this amount of unreleasable free space is significantly higher. Most likely this is not wasted space: a lot of it will be re-used for new allocations, but still with 19MB of live data, we end up with 7MB of space we can't release back. I think p=1.0 (and the corresponding implication that we use up about twice as much memory as the minimum we need) might be an acceptable tradeoff (for batch use), but I don't think I'd be comfortable going beyond that. Stefan