From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Mikael Djurfeldt Newsgroups: gmane.lisp.guile.devel Subject: Re: garbage collection slowdown Date: Thu, 6 Feb 2020 15:06:14 +0100 Message-ID: References: <877e11m2ao.fsf@gnu.org> Reply-To: mikael@djurfeldt.com Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="00000000000096331a059de8c7e4" Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="115407"; mail-complaints-to="usenet@ciao.gmane.io" Cc: =?UTF-8?Q?Ludovic_Court=C3=A8s?= , Guile Devel To: Han-Wen Nienhuys Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Thu Feb 06 15:07:04 2020 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1izho7-000Tuz-R3 for guile-devel@m.gmane-mx.org; Thu, 06 Feb 2020 15:07:03 +0100 Original-Received: from localhost ([::1]:39688 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izho6-0008HL-TW for guile-devel@m.gmane-mx.org; Thu, 06 Feb 2020 09:07:02 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:45487) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izhnq-0008Df-9s for guile-devel@gnu.org; Thu, 06 Feb 2020 09:06:47 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izhng-0004Lu-Tr for guile-devel@gnu.org; Thu, 06 Feb 2020 09:06:43 -0500 Original-Received: from mail-vs1-f41.google.com ([209.85.217.41]:43581) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izhnb-00045f-1j; Thu, 06 Feb 2020 09:06:31 -0500 Original-Received: by mail-vs1-f41.google.com with SMTP id 7so3803171vsr.10; Thu, 06 Feb 2020 06:06:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=K+1Z6K/NW4sc7lWP222zrFKiA8HhzEwPiK8pZ3FdZWs=; b=mIVQy8sEWjSW2VmfKbun4+17PxtIQDVI9t6tz0jdD0otMhTxKK/TjKk+X5yGaVQXdu x4Sa/vk7GF+HRNW1g51oKsYS1KrdFaG9zwtlmdrMrSBONbE8UAtpskqGNxTN7D/rTwlW 5HkC+dcL52hFoYKeLzjsDB2UL0xkYrbACQrC2SUI+QRK6uQS4yPBPs+6s78qfeWuORpe triKfHY2I/9qNn3rNKTn42osYeJk9wixqi9UYb7E0VIIFqMGslgUU5/SCizrq2Q1rXTy rTextpWHpUUXahiHy9KDN+RWDRB6jknQRixIViovohUOXPwLeIUM/AlqKMk3Yin5/1WW 00nA== X-Gm-Message-State: APjAAAUTwi9/AwyV5G8PUHMJ8vcrNeO1RDodgYAetCT4p/B6aEHs+shf 4ZzeCl3OkSju5JIDqQlY2Z5wjZUm5ynFijv+Qik= X-Google-Smtp-Source: APXvYqyBSDLEq/Ep0G0euWVCrvSyK2qWCapBM+4MPjhijSm992CZiFVa22F4p12WImXDaS5iAEM+7wMLxJ8OXjPSQVg= X-Received: by 2002:a05:6102:90:: with SMTP id t16mr1817560vsp.140.1580997990034; Thu, 06 Feb 2020 06:06:30 -0800 (PST) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.217.41 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.io gmane.lisp.guile.devel:20397 Archived-At: --00000000000096331a059de8c7e4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Den ons 5 feb. 2020 23:32Han-Wen Nienhuys skrev: > > > On Wed, Feb 5, 2020 at 5:23 PM Ludovic Court=C3=A8s wrote: > >> Weird. It would be interesting to see where the slowdown comes from. >> Overall, my recollection of the 1.8 to 2.0 transition (where we >> introduced libgc) is that GC was a bit faster, definitely not slower. >> >> That said, does LilyPond happen to use lots of bignums and/or lots of >> finalizers? Finalizers, including those on bignums, end up being very >> GC-intensive, as discussed in my recent message. Perhaps that=E2=80=99s= what=E2=80=99s >> happening here, for instance if you create lots of SMOBs with a free >> function. >> > > No, I think it's because in some phases of the program, there is a lot of > heap growth, with little garbage generation. This causes frequent > (expensive) GCs that don't reclaim anything. > When programming dynamic vectors, it is common to adapt the size of newly allocated chunks by letting them grow in proportion to vector size. Could the frequency of GC be adapted similarly such that the balance between GC and allocation is shifted towards allocation in phases with a lot of heap growth? More concretely, this could either be achieved by letting the newly allocated chunks grow in proportion to allocated memory (as I think it was in the 1.8 GC---don't know about now) or by choosing to not do a GC every time, but instead directly allocate if running average of GC gain is small compared to allocated memory. Of course these are issues at research level... > > -- > Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen > --00000000000096331a059de8c7e4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Den ons 5 feb. 2020 23:32Han-Wen Nienhuys <hanwenn@gmail.com> skrev:

On Wed, F= eb 5, 2020 at 5:23 PM Ludovic Court=C3=A8s <ludo@gnu.org> wrote:
Weird.=C2=A0 It would = be interesting to see where the slowdown comes from.
Overall, my recollection of the 1.8 to 2.0 transition (where we
introduced libgc) is that GC was a bit faster, definitely not slower.

That said, does LilyPond happen to use lots of bignums and/or lots of
finalizers?=C2=A0 Finalizers, including those on bignums, end up being very=
GC-intensive, as discussed in my recent message.=C2=A0 Perhaps that=E2=80= =99s what=E2=80=99s
happening here, for instance if you create lots of SMOBs with a free
function.

No, I think it's because = in some phases of the program, there is a lot of heap growth, with little g= arbage generation. This causes frequent (expensive) GCs that don't recl= aim anything.

When programmin= g dynamic vectors, it is common to adapt the size of newly allocated chunks= by letting them grow in proportion to vector size.

Could the frequency of GC be adapted similarly such tha= t the balance between GC and allocation is shifted towards allocation in ph= ases with a lot of heap growth?

M= ore concretely, this could either be achieved by letting the newly allocate= d chunks grow in proportion to allocated memory (as I think it was in the 1= .8 GC---don't know about now) or by choosing to not do a GC every time,= but instead directly allocate if running average of GC gain is small compa= red to allocated memory.

Of cours= e these are issues at research level...

=


<= /div>--
--00000000000096331a059de8c7e4--