From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: =?utf-8?Q?Hans_=C3=85berg?= Newsgroups: gmane.lisp.guile.user Subject: Re: out-of-control GC Date: Mon, 11 Sep 2017 20:13:25 +0200 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1505154228 31479 195.159.176.226 (11 Sep 2017 18:23:48 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 11 Sep 2017 18:23:48 +0000 (UTC) Cc: "guile-user@gnu.org" To: linasvepstas@gmail.com Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Mon Sep 11 20:23:43 2017 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drTMl-0007Gz-Sj for guile-user@m.gmane.org; Mon, 11 Sep 2017 20:23:27 +0200 Original-Received: from localhost ([::1]:59762 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTMt-0007BF-0I for guile-user@m.gmane.org; Mon, 11 Sep 2017 14:23:35 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:59583) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTDA-0006Ad-EY for guile-user@gnu.org; Mon, 11 Sep 2017 14:13:33 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1drTD7-0001L6-7c for guile-user@gnu.org; Mon, 11 Sep 2017 14:13:32 -0400 Original-Received: from v-smtpout1.han.skanova.net ([81.236.60.154]:34266) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1drTD7-0001JN-0C for guile-user@gnu.org; Mon, 11 Sep 2017 14:13:29 -0400 Original-Received: from [10.0.1.4] ([90.227.203.185]) by cmsmtp with SMTP id rTD4d5N29PPY2rTD4d1W4w; Mon, 11 Sep 2017 20:13:26 +0200 In-Reply-To: X-Mailer: Apple Mail (2.3273) X-CMAE-Envelope: MS4wfPi8mZHSrrqknwlajkhwEJtCzix48efE3s6WQvQmlhNDfqsXsZ/8DRNEeZ6sNzr3ltOx0go5gsa1KnRcgQQfiPz6AtWLTnxquiEHI2dL02YcUXOq4V3E 6PJ2BtYtRHfaIh8S+tYjK9CiepdcQoo/21pqtEXQDShRDbi/09WHK3MO8QaUHenq2/BvJGZluZs3YOsv3OfrEeNxWfAEwe8nrEU= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 81.236.60.154 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:14121 Archived-At: > On 9 Sep 2017, at 21:47, Linas Vepstas wrote: >=20 > I've been experiencing problems with guile GC for a while now, that = I've > mostly been able to ignore, but that are becoming debilitating, and I = need > help, or need to find some solution. >=20 > Background: I have a C++ library that has guile bindings; it is = wrapped up > as a guile module. It manages a lot of data, anywhere from a few = gigabytes, > to a over a hundred gigabytes. Almost all this RAM usage should be > invisible, opaque to guile: so, for example, while my library is = managing > 5 or 10 GBytes, guile itself is managing maybe 20 or 50 or 100MBytes, > according to (gc-stats). This is normal and expected. Some of my = programs > have guile use considerably more than this, in the multi-GByte range >=20 > However, the larger the RAM usage, the more frequently guile GC seems = to > run, and the longer it takes, so that by the time my dataset is = 100GBytes, > guile is using maybe 1GBytes, and is spending 99% of its time in GC. = My app > slows to a crawl, and nothing gets done. I need to get GC under = control, > because it is running pointlessly, because its clear that (almost) = nothing > can be reclaimed, (usually) nothing is being reclaimed. Does it have to do with this issue [1], that the Boehm GC initially uses = a small amount of heap. One can use say export GC_INITIAL_HEAP_SIZE=3D50G on the shell command line before running the program. 1. https://github.com/Macaulay2/M2/issues/500