From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Linas Vepstas Newsgroups: gmane.lisp.guile.bugs Subject: bug#25386: Manual gc helps Date: Sat, 7 Jan 2017 20:57:26 -0600 Message-ID: References: Reply-To: linasvepstas@gmail.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Trace: blaine.gmane.org 1483844293 30759 195.159.176.226 (8 Jan 2017 02:58:13 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 8 Jan 2017 02:58:13 +0000 (UTC) To: 25386@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Sun Jan 08 03:58:10 2017 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cQ3gN-0006wS-3m for guile-bugs@m.gmane.org; Sun, 08 Jan 2017 03:58:07 +0100 Original-Received: from localhost ([::1]:60021 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cQ3gR-0004g1-CT for guile-bugs@m.gmane.org; Sat, 07 Jan 2017 21:58:11 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:56264) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cQ3gL-0004fp-FA for bug-guile@gnu.org; Sat, 07 Jan 2017 21:58:06 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cQ3gI-0008NP-CD for bug-guile@gnu.org; Sat, 07 Jan 2017 21:58:05 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:59200) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cQ3gI-0008NE-7J for bug-guile@gnu.org; Sat, 07 Jan 2017 21:58:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1cQ3gH-0007EC-U7 for bug-guile@gnu.org; Sat, 07 Jan 2017 21:58:01 -0500 X-Loop: help-debbugs@gnu.org In-Reply-To: Resent-From: Linas Vepstas Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Sun, 08 Jan 2017 02:58:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 25386 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 25386-submit@debbugs.gnu.org id=B25386.148384427527770 (code B ref 25386); Sun, 08 Jan 2017 02:58:01 +0000 Original-Received: (at 25386) by debbugs.gnu.org; 8 Jan 2017 02:57:55 +0000 Original-Received: from localhost ([127.0.0.1]:46366 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cQ3gB-0007Dq-4R for submit@debbugs.gnu.org; Sat, 07 Jan 2017 21:57:55 -0500 Original-Received: from mail-qk0-f171.google.com ([209.85.220.171]:33444) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cQ3g9-0007Dd-0T for 25386@debbugs.gnu.org; Sat, 07 Jan 2017 21:57:53 -0500 Original-Received: by mail-qk0-f171.google.com with SMTP id s140so88517545qke.0 for <25386@debbugs.gnu.org>; Sat, 07 Jan 2017 18:57:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:from:date:message-id:subject:to; bh=1kS3gkJIlLACpsGjzdh9CyHzsg5munvTtIYr1TJt+ag=; b=nsk+GXFUFbkVfZP6Dhw4BLBI4LGb2bYjpYHavxS9TVfZgHNXraCxnc9CRuM8VIHTPd mgiOYiBkazCqqb0YSZSJT6W5UtR8rjWGuzh9DhCjqw9WXpahbZS/1NWjLLmMw2nlXd7Z ERSYcZ7YpqMp+XGEy/FOnLPwP44raUg876z1N+OjMbzBPg6yQqGEJDKZ8z1ErQ5jNctC CzGSpJvR8CBMippCn/ca8aCFJa5epLPOshJM45F/qnVNuq/UhCTynE8ZKAlBNVRqhS5B HFZTk2wAZ9gDk+Pg0bXMx/j3jqIAsmgaSWZI9jmdWjuRn0NVGyPcbfwTW5yZhWh1qAAC Hs9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:from:date:message-id :subject:to; bh=1kS3gkJIlLACpsGjzdh9CyHzsg5munvTtIYr1TJt+ag=; b=IYv4ANjLHc7MxW9SOVbsOz3S1IC5bffueIZeK32HQAbT4FIldCDrKUUpT1nLg+X7/w kDXDk5zuGWqQPwwpEUazd/Sy8DL2BDoBNhRnvSB8u3PyYPM8zXf1jrwkLKFHc6H2rKwp Slzd9NPERVipX2UvHt46rKpQr89jwLX+3qjb66C4SvO80Kvq/nn8EZr9ElQOB4MMBe8S KCFvr6hZGG8t7BAgY8DHOerwImhDYxZTxEafpvbSEKPBvLQxu/ujNcduApyMAPZ5/tk8 /W0OWJVjyZm79aaATruxL7C0oOH6RDKI2Id20quA/5r6Xei4ZncLnWZ2xUJLYpv84EV3 6lWQ== X-Gm-Message-State: AIkVDXJNU1qz9fxMXDCCbD1ZGteBOUQ18X+iIT6dKq59UyvhHLsQFs4/TTSMn3bZG7BxSr0rjBa0XawQ58Gjew== X-Received: by 10.55.198.149 with SMTP id s21mr89211348qkl.196.1483844267290; Sat, 07 Jan 2017 18:57:47 -0800 (PST) Original-Received: by 10.12.128.78 with HTTP; Sat, 7 Jan 2017 18:57:26 -0800 (PST) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:8498 Archived-At: I did a fairly through review of the thread-creation and thread-join code in the git master branch, and it looks to be just fine. Thus, some experimentation is in order: Going back to guile-2.0, I see this behavior: guile -v guile (GNU Guile) 2.0.11 Packaged by Debian (2.0.11-deb+1-10) If I add a manual gc to the exit of the thread, like so: (define (mkthr v) (call-with-new-thread (lambda () (set! junk (+ junk 1)) (gc) ))) then the heap blows up, in minutes, to about 180MB but then stops growing, even after hours and millions of thread creates: (heap-size . 183734272) (gc-times . 1957954) If I gc only every third thread create, it quickly blows up to about 400MB, and then stablizes, for hours: (heap-size . 428638208) (gc-times . 1292663) If I gc every 17th thread, it blows up to about 1.8GB and then is stable: (heap-size . 1875902464) (gc-times . 327462) This last one after about 5.5 million thread creates and joins. The counting is done like so: (define (mkthr v) (call-with-new-thread (lambda () (lock-mutex mtx) (if (eq? 0 (modulo junk 17)) (gc)) (set! junk (+ junk 1)) (unlock-mutex mtx) ))) In each case, it seems to hit a plateau at about (n+1)*100MB when gc is done on one out of every n threads. This seems quite bizarre to me: why does this inverted relation on number of gcs vs number of thread creates? What's magic about 100MB? Clearly 100MB is wayyy too large for this very simple program. I mean, even if I gc at *every* thread-exit ... (I have not yet explored above in guile-2.2) Since I cannot find any 'obvious' bugs in guile, this suggests some strange stochastic behavior in bdw-gc?