From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Roland Orre Newsgroups: gmane.lisp.guile.user Subject: Re: What is best way to limit memory alloc? Date: Wed, 29 Aug 2007 12:22:47 +0200 Message-ID: <1188382968.8043.578.camel@localhost.localdomain> References: <1188367992.8043.508.camel@localhost.localdomain> NNTP-Posting-Host: dough.gmane.org Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Trace: sea.gmane.org 1188383101 20748 80.91.229.10 (29 Aug 2007 10:25:01 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Wed, 29 Aug 2007 10:25:01 +0000 (UTC) To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Wed Aug 29 12:29:32 2007 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by dough.gmane.org with esmtp (Exim 4.50) id 1IQKnj-00055Z-6U for guile-user@m.gmane.org; Wed, 29 Aug 2007 12:29:31 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1IQKjI-0000ZZ-7M for guile-user@m.gmane.org; Wed, 29 Aug 2007 06:24:56 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1IQKhL-00005Z-Ko for guile-user@gnu.org; Wed, 29 Aug 2007 06:22:56 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1IQKhH-0008WR-Ug for guile-user@gnu.org; Wed, 29 Aug 2007 06:22:53 -0400 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1IQKhH-0008W8-7G for guile-user@gnu.org; Wed, 29 Aug 2007 06:22:51 -0400 Original-Received: from [195.47.247.21] (helo=csmtp1.b-one.net) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1IQKhG-0005cw-QP for guile-user@gnu.org; Wed, 29 Aug 2007 06:22:50 -0400 Original-Received: from dyna224-140.nada.kth.se (dyna224-140.nada.kth.se [130.237.224.140]) by csmtp1.b-one.net (Postfix) with ESMTP id 84F72106902D; Wed, 29 Aug 2007 12:22:49 +0200 (CEST) In-Reply-To: <1188367992.8043.508.camel@localhost.localdomain> X-Mailer: Evolution 2.6.1 X-Detected-Kernel: Linux 2.6 (newer, 3) X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-user-bounces+guile-user=m.gmane.org@gnu.org Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.user:6113 Archived-At: I found out that I had already solved the memory allocation problem in one way a few years ago. With the help of a small routine gc-heap-size, which accesses scm_i_master_freelist.heap_size scm_i_master_freelist2.heap_size I did: (define gc-heap1 (gc-heap-size 1)) (define gc-heap2 (gc-heap-size 2)) (let loop .... (gc-heap-size 1 gc-heap1) (gc-heap-size 2 gc-heap2) (loop ...)) By not allowing the heap size to increase. Then it worked, the memory just increased to 7.1 GB instead of 8.8 GB and it was then stable there during the whole run. Is there any other better way to solve this? I would like to be able to say something like: don't allocate any memory above nn GB otherwise abort, as a swapping guile process is just dead anyway. The job now took just 19 minutes to finish, the other one which is still swapping, has been running since Monday (52 hours) and is just half ready... /Roland On Wed, 2007-08-29 at 08:13 +0200, Roland Orre wrote: > What is the best way to limit the memory allocation in guile? > > I have a process that has allocated 6GB when the first step > is finished, this is much, but is fine as the machine has 8 GB. > > Then when performing the last step, just involving reading > from a file, doing some string split for each line, adding > some data and writing it back to the file line per line, the > memory suddenly increases to 8.8 GB and then the job starts > swapping ridicously and stops of course. > > It seems as the memory allocator in guile doesn't care about > the physical memory limits and it seems as it prefers to > allocate big chunks of more memory instead of performing gc > despite very little extra memory should be needed. > > I'm still running 1.7 as I haven't got the time and energy > to change the array implementation yet. > > I assume I have to change something in gc.c, the simplest I > would like to say is, don't allocate more memory than this! > > /Roland Orre > > _______________________________________________ Guile-user mailing list Guile-user@gnu.org http://lists.gnu.org/mailman/listinfo/guile-user