From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Marko Rauhamaa Newsgroups: gmane.lisp.guile.user Subject: Re: How to make GNU Guile more successful Date: Sun, 16 Jul 2017 12:18:18 +0300 Message-ID: <87h8ychh9h.fsf@elektro.pacujo.net> References: <87lgtajpkc.fsf@web.de> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1500196738 17585 195.159.176.226 (16 Jul 2017 09:18:58 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 16 Jul 2017 09:18:58 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) Cc: "guile-user@gnu.org" To: Freja Nordsiek Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Sun Jul 16 11:18:46 2017 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dWfhK-0003lc-79 for guile-user@m.gmane.org; Sun, 16 Jul 2017 11:18:42 +0200 Original-Received: from localhost ([::1]:44575 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dWfhP-0006gK-IZ for guile-user@m.gmane.org; Sun, 16 Jul 2017 05:18:47 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:35126) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dWfh1-0006g3-GE for guile-user@gnu.org; Sun, 16 Jul 2017 05:18:24 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dWfh0-0001Bu-EJ for guile-user@gnu.org; Sun, 16 Jul 2017 05:18:23 -0400 Original-Received: from [2001:1bc8:1a0:5384:7a2b:cbff:fe9f:e508] (port=57756 helo=pacujo.net) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dWfh0-00017G-5q for guile-user@gnu.org; Sun, 16 Jul 2017 05:18:22 -0400 Original-Received: from elektro.pacujo.net (192.168.1.200) by elektro.pacujo.net; Sun, 16 Jul 2017 12:18:18 +0300 Original-Received: by elektro.pacujo.net (sSMTP sendmail emulation); Sun, 16 Jul 2017 12:18:18 +0300 In-Reply-To: (Freja Nordsiek's message of "Sun, 16 Jul 2017 10:30:25 +0200") X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2001:1bc8:1a0:5384:7a2b:cbff:fe9f:e508 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:13936 Archived-At: Freja Nordsiek : > If I was to hazard a reason for why Guile gets very slow when loading > 20 GB or more (may or may not be related to it being buggy and > crashy), my guesses would be a lot of the data when loaded into Guile > was allocated such that the GC scans it for pointers (using > scm_gc_malloc instead of scm_gc_malloc_pointerless) which would vastly > increase the amount of memory the GC needs to scan every time it runs. Good point! If you didn't to any C programming, what kind of native Guile data structures are good for such large random-access storage? At least arrays haven't specifically been documented for such GC optimization: . Maybe bytevectors would do: . Of course constantly encoding to and decoding from a bytevector using scheme code might be very slow without the help of some binary bulk formatting facilities for the data records. Marko