From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Linas Vepstas Newsgroups: gmane.lisp.guile.user Subject: Re: Matrix or array operations library Date: Sun, 2 Jun 2019 16:44:56 -0500 Message-ID: References: <7a362a59-3a4d-44c8-f4cd-5b232b6dffce@gmail.com> Reply-To: linasvepstas@gmail.com Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="238231"; mail-complaints-to="usenet@blaine.gmane.org" Cc: Guile User To: Zelphir Kaltstahl Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Sun Jun 02 23:45:39 2019 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hXYIM-000zq8-KW for guile-user@m.gmane.org; Sun, 02 Jun 2019 23:45:39 +0200 Original-Received: from localhost ([127.0.0.1]:53560 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXYIK-0002bA-Sz for guile-user@m.gmane.org; Sun, 02 Jun 2019 17:45:36 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:59949) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXYHv-0002ao-Iv for guile-user@gnu.org; Sun, 02 Jun 2019 17:45:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hXYHu-0001M2-0F for guile-user@gnu.org; Sun, 02 Jun 2019 17:45:11 -0400 Original-Received: from mail-lf1-x131.google.com ([2a00:1450:4864:20::131]:39786) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hXYHt-0001L3-Ni for guile-user@gnu.org; Sun, 02 Jun 2019 17:45:09 -0400 Original-Received: by mail-lf1-x131.google.com with SMTP id p24so5517434lfo.6 for ; Sun, 02 Jun 2019 14:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=25AdWqMVfRJNLQ6iuAHADGI7mDOQOjmpt/gGHaJGvq0=; b=Kwwv+gqXNR3SN2siCMnX+CBN8rFzw6kJ8qY8tTfS6EZugjEAHSOX5I5R41ceEY2ntn kaUEdb5dYrb8zV3L7qAu6kJ91UbznLxj4fHXR+ALw6G9PKgERuXwOO0QJJqd4tyUXb3m NGX3sdTDsVDKdVsKRf7QP5V80mmrUTL6a1MqxqMmyjsnQUBwNYOph+6FalCKAXkmJ7Su ESdPK51sLdhQ5g216R5bys1meDlqEftuZc6wubt7Q/ZYUrOcvSu0CSMOBCeyHhdiAmzs NqNSZMft24qIVeCJ2X3YVnL/4f11qLfyGpRLnd+Xe53CgYZIyjjMVCW1jgxAQnXe1Zvz WBNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=25AdWqMVfRJNLQ6iuAHADGI7mDOQOjmpt/gGHaJGvq0=; b=SU+A7GzGeczrAYjRhyC+wcl60Vo+RsJ9z1VfphS+N5WC4cYCasUp1UOZPIWPlNTqZY Lg2tk0X9yiWHqqRU4egxvWmpOwgHBaUXsh7Yn2Z1gQLFvbfmhh6BBFE3GScm2InhxeiE ZEhG546QHwKXxCy5B1uy/rwMVcBlyfL86pwtAodZVGd4EkFxsuDbf92B087SdK8PF9ii LnaYX+L39yYzzr/rlFNuUK7p1p6dGKhQzAaJhk/HLx9H5l+u0CnNhH7M+TB2X59Q47E3 J/Es6IkOggZooPuMwvGAIIAMM2S6zxA/ucK3jGF5aXumhCQES6sxKlGF2w1zGm0Oklmy Iaow== X-Gm-Message-State: APjAAAWIUhhXqSCvGzPVyLmNFDLq5MeDZDJDxc7fGwvSXUozkKHmzdd7 sSQTjWnjB7s9USM1YYfJ9px7+iKo188Pm1P4hXmg8Mu5qgI= X-Google-Smtp-Source: APXvYqyUwunwedxhddD1vrZjEwNIMgRDhVuvnMEEosyNhwsE1MLr3MNUhSPMZIfX0aVyrb2XRv+9udiJPhQ4IHxT/A4= X-Received: by 2002:a19:ed07:: with SMTP id y7mr12494505lfy.56.1559511908097; Sun, 02 Jun 2019 14:45:08 -0700 (PDT) In-Reply-To: <7a362a59-3a4d-44c8-f4cd-5b232b6dffce@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::131 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:15524 Archived-At: Me too. With one significant difference: I have extremely sparse arrays. Like, only one-in-a-million array entries are non-zero. And my arrays are hug -- say 2M by 2M, for a total of 4 tera-entries, of which only one in a million are non-zero, so in fact, my data might fit in a gigabyte or less. (instead of 32 terabytes of RAM) I solved this by writing my own ad-hoc library. The library user needs to declare a half-dozen methods, indicating how to fetch the matrix entries, what the dimensions of the array are, and methods to loop over the non-zero row and column entries. The library does the rest. My code is quasi-special-purpose, and, well, just right for me, but a serious dedicated library programmer could do a much better job than I can. My code is here: https://github.com/opencog/atomspace/tree/master/opencog/matrix The low-level API that the user must declare is here: https://github.com/opencog/atomspace/blob/master/opencog/matrix/object-api.scm Hmm, maybe I should pull this out of the project, and make it generic.... -- Linas On Mon, Dec 24, 2018 at 4:01 PM Zelphir Kaltstahl < zelphirkaltstahl@gmail.com> wrote: > Hello Guile Users, > > Is there some library, that enables high performance matrix operations > or even n-dimensional array operations? I am thinking of something like > NumPy in the Python ecosystem. I think NumPy in turn also uses some > lower level thing to do what it does. I think OpenBLAS or MKL, depending > on the architecture. I wonder if there is any wrapper around OpenBLAS > for Guile or something similar. > > I am writing a little code for matrix operations and currently I am > using Guile arrays, as they are made of vectors and have constant access > time, which is already great. My guess is, that this would be the right > choice if using pure Guile. I am writing data abstraction procedures, so > that later on I could exchange what is used to represent the data. > > Maybe, if there is something like NumPy or lower level, I should use > that instead? (Would I have to learn how to use FFI first?) > > Or maybe Guile's implementation is already so fast, that it would not > make that much difference to use a lower level thing? > > Currently I have only a little experimental program, started today, so > no huge plan. OK, one can fantasize about stuff like Pandas data frames > in Guile, but I have no illusion, that it is a work of a few days or > even weeks. It would be nice to learn, how to use a low level thing or > maybe even Pandas, if there are any such bindings for Guile. I could > make the implementation use different representations, depending on a > parameter or something like that. > > Regards, > > Zelphir > > > -- cassette tapes - analog TV - film cameras - you