From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Daniel Llorens Newsgroups: gmane.lisp.guile.user Subject: Re: Matrix or array operations library Date: Thu, 27 Dec 2018 19:43:18 +0100 Message-ID: <2FB4CE72-80C9-46CE-B33F-8D133CD77A84@bluewin.ch> References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1545936221 12977 195.159.176.226 (27 Dec 2018 18:43:41 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 27 Dec 2018 18:43:41 +0000 (UTC) To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Thu Dec 27 19:43:36 2018 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcad5-0003G3-VQ for guile-user@m.gmane.org; Thu, 27 Dec 2018 19:43:36 +0100 Original-Received: from localhost ([127.0.0.1]:54701 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gcafC-00031x-Ik for guile-user@m.gmane.org; Thu, 27 Dec 2018 13:45:46 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:46662) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gcado-0002SG-JL for guile-user@gnu.org; Thu, 27 Dec 2018 13:44:24 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gcacs-0001xC-PO for guile-user@gnu.org; Thu, 27 Dec 2018 13:43:26 -0500 Original-Received: from vimdzmsp-sfwd05.bluewin.ch ([195.186.120.133]:60122) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gcacs-0001vD-Fz for guile-user@gnu.org; Thu, 27 Dec 2018 13:43:22 -0500 Original-Received: from 4box.home ([100.91.106.162]) by vimdzmsp-sfwd05.bluewin.ch Swisscom AG with SMTP id cacog2WFHuZRucacpgiZ85; Thu, 27 Dec 2018 19:43:19 +0100 X-Bluewin-Spam-Analysis: v=2.1 cv=ZuUgU4PG c=1 sm=1 tr=0 a=E/cB+Ag6Owp6h9AtTK67vw==:117 a=E/cB+Ag6Owp6h9AtTK67vw==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=IkcTkHD0fZMA:10 a=4CtUzyQ7cbxiMAEdVpIA:9 a=QEXdDO2ut3YA:10 X-Bluewin-Spam-Score: 0.00 X-FXIT-IP: IPv4[100.91.106.162] Epoch[1545936199] X-Bluewin-AuthAs: dll@bluewin.ch In-Reply-To: X-Mailer: Apple Mail (2.3445.102.3) X-CMAE-Envelope: MS4wfDCcD9rQ4C2w08fv5Ow+CbpvxNCIhfi3DNk/t6mhx8U1qcqp9jB26E+z6j+L7XfRyG7PIc3vp5lem8oy+otqIsWMsL/g3ZYL3Z2obWbhuv275zMOsF0b pVZ2e6PMhfRfL20WHV0spKEZSjg7TV3xwQoIlMpvQzUMB7mVsSD9Qmla X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 195.186.120.133 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:15155 Archived-At: >> With regard to NumPy, especially the stuff about the "strides" = (whatever >> that is) in the data buffer of a NumPy array and how what changes = when >> creating a view and changing indices (For example I cannot imagine a >> single reason, why the ordering of how one writes the indices would >> matter, except for human readability?) =E2=80=A6 If I need to keep = track of that >> =E2=80=A6 Let's just say I dislike working on a low level with a lot = of >> pointers, addresses and segfaults ; ) So I think there are three sides to this. 1) the array type, 2) using = the array type in Scheme, 3) interfacing with C or Fortran (or = whavever). 1) Guile already has a multidimensional array type with strides and all = that. Has had it forever. The base library (array-map! etc) is low level = and doesn't do much, and the implementation has some cruft, but the type = is there, it's fairly simple stuff and and there's nothing fundamentally = wrong with it. Should be safe too. People love to write their own matrix type on top of vectors or = bytevectors or even lists. Sometimes there's a good reason for that but = often there isn't. 2) IMO the only real flaw with the current Guile array type is that it's = all in C so the array descriptors are opaque to Scheme, and the compiler = isn't smart enough yet to remove the type dispatches that result from = the base library being type generic [*]. This is an actual problem when = you do everything in Scheme, but it's just the difference between things = being slow or very slow. AFAIK this isn't any worse than what you get in Numpy. If you operate = with arrays =E2=80=98in the large=E2=80=99 (manipulating strides, etc.) = then everything is fast. Of course Numpy gives you a ton of ready made = functions that would be nice to have in Guile as well [**]. 3) If you're using Guile arrays as an interface to C or Fortran, the = above doesn't matter. Neither C nor Fortran (<90) have a standard way of = passing multidimensional arrays to a function, so you are using pointers = whether you like it or not. Every library has its own particular way of = passing strides and its own quirks about what strides they accept, etc. = You have to deal with that one way or another. Might as well have a = nice, already available array type on the Scheme side! [*] [**] I have started efforts to fix these two problems (both fizzled = due to lack of time, although I think they contain some valid stuff =E2=80= =94 e.g. if you need to slice arrays, I prefer from and amend! in = guile-ploy to the Numpy mess). I'd be happy to contribute to a common = effort if anyone else is interested. PS. Does anyone want to collaborate in wrapping HDF5? I've looked into = mwette's FFI helper for like two seconds but I couldn't get it to work.=