From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Nala Ginrut Newsgroups: gmane.lisp.guile.user Subject: Re: How to make GNU Guile more successful Date: Sat, 15 Jul 2017 20:55:39 +0800 Message-ID: References: <87lgtajpkc.fsf@web.de> <408d91ab-83e4-44ab-bf43-7b46c311a14e@email.android.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1500123379 1218 195.159.176.226 (15 Jul 2017 12:56:19 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 15 Jul 2017 12:56:19 +0000 (UTC) Cc: "guile-user@gnu.org" To: Jan Wedekind Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Sat Jul 15 14:56:14 2017 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dWMcF-0008H3-9O for guile-user@m.gmane.org; Sat, 15 Jul 2017 14:56:11 +0200 Original-Received: from localhost ([::1]:42064 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dWMcK-00055y-TB for guile-user@m.gmane.org; Sat, 15 Jul 2017 08:56:16 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39006) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dWMbq-00055f-66 for guile-user@gnu.org; Sat, 15 Jul 2017 08:55:48 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dWMbo-0006Mv-IQ for guile-user@gnu.org; Sat, 15 Jul 2017 08:55:46 -0400 Original-Received: from mail-it0-x230.google.com ([2607:f8b0:4001:c0b::230]:37793) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dWMbo-0006KO-7c for guile-user@gnu.org; Sat, 15 Jul 2017 08:55:44 -0400 Original-Received: by mail-it0-x230.google.com with SMTP id m84so36767474ita.0 for ; Sat, 15 Jul 2017 05:55:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=I5YM1L/HNOpVP590EbpvNRowMpW8M682AfsfUgmLlIE=; b=Ueqf57+L9kvseCLyzFM0UKYeXkE6Y4Ld4bUJNeqW93nEXQPkS7ZrjYlczLK7lKMOXe UESd96WPp/BAEwNwoH5wTnAODcZgpVZdtoQhau3ozsAZQ8ekiwjAZ2qFCW1bI4P0n5uE YY6FbG68L77LARR/pZwXPM/gdr68/IzstcnVeq7Ebm7K+KHaFKeetAT0GitYddyZUFqW J80RhJPIky80AoWdUQcAZk1zZm8iQ1xwwFiOC4u+UDTnbrtWX20/9SL7daYfiXTZjp3x 9dpR4K8W0comQ8by7cUfjpRdQdi3uO7Q6VaJT20iEOfYdNxf/Is/4jcQ7As7gwWAQDDz YuJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=I5YM1L/HNOpVP590EbpvNRowMpW8M682AfsfUgmLlIE=; b=aMvcM8nCkDfhX63va1jpOd1QqiAFTMce1X6L3sodbS3RvGZkfO5mFNt+Xw6hkUoFAH uOYA9zwn1nE6aTTMhp4Cx/ZhDnpsavvYCHOUhdfZNmACvnmSfpFTE96nIRIoCCsRVXAm z5dEuY93JYAa1s1225MYml+tzUWwg0PYvr59x8gjPy365dGumJYYftq6SshJvpf5O50X 3X8GtN3fheIgt7YMwEz4/Nf1jvw6t4WmOZc5aSQbCFw0CxmjDNvNS+EB9cfxXei+LeA6 F6UnxpJ+ENcrXuxGSP4z0islS9daSMcyfiMlmkbPINyNx7WmMgryBslU5UwDFcliWBSl oU2A== X-Gm-Message-State: AIVw113J5xoZKaZ85xaXZEFjzPqY2d5VsLK1NpMbWM4K5vdX2DfZ6VuD ugAY3NZfq/CV5XtRgiWOmQWPkUPzCQ== X-Received: by 10.36.51.208 with SMTP id k199mr1056806itk.8.1500123340536; Sat, 15 Jul 2017 05:55:40 -0700 (PDT) Original-Received: by 10.107.199.1 with HTTP; Sat, 15 Jul 2017 05:55:39 -0700 (PDT) In-Reply-To: <408d91ab-83e4-44ab-bf43-7b46c311a14e@email.android.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4001:c0b::230 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:13930 Archived-At: @Linas Thanks for sharing your experiences on large data processing using Guile! I'm developing a framework to do the similar work for our product too. But what I'm planning is to do map/reduce for large data in a distributed system. Anyway, I don't think it's a good way to analysis massive data in just one very-strong machine. It requires too much for the language compiler. I'm still very confident to use Guile on machine learning work. Please don't forget that no any dynamic language do the heavy computation directly, they wrapped BLAS or MKL, just like what numpy does. And I have chance to get a very strong Nvidia GPU cluster server for the work on the next stage. I don't think the computing speed will be the issue for me. We're using Python and C++14 at present, but I don't think Python is the idea glue language, no one wants to use Cython, trust me. The tail-recursive and delimited-continuation for lightweight task scheduling is preferred too. The key issue for me is the expressiveness, which could let us type less code to do more work. Actually, we use Guile to generate many C++ code in compiling time to save lot work coding work. This does matter, especially when you have few people but much work to finish in a tight deadline. There should be better paradigm to use Guile to glue C/C++ code. Implement algorithm in Guile directly is not the good way. And wrap C++ interface with FFI simply is not good too. Sometimes we need to do more abstract work to get both advantages of Guile and C/C++. This is about how we use Guile, not the compiler issue. Of course, I confess there're many problems in our current Guile compiler. The problem is that we have to use Guile a lot to get to know what is the problem exactly, and give compelling reasons for the improvement. @Jan Yes, that should be a way to go. And I have a new idea which is just an idea at present. Many we could find a way to read PyObject to Gulie, and call Python module directly (say, numpy). There should be a type-compatible abstract level between Guile and PyObject. If it works, we may implement Python3 on Guile. Although it seems a large work to implement complete Python3 frontend, we may save lot of work to write alternative Python modules for Guile. Julia language does in this idea, but it's backend is compatible with Python. My idea is not to convert all Python types to Guile, just wrap some types to a special object like is enough, then Guile could be glue language for Python too. Maybe people ask, why bother to glue Python? Python rocks! Rocks? I need complete lambda calculus, proper tail call, static scoping, and delimited-continuations (not the generators). In our product code, we use lot of lambdas in C++ code, so I expect the programming mind could be consistent between glue language and main language. But Python can't provide multi-lines lambda expression only because it uses whitespaces for delimiters, there's no any chance to delimit multi expression within a lambda unless introducing more complexity to the parser. Back to the topic, Guile lacks sufficient practical usage in industry to reveal its disadvantages. But its advantages are apparent enough for me so that it worth for me to go further with it. Best regards. On Sat, Jul 15, 2017 at 6:10 PM, Jan Wedekind wrote: > One could implement something like Theano+NumPy with GNU Guile. I am trying to do something like that (github.com/wedesoft/aiscm) but I am doing it in my spare time only. Theoretically GNU Guile is better suited for this than Python because of macros. > > On July 14, 2017 10:54:45 PM GMT+01:00, Linas Vepstas wrote: >>On Mon, Feb 13, 2017 at 2:28 PM, Panicz Maciej Godek >>>> wrote: >> >>> >>> someone >>> responded critically: "are there out of the box libraries to estimate >>a >>> zero inflated negative >>> binomial regression model in guile". Of course, if I knew what a >>> zero-inflated >>> negative binomial regression model, I could deliver an implementation >>by >>> just explaining >>> the notions used in that phrase. >> >> >>Caution: the message below sounds negative. Sorry, I use guile daily >>and >>almost exclusively now. So there ... >> >>Lack of decent science libraries for scheme is a major stumbling block, >>for >>me. Simply having sine and cosine is not enough. I got excited (a >>decade >>ago) when I realized that guile supported GnuMP, and then rapidly >>deflated >>when I realized it only supported integers and rationals in GnuMP .. I >>work >>with arbitrary-precision floats. Or, I did back then. >> >>Maybe more important is making guile work well with large-RAM setups. >>Currently, I do data analysis, every day, in guile, on datasets that >>take >>20GB or 40GB -- my current one is 110GB when loaded in RAM, and guile >>starts getting buggy, crashy and slow when working at that size. >>Sometimes, it starts calling GC half-a-dozen times per second, for no >>apparent reason, eating up 6 cores (or more!) doing nothing but GC. >>Why? >>Who knows? Who can tell? >> >>Yes, I have a machine with 256 GB RAM and a few dozen cores, and SSD's >>that >>hold the data, but every time guile crashes, I have to wait an hour for >>the >>data to reload. I can live with it, but its a dirty secret I would not >>share with guile wannabe users. >> >>String handling in guile is a disaster area: If I give it a >>10-megabyte-long string in utf8, it promptly tries to convert all of >>that >>string in utf32, for utterly pointless reasons. This just makes it >>slow. >> >>There are still bugs between GC and the compiler: if call (eval "(some >>stuff) (other stuff)") the compiler will try to compile that string >>(after >>it was converted ti utf32!) and if GC happens to run at just that >>moment, >>guile crashes or hangs. These bugs need to be fixed. >> >>So although its a good start, there's a lot of work left until it can >>get >>to "the next level". And that work can't happen until guile is more >>popular. So it's very much chicken-and-egg scenario. >> >>--linas > > -- > Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.