From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Amirouche Newsgroups: gmane.lisp.guile.user Subject: Re: How to make GNU Guile more successful Date: Fri, 10 Mar 2017 21:17:37 +0100 Message-ID: <4b508a2c-92fa-aa24-5780-e13397819261@hypermove.net> References: <58C20A87.9010908@sapo.pt> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1489177099 18237 195.159.176.226 (10 Mar 2017 20:18:19 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 10 Mar 2017 20:18:19 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Fri Mar 10 21:18:11 2017 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cmQzE-0003N0-QH for guile-user@m.gmane.org; Fri, 10 Mar 2017 21:18:05 +0100 Original-Received: from localhost ([::1]:40830 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cmQzK-0002pK-OF for guile-user@m.gmane.org; Fri, 10 Mar 2017 15:18:10 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:49521) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cmQyu-0002pD-Dw for guile-user@gnu.org; Fri, 10 Mar 2017 15:17:46 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cmQyq-0005pV-9u for guile-user@gnu.org; Fri, 10 Mar 2017 15:17:44 -0500 Original-Received: from relay5-d.mail.gandi.net ([217.70.183.197]:43311) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cmQyp-0005pO-Vr for guile-user@gnu.org; Fri, 10 Mar 2017 15:17:40 -0500 Original-Received: from mfilter24-d.gandi.net (mfilter24-d.gandi.net [217.70.178.152]) by relay5-d.mail.gandi.net (Postfix) with ESMTP id D624E41C04C for ; Fri, 10 Mar 2017 21:17:36 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mfilter24-d.gandi.net Original-Received: from relay5-d.mail.gandi.net ([IPv6:::ffff:217.70.183.197]) by mfilter24-d.gandi.net (mfilter24-d.gandi.net [::ffff:10.0.15.180]) (amavisd-new, port 10024) with ESMTP id OTAsnKPOyJgD for ; Fri, 10 Mar 2017 21:17:34 +0100 (CET) X-Originating-IP: 82.239.61.147 Original-Received: from [192.168.0.7] (aul93-4-82-239-61-147.fbx.proxad.net [82.239.61.147]) (Authenticated sender: amirouche@hypermove.net) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 4EBED41C08E for ; Fri, 10 Mar 2017 21:17:34 +0100 (CET) In-Reply-To: <58C20A87.9010908@sapo.pt> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 217.70.183.197 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:13485 Archived-At: Héllo! Le 10/03/2017 à 03:08, Vítor De Araújo a écrit : > Hi, > > I'm relatively new to Guile and new to this list, but I saw this > thread in the archives and I'd like to make some comments. Welcome! > > First, I think it would be nice to have a Guile wiki, in the likes of > cliki.net. A publicly-editable reference place where people can share > knowledge about Guile and Guile libraries and projects. I agree. > Sure, having a > centralized package manager/repository would be nice, but just having > a common place where people can look up what libraries exist and where > they can be downloaded would be nice. (I don't know how easy would it > be to set up a wiki at the Guile website.) The current most comprehensive list of GNU Guile package is available at http://sph.mn/content/3e73 There is also the github org: https://github.com/guildhall > On the topic of package management, I think that, more important than > a central repository, is a standard format in which to distribute > packages for easy installation. I agree. The standard could be to have a tarball named using semantic versionning, from which one can extract the PACKAGE-NAME. Inside the tarball, the package manager will expect a PACKAGE-NAME.scm or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile WDYT? > I've never used guildhall, and I guess > it defines something like this, Probably but it's not documented. > but I think just having an > installation command where you can pass the URL of a package to > download and install (say, guild install http://some.url/somepackage.zip, > or even a git URL) This will still require the user to set GUILE_LOAD_PATH. Maybe we can add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things easier. A simple guild install https://url/to/patch can be done in mater of hours (if not minutes). WDYT? > would already go a long way towards making it > easier for people to use third-party libraries in Guile. I agree. > (Or maybe package _names_ could be URLs pointing to an index in a > well-defined format, which tells how to download each version of a > package; then we could have even dependency management without a > central repository. I'm pipe-dreaming here, though.) This seems fun, but at the same time odd. FWIW, I've started a new effort to build a package manager for GNU Guile. You can find a demo at http://guildhall.hypermove.net/ It started as a guildhall web frontend but I noticed that guildhall: 1) doesn't work with guile 2.2 and I can not make it work 2) The solver is too complicated, even if it's based on aptitude algorithm it's still complicated. 3) use a lot of other scheme libraries [0] (like foof-loop) which doesn't help noobs like me to dive into the code. Maybe those libraries are *very* neat but why not include them in Guile proper then and make them default. [0] https://github.com/ijp/guildhall/tree/master/guildhall/ext For all this reasons I forked the effort. I address each previous point as follow: 1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's bad this should be addressed at some point. AFAIR the issue is in guile-wiredtiger (or it's an issue with setlocale (anyway this must be addressed))) 2) I use a logic programming library called minikanren to solve the dependency problem. This is the first real problem I can use minikanren to solve a problem that is craft. This is the logic programming library in scheme. While it's not very advanced compared to core.logic, the version I use called microkaren has a straight forward implementation that can hold all at once in a single head. Logic programming, like probabilistic programming (cf. ) are two research areas in programming languages that could improve the way we craft algorithms today. The logic approach being more useful to me (but someone doing machine learning stuff, might find the probabilistic scheme very useful. Ask me for a link!). OpenCog does have a logic language embedded in it's language in their hyper graph database (AFAIU it works using backward / forward inference but implements on top of it higher logic constructs, like abduction, anaphora resolution...). It's something they call PLN as Probabilistic Logic Network. AFAIK it's some kind of probabilistic Datalog (which is one of the API Datomic support). There is also probabilistic minikanren (albeit not useable). So, to sum up (because I can (because I am not a robot)): programming language research + guile + logic = minikanren. Mind the fact that there is also a guile-log. But I still don't get what guile-log does (hints: http://c-lambda.se/) whereas I understand how microkanren does its stuff (I still need to benchmark it). 3) I don't use foof-loop but I bring my own dependencies. I use my own database library that is based on wiredtiger. I've been working on this database library for 3 years, it's well documented and various example codes. That said, AFAIK, I am the only one to use it. I've built a clone of Datomic (with the persistence part, patch welcome) which use a query engine similar to Datomic I guess based on minikanren. It's performance on a middle end laptop run Guile 2.2 are the followings: - 1500 document reads per seconds - 1000 to 500 document write per second Documents are scheme assoc, and those are inserted 1000 per 1000 and read 1000 per 1000 until it reach 50Go of data. At that point writes take 1/500 seconds or 0.002 seconds. The biggest dataset I *loaded* into that database is wikidata which is 50Go, I don't remember how much time it takes to load it. I had issues with wiredtiger using the Python bindings but only during reads over a gigantic dataset of 60Giga. I say gigantic for a blog not for Google scale. Also upstream can solve issues if we can have a way to reproduce the issues (which I plan to do once guile 2.2 is out (which means I will redo the benchmarks against wikidata and read/write)). I understand the problem that wiredtiger being part of MongoDB is problematic as MongoDB might not care much for the same problem as ours. They will always be interested by bigger free software database, tho. As it make free publicity of how successful is their software. Also there is much documentation about this library. I created several Guile projects using it (albeit not big) they document several layers of the library and one Guile user reported using it for doing human/social science research. Search for guile wiredtiger in you favorite duckduckgo search engine. Some people claim that PostgreSQL has all the required feature that someone wants to store data and that this mongodb/wiredtiger is a bad. I recognize that wiredtiger can be poor man's database right now. I don't have the required expertise to verify whether it's good enough for your usecase. PostgreSQL is used in "production" in all the world using multiple workload and stuff. PostgreSQL is good. 1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you can't use pgsql from Guile RIGHT NOW. 2) wiredtiger is GOOD enough, benchmark it before saying it's not good ENOUGH. 3) it's not SQL, it's Schemey! It's inspired from datomic which the goto database for clojure with multiple implementation in the browser. This is killer feature of the clojure ecosystem that is closed source. 4) I find it more funny to fiddle with logic programming that set pseudo theory of SQL. 5) The first thing I do when pgsql bindings will be out, is to port feature space to PostgreSQL and compare both pgsql and wiredtiger. 6) wiredtiger will always be simpler to use that Postgresql it's like comparing sqlite and Datomic. What happens next? You choose to hack on this project and find something interesting in this TODO list: - fix guile-wiredtiger to work on guile 2.0 - make it possible in guile-wiredtiger to stream the results of a look up using traversi using multiple cursors - implement disequality in microkanren (for implementing the package dependency resolver and for fun because logic is awesome). - implement a pastebin service using GNU Guile, yes because I think this a good bad reason to start another simple project that is not a static blog generator. - Port the wiki of http://culturia.one to use feature space library. - implement or find a scheme library that does the diff two lists (to finish wiki implementation and for the package dependency resolver). - Help Matt Wette to complete his ffi helper http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html (because we will need it for guix). - add guile-fibers as a submodule and use it - there is also a simple boolean keyword search engine that must be ported to feature space - find the code that implements trigram transderivational search and put it in the repository. Bridging the gap with tsearch2. - index the wiki - index the packages - index the pastebin - index the web And add gnunet to the mix. There is not a lot of get-together pure guile project out there. There is guix. I hope this new project can be the occasion for new guilers to submit patches or ideas. The code is currently hosted at https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/ The codename is "primary". Also I'd like to point out, that I don't need primary. I do it for the community only. The road is fascinating, tho. I could make mistakes so please input your feedback if you want to use a tool like that. You can: guix package -i wiredtiger git clone https://framagit.org/a-guile-mind/culturia.one.git git checkout guildhall emacs culturia.one/src/webui.scm or cd culturia.one/src/ && guile -L . webui.scm