From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Amirouche Newsgroups: gmane.lisp.guile.user Subject: Re: Yet another GNU Guile package manager (Fwd: Re: How to make GNU Guile more successful) Date: Sat, 11 Mar 2017 17:49:08 +0100 Message-ID: <179cbaf2-6327-2b1b-c963-44bb0902fb0d@hypermove.net> References: <4b508a2c-92fa-aa24-5780-e13397819261@hypermove.net> <02da6143-6733-683c-6d07-0fdc274413f2@hypermove.net> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1489250993 10695 195.159.176.226 (11 Mar 2017 16:49:53 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 11 Mar 2017 16:49:53 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 To: Nala Ginrut , "guile-user@gnu.org" Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Sat Mar 11 17:49:45 2017 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cmkD9-0001qG-Ts for guile-user@m.gmane.org; Sat, 11 Mar 2017 17:49:44 +0100 Original-Received: from localhost ([::1]:43960 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cmkDG-0005KN-2B for guile-user@m.gmane.org; Sat, 11 Mar 2017 11:49:50 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:49813) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cmkCe-0005I2-H2 for guile-user@gnu.org; Sat, 11 Mar 2017 11:49:14 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cmkCa-0008NG-Hx for guile-user@gnu.org; Sat, 11 Mar 2017 11:49:12 -0500 Original-Received: from relay6-d.mail.gandi.net ([217.70.183.198]:35351) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cmkCa-0008Lu-7G for guile-user@gnu.org; Sat, 11 Mar 2017 11:49:08 -0500 Original-Received: from mfilter5-d.gandi.net (mfilter5-d.gandi.net [217.70.178.132]) by relay6-d.mail.gandi.net (Postfix) with ESMTP id D509BFB88B; Sat, 11 Mar 2017 17:49:05 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mfilter5-d.gandi.net Original-Received: from relay6-d.mail.gandi.net ([IPv6:::ffff:217.70.183.198]) by mfilter5-d.gandi.net (mfilter5-d.gandi.net [::ffff:10.0.15.180]) (amavisd-new, port 10024) with ESMTP id Lb31ukmRQKuw; Sat, 11 Mar 2017 17:49:03 +0100 (CET) X-Originating-IP: 82.239.61.147 Original-Received: from [192.168.0.7] (aul93-4-82-239-61-147.fbx.proxad.net [82.239.61.147]) (Authenticated sender: amirouche@hypermove.net) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 5B19EFB87E; Sat, 11 Mar 2017 17:49:02 +0100 (CET) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 217.70.183.198 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:13498 Archived-At: Le 11/03/2017 à 02:56, Nala Ginrut a écrit : > > Hi Amiroiche! > Héllo Nala! > FWIW, usually there are 3 kind s of pkg-manager: > > 1. SPM(system pkg manager) > It's used for building a distro, like RPM, deb, Guix. > > 2. LPM(language pkg manager) > It's used for specific language to install optional packages. Like > npm, gem, pip, etc. > Guix could do that too, IMO. > > 3. APM(application pkg manager) > It's used for specific framework to install packages in local > application directory. This is important for deployment. > Ruby has "bundle" for that, and Python usually use pip+virtualenv for > that, and npm is flexible enough to do that for JS too. Of course > there are alternative tools do the similar thing in each community. > > For now, I'm concerning an APM for Artanis. > I think right now I will do Language Package Manager. I'd like to focus on community building feature that helps document GNU Guile, like: - wiki - discussion pages - pastebin Code is always code but I need to choose which feature I will implement given I have not an infinite set of time. I have all those feature in some project, but now I need to put them together. > I've discussed with @Ludo the possibility to take advantage of Guix > (as a core lib) to implement it. > > If you want to write yet another pkg manager, it's better to confirm > which kind are you going to do, and don't forget to reuse some cool > features exists in Guix. :-) > I don't think Guix can help right now. The only issue that intersect with guix is dependency resolution, and AFAIK Guix does hardcode dependencies in the sens it's a simple breath first search. And what I planned to do is implement semantic versioning à la npmjs (or aptitude) https://docs.npmjs.com/misc/semver I think guix web, is enough for guix. I am not a regular guixsd user anymore, so I am not sure how to help guix/sd. A far reaching goal can be to link against guile-git to mix both package management with forge. > > Amirouche >于2017年3月11日周六 04:27写道： > > There is compan, there is guildhall, there is guix. They shall be > primary. > > > > -------- Message transféré -------- > Sujet : Re: How to make GNU Guile more successful > Date : Fri, 10 Mar 2017 21:17:37 +0100 > De : Amirouche > > Pour : guile-user@gnu.org > > > > Héllo! > > > Le 10/03/2017 à 03:08, Vítor De Araújo a écrit : > > Hi, > > > > I'm relatively new to Guile and new to this list, but I saw this > > thread in the archives and I'd like to make some comments. > > Welcome! > > > > > First, I think it would be nice to have a Guile wiki, in the > likes of > > cliki.net . A publicly-editable reference > place where people can share > > knowledge about Guile and Guile libraries and projects. > I agree. > > Sure, having a > > centralized package manager/repository would be nice, but just > having > > a common place where people can look up what libraries exist and > where > > they can be downloaded would be nice. (I don't know how easy > would it > > be to set up a wiki at the Guile website.) > > The current most comprehensive list of GNU Guile package is > available at http://sph.mn/content/3e73 > > There is also the github org: > > https://github.com/guildhall > > > On the topic of package management, I think that, more important > than > > a central repository, is a standard format in which to distribute > > packages for easy installation. > I agree. The standard could be to have a tarball named using > semantic versionning, from which one can extract the PACKAGE-NAME. > Inside the tarball, the package manager will expect a PACKAGE-NAME.scm > or a PACKAGE-NAME directory which can be installed into > ~/.local/lib/guile > > WDYT? > > > I've never used guildhall, and I guess > > it defines something like this, > Probably but it's not documented. > > > but I think just having an > > installation command where you can pass the URL of a package to > > download and install (say, guild install > http://some.url/somepackage.zip, > > or even a git URL) > This will still require the user to set GUILE_LOAD_PATH. Maybe we can > add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things > easier. > > A simple guild install https://url/to/patch can be done in mater > of hours > (if not minutes). > > WDYT? > > > would already go a long way towards making it > > easier for people to use third-party libraries in Guile. > I agree. > > (Or maybe package _names_ could be URLs pointing to an index in a > > well-defined format, which tells how to download each version of a > > package; then we could have even dependency management without a > > central repository. I'm pipe-dreaming here, though.) > This seems fun, but at the same time odd. > > FWIW, I've started a new effort to build a package manager for GNU > Guile. You can find a demo at http://guildhall.hypermove.net/ > > It started as a guildhall web frontend but I noticed that guildhall: > > 1) doesn't work with guile 2.2 and I can not make it work > 2) The solver is too complicated, even if it's based on aptitude > algorithm > it's still complicated. > 3) use a lot of other scheme libraries [0] (like foof-loop) which > doesn't help noobs like me to dive into the code. Maybe those > libraries are *very* neat but why not include them in Guile proper > then and make them default. > > [0] https://github.com/ijp/guildhall/tree/master/guildhall/ext > > For all this reasons I forked the effort. > > I address each previous point as follow: > > 1) I only use guile 2.2, right now guile 2.1 is not supported. I > think it's > bad this should be addressed at some point. AFAIR the issue is in > guile-wiredtiger (or it's an issue with setlocale (anyway this must be > addressed))) > > 2) I use a logic programming library called minikanren to solve the > dependency problem. This is the first real problem I can use > minikanren > to solve a problem that is craft. This is the logic programming > library in scheme. While it's not very advanced compared to > core.logic, > the version I use called microkaren has a straight forward > implementation that can > hold all at once in a single head. Logic programming, like > probabilistic > programming (cf. ) are two research areas in programming languages > that could improve the way we craft algorithms today. The logic > approach > being more useful to me (but someone doing machine learning stuff, > might find the probabilistic scheme very useful. Ask me for a link!). > OpenCog > does have a logic language embedded in it's language in their hyper > graph database > (AFAIU it works using backward / forward inference but implements > on top > of it higher logic constructs, like abduction, anaphora > resolution...). It's > something they call PLN as Probabilistic Logic Network. AFAIK it's > some kind > of probabilistic Datalog (which is one of the API Datomic support). > There is > also probabilistic minikanren (albeit not useable). So, to sum up > (because > I can (because I am not a robot)): > > programming language research + guile + logic = minikanren. > > Mind the fact that there is also a guile-log. But I still don't > get what > guile-log does (hints: http://c-lambda.se/) whereas I understand how > microkanren does its stuff (I still need to benchmark it). > > 3) I don't use foof-loop but I bring my own dependencies. I use my > own database library that is based on wiredtiger. I've been working > on this database library for 3 years, it's well documented and various > example codes. That said, AFAIK, I am the only one to use it. I've > built > a clone of Datomic (with the persistence part, patch welcome) which > use a query engine similar to Datomic I guess based on minikanren. > > It's performance on a middle end laptop run Guile 2.2 are the > followings: > > - 1500 document reads per seconds > - 1000 to 500 document write per second > > Documents are scheme assoc, and those are inserted 1000 per 1000 and > read 1000 per 1000 until it reach 50Go of data. At that point writes > take 1/500 > seconds or 0.002 seconds. > > The biggest dataset I *loaded* into that database is wikidata which is > 50Go, I > don't remember how much time it takes to load it. > > I had issues with wiredtiger using the Python bindings but only > during reads > over a gigantic dataset of 60Giga. I say gigantic for a blog not > for Google > scale. Also upstream can solve issues if we can have a way to > reproduce the > issues (which I plan to do once guile 2.2 is out (which means I will > redo the > benchmarks against wikidata and read/write)). I understand the problem > that wiredtiger being part of MongoDB is problematic as MongoDB > might not > care much for the same problem as ours. They will always be > interested by > bigger free software database, tho. As it make free publicity of how > successful > is their software. > > Also there is much documentation about this library. I created several > Guile projects > using it (albeit not big) they document several layers of the library > and one > Guile user reported using it for doing human/social science research. > > Search for guile wiredtiger in you favorite duckduckgo search engine. > > Some people claim that PostgreSQL has all the required feature that > someone wants > to store data and that this mongodb/wiredtiger is a bad. I recognize > that wiredtiger > can be poor man's database right now. I don't have the required > expertise to verify > whether it's good enough for your usecase. PostgreSQL is used in > "production" in > all the world using multiple workload and stuff. PostgreSQL is good. > > 1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise > said you > can't use pgsql from Guile RIGHT NOW. > 2) wiredtiger is GOOD enough, benchmark it before saying it's not good > ENOUGH. > 3) it's not SQL, it's Schemey! It's inspired from datomic which > the goto > database > for clojure with multiple implementation in the browser. This is > killer feature > of the clojure ecosystem that is closed source. > 4) I find it more funny to fiddle with logic programming that set > pseudo > theory of SQL. > 5) The first thing I do when pgsql bindings will be out, is to port > feature space to PostgreSQL > and compare both pgsql and wiredtiger. > 6) wiredtiger will always be simpler to use that Postgresql it's like > comparing sqlite > and Datomic. > > What happens next? You choose to hack on this project and find > something > interesting in this > TODO list: > > - fix guile-wiredtiger to work on guile 2.0 > > - make it possible in guile-wiredtiger to stream the results of a look > up using traversi using multiple cursors > > - implement disequality in microkanren (for implementing the package > dependency resolver > and for fun because logic is awesome). > > - implement a pastebin service using GNU Guile, yes because I > think this > a good bad reason to start > another simple project that is not a static blog generator. > > - Port the wiki of http://culturia.one to use feature space library. > > - implement or find a scheme library that does the diff two lists (to > finish wiki implementation > and for the package dependency resolver). > > - Help Matt Wette to complete his ffi helper > http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html > (because we will need it for guix). > > - add guile-fibers as a submodule and use it > > - there is also a simple boolean keyword search engine that must be > ported to feature space > > - find the code that implements trigram transderivational search > and put > it in the repository. Bridging > the gap with tsearch2. > > - index the wiki > > - index the packages > > - index the pastebin > > - index the web > > And add gnunet to the mix. > > There is not a lot of get-together pure guile project out there. There > is guix. I hope this new project > can be the occasion for new guilers to submit patches or ideas. > > The code is currently hosted at > https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/ > > The codename is "primary". > > Also I'd like to point out, that I don't need primary. I do it for the > community only. The > road is fascinating, tho. I could make mistakes so please input your > feedback if you want > to use a tool like that. > > You can: > > guix package -i wiredtiger > git clone https://framagit.org/a-guile-mind/culturia.one.git > git checkout guildhall > emacs culturia.one/src/webui.scm > > or > > cd culturia.one/src/ && guile -L . webui.scm > >