Re: How to make GNU Guile more successful

Re: How to make GNU Guile more successful

[Vítor De Araújo]

Hi,

I'm relatively new to Guile and new to this list, but I saw this
thread in the archives and I'd like to make some comments.

First, I think it would be nice to have a Guile wiki, in the likes of
cliki.net. A publicly-editable reference place where people can share
knowledge about Guile and Guile libraries and projects. Sure, having a
centralized package manager/repository would be nice, but just having
a common place where people can look up what libraries exist and where
they can be downloaded would be nice. (I don't know how easy would it
be to set up a wiki at the Guile website.)

On the topic of package management, I think that, more important than
a central repository, is a standard format in which to distribute
packages for easy installation. I've never used guildhall, and I guess
it defines something like this, but I think just having an
installation command where you can pass the URL of a package to
download and install (say, guild install http://some.url/somepackage.zip,
or even a git URL) would already go a long way towards making it
easier for people to use third-party libraries in Guile.

(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

Best regards,

Vítor De Araújo

-- 
Vítor De Araújo
http://inf.ufrgs.br/~vbuaraujo

Re: How to make GNU Guile more successful

[Arne Babenhauserheide]

Vítor De Araújo writes:

> (Or maybe package _names_ could be URLs pointing to an index in a
> well-defined format, which tells how to download each version of a
> package; then we could have even dependency management without a
> central repository. I'm pipe-dreaming here, though.)

You’re not just dreaming:

https://github.com/plande/compan/blob/master/compan.scm

    ;; clone the repository and load (lib) module from the top level:
    (load-modules ("https://bitbucket.org/panicz/dogma" (lib)))

    ;; as above, but load modules from "libraries" directory:
    (load-modules (("https://github.com/panicz/pamphlet" "libraries") (pamphlet)))

    ;; one can also fix on a particular tag/branch/commit:
    (load-modules (("https://bitbucket.org/panicz/dogma" "." "3884445191c2") (lib)))

We’d just have to decide to use this.

Best wishes,
Arne
-- 
Sent with my mu4e

Re: How to make GNU Guile more successful

[Panicz Maciej Godek]

2017-03-10 10:55 GMT+01:00 Arne Babenhauserheide <arne_bab@web.de>:

>
> Vítor De Araújo writes:
>
> > (Or maybe package _names_ could be URLs pointing to an index in a
> > well-defined format, which tells how to download each version of a
> > package; then we could have even dependency management without a
> > central repository. I'm pipe-dreaming here, though.)
>
> You’re not just dreaming:
>
> https://github.com/plande/compan/blob/master/compan.scm
>
>     ;; clone the repository and load (lib) module from the top level:
>     (load-modules ("https://bitbucket.org/panicz/dogma" (lib)))
>
>     ;; as above, but load modules from "libraries" directory:
>     (load-modules (("https://github.com/panicz/pamphlet" "libraries")
> (pamphlet)))
>
>     ;; one can also fix on a particular tag/branch/commit:
>     (load-modules (("https://bitbucket.org/panicz/dogma" "."
> "3884445191c2") (lib)))
>
> We’d just have to decide to use this.
>
> I think that the situation isn't quite as good as one could think. The
code works fine from
the interpreter, but if fails to work for compiled code, and I think that
we'd need to come up
with some solution of delaing module imports in compiled code until they're
available.

(Note also that the source code of Compan is extremely straightforward, as
it boils down
to a few calls of the "system" function which in turns calls some "hg"
commands)

So while I think that the author's idea was very nice, it clearly needs
some elaboration :)

Re: How to make GNU Guile more successful

[vbuaraujo]

Citando Panicz Maciej Godek <godek.maciek@gmail.com>:

> 2017-03-10 10:55 GMT+01:00 Arne Babenhauserheide <arne_bab@web.de>:
>
>>
>> Vítor De Araújo writes:
>>
>> > (Or maybe package _names_ could be URLs pointing to an index in a
>> > well-defined format, which tells how to download each version of a
>> > package; then we could have even dependency management without a
>> > central repository. I'm pipe-dreaming here, though.)
>>
>> You’re not just dreaming:
>>
>> https://github.com/plande/compan/blob/master/compan.scm
>>
>>     ;; clone the repository and load (lib) module from the top level:
>>     (load-modules ("https://bitbucket.org/panicz/dogma" (lib)))
>>
>>     ;; as above, but load modules from "libraries" directory:
>>     (load-modules (("https://github.com/panicz/pamphlet" "libraries")
>> (pamphlet)))
>>
>>     ;; one can also fix on a particular tag/branch/commit:
>>     (load-modules (("https://bitbucket.org/panicz/dogma" "."
>> "3884445191c2") (lib)))
>>
>> We’d just have to decide to use this.

That's very cool :) And also supports my point that there should be something
like a wiki for publishing information about Guile packages. It'd be hard
for me to find this on my own.

>> I think that the situation isn't quite as good as one could think. The
> code works fine from
> the interpreter, but if fails to work for compiled code, and I think that
> we'd need to come up
> with some solution of delaing module imports in compiled code until they're
> available.
>
> (Note also that the source code of Compan is extremely straightforward, as
> it boils down
> to a few calls of the "system" function which in turns calls some "hg"
> commands)
>
> So while I think that the author's idea was very nice, it clearly needs
> some elaboration :)

Yeah, for one I don't think the URLs should go directly into the code, but
rather in a package metadata file. The reason is I don't think we should
hard-code the package manager in the code. People should be able to use
the code without having the package manager, if they already have the
dependencies installed by some other means. But the basic idea is cool. :)

--
Vítor De Araújo

Re: How to make GNU Guile more successful

[Panicz Maciej Godek]

2017-03-10 15:27 GMT+01:00 <vbuaraujo@sapo.pt>:


> That's very cool :) And also supports my point that there should be
> something
> like a wiki for publishing information about Guile packages. It'd be hard
> for me to find this on my own.
>
> [...]
>>
>
> Yeah, for one I don't think the URLs should go directly into the code, but
> rather in a package metadata file. The reason is I don't think we should
> hard-code the package manager in the code. People should be able to use
> the code without having the package manager, if they already have the
> dependencies installed by some other means. But the basic idea is cool. :)
>

There is a reason why I like the idea of having URLs directly in the code,
namely that it makes you independent from some big institutions/particular
maintainers. Of course it is risky if your code refers to repositories that
you do not control, but you can always make your private forks.

Essentially the power of this idea is that it suddenly turns the whole web
into a Guile Incomprehensive Archive Network (clearly, a word beginning
with a letter T is missing to get a nice acronym).

Of course, it should be very easy to build a translation layer that
would map package names to addresses, but it requires a middleman
which always makes the costs higher (but perhaps having a proxy that
would make the necessary forks automatically would be a nice idea.
Plus if it would be able to check whether the changes between
some commits break compatibility then, well, that would really be
something. Which brings to mind the recent talk by Rich Hickey:
https://www.youtube.com/watch?v=oyLBGkS5ICk )

I do agree that something wiki-like would be a lovely thing.

Re: How to make GNU Guile more successful

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 948 bytes --]


() Panicz Maciej Godek <godek.maciek@gmail.com>
() Fri, 10 Mar 2017 16:08:23 +0100

   Essentially the power of this idea is that it suddenly turns
   the whole web into a Guile Incomprehensive Archive Network
   (clearly, a word beginning with a letter T is missing to get
   a nice acronym).

Cool.  Some suggestions:

tactic
tap
tapestry
tatter
tattoo
taunt
teacup
teardrop
tech
tee
temperament
tendency
tenet
thanks
thicket
thing
thrall
thwack
tine
tofu
toke
torpor
torrent
toy
tracks
traipse
travail
trawl
tree
tremor
trick
trinkets
triumph
troth
truffle
tsunami
tub
tufts
twig
twists
typo

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: How to make GNU Guile more successful

[Nala Ginrut]

Well, I like tofu :-)

Thien-Thi Nguyen <ttn@gnu.org>于2017年3月11日周六 16:48写道：

>
> () Panicz Maciej Godek <godek.maciek@gmail.com>
> () Fri, 10 Mar 2017 16:08:23 +0100
>
>    Essentially the power of this idea is that it suddenly turns
>    the whole web into a Guile Incomprehensive Archive Network
>    (clearly, a word beginning with a letter T is missing to get
>    a nice acronym).
>
> Cool.  Some suggestions:
>
> tactic
> tap
> tapestry
> tatter
> tattoo
> taunt
> teacup
> teardrop
> tech
> tee
> temperament
> tendency
> tenet
> thanks
> thicket
> thing
> thrall
> thwack
> tine
> tofu
> toke
> torpor
> torrent
> toy
> tracks
> traipse
> travail
> trawl
> tree
> tremor
> trick
> trinkets
> triumph
> troth
> truffle
> tsunami
> tub
> tufts
> twig
> twists
> typo
>
> --
> Thien-Thi Nguyen -----------------------------------------------
>  (defun responsep (query)
>    (pcase (context query)
>      (`(technical ,ml) (correctp ml))
>      ...))                              748E A0E8 1CB8 A748 9BFA
> --------------------------------------- 6CE4 6703 2224 4C80 7502
>
>

Re: How to make GNU Guile more successful

[Panicz Maciej Godek]

My mind has actually been oscillating between "trick", "toy", "thing" and
"torrent".
The last one would go particularly nice with the concept of "GitTorrent"

http://blog.printf.net/articles/2015/05/29/announcing-gittorrent-a-decentralized-github/


2017-03-13 16:55 GMT+01:00 Nala Ginrut <nalaginrut@gmail.com>:

> Well, I like tofu :-)
>
> Thien-Thi Nguyen <ttn@gnu.org>于2017年3月11日周六 16:48写道：
>
> >
> > () Panicz Maciej Godek <godek.maciek@gmail.com>
> > () Fri, 10 Mar 2017 16:08:23 +0100
> >
> >    Essentially the power of this idea is that it suddenly turns
> >    the whole web into a Guile Incomprehensive Archive Network
> >    (clearly, a word beginning with a letter T is missing to get
> >    a nice acronym).
> >
> > Cool.  Some suggestions:
> >
> > tactic
> > tap
> > tapestry
> > tatter
> > tattoo
> > taunt
> > teacup
> > teardrop
> > tech
> > tee
> > temperament
> > tendency
> > tenet
> > thanks
> > thicket
> > thing
> > thrall
> > thwack
> > tine
> > tofu
> > toke
> > torpor
> > torrent
> > toy
> > tracks
> > traipse
> > travail
> > trawl
> > tree
> > tremor
> > trick
> > trinkets
> > triumph
> > troth
> > truffle
> > tsunami
> > tub
> > tufts
> > twig
> > twists
> > typo
> >
> > --
> > Thien-Thi Nguyen -----------------------------------------------
> >  (defun responsep (query)
> >    (pcase (context query)
> >      (`(technical ,ml) (correctp ml))
> >      ...))                              748E A0E8 1CB8 A748 9BFA
> > --------------------------------------- 6CE4 6703 2224 4C80 7502
> >
> >
>

Re: How to make GNU Guile more successful

[Amirouche]

Héllo!


Le 10/03/2017 à 03:08, Vítor De Araújo a écrit :
> Hi,
>
> I'm relatively new to Guile and new to this list, but I saw this
> thread in the archives and I'd like to make some comments.

Welcome!

>
> First, I think it would be nice to have a Guile wiki, in the likes of
> cliki.net. A publicly-editable reference place where people can share
> knowledge about Guile and Guile libraries and projects.
I agree.
> Sure, having a
> centralized package manager/repository would be nice, but just having
> a common place where people can look up what libraries exist and where
> they can be downloaded would be nice. (I don't know how easy would it
> be to set up a wiki at the Guile website.)

The current most comprehensive list of GNU Guile package is
available at http://sph.mn/content/3e73

There is also the github org:

https://github.com/guildhall

> On the topic of package management, I think that, more important than
> a central repository, is a standard format in which to distribute
> packages for easy installation.
I agree. The standard could be to have a tarball named using
semantic versionning, from which one can extract the PACKAGE-NAME.
Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile

WDYT?

> I've never used guildhall, and I guess
> it defines something like this,
Probably but it's not documented.

> but I think just having an
> installation command where you can pass the URL of a package to
> download and install (say, guild install http://some.url/somepackage.zip,
> or even a git URL)
This will still require the user to set GUILE_LOAD_PATH. Maybe we can
add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
easier.

A simple guild install https://url/to/patch can be done in mater of hours
(if not minutes).

WDYT?

> would already go a long way towards making it
> easier for people to use third-party libraries in Guile.
I agree.
> (Or maybe package _names_ could be URLs pointing to an index in a
> well-defined format, which tells how to download each version of a
> package; then we could have even dependency management without a
> central repository. I'm pipe-dreaming here, though.)
This seems fun, but at the same time odd.

FWIW, I've started a new effort to build a package manager for GNU
Guile. You can find a demo at http://guildhall.hypermove.net/

It started as a guildhall web frontend but I noticed that guildhall:

1) doesn't work with guile 2.2 and I can not make it work
2) The solver is too complicated, even if it's based on aptitude algorithm
   it's still complicated.
3) use a lot of other scheme libraries [0] (like foof-loop) which
   doesn't help noobs like me to dive into the code. Maybe those
   libraries are *very* neat but why not include them in Guile proper
   then and make them default.

[0] https://github.com/ijp/guildhall/tree/master/guildhall/ext

For all this reasons I forked the effort.

I address each previous point as follow:

1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
bad this should be addressed at some point. AFAIR the issue is in
guile-wiredtiger (or it's an issue with setlocale (anyway this must be
addressed)))

2) I use a logic programming library called minikanren to solve the
dependency problem. This is the first real problem I can use minikanren
to solve a problem that is craft. This is the logic programming
library in scheme. While it's not very advanced compared to core.logic,
the version I use called microkaren has a straight forward 
implementation that can
hold all at once in a single head. Logic programming, like probabilistic
programming (cf. ) are two research areas in programming languages
that could improve the way we craft algorithms today. The logic approach
being more useful to me (but someone doing machine learning stuff,
might find the probabilistic scheme very useful. Ask me for a link!). 
OpenCog
does have a logic language embedded in it's language in their hyper 
graph database
(AFAIU it works using backward / forward inference but implements on top
of it higher logic constructs, like abduction, anaphora resolution...). It's
something they call PLN as Probabilistic Logic Network. AFAIK it's some kind
of probabilistic Datalog (which is one of the API Datomic support). 
There is
also probabilistic minikanren (albeit not useable). So, to sum up (because
I can (because I am not a robot)):

programming language research + guile + logic = minikanren.

Mind the fact that there is also a guile-log. But I still don't get what
guile-log does (hints: http://c-lambda.se/) whereas I understand how
microkanren does its stuff (I still need to benchmark it).

3) I don't use foof-loop but I bring my own dependencies. I use my
own database library that is based on wiredtiger. I've been working
on this database library for 3 years, it's well documented and various
example codes. That said, AFAIK, I am the only one to use it. I've built
a clone of Datomic (with the persistence part, patch welcome) which
use a query engine similar to Datomic I guess based on minikanren.

It's performance on a middle end laptop run Guile 2.2 are the followings:

- 1500 document reads per seconds
- 1000 to 500 document write per second

Documents are scheme assoc, and those are inserted 1000 per 1000 and
read 1000 per 1000 until it reach 50Go of data. At that point writes 
take 1/500
seconds or 0.002 seconds.

The biggest dataset I *loaded* into that database is wikidata which is 
50Go, I
don't remember how much time it takes to load it.

I had issues with wiredtiger using the Python bindings but only during reads
over a gigantic dataset of 60Giga. I say gigantic for a blog  not for Google
scale. Also upstream can solve issues if we can have a way to reproduce the
issues (which I plan to do once guile 2.2 is out (which means I will 
redo the
benchmarks against wikidata and read/write)). I understand the problem
that wiredtiger being part of MongoDB is problematic as MongoDB might not
care much for the same problem as ours. They will always be interested by
bigger free software database, tho. As it make free publicity of how 
successful
is their software.

Also there is much documentation about this library. I created several 
Guile projects
using it (albeit not big) they document several layers of the library 
and one
Guile user reported using it for doing human/social science research.

Search for guile wiredtiger in you favorite duckduckgo search engine.

Some people claim that PostgreSQL has all the required feature that 
someone wants
to store data and that this mongodb/wiredtiger is a bad. I recognize 
that wiredtiger
can be poor man's database right now. I don't have the required 
expertise to verify
whether it's good enough for your usecase. PostgreSQL is used in 
"production" in
all the world using multiple workload and stuff. PostgreSQL is good.

1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
     can't use pgsql from Guile RIGHT NOW.
2) wiredtiger is GOOD enough, benchmark it before saying it's not good 
ENOUGH.
3) it's not SQL, it's Schemey! It's inspired from datomic which the goto 
database
     for clojure with multiple implementation in the browser. This is 
killer feature
     of the clojure ecosystem that is closed source.
4) I find it more funny to fiddle with logic programming that set pseudo 
theory of SQL.
5) The first thing I do when pgsql bindings will be out, is to port 
feature space to PostgreSQL
and compare both pgsql and wiredtiger.
6) wiredtiger will always be simpler to use that Postgresql it's like 
comparing sqlite
and Datomic.

What happens next? You choose to hack on this project and find something 
interesting in this
TODO list:

- fix guile-wiredtiger to work on guile 2.0

- make it possible in guile-wiredtiger to stream the results of a look 
up using traversi using multiple cursors

- implement disequality in microkanren (for implementing the package 
dependency resolver
   and for fun because logic is awesome).

- implement a pastebin service using GNU Guile, yes because I think this 
a good bad reason to start
   another simple project that is not a static blog generator.

- Port the wiki of http://culturia.one to use feature space library.

- implement or find a scheme library that does the diff two lists (to 
finish wiki implementation
   and for the package dependency resolver).

- Help Matt Wette to complete his ffi helper 
http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
   (because we will need it for guix).

- add guile-fibers as a submodule and use it

- there is also a simple boolean keyword search engine that must be 
ported to feature space

- find the code that implements trigram transderivational search and put 
it in the repository. Bridging
   the gap with tsearch2.

- index the wiki

- index the packages

- index the pastebin

- index the web

And add gnunet to the mix.

There is not a lot of get-together pure guile project out there. There 
is guix. I hope this new project
can be the occasion for new guilers to submit patches or ideas.

The code is currently hosted at 
https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/

The codename is "primary".

Also I'd like to point out, that I don't need primary. I do it for the 
community only. The
road is fascinating, tho. I could make mistakes so please input your 
feedback if you want
to use a tool like that.

You can:

guix package -i wiredtiger
git clone https://framagit.org/a-guile-mind/culturia.one.git
git checkout guildhall
emacs culturia.one/src/webui.scm

or

cd culturia.one/src/ && guile -L . webui.scm

Yet another GNU Guile package manager (Fwd: Re: How to make GNU Guile more successful)

[Amirouche]

There is compan, there is guildhall, there is guix. They shall be primary.



-------- Message transféré --------
Sujet : 	Re: How to make GNU Guile more successful
Date : 	Fri, 10 Mar 2017 21:17:37 +0100
De : 	Amirouche <amirouche@hypermove.net>
Pour : 	guile-user@gnu.org



Héllo!


Le 10/03/2017 à 03:08, Vítor De Araújo a écrit :
> Hi,
>
> I'm relatively new to Guile and new to this list, but I saw this
> thread in the archives and I'd like to make some comments.

Welcome!

>
> First, I think it would be nice to have a Guile wiki, in the likes of
> cliki.net. A publicly-editable reference place where people can share
> knowledge about Guile and Guile libraries and projects.
I agree.
> Sure, having a
> centralized package manager/repository would be nice, but just having
> a common place where people can look up what libraries exist and where
> they can be downloaded would be nice. (I don't know how easy would it
> be to set up a wiki at the Guile website.)

The current most comprehensive list of GNU Guile package is
available at http://sph.mn/content/3e73

There is also the github org:

https://github.com/guildhall

> On the topic of package management, I think that, more important than
> a central repository, is a standard format in which to distribute
> packages for easy installation.
I agree. The standard could be to have a tarball named using
semantic versionning, from which one can extract the PACKAGE-NAME.
Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile

WDYT?

> I've never used guildhall, and I guess
> it defines something like this,
Probably but it's not documented.

> but I think just having an
> installation command where you can pass the URL of a package to
> download and install (say, guild install http://some.url/somepackage.zip,
> or even a git URL)
This will still require the user to set GUILE_LOAD_PATH. Maybe we can
add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
easier.

A simple guild install https://url/to/patch can be done in mater of hours
(if not minutes).

WDYT?

> would already go a long way towards making it
> easier for people to use third-party libraries in Guile.
I agree.
> (Or maybe package _names_ could be URLs pointing to an index in a
> well-defined format, which tells how to download each version of a
> package; then we could have even dependency management without a
> central repository. I'm pipe-dreaming here, though.)
This seems fun, but at the same time odd.

FWIW, I've started a new effort to build a package manager for GNU
Guile. You can find a demo at http://guildhall.hypermove.net/

It started as a guildhall web frontend but I noticed that guildhall:

1) doesn't work with guile 2.2 and I can not make it work
2) The solver is too complicated, even if it's based on aptitude algorithm
   it's still complicated.
3) use a lot of other scheme libraries [0] (like foof-loop) which
   doesn't help noobs like me to dive into the code. Maybe those
   libraries are *very* neat but why not include them in Guile proper
   then and make them default.

[0] https://github.com/ijp/guildhall/tree/master/guildhall/ext

For all this reasons I forked the effort.

I address each previous point as follow:

1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
bad this should be addressed at some point. AFAIR the issue is in
guile-wiredtiger (or it's an issue with setlocale (anyway this must be
addressed)))

2) I use a logic programming library called minikanren to solve the
dependency problem. This is the first real problem I can use minikanren
to solve a problem that is craft. This is the logic programming
library in scheme. While it's not very advanced compared to core.logic,
the version I use called microkaren has a straight forward
implementation that can
hold all at once in a single head. Logic programming, like probabilistic
programming (cf. ) are two research areas in programming languages
that could improve the way we craft algorithms today. The logic approach
being more useful to me (but someone doing machine learning stuff,
might find the probabilistic scheme very useful. Ask me for a link!).
OpenCog
does have a logic language embedded in it's language in their hyper
graph database
(AFAIU it works using backward / forward inference but implements on top
of it higher logic constructs, like abduction, anaphora resolution...). It's
something they call PLN as Probabilistic Logic Network. AFAIK it's some kind
of probabilistic Datalog (which is one of the API Datomic support).
There is
also probabilistic minikanren (albeit not useable). So, to sum up (because
I can (because I am not a robot)):

programming language research + guile + logic = minikanren.

Mind the fact that there is also a guile-log. But I still don't get what
guile-log does (hints: http://c-lambda.se/) whereas I understand how
microkanren does its stuff (I still need to benchmark it).

3) I don't use foof-loop but I bring my own dependencies. I use my
own database library that is based on wiredtiger. I've been working
on this database library for 3 years, it's well documented and various
example codes. That said, AFAIK, I am the only one to use it. I've built
a clone of Datomic (with the persistence part, patch welcome) which
use a query engine similar to Datomic I guess based on minikanren.

It's performance on a middle end laptop run Guile 2.2 are the followings:

- 1500 document reads per seconds
- 1000 to 500 document write per second

Documents are scheme assoc, and those are inserted 1000 per 1000 and
read 1000 per 1000 until it reach 50Go of data. At that point writes
take 1/500
seconds or 0.002 seconds.

The biggest dataset I *loaded* into that database is wikidata which is
50Go, I
don't remember how much time it takes to load it.

I had issues with wiredtiger using the Python bindings but only during reads
over a gigantic dataset of 60Giga. I say gigantic for a blog  not for Google
scale. Also upstream can solve issues if we can have a way to reproduce the
issues (which I plan to do once guile 2.2 is out (which means I will
redo the
benchmarks against wikidata and read/write)). I understand the problem
that wiredtiger being part of MongoDB is problematic as MongoDB might not
care much for the same problem as ours. They will always be interested by
bigger free software database, tho. As it make free publicity of how
successful
is their software.

Also there is much documentation about this library. I created several
Guile projects
using it (albeit not big) they document several layers of the library
and one
Guile user reported using it for doing human/social science research.

Search for guile wiredtiger in you favorite duckduckgo search engine.

Some people claim that PostgreSQL has all the required feature that
someone wants
to store data and that this mongodb/wiredtiger is a bad. I recognize
that wiredtiger
can be poor man's database right now. I don't have the required
expertise to verify
whether it's good enough for your usecase. PostgreSQL is used in
"production" in
all the world using multiple workload and stuff. PostgreSQL is good.

1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
     can't use pgsql from Guile RIGHT NOW.
2) wiredtiger is GOOD enough, benchmark it before saying it's not good
ENOUGH.
3) it's not SQL, it's Schemey! It's inspired from datomic which the goto
database
     for clojure with multiple implementation in the browser. This is
killer feature
     of the clojure ecosystem that is closed source.
4) I find it more funny to fiddle with logic programming that set pseudo
theory of SQL.
5) The first thing I do when pgsql bindings will be out, is to port
feature space to PostgreSQL
and compare both pgsql and wiredtiger.
6) wiredtiger will always be simpler to use that Postgresql it's like
comparing sqlite
and Datomic.

What happens next? You choose to hack on this project and find something
interesting in this
TODO list:

- fix guile-wiredtiger to work on guile 2.0

- make it possible in guile-wiredtiger to stream the results of a look
up using traversi using multiple cursors

- implement disequality in microkanren (for implementing the package
dependency resolver
   and for fun because logic is awesome).

- implement a pastebin service using GNU Guile, yes because I think this
a good bad reason to start
   another simple project that is not a static blog generator.

- Port the wiki of http://culturia.one to use feature space library.

- implement or find a scheme library that does the diff two lists (to
finish wiki implementation
   and for the package dependency resolver).

- Help Matt Wette to complete his ffi helper
http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
   (because we will need it for guix).

- add guile-fibers as a submodule and use it

- there is also a simple boolean keyword search engine that must be
ported to feature space

- find the code that implements trigram transderivational search and put
it in the repository. Bridging
   the gap with tsearch2.

- index the wiki

- index the packages

- index the pastebin

- index the web

And add gnunet to the mix.

There is not a lot of get-together pure guile project out there. There
is guix. I hope this new project
can be the occasion for new guilers to submit patches or ideas.

The code is currently hosted at
https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/

The codename is "primary".

Also I'd like to point out, that I don't need primary. I do it for the
community only. The
road is fascinating, tho. I could make mistakes so please input your
feedback if you want
to use a tool like that.

You can:

guix package -i wiredtiger
git clone https://framagit.org/a-guile-mind/culturia.one.git
git checkout guildhall
emacs culturia.one/src/webui.scm

or

cd culturia.one/src/ && guile -L . webui.scm

Re: Yet another GNU Guile package manager (Fwd: Re: How to make GNU Guile more successful)

[Nala Ginrut]

Hi Amiroiche!
FWIW, usually there are 3 kind s of pkg-manager:

1. SPM(system pkg manager)
It's used for building a distro, like RPM, deb, Guix.

2. LPM(language pkg manager)
It's used for specific language to install optional packages. Like npm,
gem, pip, etc.
Guix could do that too, IMO.

3. APM(application pkg manager)
It's used for specific framework to install packages in local application
directory. This is important for deployment.
Ruby has "bundle" for that, and Python usually use pip+virtualenv for that,
and npm is flexible enough to do that for JS too. Of course there are
alternative tools do the similar thing in each community.

For now, I'm concerning an APM for Artanis.
I've discussed with @Ludo the possibility to take advantage of Guix (as a
core lib) to implement it. If you want to write yet another pkg manager,
it's better to confirm which kind are you going to do, and don't forget to
reuse some cool features exists in Guix. :-)

Amirouche <amirouche@hypermove.net>于2017年3月11日周六 04:27写道：

> There is compan, there is guildhall, there is guix. They shall be primary.
>
>
>
> -------- Message transféré --------
> Sujet :         Re: How to make GNU Guile more successful
> Date :  Fri, 10 Mar 2017 21:17:37 +0100
> De :    Amirouche <amirouche@hypermove.net>
> Pour :  guile-user@gnu.org
>
>
>
> Héllo!
>
>
> Le 10/03/2017 à 03:08, Vítor De Araújo a écrit :
> > Hi,
> >
> > I'm relatively new to Guile and new to this list, but I saw this
> > thread in the archives and I'd like to make some comments.
>
> Welcome!
>
> >
> > First, I think it would be nice to have a Guile wiki, in the likes of
> > cliki.net. A publicly-editable reference place where people can share
> > knowledge about Guile and Guile libraries and projects.
> I agree.
> > Sure, having a
> > centralized package manager/repository would be nice, but just having
> > a common place where people can look up what libraries exist and where
> > they can be downloaded would be nice. (I don't know how easy would it
> > be to set up a wiki at the Guile website.)
>
> The current most comprehensive list of GNU Guile package is
> available at http://sph.mn/content/3e73
>
> There is also the github org:
>
> https://github.com/guildhall
>
> > On the topic of package management, I think that, more important than
> > a central repository, is a standard format in which to distribute
> > packages for easy installation.
> I agree. The standard could be to have a tarball named using
> semantic versionning, from which one can extract the PACKAGE-NAME.
> Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
> or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile
>
> WDYT?
>
> > I've never used guildhall, and I guess
> > it defines something like this,
> Probably but it's not documented.
>
> > but I think just having an
> > installation command where you can pass the URL of a package to
> > download and install (say, guild install http://some.url/somepackage.zip
> ,
> > or even a git URL)
> This will still require the user to set GUILE_LOAD_PATH. Maybe we can
> add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
> easier.
>
> A simple guild install https://url/to/patch can be done in mater of hours
> (if not minutes).
>
> WDYT?
>
> > would already go a long way towards making it
> > easier for people to use third-party libraries in Guile.
> I agree.
> > (Or maybe package _names_ could be URLs pointing to an index in a
> > well-defined format, which tells how to download each version of a
> > package; then we could have even dependency management without a
> > central repository. I'm pipe-dreaming here, though.)
> This seems fun, but at the same time odd.
>
> FWIW, I've started a new effort to build a package manager for GNU
> Guile. You can find a demo at http://guildhall.hypermove.net/
>
> It started as a guildhall web frontend but I noticed that guildhall:
>
> 1) doesn't work with guile 2.2 and I can not make it work
> 2) The solver is too complicated, even if it's based on aptitude algorithm
>    it's still complicated.
> 3) use a lot of other scheme libraries [0] (like foof-loop) which
>    doesn't help noobs like me to dive into the code. Maybe those
>    libraries are *very* neat but why not include them in Guile proper
>    then and make them default.
>
> [0] https://github.com/ijp/guildhall/tree/master/guildhall/ext
>
> For all this reasons I forked the effort.
>
> I address each previous point as follow:
>
> 1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
> bad this should be addressed at some point. AFAIR the issue is in
> guile-wiredtiger (or it's an issue with setlocale (anyway this must be
> addressed)))
>
> 2) I use a logic programming library called minikanren to solve the
> dependency problem. This is the first real problem I can use minikanren
> to solve a problem that is craft. This is the logic programming
> library in scheme. While it's not very advanced compared to core.logic,
> the version I use called microkaren has a straight forward
> implementation that can
> hold all at once in a single head. Logic programming, like probabilistic
> programming (cf. ) are two research areas in programming languages
> that could improve the way we craft algorithms today. The logic approach
> being more useful to me (but someone doing machine learning stuff,
> might find the probabilistic scheme very useful. Ask me for a link!).
> OpenCog
> does have a logic language embedded in it's language in their hyper
> graph database
> (AFAIU it works using backward / forward inference but implements on top
> of it higher logic constructs, like abduction, anaphora resolution...).
> It's
> something they call PLN as Probabilistic Logic Network. AFAIK it's some
> kind
> of probabilistic Datalog (which is one of the API Datomic support).
> There is
> also probabilistic minikanren (albeit not useable). So, to sum up (because
> I can (because I am not a robot)):
>
> programming language research + guile + logic = minikanren.
>
> Mind the fact that there is also a guile-log. But I still don't get what
> guile-log does (hints: http://c-lambda.se/) whereas I understand how
> microkanren does its stuff (I still need to benchmark it).
>
> 3) I don't use foof-loop but I bring my own dependencies. I use my
> own database library that is based on wiredtiger. I've been working
> on this database library for 3 years, it's well documented and various
> example codes. That said, AFAIK, I am the only one to use it. I've built
> a clone of Datomic (with the persistence part, patch welcome) which
> use a query engine similar to Datomic I guess based on minikanren.
>
> It's performance on a middle end laptop run Guile 2.2 are the followings:
>
> - 1500 document reads per seconds
> - 1000 to 500 document write per second
>
> Documents are scheme assoc, and those are inserted 1000 per 1000 and
> read 1000 per 1000 until it reach 50Go of data. At that point writes
> take 1/500
> seconds or 0.002 seconds.
>
> The biggest dataset I *loaded* into that database is wikidata which is
> 50Go, I
> don't remember how much time it takes to load it.
>
> I had issues with wiredtiger using the Python bindings but only during
> reads
> over a gigantic dataset of 60Giga. I say gigantic for a blog  not for
> Google
> scale. Also upstream can solve issues if we can have a way to reproduce the
> issues (which I plan to do once guile 2.2 is out (which means I will
> redo the
> benchmarks against wikidata and read/write)). I understand the problem
> that wiredtiger being part of MongoDB is problematic as MongoDB might not
> care much for the same problem as ours. They will always be interested by
> bigger free software database, tho. As it make free publicity of how
> successful
> is their software.
>
> Also there is much documentation about this library. I created several
> Guile projects
> using it (albeit not big) they document several layers of the library
> and one
> Guile user reported using it for doing human/social science research.
>
> Search for guile wiredtiger in you favorite duckduckgo search engine.
>
> Some people claim that PostgreSQL has all the required feature that
> someone wants
> to store data and that this mongodb/wiredtiger is a bad. I recognize
> that wiredtiger
> can be poor man's database right now. I don't have the required
> expertise to verify
> whether it's good enough for your usecase. PostgreSQL is used in
> "production" in
> all the world using multiple workload and stuff. PostgreSQL is good.
>
> 1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
>      can't use pgsql from Guile RIGHT NOW.
> 2) wiredtiger is GOOD enough, benchmark it before saying it's not good
> ENOUGH.
> 3) it's not SQL, it's Schemey! It's inspired from datomic which the goto
> database
>      for clojure with multiple implementation in the browser. This is
> killer feature
>      of the clojure ecosystem that is closed source.
> 4) I find it more funny to fiddle with logic programming that set pseudo
> theory of SQL.
> 5) The first thing I do when pgsql bindings will be out, is to port
> feature space to PostgreSQL
> and compare both pgsql and wiredtiger.
> 6) wiredtiger will always be simpler to use that Postgresql it's like
> comparing sqlite
> and Datomic.
>
> What happens next? You choose to hack on this project and find something
> interesting in this
> TODO list:
>
> - fix guile-wiredtiger to work on guile 2.0
>
> - make it possible in guile-wiredtiger to stream the results of a look
> up using traversi using multiple cursors
>
> - implement disequality in microkanren (for implementing the package
> dependency resolver
>    and for fun because logic is awesome).
>
> - implement a pastebin service using GNU Guile, yes because I think this
> a good bad reason to start
>    another simple project that is not a static blog generator.
>
> - Port the wiki of http://culturia.one to use feature space library.
>
> - implement or find a scheme library that does the diff two lists (to
> finish wiki implementation
>    and for the package dependency resolver).
>
> - Help Matt Wette to complete his ffi helper
> http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
>    (because we will need it for guix).
>
> - add guile-fibers as a submodule and use it
>
> - there is also a simple boolean keyword search engine that must be
> ported to feature space
>
> - find the code that implements trigram transderivational search and put
> it in the repository. Bridging
>    the gap with tsearch2.
>
> - index the wiki
>
> - index the packages
>
> - index the pastebin
>
> - index the web
>
> And add gnunet to the mix.
>
> There is not a lot of get-together pure guile project out there. There
> is guix. I hope this new project
> can be the occasion for new guilers to submit patches or ideas.
>
> The code is currently hosted at
> https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/
>
> The codename is "primary".
>
> Also I'd like to point out, that I don't need primary. I do it for the
> community only. The
> road is fascinating, tho. I could make mistakes so please input your
> feedback if you want
> to use a tool like that.
>
> You can:
>
> guix package -i wiredtiger
> git clone https://framagit.org/a-guile-mind/culturia.one.git
> git checkout guildhall
> emacs culturia.one/src/webui.scm
>
> or
>
> cd culturia.one/src/ && guile -L . webui.scm
>
>
>

Re: Yet another GNU Guile package manager (Fwd: Re: How to make GNU Guile more successful)

[Amirouche]

Le 11/03/2017 à 02:56, Nala Ginrut a écrit :
>
> Hi Amiroiche!
>

Héllo Nala!

> FWIW, usually there are 3 kind s of pkg-manager:
>
> 1. SPM(system pkg manager)
> It's used for building a distro, like RPM, deb, Guix.
>
> 2. LPM(language pkg manager)
> It's used for specific language to install optional packages. Like 
> npm, gem, pip, etc.
> Guix could do that too, IMO.
>
> 3. APM(application pkg manager)
> It's used for specific framework to install packages in local 
> application directory. This is important for deployment.
> Ruby has "bundle" for that, and Python usually use pip+virtualenv for 
> that, and npm is flexible enough to do that for JS too. Of course 
> there are alternative tools do the similar thing in each community.
>
> For now, I'm concerning an APM for Artanis.
>

I think right now I will do Language Package Manager.
I'd like to focus on community building feature that helps
document GNU Guile, like:

- wiki
- discussion pages
- pastebin

Code is always code but I need to choose which feature I will
implement given I have not an infinite set of time.

I have all those feature in some project, but now I need to put
them together.

> I've discussed with @Ludo the possibility to take advantage of Guix 
> (as a core lib) to implement it.
>
> If you want to write yet another pkg manager, it's better to confirm 
> which kind are you going to do, and don't forget to reuse some cool 
> features exists in Guix. :-)
>
I don't think Guix can help right now. The only issue that intersect with
guix is dependency resolution, and AFAIK Guix does hardcode dependencies
in the sens it's a simple breath first search. And what I planned to do is
implement semantic versioning à la npmjs (or aptitude) 
https://docs.npmjs.com/misc/semver

I think guix web, is enough for guix. I am not a regular guixsd user 
anymore,
so I am not sure how to help guix/sd.

A far reaching goal can be to link against guile-git to mix both package
management with forge.

>
> Amirouche <amirouche@hypermove.net 
> <mailto:amirouche@hypermove.net>>于2017年3月11日周六 04:27写道：
>
>     There is compan, there is guildhall, there is guix. They shall be
>     primary.
>
>
>
>     -------- Message transféré --------
>     Sujet :         Re: How to make GNU Guile more successful
>     Date :  Fri, 10 Mar 2017 21:17:37 +0100
>     De :    Amirouche <amirouche@hypermove.net
>     <mailto:amirouche@hypermove.net>>
>     Pour : guile-user@gnu.org <mailto:guile-user@gnu.org>
>
>
>
>     Héllo!
>
>
>     Le 10/03/2017 à 03:08, Vítor De Araújo a écrit :
>     > Hi,
>     >
>     > I'm relatively new to Guile and new to this list, but I saw this
>     > thread in the archives and I'd like to make some comments.
>
>     Welcome!
>
>     >
>     > First, I think it would be nice to have a Guile wiki, in the
>     likes of
>     > cliki.net <http://cliki.net>. A publicly-editable reference
>     place where people can share
>     > knowledge about Guile and Guile libraries and projects.
>     I agree.
>     > Sure, having a
>     > centralized package manager/repository would be nice, but just
>     having
>     > a common place where people can look up what libraries exist and
>     where
>     > they can be downloaded would be nice. (I don't know how easy
>     would it
>     > be to set up a wiki at the Guile website.)
>
>     The current most comprehensive list of GNU Guile package is
>     available at http://sph.mn/content/3e73
>
>     There is also the github org:
>
>     https://github.com/guildhall
>
>     > On the topic of package management, I think that, more important
>     than
>     > a central repository, is a standard format in which to distribute
>     > packages for easy installation.
>     I agree. The standard could be to have a tarball named using
>     semantic versionning, from which one can extract the PACKAGE-NAME.
>     Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
>     or a PACKAGE-NAME directory which can be installed into
>     ~/.local/lib/guile
>
>     WDYT?
>
>     > I've never used guildhall, and I guess
>     > it defines something like this,
>     Probably but it's not documented.
>
>     > but I think just having an
>     > installation command where you can pass the URL of a package to
>     > download and install (say, guild install
>     http://some.url/somepackage.zip,
>     > or even a git URL)
>     This will still require the user to set GUILE_LOAD_PATH. Maybe we can
>     add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
>     easier.
>
>     A simple guild install https://url/to/patch can be done in mater
>     of hours
>     (if not minutes).
>
>     WDYT?
>
>     > would already go a long way towards making it
>     > easier for people to use third-party libraries in Guile.
>     I agree.
>     > (Or maybe package _names_ could be URLs pointing to an index in a
>     > well-defined format, which tells how to download each version of a
>     > package; then we could have even dependency management without a
>     > central repository. I'm pipe-dreaming here, though.)
>     This seems fun, but at the same time odd.
>
>     FWIW, I've started a new effort to build a package manager for GNU
>     Guile. You can find a demo at http://guildhall.hypermove.net/
>
>     It started as a guildhall web frontend but I noticed that guildhall:
>
>     1) doesn't work with guile 2.2 and I can not make it work
>     2) The solver is too complicated, even if it's based on aptitude
>     algorithm
>        it's still complicated.
>     3) use a lot of other scheme libraries [0] (like foof-loop) which
>        doesn't help noobs like me to dive into the code. Maybe those
>        libraries are *very* neat but why not include them in Guile proper
>        then and make them default.
>
>     [0] https://github.com/ijp/guildhall/tree/master/guildhall/ext
>
>     For all this reasons I forked the effort.
>
>     I address each previous point as follow:
>
>     1) I only use guile 2.2, right now guile 2.1 is not supported. I
>     think it's
>     bad this should be addressed at some point. AFAIR the issue is in
>     guile-wiredtiger (or it's an issue with setlocale (anyway this must be
>     addressed)))
>
>     2) I use a logic programming library called minikanren to solve the
>     dependency problem. This is the first real problem I can use
>     minikanren
>     to solve a problem that is craft. This is the logic programming
>     library in scheme. While it's not very advanced compared to
>     core.logic,
>     the version I use called microkaren has a straight forward
>     implementation that can
>     hold all at once in a single head. Logic programming, like
>     probabilistic
>     programming (cf. ) are two research areas in programming languages
>     that could improve the way we craft algorithms today. The logic
>     approach
>     being more useful to me (but someone doing machine learning stuff,
>     might find the probabilistic scheme very useful. Ask me for a link!).
>     OpenCog
>     does have a logic language embedded in it's language in their hyper
>     graph database
>     (AFAIU it works using backward / forward inference but implements
>     on top
>     of it higher logic constructs, like abduction, anaphora
>     resolution...). It's
>     something they call PLN as Probabilistic Logic Network. AFAIK it's
>     some kind
>     of probabilistic Datalog (which is one of the API Datomic support).
>     There is
>     also probabilistic minikanren (albeit not useable). So, to sum up
>     (because
>     I can (because I am not a robot)):
>
>     programming language research + guile + logic = minikanren.
>
>     Mind the fact that there is also a guile-log. But I still don't
>     get what
>     guile-log does (hints: http://c-lambda.se/) whereas I understand how
>     microkanren does its stuff (I still need to benchmark it).
>
>     3) I don't use foof-loop but I bring my own dependencies. I use my
>     own database library that is based on wiredtiger. I've been working
>     on this database library for 3 years, it's well documented and various
>     example codes. That said, AFAIK, I am the only one to use it. I've
>     built
>     a clone of Datomic (with the persistence part, patch welcome) which
>     use a query engine similar to Datomic I guess based on minikanren.
>
>     It's performance on a middle end laptop run Guile 2.2 are the
>     followings:
>
>     - 1500 document reads per seconds
>     - 1000 to 500 document write per second
>
>     Documents are scheme assoc, and those are inserted 1000 per 1000 and
>     read 1000 per 1000 until it reach 50Go of data. At that point writes
>     take 1/500
>     seconds or 0.002 seconds.
>
>     The biggest dataset I *loaded* into that database is wikidata which is
>     50Go, I
>     don't remember how much time it takes to load it.
>
>     I had issues with wiredtiger using the Python bindings but only
>     during reads
>     over a gigantic dataset of 60Giga. I say gigantic for a blog not
>     for Google
>     scale. Also upstream can solve issues if we can have a way to
>     reproduce the
>     issues (which I plan to do once guile 2.2 is out (which means I will
>     redo the
>     benchmarks against wikidata and read/write)). I understand the problem
>     that wiredtiger being part of MongoDB is problematic as MongoDB
>     might not
>     care much for the same problem as ours. They will always be
>     interested by
>     bigger free software database, tho. As it make free publicity of how
>     successful
>     is their software.
>
>     Also there is much documentation about this library. I created several
>     Guile projects
>     using it (albeit not big) they document several layers of the library
>     and one
>     Guile user reported using it for doing human/social science research.
>
>     Search for guile wiredtiger in you favorite duckduckgo search engine.
>
>     Some people claim that PostgreSQL has all the required feature that
>     someone wants
>     to store data and that this mongodb/wiredtiger is a bad. I recognize
>     that wiredtiger
>     can be poor man's database right now. I don't have the required
>     expertise to verify
>     whether it's good enough for your usecase. PostgreSQL is used in
>     "production" in
>     all the world using multiple workload and stuff. PostgreSQL is good.
>
>     1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise
>     said you
>          can't use pgsql from Guile RIGHT NOW.
>     2) wiredtiger is GOOD enough, benchmark it before saying it's not good
>     ENOUGH.
>     3) it's not SQL, it's Schemey! It's inspired from datomic which
>     the goto
>     database
>          for clojure with multiple implementation in the browser. This is
>     killer feature
>          of the clojure ecosystem that is closed source.
>     4) I find it more funny to fiddle with logic programming that set
>     pseudo
>     theory of SQL.
>     5) The first thing I do when pgsql bindings will be out, is to port
>     feature space to PostgreSQL
>     and compare both pgsql and wiredtiger.
>     6) wiredtiger will always be simpler to use that Postgresql it's like
>     comparing sqlite
>     and Datomic.
>
>     What happens next? You choose to hack on this project and find
>     something
>     interesting in this
>     TODO list:
>
>     - fix guile-wiredtiger to work on guile 2.0
>
>     - make it possible in guile-wiredtiger to stream the results of a look
>     up using traversi using multiple cursors
>
>     - implement disequality in microkanren (for implementing the package
>     dependency resolver
>        and for fun because logic is awesome).
>
>     - implement a pastebin service using GNU Guile, yes because I
>     think this
>     a good bad reason to start
>        another simple project that is not a static blog generator.
>
>     - Port the wiki of http://culturia.one to use feature space library.
>
>     - implement or find a scheme library that does the diff two lists (to
>     finish wiki implementation
>        and for the package dependency resolver).
>
>     - Help Matt Wette to complete his ffi helper
>     http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
>        (because we will need it for guix).
>
>     - add guile-fibers as a submodule and use it
>
>     - there is also a simple boolean keyword search engine that must be
>     ported to feature space
>
>     - find the code that implements trigram transderivational search
>     and put
>     it in the repository. Bridging
>        the gap with tsearch2.
>
>     - index the wiki
>
>     - index the packages
>
>     - index the pastebin
>
>     - index the web
>
>     And add gnunet to the mix.
>
>     There is not a lot of get-together pure guile project out there. There
>     is guix. I hope this new project
>     can be the occasion for new guilers to submit patches or ideas.
>
>     The code is currently hosted at
>     https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/
>
>     The codename is "primary".
>
>     Also I'd like to point out, that I don't need primary. I do it for the
>     community only. The
>     road is fascinating, tho. I could make mistakes so please input your
>     feedback if you want
>     to use a tool like that.
>
>     You can:
>
>     guix package -i wiredtiger
>     git clone https://framagit.org/a-guile-mind/culturia.one.git
>     git checkout guildhall
>     emacs culturia.one/src/webui.scm
>
>     or
>
>     cd culturia.one/src/ && guile -L . webui.scm
>
>

Re: Yet another GNU Guile package manager (Fwd: Re: How to make GNU Guile more successful)

[carl hansen]

​FYI:
​

​"​
guile-lib

guile-lib is intended as an accumulation place for pure-scheme Guile
modules, allowing for people to cooperate integrating their generic Guile
modules into a coherent library. Think "a down-scaled, limited-scope CPAN
<http://www.cpan.org/> for Guile".

Also, it can be seen as a code staging area for Guile; the Guile developers
could decide to integrate some of the code into guile-core. An example for
a possible candidate is SRFI-35."​

http://www.nongnu.org/guile-lib/

Re: Yet another GNU Guile package manager (Fwd: Re: How to make GNU Guile more successful)

[Matt Wette]

> On Mar 16, 2017, at 2:03 AM, carl hansen <carlhansen1234@gmail.com> wrote:
> 
> ​FYI:
> ​
> 
> ​"​
> guile-lib
> 
> guile-lib is intended as an accumulation place for pure-scheme Guile
> modules, allowing for people to cooperate integrating their generic Guile
> modules into a coherent library. Think "a down-scaled, limited-scope CPAN
> <http://www.cpan.org/> for Guile".
> 
> Also, it can be seen as a code staging area for Guile; the Guile developers
> could decide to integrate some of the code into guile-core. An example for
> a possible candidate is SRFI-35."​
> 
> http://www.nongnu.org/guile-lib/

How does one submit proposals for contributions to guile-lib?

I have a few items: regex-match, struct (like struct in python), maybe others

Re: Yet another GNU Guile package manager (Fwd: Re: How to make GNU Guile more successful)

[Alex Kost]

Matt Wette (2017-03-16 05:07 -0700) wrote:

>> On Mar 16, 2017, at 2:03 AM, carl hansen <carlhansen1234@gmail.com> wrote:
>> ​"
>> guile-lib
>>
>> guile-lib is intended as an accumulation place for pure-scheme Guile
>> modules, allowing for people to cooperate integrating their generic Guile
>> modules into a coherent library. Think "a down-scaled, limited-scope CPAN
>> <http://www.cpan.org/> for Guile".
>>
>> Also, it can be seen as a code staging area for Guile; the Guile developers
>> could decide to integrate some of the code into guile-core. An example for
>> a possible candidate is SRFI-35."
>>
>> http://www.nongnu.org/guile-lib/
>
> How does one submit proposals for contributions to guile-lib?

I also wonder.  I asked about adding a plist library on this list once¹
but didn't get an answer.

¹ http://lists.gnu.org/archive/html/guile-user/2016-09/msg00052.html

-- 
Alex

Re: How to make GNU Guile more successful

[Vítor De Araújo]

On 10/03/2017 17:17, Amirouche wrote:
> Héllo!
> 
> 
> Le 10/03/2017 à 03:08, Vítor De Araújo a écrit :
>> Hi,
>>
>> I'm relatively new to Guile and new to this list, but I saw this
>> thread in the archives and I'd like to make some comments.
> 
> Welcome!

Thanks :)

[...]

>> On the topic of package management, I think that, more important than
>> a central repository, is a standard format in which to distribute
>> packages for easy installation.
> I agree. The standard could be to have a tarball named using
> semantic versionning, from which one can extract the PACKAGE-NAME.
> Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
> or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile
> 
> WDYT?

That's good, but it'd also have to contain a metadata file containing at
least the package's dependencies and what commands should be run to
install (for example, ./configure && make && make install, for packages
which need some compilation step before copying to ~/.local/lib/guile).

>> I've never used guildhall, and I guess
>> it defines something like this,
> Probably but it's not documented.
> 
>> but I think just having an
>> installation command where you can pass the URL of a package to
>> download and install (say, guild install http://some.url/somepackage.zip,
>> or even a git URL)
> This will still require the user to set GUILE_LOAD_PATH. Maybe we can
> add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
> easier.

Yes, ~/.local/lib/guile/ (or other user-level directory) definitely
should be in the default GUILE_LOAD_PATH.

> A simple guild install https://url/to/patch can be done in mater of hours
> (if not minutes).
> 
> WDYT?

Sure. Adding dependency handling would make things somewhat more
complex, though.

I thought a little bit about the packages-as-urls idea and I had some
ideas I'll just throw out here.

First, I realized using the URL as the package _name_ is problematic,
because it hard-codes the source of the package; that complicates things
if someone wants to make a fork of the package and install it in place
of the original. However, packages can still have a normal name and a
"canonical URL" for fetching it.

The package would come with a metadata file (named something like
metadata.foopkg, where 'foopkg' is a placeholder for the name of the
package manager (and I'd rather have the name of the package manager as
an extension, because I'd rather not assume I'm the only package manager
in the world)). This metadata would contain the package name, version,
and dependencies. For each dependency, it would specify the package
name, the version, *and the canonical URL* for that package. Upoin
installation, the package manager would go through each dependency and
check if a package of that name and compatible version (assuming
semantic versioning) is already installed; if it's not, it would fetch
it from the canonical URL.

In this way, we can have decentralized package management which can
handle dependencies.

And commenting on something I said previously:

On 10/03/2017 12:08, Panicz Maciej Godek wrote:
> 2017-03-10 15:27 GMT+01:00 <vbuaraujo@sapo.pt>:
>
>     Yeah, for one I don't think the URLs should go directly into the
>     code, but
>     rather in a package metadata file. The reason is I don't think we
should
>     hard-code the package manager in the code. People should be able
to use
>     the code without having the package manager, if they already have the
>     dependencies installed by some other means. But the basic idea is
>     cool. :)
>
>
> There is a reason why I like the idea of having URLs directly in the code,
> namely that it makes you independent from some big institutions/particular
> maintainers. Of course it is risky if your code refers to repositories
that
> you do not control, but you can always make your private forks.

I'm totally in favor of independence from particular institutions or
maintainers, but I think this can be achieved by putting the URLs in the
metadata file rather than directly in the code. Code would just use
(use-modules ...) as usual to load code, and the business of dowloading
and installing would be solely the task of the package manager. I think
that enables a friendlier co-existence with other package managers, and
allows the same library to be packaged for multiple package managers at
the same time (e.g., guix, compan, guildhall, etc.). It also allows
people to just unpack and install the code manually if they want to.

(A problem that still remains if packages have names separate from the
URLs is avoiding name clashes. The wiki would go some way towards that
(people could check more easily if the name they want is already in
use), but maybe some conventions for globally unique names (e.g.,
prepending a domain prefix like Java people do) could ameliorate this. I
don't think this is really important at this point.)

[...]
> FWIW, I've started a new effort to build a package manager for GNU
> Guile. You can find a demo at http://guildhall.hypermove.net/
> 
> It started as a guildhall web frontend but I noticed that guildhall:
> 
> 1) doesn't work with guile 2.2 and I can not make it work
> 2) The solver is too complicated, even if it's based on aptitude algorithm
>   it's still complicated.
> 3) use a lot of other scheme libraries [0] (like foof-loop) which
>   doesn't help noobs like me to dive into the code. Maybe those
>   libraries are *very* neat but why not include them in Guile proper
>   then and make them default.
> 
> [0] https://github.com/ijp/guildhall/tree/master/guildhall/ext
> 
> For all this reasons I forked the effort.
> 
> I address each previous point as follow:
> 
> 1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
> bad this should be addressed at some point. AFAIR the issue is in
> guile-wiredtiger (or it's an issue with setlocale (anyway this must be
> addressed)))
> 
> 2) I use a logic programming library called minikanren to solve the
> dependency problem. This is the first real problem I can use minikanren
> to solve a problem that is craft. This is the logic programming
> library in scheme. While it's not very advanced compared to core.logic,
> the version I use called microkaren has a straight forward
> implementation that can
> hold all at once in a single head. Logic programming, like probabilistic
> programming (cf. ) are two research areas in programming languages
> that could improve the way we craft algorithms today. The logic approach
> being more useful to me (but someone doing machine learning stuff,
> might find the probabilistic scheme very useful. Ask me for a link!).
> OpenCog
> does have a logic language embedded in it's language in their hyper
> graph database
> (AFAIU it works using backward / forward inference but implements on top
> of it higher logic constructs, like abduction, anaphora resolution...).
> It's
> something they call PLN as Probabilistic Logic Network. AFAIK it's some
> kind
> of probabilistic Datalog (which is one of the API Datomic support).
> There is
> also probabilistic minikanren (albeit not useable). So, to sum up (because
> I can (because I am not a robot)):
> 
> programming language research + guile + logic = minikanren.
> 
> Mind the fact that there is also a guile-log. But I still don't get what
> guile-log does (hints: http://c-lambda.se/) whereas I understand how
> microkanren does its stuff (I still need to benchmark it).
> 
> 3) I don't use foof-loop but I bring my own dependencies. I use my
> own database library that is based on wiredtiger. I've been working
> on this database library for 3 years, it's well documented and various
> example codes. That said, AFAIK, I am the only one to use it. I've built
> a clone of Datomic (with the persistence part, patch welcome) which
> use a query engine similar to Datomic I guess based on minikanren.
> 
> It's performance on a middle end laptop run Guile 2.2 are the followings:
> 
> - 1500 document reads per seconds
> - 1000 to 500 document write per second
> 
> Documents are scheme assoc, and those are inserted 1000 per 1000 and
> read 1000 per 1000 until it reach 50Go of data. At that point writes
> take 1/500
> seconds or 0.002 seconds.
> 
> The biggest dataset I *loaded* into that database is wikidata which is
> 50Go, I
> don't remember how much time it takes to load it.
> 
> I had issues with wiredtiger using the Python bindings but only during
> reads
> over a gigantic dataset of 60Giga. I say gigantic for a blog  not for
> Google
> scale. Also upstream can solve issues if we can have a way to reproduce the
> issues (which I plan to do once guile 2.2 is out (which means I will
> redo the
> benchmarks against wikidata and read/write)). I understand the problem
> that wiredtiger being part of MongoDB is problematic as MongoDB might not
> care much for the same problem as ours. They will always be interested by
> bigger free software database, tho. As it make free publicity of how
> successful
> is their software.
> 
> Also there is much documentation about this library. I created several
> Guile projects
> using it (albeit not big) they document several layers of the library
> and one
> Guile user reported using it for doing human/social science research.
> 
> Search for guile wiredtiger in you favorite duckduckgo search engine.
> 
> Some people claim that PostgreSQL has all the required feature that
> someone wants
> to store data and that this mongodb/wiredtiger is a bad. I recognize
> that wiredtiger
> can be poor man's database right now. I don't have the required
> expertise to verify
> whether it's good enough for your usecase. PostgreSQL is used in
> "production" in
> all the world using multiple workload and stuff. PostgreSQL is good.
> 
> 1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
>     can't use pgsql from Guile RIGHT NOW.
> 2) wiredtiger is GOOD enough, benchmark it before saying it's not good
> ENOUGH.
> 3) it's not SQL, it's Schemey! It's inspired from datomic which the goto
> database
>     for clojure with multiple implementation in the browser. This is
> killer feature
>     of the clojure ecosystem that is closed source.
> 4) I find it more funny to fiddle with logic programming that set pseudo
> theory of SQL.
> 5) The first thing I do when pgsql bindings will be out, is to port
> feature space to PostgreSQL
> and compare both pgsql and wiredtiger.
> 6) wiredtiger will always be simpler to use that Postgresql it's like
> comparing sqlite
> and Datomic.
> 
> What happens next? You choose to hack on this project and find something
> interesting in this
> TODO list:
> 
> - fix guile-wiredtiger to work on guile 2.0
> 
> - make it possible in guile-wiredtiger to stream the results of a look
> up using traversi using multiple cursors
> 
> - implement disequality in microkanren (for implementing the package
> dependency resolver
>   and for fun because logic is awesome).
> 
> - implement a pastebin service using GNU Guile, yes because I think this
> a good bad reason to start
>   another simple project that is not a static blog generator.
> 
> - Port the wiki of http://culturia.one to use feature space library.
> 
> - implement or find a scheme library that does the diff two lists (to
> finish wiki implementation
>   and for the package dependency resolver).
> 
> - Help Matt Wette to complete his ffi helper
> http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
>   (because we will need it for guix).
> 
> - add guile-fibers as a submodule and use it
> 
> - there is also a simple boolean keyword search engine that must be
> ported to feature space
> 
> - find the code that implements trigram transderivational search and put
> it in the repository. Bridging
>   the gap with tsearch2.
> 
> - index the wiki
> 
> - index the packages
> 
> - index the pastebin
> 
> - index the web
> 
> And add gnunet to the mix.
> 
> There is not a lot of get-together pure guile project out there. There
> is guix. I hope this new project
> can be the occasion for new guilers to submit patches or ideas.
> 
> The code is currently hosted at
> https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/
> 
> The codename is "primary".
> 
> Also I'd like to point out, that I don't need primary. I do it for the
> community only. The
> road is fascinating, tho. I could make mistakes so please input your
> feedback if you want
> to use a tool like that.
> 
> You can:
> 
> guix package -i wiredtiger
> git clone https://framagit.org/a-guile-mind/culturia.one.git
> git checkout guildhall
> emacs culturia.one/src/webui.scm
> 
> or
> 
> cd culturia.one/src/ && guile -L . webui.scm

There are certainly lots of fun subprojects to tackle here, but I'm
afraid it would take really long to get ready. I'd rather have a really
simple package manager, preferably with zero dependencies other than
Guile itself. But then again I think multiple package managers should be
able to co-exist, so there's no problem for different people to tackle
multiple approaches to the problem.

-- 
Vítor De Araújo

Re: How to make GNU Guile more successful

[Vítor De Araújo]

On 10/03/2017 21:50, Vítor De Araújo wrote:
[...]
> I thought a little bit about the packages-as-urls idea and I had some
> ideas I'll just throw out here.
> 
> First, I realized using the URL as the package _name_ is problematic,
> because it hard-codes the source of the package; that complicates things
> if someone wants to make a fork of the package and install it in place
> of the original. However, packages can still have a normal name and a
> "canonical URL" for fetching it.
> 
> The package would come with a metadata file (named something like
> metadata.foopkg, where 'foopkg' is a placeholder for the name of the
> package manager (and I'd rather have the name of the package manager as
> an extension, because I'd rather not assume I'm the only package manager
> in the world)). This metadata would contain the package name, version,
> and dependencies. For each dependency, it would specify the package
> name, the version, *and the canonical URL* for that package. Upoin
> installation, the package manager would go through each dependency and
> check if a package of that name and compatible version (assuming
> semantic versioning) is already installed; if it's not, it would fetch
> it from the canonical URL.
> 
> In this way, we can have decentralized package management which can
> handle dependencies.
[...]

Just one more little thought on package names vs. URLs. Guile already
has a standard module system, and modules already have names which are
supposed to be globally unique, and even they even have version numbers.
So module names can double as package names. The advantage is that then
you can check whether a dependency is installed *whether or not it was
installed via the package manager*. Basically you'd describe your
dependencies in terms of modules, and specify where the package that
implements that module can be fetched from.

I have to think this through a bit more, though.

-- 
Vítor De Araújo

Re: How to make GNU Guile more successful

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]


() Amirouche <amirouche@hypermove.net>
() Fri, 10 Mar 2017 21:17:37 +0100

   1) There is no dynamic ffi bindings for PostgreSQL yet,
      otherwise said you can't use pgsql from Guile RIGHT NOW.

Plug: Guile-PG[0] does not provide ffi bindings for PostgreSQL,
but it does work w/ Guile 2.0 today (last i checked: 2.0.11).
Paren lovers might like the query construction DSL[1], too.

[0] http://www.nongnu.org/guile-pg/
[1] http://www.nongnu.org/guile-pg/doc/Query-Construction.html

Please report any problems you find w/ it to this mailing list.
I'm especially interested in experiences w/ Guile 2.x (x > 0).

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: How to make GNU Guile more successful

[Christopher Allan Webber]

Vítor De Araújo writes:

> First, I think it would be nice to have a Guile wiki, in the likes of
> cliki.net. A publicly-editable reference place where people can share
> knowledge about Guile and Guile libraries and projects. Sure, having a
> centralized package manager/repository would be nice, but just having
> a common place where people can look up what libraries exist and where
> they can be downloaded would be nice. (I don't know how easy would it
> be to set up a wiki at the Guile website.)

A Guile wiki would be pretty nice, but my experience is that running a
wiki is a maintenance drag, and nobody really wants to keep it going for
a project of Guile's size (I wish this weren't the case).  Of course, if
someone wants to run one, great!  But IMO it would be better to "team
up" with something/someone that's already running a wiki.  I would
suggest using a namespace under the LibrePlanet wiki?  That one is
already maintained by the FSF sysadmins and/or GNU sysadmin
volunteers (I don't quite know the setup).