From: Amirouche Boubekki <amirouche.boubekki@gmail.com>
To: Guile User <guile-user@gnu.org>
Subject: Re: babelia
Date: Fri, 6 Dec 2019 14:19:59 +0100 [thread overview]
Message-ID: <CAL7_Mo8Xha3hMsQS4mmZV6GQq4-Ntqpqkm2D6rDO1xNPyN6JPA@mail.gmail.com> (raw)
In-Reply-To: <CAL7_Mo85f11cncZOcN7V8Uc2=QRGRkK=84XvJuApNeBoLkXtVQ@mail.gmail.com>
Hello all!
Le sam. 16 nov. 2019 à 11:06, Amirouche Boubekki <
amirouche.boubekki@gmail.com> a écrit :
> I restarted working on my personal search engine.
>
After two weeks of work, 41 files changed, 1845 insertions(+), 441
deletions(-) and 97 commits, I tagged a v0.2.0 in the repository at:
https://git.sr.ht/~amz3/guile-babelia
The babelia index and babelia search subcommands were removed. Instead, one
has to `make web` to spawn a server and then hit the
/api/search?query=foobar to make a search. To index stuff, one can POST a
file like test.scm to /api/index or rely on babelia crawler subcommands.
The crawler is still a work-in-progress. Do not expect the index to be
compatible with future releases.
>
> The last iteration, gotofish, was not too bad even if it has bitrot.
> Based on my research and practical experiment, it seems very clear
> that there is no workaround the use of map-reduce, that might be known
> as n-par-for-each [3].
>
> [3]
> https://www.gnu.org/software/guile/manual/html_node/Parallel-Forms.html#index-n_002dpar_002dfor_002deach
>
> I made a prototype similar to that n-par-for-each, except it works
> with guile-fibers, is asynchronous and works with a shared pool of
> threads instead of spawning N threads for each incoming query like
> gotofish does.
Actually, what I need is n-for-each-par-map where map happens in parallel.
The implementation can be found in the babelia/pool.scm file [4].
[4] https://git.sr.ht/~amz3/guile-babelia/tree/v0.2.0/babelia/pool.scm
The installation process is still a little bit akward, because one needs to
change the path to wiredtiger-3.2.0-0 shared library in the source. Add my
channel [5] and do `make init` to get started.
[5] https://git.sr.ht/~amz3/guix-amz3-channel
Happy hacking!
prev parent reply other threads:[~2019-12-06 13:19 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-16 10:06 babelia Amirouche Boubekki
2019-11-16 10:19 ` babelia Amirouche Boubekki
2019-11-16 12:08 ` babelia Arne Babenhauserheide
2019-11-18 17:12 ` babelia Amirouche Boubekki
2019-11-22 17:38 ` babelia Amirouche Boubekki
2019-12-06 13:19 ` Amirouche Boubekki [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/guile/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAL7_Mo8Xha3hMsQS4mmZV6GQq4-Ntqpqkm2D6rDO1xNPyN6JPA@mail.gmail.com \
--to=amirouche.boubekki@gmail.com \
--cc=guile-user@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).