babelia

[Amirouche Boubekki <amirouche.boubekki@gmail.com>]

I restarted working on my personal search engine.

It used to be called culturia [0] with too many planned features. At
some point, I called it asylum [1] and focused on personal knowledge
base aspects and the last iteration was called gotofish [2]

[0] https://framagit.org/a-guile-mind/culturia
[1] https://framagit.org/a-guile-mind/culturia.next
[2] https://git.sr.ht/~amz3/guile-gotofish

I learned much from all this projects. In particular, I learned that
it will be a long long long project, even if I focus only on "personal
search engine" line of work.

The last iteration, gotofish, was not too bad even if it has bitrot.
Based on my research and practical experiment, it seems very clear
that there is no workaround the use of map-reduce, that might be known
as n-par-for-each [3].

[3] https://www.gnu.org/software/guile/manual/html_node/Parallel-Forms.html#index-n_002dpar_002dfor_002deach

I made a prototype similar to that n-par-for-each, except it works
with guile-fibers, is asynchronous and works with a shared pool of
threads instead of spawning N threads for each incoming query like
gotofish does.

Related blog post: https://hyper.dev/blog/on-the-road-to-babelia.html

If you want to help or discuss those matters, do not hesitate to reply
to this message.


Cheers,

Amirouche ~ amz3 ~ https://hyper.dev