unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: Ricardo Wurmus <rekado@elephly.net>
To: Guix Devel <guix-devel@gnu.org>
Subject: Help wanted for mumi (issues.guix.gnu.org)
Date: Sat, 19 Oct 2019 23:12:30 +0200	[thread overview]
Message-ID: <87o8ycjv7l.fsf@elephly.net> (raw)

Hello Guix,

our bug tracker web interface at issues.guix.gnu.org could really
benefit from a more reliable, faster search.

Currently, mumi (the application behind issues.guix.gnu.org) uses a slow
interface to Debbugs, the bug tracker service that runs at
debbugs.gnu.org.  The search isn’t great as it returns duplicates and is
paginated, which makes it unsuitable for processing.  Mumi may need to
further filter the search results by status or activity, or any other
metric that the Debbugs search API doesn’t let us do.

So I decided to switch away from using the Debbugs API and instead
operate on a *local* copy of all messages that reach Debbugs.  Debbugs
operates on email messages, and luckily it allows us to download these
original messages.  Whenever someone visits an issue page, all related
messages are downloaded by mumi, so it amasses a sizeable stash of
emails over time.

Mumi is using a modified version of “mu”, the mail indexer and search
tool, to continuously index the contents of all messages.  (“mu” is
modified only so that the issue number is indexed alongside the message
contents.)

Unfortunately, that’s as far as I got before life intervened.  The next
step is really close, but getting there requires more contiguous
segments of time than I can free at the moment.  We really only need to
do the following things next:

1) keep updating the mu database as new messages are stored
2) using the mu Guile bindings to search messages via mu instead of
using the slow Debbugs API.

While working on 2 we may find that more properties should be stored in
the mu database, and that’s fine.  Our variant of mu is easily patched
to accomodate our needs.

Does anyone here have an interest in playing with and improving mumi?
It’s a very simple code base and it’s very easy to get started.

The code is here:

    https://git.elephly.net/software/mumi.git

--
Ricardo

             reply	other threads:[~2019-10-19 21:12 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-19 21:12 Ricardo Wurmus [this message]
2019-10-22 14:04 ` Help wanted for mumi (issues.guix.gnu.org) Ludovic Courtès
2019-11-28 18:05 ` zimoun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87o8ycjv7l.fsf@elephly.net \
    --to=rekado@elephly.net \
    --cc=guix-devel@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).