BuGit

BuGit

[Stefan Monnier]

For those curious about what I've been upto lately, one of the things
I've been working on is a new distributed issue tracking system which
I call BuGit:

   https://gitlab.com/monnier/bugit

It's still primitive (only the command-line UI and the email
notification system are more or less complete: the web UI is still
read-only, and there is no email UI at all).


        Stefan

Re: BuGit

[Aurélien Aptel]

On Tue, Feb 2, 2016 at 4:09 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> For those curious about what I've been upto lately, one of the things
> I've been working on is a new distributed issue tracking system which
> I call BuGit:
>
>    https://gitlab.com/monnier/bugit
>
> It's still primitive (only the command-line UI and the email
> notification system are more or less complete: the web UI is still
> read-only, and there is no email UI at all).

I like the offline work+sync later feature. This looks really
interesting. I see in the source you have code to import debbugs
issues. Do you have an emacs-bugit repo?

How does querying for say open bugs work? Do you think it can scale to
emacs bugs database?

The web and especially the email UI are going to be a PITA to write
but they are essential so keep up the good work!

Re: BuGit

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 491 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> For those curious about what I've been upto lately, one of the things
> I've been working on is a new distributed issue tracking system which
> I call BuGit:
>
>    https://gitlab.com/monnier/bugit
>
> It's still primitive (only the command-line UI and the email
> notification system are more or less complete: the web UI is still
> read-only, and there is no email UI at all).

Nice!  ...but if I may, why writing it all in bash?

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: BuGit

[Stefan Monnier]

>> For those curious about what I've been upto lately, one of the things
>> I've been working on is a new distributed issue tracking system which
>> I call BuGit:
>> https://gitlab.com/monnier/bugit
>> It's still primitive (only the command-line UI and the email
>> notification system are more or less complete: the web UI is still
>> read-only, and there is no email UI at all).
> I like the offline work+sync later feature.  This looks really
> interesting.  I see in the source you have code to import debbugs
> issues.  Do you have an emacs-bugit repo?

I wrote the debbugs conversion code in order to see how the system
scales to largish databases (using debbugs.gnu.org as the largish
database).  So yes, I have an "emacs-bugit" repo (actually it contains
a bit more since debbugs.gnu.org is used for more than just Emacs).

But the debbugs conversion code doesn't know anything about MIME, so the
result is not really usable: the majority of messages are OK, but many
appear as base64 line-noise, and none of the attachments are recognized
as attachments.

My next goal is to add an email UI so it can be used "as
a mailing-list", just like we do for debbugs (the "control@debbugs" part
of debbugs's UI is of lesser importance to me).  Once that code works,
it should be easy to improve the debbugs conversion so as to make an
actually usable "emacs-bugit" repo (since the email UI will have to
deal with MIME).

But that'd only be of interest if Emacs wants to switch to BuGit, which
seems unlikely in the short term.  Again, the only reason for this
debbugs conversion code was to get a realistic database to play with.

> How does querying for say open bugs work?

"bugit list" or click on "list all issues" in the web UI.

> Do you think it can scale to emacs bugs database?

My tests on the emacs-repo so far indicate that it should be possible to
make it scale to that kind of size.  There are a few cases where it
currently gets slow at such sizes, tho.  We may need to add caching of
some sort at a few places.

> The web and especially the email UI are going to be a PITA to write
> but they are essential so keep up the good work!

The code is currently written 100% in /bin/sh, which was great for the
command-line UI (I very quickly managed to get something usable), and
indeed handling MIME with /bin/sh is a PITA.

So I'll probably use Python for that (I actually expect most of the code
to be rewritten in Python over time), where it should actually be fairly
easy to write the email UI.  All we really need is, give a single
email:
- figure out if it's a new bug or a reply.
- find the corresponding issue id.
- split it into "body + attachments" (that's where sh sucks compared to
  something like Python's mail library).
- pass the result to Git.
- that's about it, tho afterwards we'll want to add support for things like
  closing-via-email and things like that, but these should also be easy.

I don't foresee any real PITA on the web UI side either, actually (tho
I'm very much unversed in web technologies, so I might be
underestimating the problems).

Any help welcome, of course.


        Stefan

Re: BuGit

[Stefan Monnier]

>> For those curious about what I've been upto lately, one of the things
>> I've been working on is a new distributed issue tracking system which
>> I call BuGit:
>> https://gitlab.com/monnier/bugit
>> It's still primitive (only the command-line UI and the email
>> notification system are more or less complete: the web UI is still
>> read-only, and there is no email UI at all).
> Nice!  ...but if I may, why writing it all in bash?

[ Actually it works in dash as well.  AFAIK it's almost 99% POSIX shell
code (IIRC the only extension used is to use "type <foo>" to figure out
if <foo> is a valid function/command).  ]

The real reason is rather silly: I'd been thinking about how to
represent the database in such a way that all the merging is done "for
free" by the underlying VCS, and when it finally occurred to me, I was
all excited at how simple it was and started to write an email to
a friend outlining how it worked, and since describing it in text was
cumbersome, I started to write it in pseudo-code which quickly turned
into executable sh code.

The next planned feature (email UI) will not be written in sh, and
I expect that after that, all the new code will be in Python.


        Stefan

Re: BuGit

[Aaron Conole]

Stefan,

On Tue, Feb 02, 2016 at 10:09:00AM -0500, Stefan Monnier wrote:
> For those curious about what I've been upto lately, one of the things
> I've been working on is a new distributed issue tracking system which
> I call BuGit:
> 
>    https://gitlab.com/monnier/bugit
> 
> It's still primitive (only the command-line UI and the email
> notification system are more or less complete: the web UI is still
> read-only, and there is no email UI at all).
>

This looks great! Is there an intent to integrate this as another set of
"git branch metadata" on existing repositories? IE: can I take a repo like
emacs, and just use bugit on top of it? Is that ever intended as a workflow?

-Aaron 

Re: BuGit

[Dmitry Gutov]

On 02/02/2016 06:09 PM, Stefan Monnier wrote:

>     https://gitlab.com/monnier/bugit
>
> It's still primitive (only the command-line UI and the email
> notification system are more or less complete: the web UI is still
> read-only, and there is no email UI at all).

Looks like an interesting experiment.

These lines look a bit contradictory to me:

 > ...the user can also share the same repository with the main project.

 > Every issue lives in its own branch named bugit/<id>.

I imagine it might be uncomfortable working on a project which contains 
thousands of branches. You do a pull--and Git shows you all the remotes 
that have changed. With lots of branches, that would be a lot of output, 
even if you don't check them out locally.

Not sure if there's an option to ignore only certain branches, for 
certain commands.

Re: BuGit

[Stefan Monnier]

>> For those curious about what I've been upto lately, one of the things
>> I've been working on is a new distributed issue tracking system which
>> I call BuGit:
>> https://gitlab.com/monnier/bugit
>> It's still primitive (only the command-line UI and the email
>> notification system are more or less complete: the web UI is still
>> read-only, and there is no email UI at all).
> This looks great! Is there an intent to integrate this as another set of
> "git branch metadata" on existing repositories? IE: can I take a repo like
> emacs, and just use bugit on top of it? Is that ever intended as a workflow?

You can indeed store the bug database in the same repository as the
project's source code.  Not in the same branch, OTOH.


        Stefan

Re: BuGit

[Stefan Monnier]

>> https://gitlab.com/monnier/bugit
>> It's still primitive (only the command-line UI and the email
>> notification system are more or less complete: the web UI is still
>> read-only, and there is no email UI at all).
> Looks like an interesting experiment.
> These lines look a bit contradictory to me:
>> ...the user can also share the same repository with the main project.
>> Every issue lives in its own branch named bugit/<id>.
> I imagine it might be uncomfortable working on a project which contains
> thousands of branches. You do a pull--and Git shows you all the remotes that
> have changed.  With lots of branches, that would be a lot of output, even if
> you don't check them out locally.

Indeed.  I don't have much experience with such a situation yet, but
I expect "branch-namespace-pollution" to be a problem.

Currently, for BuGit itself, I have 2 central repositories (one for the
code, one for the bug database), but I clone them into a shared local
repository.  So when I do "git pull/push" I access the source repository
(and corresponding branches), and when I do "bugit sync" I access from
the bug repository (and corresponding branches).

That takes care of the use-case you mention, but it still leaves another
problem "git branch -a" includes all the bugit branches, which
is inconvenient.  I haven't tried to solve this issue yet, but I'm
thinking of not sharing the local repository, and instead store the
local bugit repository in a subdirectory like .git/bugit.

> Not sure if there's an option to ignore only certain branches, for
> certain commands.

I use Git's support for multiple remote repositories for that.


        Stefan

Re: BuGit

[Aurélien Aptel]

I have found other similar projects (git based issue management) which
you might be interested in:

    https://github.com/dspinellis/gi

The README also links to similar projects:

> - deft developed in 2011 is based on the same idea. It requires Python and offers a GUI.
> - Bugs Everywhere, also written in Python, supports many version control backends and offers a web interface.
> - git-appraise is a distributed code review system for Git repos based again on Git.
> - Fossil is a distributed version control software that also supports issue tracking and a wiki. It runs as a single executable.

Re: BuGit

[Aurélien Aptel]

On Tue, Feb 2, 2016 at 5:43 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> So I'll probably use Python for that (I actually expect most of the code
> to be rewritten in Python over time), where it should actually be fairly
> easy to write the email UI.  All we really need is, give a single
> email:

You probably don't want to handle the actual sending&receiving part
yourself. It's much more arcane and complex than it seems. That means
you have to setup an MDA and configure it so that it calls your stuff
when it recieves emails (unless you want to do polling...). You will
need to configure a test setup for developement where you can send and
receive fake mails, etc. For some reason most MDA are terrible to use
and configure. Maybe it's because I'm not well versed in all of this
but for me all of this is still a PITA.

Re: BuGit

[Stefan Monnier]

> You probably don't want to handle the actual sending&receiving part
> yourself.

For sending, it's easy: /usr/sbin/sendmail.
This is already working (since the current code provides email notifications).

But for receiving it's more messy, indeed, so BuGit can't take care of the
setup itself.  Instead, the user will have to arrange its MTA so that the
relevant email messages get piped to "bugit mua" or something like that.


        Stefan

Re: BuGit

[Stefan Monnier]

> I have found other similar projects (git based issue management) which
> you might be interested in:
>     https://github.com/dspinellis/gi
> The README also links to similar projects:

Indeed, there are other distributed issue tracking systems out there
(the most mature being maybe Bugs Everywhere, IIRC).

What distinguishes BuGit is that it uses a representation that lets Git
take care of all the merging that can be done automatically (and there
are only very few cases where conflicts can arise, the only important
one so far is when the bug's name is changed concurrently), and that
individual issues can have their own life rather than being tied to
a particular database (so in an ideal world, when Debian gets an Emacs
bug and forwards it to us, we could share the same BuGit issue for it
(not the same issue Number, but same issue ID) so any comment added from
the Debian side appears in Emacs's database and vice versa).


        Stefan

Re: BuGit

[John Wiegley]

>>>>> Aurélien Aptel <aurelien.aptel+emacs@gmail.com> writes:

> I have found other similar projects (git based issue management) which you
> might be interested in:

>     https://github.com/dspinellis/gi

I even started one of my own a while back, that was later taken over by
another maintainer:

    https://github.com/duplys/git-issues

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2