Bug tracker choices for Emacs.

Bug tracker choices for Emacs.

[Karl Fogel]

This may be a controversial post.  I'll start out by saying that it is
is *no* way a criticism of Don Armstrong, whose work in setting up
emacsbugs.donarmstrong.com I appreciate very much, as do many others.

When we chose a bug tracker, the criteria were: it must be free
software, and must be manipulable by email.  Debbugs was the only one
that fit the bill, IIRC -- there was some consideration of RT, but it
turned out not all of RT's functionality was available by email.

Since then, another one has appeared: the Launchpad bug tracker,
https://bugs.launchpad.net/.  It is now free software under the GNU
Affero General Public License, along with the rest of Launchpad.

It can be completely operated by email:

  https://help.launchpad.net/Bugs/EmailInterface

(I know developers who interact with it solely by email.)  Also, it has
APIs:

  https://help.launchpad.net/API
  https://help.launchpad.net/API/Uses

The APIs can be driven by using the 'launchpadlib' Python library, or
(less commonly) through direct "ReST"-style calls.

So, are we happy with debbugs?  Here are problems that I've found
discourage me from using debbugs:

  - Not really operable via the web -- just read-only operations.
    This is huge.

  - Does not do automatic duplicate-finding when a new bug is submitted.
    Launchpad bugs does, and it's a real gift.  (Actually, from reading
    the documentation, it's not clear to me how to handle duplicates in
    debbugs at all -- there doesn't seem to be a simple way to say
    "Close bug #Y because it's a duplicate of #X".  And the mere fact
    that one has to read the documentation is already a disadvantage.)

  - Interface can be a bit unintuitive ("Toggle useless messages", for
    example; or consider the number of choices one must make before
    doing a simple search).

  - A bit Debian-centric.  See the long list of checkboxes for
    distributions in the search form on the front page, for example.

When we chose debbugs, it was effectively the only choice we had.  It
certainly gets the job done.  But I'd like to know if anyone else is
tempted by the thought of switching our bug tracking to Launchpad.

Launchpad already has code for converting debbugs to Launchpad bugs, and
we could easily leave forwarding pointers.  I don't think conversion
costs would be a huge problem, if we chose to convert.

Thoughts?

Full disclosure: I work for Canonical (obvious from my email address),
the company that runs Launchpad.

-Karl

Re: Bug tracker choices for Emacs.

[Michael Albinus]

Karl Fogel <karl.fogel@canonical.com> writes:

> So, are we happy with debbugs?  Here are problems that I've found
> discourage me from using debbugs:

A while ago I wrote xesam-debbugs.el, a search engine for
debbugs. Finally I've stopped it, because debbugs does not offer an API
for full text search.

Does Launchpad supports it per API? If yes, I would be in favor of this,
because this would ease bug handling from inside Emacs.

> -Karl

Best regards, Michael.

Re: Bug tracker choices for Emacs.

[Karl Fogel]

Michael Albinus <michael.albinus@gmx.de> writes:
> A while ago I wrote xesam-debbugs.el, a search engine for
> debbugs. Finally I've stopped it, because debbugs does not offer an API
> for full text search.
>
> Does Launchpad supports it per API? If yes, I would be in favor of this,
> because this would ease bug handling from inside Emacs.

Yes.  See https://launchpad.net/+apidoc/#bug_target-searchTasks for
details, and https://help.launchpad.net/API for an introduction.

The "searchTasks()" API provides basically the same functionality as the
Bugs Advanced Search web screen, so to get a quick overview, just visit
https://bugs.edge.launchpad.net/emacs/+bugs?advanced=1 .

Best,
-Karl

Re: Bug tracker choices for Emacs.

[Stefan Monnier]

> It can be completely operated by email:
>   https://help.launchpad.net/Bugs/EmailInterface

Can we make bug-gnu-emacs@gnu.org redirect to it (i.e. accept email
submissions from any random user without prior registration, which is
*very* important).
Can we install it in debbugs.gnu.org?
Can it be operated from Rmail (i.e. MUAs without support for GPG signing)?

>   - Not really operable via the web -- just read-only operations.
>     This is huge.

This is not huge for me, but it's clearly a shortcoming.

>   - Does not do automatic duplicate-finding when a new bug is submitted.
>     Launchpad bugs does, and it's a real gift.  (Actually, from reading
>     the documentation, it's not clear to me how to handle duplicates in
>     debbugs at all -- there doesn't seem to be a simple way to say
>     "Close bug #Y because it's a duplicate of #X".  And the mere fact
>     that one has to read the documentation is already a disadvantage.)

It's called "merge".  It's worked OK for me.

>   - Interface can be a bit unintuitive ("Toggle useless messages", for
>     example; or consider the number of choices one must make before
>     doing a simple search).
>   - A bit Debian-centric.  See the long list of checkboxes for
>     distributions in the search form on the front page, for example.

You mean the Web interface?  Yes, the web interface is not great.

> When we chose debbugs, it was effectively the only choice we had.  It
> certainly gets the job done.  But I'd like to know if anyone else is
> tempted by the thought of switching our bug tracking to Launchpad.

I'm not married to debbugs.  Launchpad's web interface is clearly more
pleasant, so if the other criterias are fulfilled, we could switch.


        Stefan

Re: Bug tracker choices for Emacs.

[Glenn Morris]

Karl Fogel wrote:

> So, are we happy with debbugs?

I'm not. Some of my reasons are listed here:

http://emacsbugs.donarmstrong.com/cgi-bin/pkgreport.cgi?pkg=emacsbugs.donarmstrong.com

The main problem is, it's effectively unmaintained, and the Emacs
developers have no administrative access. A move to a gnu machine that
would hopefully fix these issues has been waiting for the best part of
a year.

By the way, I have been tempted to suggest we just start using the gnu
version anyway, rather than waiting for the current database to get
moved there.

>  Here are problems that I've found discourage me from using debbugs:
>
>   - Not really operable via the web -- just read-only operations.
>     This is huge.

Not an issue for me.

>   - Does not do automatic duplicate-finding when a new bug is submitted.
>     Launchpad bugs does, and it's a real gift.

What does "automatic duplicate-finding" mean?

> (Actually, from reading the documentation, it's not clear to me how
> to handle duplicates in debbugs at all -- there doesn't seem to be a
> simple way to say "Close bug #Y because it's a duplicate of #X".

As pointed out, the merge/forcemerge commands do this.

>   - A bit Debian-centric.  See the long list of checkboxes for
>     distributions in the search form on the front page, for example.

Offered to fix this kind of thing a year ago... (Bug#750)

> Launchpad already has code for converting debbugs to Launchpad bugs, and
> we could easily leave forwarding pointers.  I don't think conversion
> costs would be a huge problem, if we chose to convert.
>
> Thoughts?


Questions I would ask of a bug tracker:

How well maintained is it? How many developers are there, and how
responsive are they to feature and problem requests?

If we want to customize the way it behaves for Emacs, is there someone
who can do this for us, or help us do it to our local copy if it's not
a change appropriate for the tracker in general? Failing that, how
easy is it for an outsider to modify the code?

Can the Emacs developers get full administrative access?

How does it handle spam? Unregistered users must be able to submit
Emacs bug reports. Therefore there will be spam. Some kind of human
moderation is required. Can this be integrated into the mail flow? The
current emacsbugs method (closing spam bugs after the fact) is a waste
of effort, and does not deal with spam added to existing bugs. The
state of emacsbugs is shameful in this regard (again, see Bug#750 as
an example).

How does it handle the CC problem? Currently, when people report a new
bug, they need to use X-Debbugs-CC rather than CC, lest each reply
create a new bug. Often, they don't know they need to do this. Hence,
bug 4065 and 4066, for example. Tracking of references/message-ids
might fix this?

If it sends out admin messages, can these be directed to a separate
mail list from the normal bug list?

Does it suppress duplicates? This may need message-id tracking.

Can it automatically subscribe the bug reporter to all followup discussion?

Does it obfuscate addresses in the web interface?

How good is the search function? The debbugs one is not great.


Your best bet is probably to just set up a test-bed for people to play
with, and see if they like it...

Re: Bug tracker choices for Emacs.

[Karl Fogel]

Stfan Monnier and Glenn Morris both asked questions, in separate mails;
I'll answer them together in this one mail.

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> It can be completely operated by email:
>>   https://help.launchpad.net/Bugs/EmailInterface
>
> Can we make bug-gnu-emacs@gnu.org redirect to it (i.e. accept email
> submissions from any random user without prior registration, which is
> *very* important).

Currently it requires a registered user.  I'm not sure that's a strong
policy; if we had other anti-spam measures (such as requiring a
particular string to be in each mail body), then maybe we could do away
with that.

> Can we install it in debbugs.gnu.org?

You could, but I meant hosting at Launchpad.net.  Running an instance of
Launchpad involves all sorts of production issues; Canonical is already
bearing that cost, and I certainly am not volunteering to duplicate the
cost anywhere else :-).

> Can it be operated from Rmail (i.e. MUAs without support for GPG signing)?

I think you can comment on bugs but not create them, via unsigned mail.

>>   - Not really operable via the web -- just read-only operations.
>>     This is huge.
>
> This is not huge for me, but it's clearly a shortcoming.

I should have said "huge for most users".

>>   - Does not do automatic duplicate-finding when a new bug is submitted.
>>     Launchpad bugs does, and it's a real gift.  (Actually, from reading
>>     the documentation, it's not clear to me how to handle duplicates in
>>     debbugs at all -- there doesn't seem to be a simple way to say
>>     "Close bug #Y because it's a duplicate of #X".  And the mere fact
>>     that one has to read the documentation is already a disadvantage.)
>
> It's called "merge".  It's worked OK for me.

Odd; I saw that page, but what I read there didn't seem to be about
duplicate bugs as I understand them.

  http://emacsbugs.donarmstrong.com/server-control#merge

  "Before bugs can be merged they must be in exactly the same state:
  either all open or all closed, with the same forwarded-to upstream
  author address or all not marked as forwarded, all assigned to the
  same package or package(s) (an exact string comparison is done on the
  package to which the bug is assigned), and all of the same severity.
  ..."

>>   - Interface can be a bit unintuitive ("Toggle useless messages", for
>>     example; or consider the number of choices one must make before
>>     doing a simple search).
>>   - A bit Debian-centric.  See the long list of checkboxes for
>>     distributions in the search form on the front page, for example.
>
> You mean the Web interface?  Yes, the web interface is not great.

I meant the web interface.  It goes without saying that the email
interface is unintuitive, but that's okay, as the people who would use
that are looking for an interface aimed at experts, not newcomers.

Glenn Morris writes:
> Karl Fogel wrote:
> 
> > So, are we happy with debbugs?
> 
> I'm not. Some of my reasons are listed here:
> 
> http://emacsbugs.donarmstrong.com/cgi-bin/pkgreport.cgi?pkg=emacsbugs.donarmstrong.com
> 
> The main problem is, it's effectively unmaintained, and the Emacs
> developers have no administrative access. A move to a gnu machine that
> would hopefully fix these issues has been waiting for the best part of
> a year.

Launchpad is certainly maintained, but it would still be on non-GNU
servers (unless GNU wanted to run an instance, which would be a huge
undertaking and is not what I was suggesting).

> >   - Does not do automatic duplicate-finding when a new bug is submitted.
> >     Launchpad bugs does, and it's a real gift.
> 
> What does "automatic duplicate-finding" mean?

When you file a new bug, the system first searches for bugs that look
like it, to cut down on the number of duplicate filings (which otherwise
is usually large).

> > (Actually, from reading the documentation, it's not clear to me how
> > to handle duplicates in debbugs at all -- there doesn't seem to be a
> > simple way to say "Close bug #Y because it's a duplicate of #X".
> 
> As pointed out, the merge/forcemerge commands do this.

Yup.  See my comments above, but obviously once one understands the
commands, this is what they do.

> Questions I would ask of a bug tracker:
> 
> How well maintained is it? How many developers are there, and how
> responsive are they to feature and problem requests?

Hard to say how many developers there are, since Launchpad is free
software.  Estimate 3-5 right now, since there are some full-time people
on it.

> If we want to customize the way it behaves for Emacs, is there someone
> who can do this for us, or help us do it to our local copy if it's not
> a change appropriate for the tracker in general? Failing that, how
> easy is it for an outsider to modify the code?

It's a free software project.  For code to get deployed on
Launchpad.net, it obviously has to be absorbed into upstream and go
through the rollout process, so that means no quick-n-dirty tweaks.

> Can the Emacs developers get full administrative access?

As in root?  No.

> How does it handle spam? Unregistered users must be able to submit
> Emacs bug reports. Therefore there will be spam. Some kind of human
> moderation is required. Can this be integrated into the mail flow? The
> current emacsbugs method (closing spam bugs after the fact) is a waste
> of effort, and does not deal with spam added to existing bugs. The
> state of emacsbugs is shameful in this regard (again, see Bug#750 as
> an example).

I don't think it deals well with this (see above; it could be improved).
Right now its solution is pre-registered sender addresses and
GPG-signing.

> How does it handle the CC problem? Currently, when people report a new
> bug, they need to use X-Debbugs-CC rather than CC, lest each reply
> create a new bug. Often, they don't know they need to do this. Hence,
> bug 4065 and 4066, for example. Tracking of references/message-ids
> might fix this?

Gosh.  I've never noticed any CC problem, so I guess it handles this
okay, but I'm not 100% sure.

> If it sends out admin messages, can these be directed to a separate
> mail list from the normal bug list?

Not sure what the core question is here.

> Does it suppress duplicates? This may need message-id tracking.

Again, not sure what the question is.  Does it fold duplicate mails?  I
don't know.

> Can it automatically subscribe the bug reporter to all followup discussion?

I think it does.

> Does it obfuscate addresses in the web interface?

Right now it doesn't need to, because it uses usernames instead.  

> How good is the search function? The debbugs one is not great.

The search functionality is good, IMHO.  (I'd be more specific, but it
might be easier to just try it and see what you think.)

> Your best bet is probably to just set up a test-bed for people to play
> with, and see if they like it...

Based on the above, I'm not going to pursue it right now.  There are too
many changes that would be necessary, and other things (like the Bazaar
switchover) are more pressing to me.

This discussion is in the archives now.  If we ever want to revisit it,
I hope we'll remember to start here.

-Karl