Bug statistics

Bug statistics

[Glenn Morris]

For entertainment purposes only, here are some statistics derived from
the Emacs bug database. Since Feb 2008:

4594 reports  [1]
mean number of reports per day ~ 5.4
1218 distinct submitter emails  [2]
the top 10 submitters account for 30% of all reports
67% of submitters only have one report

2647 closed reports
closed by 65 distinct email addresses  [2]
the top 10 closers account for 90% of all reports

~ 23000 emails sent
the top 10 correspondents account for 60% of these mails  [2]

Of the open reports:
14% have been marked "minor"
20% have been marked "wishlist"
0.7% have a severity higher than "normal".

9% are tagged "moreinfo"
5% are tagged "wontfix".



Awards!

The report with the most correspondence is:
#865 "The directory is unsafe today"


The most reported bug (in the sense of, same issue reported
independently) is:

#588 "with-ns emacs crash"

(#4122 "GTK menu contents don't change" will probably catch up soon...)


The award for "single bug needlessly split into the most different
reports" goes to:

#914 "In CVS Emacs, calc-eval gives multiplication higher precidence
      than division"


Notes:

[1] Merged bugs are not really counted correctly.

[2] I made some attempt to merge aliases where I knew them to be the same
person using more than one email address, but some will remain.

Re: Bug statistics

[Dan Nicolaescu]

Glenn Morris <rgm@gnu.org> writes:

> For entertainment purposes only, here are some statistics derived from
> the Emacs bug database. Since Feb 2008:
>
> 4594 reports  [1]
> mean number of reports per day ~ 5.4
> 1218 distinct submitter emails  [2]
> the top 10 submitters account for 30% of all reports
> 67% of submitters only have one report
>
> 2647 closed reports
> closed by 65 distinct email addresses  [2]
> the top 10 closers account for 90% of all reports
>
> ~ 23000 emails sent
> the top 10 correspondents account for 60% of these mails  [2]

Thanks for doing this!

What is the number of open bugs with patches available? 
(It would be nice if all those pending patches would be applied...)

Re: Bug statistics

[Glenn Morris]

Dan Nicolaescu wrote:

> What is the number of open bugs with patches available? 

I meant to do that as well...

139 bugs tagged with "patch" = 7.1%

Browse them at:

http://debbugs.gnu.org/cgi/pkgreport.cgi?include=tags%3Apatch;exclude=pending%3Adone;package=emacs

Re: Bug statistics

[Richard Stallman]

    4594 reports  [1]
    2647 closed reports

Amost 2000 bug reports not closed
is rather disturbing.

    14% have been marked "minor"
    20% have been marked "wishlist"
    9% are tagged "moreinfo"
    5% are tagged "wontfix".

That is about 40%.  It seems to imply there are around 1200 bug
reports which are not marked in these ways, and not fixed either.
Is that true?

Can you compute the number of bug reports that are waiting for action
by the maintainers?

Re: Bug statistics

[Tassilo Horn]

On Thursday 24 June 2010 17:40:50 Richard Stallman wrote:

Hi Richard,

>     4594 reports  [1]
>     2647 closed reports
> 
> Amost 2000 bug reports not closed
> is rather disturbing.
> 
>     14% have been marked "minor"
>     20% have been marked "wishlist"
>     9% are tagged "moreinfo"
>     5% are tagged "wontfix".
> 
> That is about 40%.  It seems to imply there are around 1200 bug
> reports which are not marked in these ways, and not fixed either.
> Is that true?

Statistics never lie. ;-)

I guess, that there are quite a lot duplicate bug reports.  The problem
with M-x report-emacs-bug is that a bug gets filed unconditionally.
With web-based bugtrackers, after entering the bug summary the whole
database is searched for similar reports and that list is presended to
the user.  Then she can decide if it's really a new, unknown bug.  If
not, she can simply append additional information to an existing bug.

As a poor man's solution, I'd suggest to add some text to the beginning
of the bug report email M-x report-emacs-bug is creating, which asks the
user to first search the existing bugs at

  http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs

before filing a new one.  (And maybe, there should be information how to
change the "To:" in the mail, so that the report will be appended to the
existing report if there is any.)

Bye,
Tassilo

Re: Bug statistics

[Karl Fogel]

Richard Stallman <rms@gnu.org> writes:
>    4594 reports  [1]
>    2647 closed reports
>
>Amost 2000 bug reports not closed
>is rather disturbing.
>
>    14% have been marked "minor"
>    20% have been marked "wishlist"
>    9% are tagged "moreinfo"
>    5% are tagged "wontfix".
>
>That is about 40%.  It seems to imply there are around 1200 bug
>reports which are not marked in these ways, and not fixed either.
>Is that true?
>
>Can you compute the number of bug reports that are waiting for action
>by the maintainers?

Percentage of bug reports not closed is not in itself a problem.  It's
possible that bugs-not-triaged is a problem, but that really depends on
many things about the project.

  http://ostatic.com/blog/fixing-the-perception-of-bugs

(And yes, I'm quoting someone quoting me in order to give my opinions
more authority. :-) )

IMHO the poor web interface of our bug tracker is an impediment to
finding and triaging bugs.  Heck, I can't even find the tracker half the
time.  If I start at http://gnu.org/software/emacs and click on the link
to [what is purported to be] the Emacs bug database, I am taken to the
generic top of the Gnu tracker: http://debbugs.gnu.org/.  Then there is
a table there, with a row for Emacs.  Since what I want to do is search
the Emacs bug database, I take the closest option to that, which is the
"Browse bug reports" column with its cell for "Emacs reports".  Clicking
on that brings me to:

http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs;max-bugs=100;base-order=1;bug-rev=1

That page turns out to be just a random 100 out of many bug reports --
not that this is in any way obvious.  You have to actually read the text
at the top of the page ("Showing results 0 - 100 of 2003", which of
course my eye glazes over because "2003" looks like a date and I'm not
interested in dates) to realize that what follows it is unintentionally
insincere, e.g.:

   If you find a bug not listed here, please report it.

    * Outstanding bugs -- Normal bugs; Patch Available (2 bugs)
    * Outstanding bugs -- Normal bugs; Unclassified (46 bugs)
    * Outstanding bugs -- Normal bugs; More information needed (2 bugs)
    * Outstanding bugs -- Minor bugs; Unclassified (16 bugs)
    * Outstanding bugs -- Minor bugs; More information needed (1 bug)
    * Outstanding bugs -- Wishlist items; Unclassified (11 bugs)
    * Resolved bugs -- Normal bugs (16 bugs)
    * Resolved bugs -- Minor bugs (5 bugs)

That first line is wrong in an obvious way.  The remaining lines are
wrong in the same way, though it's slightly less obvious.  There are far
more than 2 outstanding bugs (one presumes).  That summary is just
referring to the bugs *on the current page*.  But for a randomly
selected listing page, I'm not really sure what the point of a summary
block at the top is.  It can, however, confuse people who might think
they're looking at project-wide stats rather than page-wide stats.

All of which misses the main point: there's no search box.  Oh wait,
there is.  It's all the way at the bottom of the page, where I'd never
think to look for it, because these days search boxes are all (sanely)
at the top of the page, as befits the single most useful feature of a
bug tracker.

I don't know if my periodic rants about our bug tracker have any
constructive effect.  I feel better after them, though.  In any case,
bug tracker web UI aside, there is nothing inherently good or bad about
a growing bug database.  An analysis of the number of unique submitters
would tell us far more.

-K

Re: Bug statistics

[Glenn Morris]

Richard Stallman wrote:

>     20% have been marked "wishlist"
>     9% are tagged "moreinfo"
>     5% are tagged "wontfix".
>
> That is about 40%.  It seems to imply there are around 1200 bug
> reports which are not marked in these ways, and not fixed either.
> Is that true?

The total number of open Emacs reports not marked wishlist, wontfix,
moreinfo, or unreproducible, is 1295. Excluding all merged bugs, it is
1143. So the real answer is somewhere inbetween. Excluding minor bugs,
the number is about 950. Further excluding bugs marked as specific to
the MS-Windows or Mac ports, the number falls to about 780. Half of
these reports were opened in the last 8 months. So it is not as bad as
it first appears. (Exclude the top 4 bug submitters, and it falls to
640...)

I'm also sure there are lots of things that haven't been given the
appropriate tags. And probably some things that were fixed but haven't
been closed (or are waiting for people to confirm they can be closed).

> Can you compute the number of bug reports that are waiting for action
> by the maintainers?

I don't know how to quantify this. I can try to answer any question
you can frame as the equivalent of one or more greps. I don't think
the precise numbers are especially meaningful though.


The short answer is that there are certainly lots of open reports, and
it would be great if people could help deal with them. This is always
going to be the answer.


About the only useful thing I take away from this exercise is that
there are many people whose only interaction with the developers is in
the form of a single bug report. Good to try and remember this.

Re: Bug statistics

[Glenn Morris]

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

Karl Fogel wrote:

> That page turns out to be just a random 100 out of many bug reports --

It is the 100 most recent reports, most recent first.
This is not "random".

The machine serving the reports is not powerful enough to show more
than about 400 at once.

> I don't know if my periodic rants about our bug tracker have any
> constructive effect.  

Not really. They just put my back up, and it's basically just me
trying to deal with these issues. I'm sure you know more constructive
ways to report problems, that are more likely to get results.

Sorry debbugs is not Launchpad, I guess.

> I feel better after them, though.

Good for you.

> An analysis of the number of unique submitters would tell us far more.

I quoted that number in my first email. 1218. Attached is a histogram.



[-- Attachment #2: submit.ps.gz --]
[-- Type: application/octet-stream, Size: 2249 bytes --]

Re: Bug statistics

[Karl Fogel]

Glenn Morris <rgm@gnu.org> writes:
>Karl Fogel wrote:
>> That page turns out to be just a random 100 out of many bug reports --
>
>It is the 100 most recent reports, most recent first.
>This is not "random".

Since the page doesn't say that, they are random to the user.  (I
actually guessed it, but wasn't sure and didn't see a way to be sure.
There isn't any date information on the page.)

>The machine serving the reports is not powerful enough to show more
>than about 400 at once.

Understood.  I wasn't saying we shouldn't paginate results.

>> I don't know if my periodic rants about our bug tracker have any
>> constructive effect.  
>
>Not really. They just put my back up, and it's basically just me
>trying to deal with these issues. I'm sure you know more constructive
>ways to report problems, that are more likely to get results.

Sorry, I didn't know it put your back up.  I'll keep that in mind in the
future.  I tried to be specific in my critique -- ranty tone aside, I
pointed out several things that would be easy improvements for someone
familiar with our bug tracker deployment.  (I am not familiar with it,
and unfortunately can't become so due to other committments.)

>Sorry debbugs is not Launchpad, I guess.

I'm not advocating Launchpad.  I think I've proposed it as *a* solution
in the past, but certainly not as the only possible solution.

>> An analysis of the number of unique submitters would tell us far more.
>
>I quoted that number in my first email. 1218. Attached is a histogram.

Thank you (and for the histogram).  I was only looking at Richard's
quoted excerpt.  That part of your original email is exactly what I had
in mind.

If the spread of submitter uniqueness goes down even as the rate of
submissions stays the same or increases, then we'll know we have a
problem.  It would mean fewer people are submitting bug reports, which
would imply either that users are losing motivation to submit bugs
(bad), or that the actual number of users is going down (worse).

-Karl

Re: Bug statistics

[Glenn Morris]

Karl Fogel wrote:

> I tried to be specific in my critique -- ranty tone aside, I pointed
> out several things that would be easy improvements for someone
> familiar with our bug tracker deployment.

I tend to just tune out the rants. It's somewhat amusing that they
make me defensive of a system I never particularly liked...

Sorry to make you use a system you dislike, but if you can report your
specific issues in separate mails to submit AT debbugs.gnu.org, with
"Package: debbugs.gnu.org" in the first line of the body, I will try
to address those that I can.

Re: Bug statistics

[Karl Fogel]

Glenn Morris <rgm@gnu.org> writes:
>Sorry to make you use a system you dislike, but if you can report your
>specific issues in separate mails to submit AT debbugs.gnu.org, with
>"Package: debbugs.gnu.org" in the first line of the body, I will try
>to address those that I can.

Sold -- will do.

I'm on a train right now and it's a bit hard to type, so I'll do this
later today or this weekend.  (Not so hard to type that I can't rant,
mind you, just hard enough to interfere with constructive action!)

-Karl

Re: Bug statistics

[Dan Nicolaescu]

Karl Fogel <kfogel@red-bean.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>>    4594 reports  [1]
>>    2647 closed reports
>>
>>Amost 2000 bug reports not closed
>>is rather disturbing.
>>
>>    14% have been marked "minor"
>>    20% have been marked "wishlist"
>>    9% are tagged "moreinfo"
>>    5% are tagged "wontfix".
>>
>>That is about 40%.  It seems to imply there are around 1200 bug
>>reports which are not marked in these ways, and not fixed either.
>>Is that true?
>>
>>Can you compute the number of bug reports that are waiting for action
>>by the maintainers?
>
> Percentage of bug reports not closed is not in itself a problem.  It's
> possible that bugs-not-triaged is a problem, but that really depends on
> many things about the project.
>
>   http://ostatic.com/blog/fixing-the-perception-of-bugs
>
> (And yes, I'm quoting someone quoting me in order to give my opinions
> more authority. :-) )
>
> IMHO the poor web interface of our bug tracker is an impediment to
> finding and triaging bugs.  Heck, I can't even find the tracker half the
> time.  If I start at http://gnu.org/software/emacs and click on the link
> to [what is purported to be] the Emacs bug database, I am taken to the
> generic top of the Gnu tracker: http://debbugs.gnu.org/.  Then there is
> a table there, with a row for Emacs.  Since what I want to do is search
> the Emacs bug database, I take the closest option to that, which is the
> "Browse bug reports" column with its cell for "Emacs reports".  Clicking
> on that brings me to:
>
> http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs;max-bugs=100;base-order=1;bug-rev=1

Although the debbugs UI is far from ideal, do you have any evidence
that is one of the more important problems?

IMO the main problem is man power.

There are >100 bugs with patches attached that have not been applied.
For a lot of these patches there's no obvious maintainer to take care
of them, so one of the maintainers would have to do it.

Same goes about bug reports, quite a few are about areas that nobody
feels particularly attached to, so they don't get any action.

Re: Bug statistics

[Karl Fogel]

Dan Nicolaescu <dann@gnu.org> writes:
>Although the debbugs UI is far from ideal, do you have any evidence
>that is one of the more important problems?

Only anecdotal, on a very limited sample (me).  I've occasionally fixed
bugs but not bothered to look for them in the bug tracker because the
process of doing so would be too time-consuming and difficult.

I don't know how many other developers are the same way, though.

>IMO the main problem is man power.
>
>There are >100 bugs with patches attached that have not been applied.
>For a lot of these patches there's no obvious maintainer to take care
>of them, so one of the maintainers would have to do it.
>
>Same goes about bug reports, quite a few are about areas that nobody
>feels particularly attached to, so they don't get any action.

Yeah.  As the post I pointed to explains, I'm not sure this is actually
a problem.  

It would be nice to apply or reject the patches, though.  I don't know
how to find them.  Doing a search on the tag "patch" (as suggested by
the text "Valid tags are patch, wontfix, moreinfo, unreproducible,
fixed, notabug") claims to find 2006 bugs.  Since that's exactly how
many bugs total are in the tracker, it's got to be wrong.

-Karl

Re: Bug statistics

[Stephen J. Turnbull]

Richard Stallman writes:

 >     4594 reports  [1]
 >     2647 closed reports
 > 
 > Amost 2000 bug reports not closed
 > is rather disturbing.

Not really.  That's 2000 (minor) contributions to be grateful for, and
2000 reasons for somebody new to volunteer to work on Emacs!

Re: Bug statistics

[Yoni Rabkin]

> Although the debbugs UI is far from ideal, do you have any evidence
> that is one of the more important problems?
>
> IMO the main problem is man power.

Having followed the Emacs development community for some time now I
think that everyone appreciates how few people can commit and how
overwhelmed with work those people are.

> There are >100 bugs with patches attached that have not been applied.
> For a lot of these patches there's no obvious maintainer to take care
> of them, so one of the maintainers would have to do it.
>
> Same goes about bug reports, quite a few are about areas that nobody
> feels particularly attached to, so they don't get any action.

I had recently posted a patch to this list (next time I'll add it to the
bug system instead). Here is my view of this from a bug-reporter's point
of view:

I do believe that if someone posts a bug and feels strongly about it
they should do a follow-up, say at most once a week, and make sure to
get an answer from someone who can commit it. An acceptable answer could
be: "I'll commit this in 3 months, please write back then to remind me."
or "I'll commit this but I don't know when, please write back in a week
and we'll see."

What I'm trying to say is that I think outstanding bugs should not be
seen as being the sole responsibility of developers who can commit. The
people who submit the bug reports should try and follow the
activity. After all, they are reporting on issues they want fixed.

Would such a policy produce noise? I don't think so; I doubt that the
majority of people would take the time to diligently follow up their bug
reports to completion.

-- 
   "Cut your own wood and it will warm you twice"

Re: Bug statistics

[Eli Zaretskii]

> From: Tassilo Horn <tassilo@member.fsf.org>
> Date: Thu, 24 Jun 2010 19:59:11 +0200
> Cc: 
> 
> I guess, that there are quite a lot duplicate bug reports.  The problem
> with M-x report-emacs-bug is that a bug gets filed unconditionally.
> With web-based bugtrackers, after entering the bug summary the whole
> database is searched for similar reports and that list is presended to
> the user.  Then she can decide if it's really a new, unknown bug.  If
> not, she can simply append additional information to an existing bug.

This is also far from ideal.  Unless you have a lot of time on your
hands, going through the bugs and trying to figure out if they are the
same as yours is a nuisance.  This should be a job of some program
that runs periodically, or, failing that, of a human (whom we
obviously lack).

Gathering all the information needed for a good bug report is already
a non-trivial job (witness how many veteran Emacs contributors fail to
do that!).  We should not burden the bug submitters with any
additional costs.

Re: Bug statistics

[Eli Zaretskii]

> From: Dan Nicolaescu <dann@gnu.org>
> Date: Thu, 24 Jun 2010 21:28:19 -0400
> Cc: emacs-devel@gnu.org
> 
> IMO the main problem is man power.

100% agreement.

Someone(TM) should check whether the switch to Bazaar made the number
and rate of contributions from non-core developers higher.  If not, we
should look for ways to attract more contributors, because the current
resources of manpower are evidently not large enough.

Re: Bug statistics

[Tassilo Horn]

On Friday 25 June 2010 12:24:32 Eli Zaretskii wrote:

> > With web-based bugtrackers, after entering the bug summary the whole
> > database is searched for similar reports and that list is presended
> > to the user.  Then she can decide if it's really a new, unknown bug.
> > If not, she can simply append additional information to an existing
> > bug.
> 
> This is also far from ideal.  Unless you have a lot of time on your
> hands, going through the bugs and trying to figure out if they are the
> same as yours is a nuisance.

At least my experience with the bugtrackes of the Gentoo GNU/Linux and
KDE projects are quite good.  After entering a bug summary, I get a
short list of possibly related bugs.  Quite often, the bug I wanted to
report is already known and in that list.

And it's no burden to the user.  If I didn't have time, I could skip
checking the suggested relevant bugs and file my bug as new bug, thus
putting the burden on the developers.

I don't say that users should be forced to search the bug database
before writing a report, but there should be an easy way to let willing
users do so.  Currently, the emacs bug reporting interface doesn't even
mention that there is something like that!

IMO, there are many users who care about writing good reports and don't
want to produce duplicates.  When there are more than one bug in
"relevant bugs list" dealing with the same issue, then I often spend
some minutes to link them with each other.  This volunteering in
managing bug reports should be encouraged and made easy for all users.
Especially this "making easy" is not the strong point of debbugs. :-(

Bye,
Tassilo

Re: Bug statistics

[Richard Stallman]

When there are duplicate bug reports, could someone close
one of them, giving a pointer to the other?

However, duplicates would not be a problem if we
had a small number of open bug reports.  We would see
them and close them when we close the others that they match.
So duplication is not the root of the problem.

I think the problem is not enough effort to fix bugs.

Re: Bug statistics

[Chong Yidong]

Tassilo Horn <tassilo@member.fsf.org> writes:

> At least my experience with the bugtrackes of the Gentoo GNU/Linux and
> KDE projects are quite good.  After entering a bug summary, I get a
> short list of possibly related bugs.  Quite often, the bug I wanted to
> report is already known and in that list.
>
> And it's no burden to the user.  If I didn't have time, I could skip
> checking the suggested relevant bugs and file my bug as new bug, thus
> putting the burden on the developers.

We don't seem to get many duplicates, so this is not something we need
to worry much about.

Re: Bug statistics

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> IMO the main problem is man power.
>
> 100% agreement.
>
> Someone(TM) should check whether the switch to Bazaar made the number
> and rate of contributions from non-core developers higher.  If not, we
> should look for ways to attract more contributors, because the current
> resources of manpower are evidently not large enough.

While new contributors are always to be welcomed, we shouldn't have
unrealistic expectations about their impact on the bug count.

Many of the unfixed bugs are "long tail" problems that are rather
difficult to find and/or fix.  Non-core contributors almost never work
on such issues, whereas core contributors have to juggle between fixing
these and working on new features.

This is not an Emacs-specific phenomenon, of course.

Re: Bug statistics

[Dan Nicolaescu]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dan Nicolaescu <dann@gnu.org>
>> Date: Thu, 24 Jun 2010 21:28:19 -0400
>> Cc: emacs-devel@gnu.org
>> 
>> IMO the main problem is man power.
>
> 100% agreement.
>
> Someone(TM) should check whether the switch to Bazaar made the number
> and rate of contributions from non-core developers higher.

Not sure how easy is it to measure that, we've also had a feature
freeze in place for a while which skews results.

Speaking for myself (TM): the switch to Bazaar made it MUCH HARDER to contribute.
It takes 5-15 minutes (sometimes even longer) to check in anything,
even for trivial changes.
If making / testing a change is quicker than the time to check it in,
it makes one think twice before trying to do anything.

I am not sure if the "smart server" for bzr would help, but the
current one is just doing some very effective negative advertising.
One has to wonder why it even exists, given that it performs so badly.

> should look for ways to attract more contributors, because the current
> resources of manpower are evidently not large enough.

Attracting contributors always helps, but it's also a question of
better using the current contributors.  
There are many many discussions on the list that end with nothing
because there's no clear authority that can make a final decision, and
nobody feels particularly responsible to do so.  
admin/MAINTAINERS is a good example: it lists a tiny amount of the
files in the tree.
IMHO there's a need for more distributed responsibility.

Re: Bug statistics

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
>This is also far from ideal.  Unless you have a lot of time on your
>hands, going through the bugs and trying to figure out if they are the
>same as yours is a nuisance.  This should be a job of some program
>that runs periodically, or, failing that, of a human (whom we
>obviously lack).

FWIW, when I've experienced this automated dup-finding in the web
interface of other bug trackers, it has not been a nuisance -- on the
contrary it was a great relief, because it helped me know I'm not
wasting the developers' time with a duplicate report.  (The majority of
the time, it did find a dup of what I was about to file.  Sometimes I
was able to go to that existing report and add useful information.)

For me it became one of those "never go back" features, like sexp motion
in Emacs.

>Gathering all the information needed for a good bug report is already
>a non-trivial job (witness how many veteran Emacs contributors fail to
>do that!).  We should not burden the bug submitters with any
>additional costs.

It wasn't a burden at all (for me) -- quite the opposite.

Re: Bug statistics

[Stephen J. Turnbull]

Tassilo Horn writes:

 > And it's no burden to the user.  If I didn't have time, I could skip
 > checking the suggested relevant bugs and file my bug as new bug, thus
 > putting the burden on the developers.

At least on Launchpad (Canonical's tracker), it *is* a burden.
Launchpad is slow and there is no way to short-circuit the process.
I've left several bugs unreported because it was too frustrating
waiting for the hand-holding.

 > Especially this "making easy" is not the strong point of debbugs. :-(

Heh.  The Bazaar developers just made the same comment about dpkg. :-)

Re: Bug statistics

[Stephen J. Turnbull]

Richard Stallman writes:

 > However, duplicates would not be a problem if we
 > had a small number of open bug reports.  We would see
 > them and close them when we close the others that they match.

No, people frequently report duplicates of closed bugs too, and this
is more likely if the normal search doesn't report closed bugs.
Fixing bugs quickly doesn't prevent duplicates, and perhaps more
important, a regression will appear to the bug tracker as a duplicate.
You really want automatic dupe-finding capability in your bug tracker.

 > I think the problem is not enough effort to fix bugs.

Easily said, but you can't provide that effort yourself, nor can
Stefan et al, not for more than a short spurt.

What you could do is revisit decisions to impose inappropriate
infrastructure on the project, inappropriate in the sense that it's
clearly inconveniencing the "long tail" of users who are devoting
small amounts of time (from 0 to a few tens of minutes a week) to
Emacs, and therefore could easily double the amount of effort they
grant Emacs.

This is especially important for bug triage, which is boring,
unpleasant work with few intrinsic rewards at best.

Re: Bug statistics

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Cc: Tassilo Horn <tassilo@member.fsf.org>,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Fri, 25 Jun 2010 17:31:56 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >This is also far from ideal.  Unless you have a lot of time on your
> >hands, going through the bugs and trying to figure out if they are the
> >same as yours is a nuisance.  This should be a job of some program
> >that runs periodically, or, failing that, of a human (whom we
> >obviously lack).
> 
> FWIW, when I've experienced this automated dup-finding in the web
> interface of other bug trackers, it has not been a nuisance -- on the
> contrary it was a great relief, because it helped me know I'm not
> wasting the developers' time with a duplicate report.  (The majority of
> the time, it did find a dup of what I was about to file.  Sometimes I
> was able to go to that existing report and add useful information.)
> 
> For me it became one of those "never go back" features, like sexp motion
> in Emacs.

That would put you into the ``have a lot of time on your hands''
category, in my book.  I have maybe 10 hours a week to work on Emacs.
I cannot invest any significant portion of that time on reading the
descriptions of bugs, without adversely affecting my productivity,
which is too low as it is.

Re: Bug statistics

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: Dan Nicolaescu <dann@gnu.org>, kfogel@red-bean.com, emacs-devel@gnu.org
> Date: Fri, 25 Jun 2010 16:23:39 -0400
> 
> Many of the unfixed bugs are "long tail" problems that are rather
> difficult to find and/or fix.

This observation should be backed up by some numbers.  Do we know for
a fact that a significant proportion of unfixed bugs belong to this
category?

Re: Bug statistics

[Tassilo Horn]

On Saturday 26 June 2010 10:53:22 Eli Zaretskii wrote:

> > FWIW, when I've experienced this automated dup-finding in the web
> > interface of other bug trackers, it has not been a nuisance -- on
> > the contrary it was a great relief, because it helped me know I'm
> > not wasting the developers' time with a duplicate report.  (The
> > majority of the time, it did find a dup of what I was about to file.
> > Sometimes I was able to go to that existing report and add useful
> > information.)
> > 
> > For me it became one of those "never go back" features, like sexp
> > motion in Emacs.
> 
> That would put you into the ``have a lot of time on your hands''
> category, in my book.  I have maybe 10 hours a week to work on Emacs.
> I cannot invest any significant portion of that time on reading the
> descriptions of bugs, without adversely affecting my productivity,
> which is too low as it is.

Well, you (as developer) should use your limited time to fix the bugs
for which users have spent their limited time for reporting.  If users
do that carefully by avoiding dupes, linking related bug and adding as
much information as possible, then you can spend your time much more
effectively.  And because a thoroughly reported bug is much more likely
to become fixed quickly, the user has a reward, too.  Looks like a
win-win situation to me.

Bye,
Tassilo

Re: Bug statistics

[Eli Zaretskii]

> From: Tassilo Horn <tassilo@member.fsf.org>
> Date: Sat, 26 Jun 2010 11:28:49 +0200
> Cc: Karl Fogel <kfogel@red-bean.com>
> 
> > That would put you into the ``have a lot of time on your hands''
> > category, in my book.  I have maybe 10 hours a week to work on Emacs.
> > I cannot invest any significant portion of that time on reading the
> > descriptions of bugs, without adversely affecting my productivity,
> > which is too low as it is.
> 
> Well, you (as developer) should use your limited time to fix the bugs
> for which users have spent their limited time for reporting.

Are you saying that I should not report any bugs when I find them?

Re: Bug statistics

[Eli Zaretskii]

> From: Dan Nicolaescu <dann@gnu.org>
> Date: Fri, 25 Jun 2010 17:01:07 -0400
> Cc: kfogel@red-bean.com, emacs-devel@gnu.org
> 
> Speaking for myself (TM): the switch to Bazaar made it MUCH HARDER to contribute.
> It takes 5-15 minutes (sometimes even longer) to check in anything,
> even for trivial changes.
> If making / testing a change is quicker than the time to check it in,
> it makes one think twice before trying to do anything.

Would using "bzr ci --local" for several changes, followed by a single
"bzr push" help here?  Would it do any harm to the master repository?

Re: Bug statistics

[Richard Stallman]

    Many of the unfixed bugs are "long tail" problems that are rather
    difficult to find and/or fix.  Non-core contributors almost never work
    on such issues, whereas core contributors have to juggle between fixing
    these and working on new features.

I think it would be good to put more effort into fixing the bugs
for 6 months or so, rather than more new features.

Re: Bug statistics

[Richard Stallman]

     > Amost 2000 bug reports not closed
     > is rather disturbing.

    Not really.  That's 2000 (minor) contributions to be grateful for, and
    2000 reasons for somebody new to volunteer to work on Emacs!

Unless they mostly came in in the last few weeks, it means
we are lagging in fixing the bugs after other people report them.

Re: Bug statistics

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Would using "bzr ci --local" for several changes, followed by a single
 > "bzr push" help here?

Probably, but it gets messy after a while, because of the likelihood
of concurrent changes to the master.

 > Would it do any harm to the master repository?

Yes, it might.  It needs checking.  I *think* that in a bound branch
the upstream repo is considered authoritative, so that with the
following history

Local                Upstream
0 -- A*              0 -- B -- C*

pulling from upstream gives

Local                Upstream
0 -- B -- C* .. D        0 -- B -- C*
  \           .
   \         .
     A ......

with Local in a "pending merge" state that will result in D when
committed, and A on a side branch.  This is good; on pushing
Upstream's mainline has been preserved.

The potential damage is that if the Local changes are considered
authoritative, the catch-up merge from Upstream gives

Local                Upstream
0 -- A* ...... D        0 -- B -- C*
  \           .
   \         .
     B -- C .

and as you can see the old mainline is about to get shunted off on a
branch, while A usurps a mainline position.

The problem is that "bzr help pull" sez:

  If branches have diverged, you can use 'bzr merge' to integrate the
  changes from one into the other.  Once one branch has merged, the
  other should be able to pull it again.

This suggests that to make things right you need to merge *from* Local
*into* Upstream, which means that you need to be able to run bzr on
Savannah.  Urk.  However, I *think* that I've heard that commits
created with commit --local are treated differently, but I can't find
any documentation of that offhand.  "bzr help update" also suggests
that the wrong things may happen:

  This will perform a merge into the working tree, and may generate
  conflicts. If you have any local changes, you will still need to
  commit them after the update for the update to be complete.
  
  If you want to discard your local changes, you can just do a 'bzr
  revert' instead of 'bzr commit' after the update.
  
  If the tree's branch is bound to a master branch, it will also
  update the branch from the master.

However, in old Arch terminology, an update was actually a rebase of
local commits on top of the upstream branch, which is exactly what you
want here.

Bottom line: maybe commit --local does exactly what you want, but it
should be checked before recommending it.

Re: Bug statistics

[Stephen J. Turnbull]

Richard Stallman writes:

 >      > Amost 2000 bug reports not closed
 >      > is rather disturbing.
 > 
 >     Not really.  That's 2000 (minor) contributions to be grateful for, and
 >     2000 reasons for somebody new to volunteer to work on Emacs!
 > 
 > Unless they mostly came in in the last few weeks, it means
 > we are lagging in fixing the bugs after other people report them.

Sure.  Get used to it.  For a long time I thought XEmacs did a
reasonable job of responding to bug reports.  Then we got a tracker,
which for the first time provided statistical information on what was
actually happening.  Pretty distressing.  I think it is also the case
that there are a lot more "drive-by" reports[1] on the tracker so it
looks worse than it would have on the ML, even if we did have some way
to get accurate stats for the ML.  I know that several of our
developers are sloppy about closing bugs on the tracker, too.  I won't
say Emacs devs are "sloppy", but I can think of a number of reasons
why a fair number of bug fixes might not result in closing the
corresponding bug, so it's possible that Emacs suffers from a slight
bug count inflation for such reasons.

OK, "we're Emacs and we should do better than this".  Let's compare to
a number of products that I consider quite robust whose projects
provide public trackers.  Having backlogs of open bugs on the same
order of magnitude of the closed ones seems pretty consistent.  This
ratio is true of Python and Mailman.  Here are the stats for
everybody's favorite VCS which happened to be open on my desktop:

Bugs in Bazaar Version Control System

4           New bugs
1905        Open bugs
70          In-progress bugs
0           Critical bugs
194         High importance bugs
177         Incomplete bugs (can expire)

26          Bugs with patches
13          Bugs fixed elsewhere
0           Open CVE bugs - CVE reports

There appear to be 2440 bugs closed by developer action, supporting
the approximate 1:1 ratio.  This is a project which has, I believe, 5
developers paid to work on it to some extent, and another 3-5 core
contributors with jobs at the same company in related projects (eg,
working on Launchpad itself).

Now, money isn't everything, but the Bazaar developers love their
project as much as Emacs hackers love theirs, and several get paid to
work on it on top of that.  And *still* they have as many open bugs as
Emacs does!

Sure, you can and should try to reduce the number of open bugs in
Emacs, but I think you're going to end up needing to accept 4-figure
counts just like everybody else.

Footnotes: 
[1]  The report appears on the channel, but the reporter doesn't
respond to requests for more information.

Re: Bug statistics

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
>> FWIW, when I've experienced this automated dup-finding in the web
>> interface of other bug trackers, it has not been a nuisance -- on the
>> contrary it was a great relief, because it helped me know I'm not
>> wasting the developers' time with a duplicate report.  (The majority of
>> the time, it did find a dup of what I was about to file.  Sometimes I
>> was able to go to that existing report and add useful information.)
>> 
>> For me it became one of those "never go back" features, like sexp motion
>> in Emacs.
>
>That would put you into the ``have a lot of time on your hands''
>category, in my book.  I have maybe 10 hours a week to work on Emacs.
>I cannot invest any significant portion of that time on reading the
>descriptions of bugs, without adversely affecting my productivity,
>which is too low as it is.

I almost always only need to read the subject of the duplicate bug to
know it's the same as the one I'm about to report.  In the cases where I
need to dive into the description, that only adds another 20 seconds or
so.  

These amounts of time are "lost in the noise" when compared to the
amount of time it would take to actually file a new report.

-K

Re: Bug statistics

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> Many of the unfixed bugs are "long tail" problems that are rather
>> difficult to find and/or fix.
>
> This observation should be backed up by some numbers.  Do we know for
> a fact that a significant proportion of unfixed bugs belong to this
> category?

It is an anecdotal observation, but you can get a good feel of the
situation by browsing some of the open bugs with numbers less than 3000.

Re: Bug statistics

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Dan Nicolaescu <dann@gnu.org>,
>     emacs-devel@gnu.org
> Date: Sat, 26 Jun 2010 23:14:12 +0900
> 
> Bottom line: maybe commit --local does exactly what you want, but it
> should be checked before recommending it.

The discussion on the Bazaar mailing list, starting here:

  https://lists.ubuntu.com/archives/bazaar/2010q2/069117.html

seems to indicate that "ci --local" could be the solution.  Caveat: I
didn't yet try this myself.

Re: Bug statistics

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Stephen J. Turnbull" <stephen@xemacs.org>
>> Cc: Dan Nicolaescu <dann@gnu.org>,
>>     emacs-devel@gnu.org
>> Date: Sat, 26 Jun 2010 23:14:12 +0900
>> 
>> Bottom line: maybe commit --local does exactly what you want, but it
>> should be checked before recommending it.
>
> The discussion on the Bazaar mailing list, starting here:
>
>   https://lists.ubuntu.com/archives/bazaar/2010q2/069117.html
>
> seems to indicate that "ci --local" could be the solution.  Caveat: I
> didn't yet try this myself.

Or just use an unbound branch, then --local is the default.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Bug statistics

[Richard Stallman]

    Sure, you can and should try to reduce the number of open bugs in
    Emacs, but I think you're going to end up needing to accept 4-figure
    counts just like everybody else.

It is not inevitable.  The number of bugs remaining unfixed is a
matter of our priorities.  We can give priority to adding features
(which tends to make more bugs) or we can give it to fixing bugs.

Bug statistics

[Eli Zaretskii]

Does debbugs*.el support compiling statistics about our bugs?  Like
how many new bugs were reported and how many were closed during a
given period of time, including distribution by severity?

I think it might make sense to have such a feature, and perhaps even
post the statistics from time to time, so that we know how we are
doing on that front.

Does it make sense?  Would someone like to work on this?

TIA

Re: Bug statistics

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Does debbugs*.el support compiling statistics about our bugs?  Like
> how many new bugs were reported and how many were closed during a
> given period of time, including distribution by severity?

It doesn't, but there's this:

https://debbugs.gnu.org/rrd/emacs.html

I thought those charts were a bit too coarse, so I've written some code
that downloads all the debbugs data and mangles the info a bit, and then
output to Javascript so that you can zoom a bit:

http://quimby.gnus.org/circus/stats-emacs/stats-emacs.html

I haven't made my code do the separation by severity, but I've been
meaning to do that...

> I think it might make sense to have such a feature, and perhaps even
> post the statistics from time to time, so that we know how we are
> doing on that front.
>
> Does it make sense?  Would someone like to work on this?

The amount of data you need to do the computations client side are a
bit...  on the large side.  I think my script takes, like, ten minutes
to download the data, so I only run it very occasionally.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Bug statistics

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sun, 20 Oct 2019 22:15:42 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Does debbugs*.el support compiling statistics about our bugs?  Like
> > how many new bugs were reported and how many were closed during a
> > given period of time, including distribution by severity?
> 
> It doesn't, but there's this:
> 
> https://debbugs.gnu.org/rrd/emacs.html

Thanks.  Maybe this is enough.  A distribution of the time since
report to solution would be nice, though.

How and by whom are these charts produced?

> I thought those charts were a bit too coarse, so I've written some code
> that downloads all the debbugs data and mangles the info a bit, and then
> output to Javascript so that you can zoom a bit:

I actually thought about producing input for something like Gnuplot.

> The amount of data you need to do the computations client side are a
> bit...  on the large side.  I think my script takes, like, ten minutes
> to download the data, so I only run it very occasionally.

Sure, but this is hardly needed every day.  Once a month or 3 months
should be good enough, I think.

Re: Bug statistics

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> The amount of data you need to do the computations client side are a
>> bit...  on the large side.  I think my script takes, like, ten minutes
>> to download the data, so I only run it very occasionally.
>
> Sure, but this is hardly needed every day.  Once a month or 3 months
> should be good enough, I think.

Hmmm, we made already an extension to the Debbugs::SOAP interface,
offered on debbugs.gnu.org (function search_est for searching in the
bugs database). So we could add another Debbugs::SOAP function to
provide statistical data.

OTOH, the long term plan is to merge our changes to upstream
Debbugs. Further extensions would be in the way.

Best regards, Michael.

Re: Bug statistics

[Lars Ingebrigtsen]

I was inspired to tweak the scripts a bit and output more information:

https://quimby.gnus.org/circus/stats-emacs/stats-emacs.html

The potentially most interesting data set it the final chart (and it's
not on the rrd page).  It shows the percentage of bugs that are closed
within a week/month/year of reporting.  It's pretty noisy, but it shows
that by this metric, things haven't changed much over the years: Since
2011 about 50% of reported bugs have been fixed within a month.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Bug statistics

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Mon, 21 Oct 2019 13:05:13 +0200
> 
> I was inspired to tweak the scripts a bit and output more information:
> 
> https://quimby.gnus.org/circus/stats-emacs/stats-emacs.html

Thanks.

Re: Bug statistics

[Stefan Monnier]

> Since 2011 about 50% of reported bugs have been fixed within a month.

Thanks Lars.  So it's not as bad as I thought,


        Stefan

Re: Bug statistics

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> Hmmm, we made already an extension to the Debbugs::SOAP interface,
> offered on debbugs.gnu.org (function search_est for searching in the
> bugs database). So we could add another Debbugs::SOAP function to
> provide statistical data.
>
> OTOH, the long term plan is to merge our changes to upstream
> Debbugs. Further extensions would be in the way.

One thing that's a bit annoying with the data the SOAP interface returns
today is that it doesn't really have a field that says when a bug was
closed.  Instead if has a status field, and a last_modified field, and
that's what I'm using in my charts.  However, a bug may be modified
after it's closed: The main reason is when it's moved to the archive one
month later, and that's easy enough to check for (just subtract a month
when it's in the archive).

But it does mean that the charts I've made are slightly inaccurate,
which is annoying.

Is there a date field in the bugs database that says when the bug was
closed?  It seems like it, because the web interface includes that data
point.  Would it be possible to return that in the SOAP response, too?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Bug statistics

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

Hi Lars,

>> Hmmm, we made already an extension to the Debbugs::SOAP interface,
>> offered on debbugs.gnu.org (function search_est for searching in the
>> bugs database). So we could add another Debbugs::SOAP function to
>> provide statistical data.
>>
>> OTOH, the long term plan is to merge our changes to upstream
>> Debbugs. Further extensions would be in the way.
>
> One thing that's a bit annoying with the data the SOAP interface returns
> today is that it doesn't really have a field that says when a bug was
> closed.  Instead if has a status field, and a last_modified field, and
> that's what I'm using in my charts.  However, a bug may be modified
> after it's closed: The main reason is when it's moved to the archive one
> month later, and that's easy enough to check for (just subtract a month
> when it's in the archive).

There is fixed_date, which is intended for this. But it doesn't seem to
be set.

> Is there a date field in the bugs database that says when the bug was
> closed?  It seems like it, because the web interface includes that data
> point.  Would it be possible to return that in the SOAP response, too?

Could you please give me an example bug where the web interface tells
this? I didn't find any bug showing this information.

Btw, if you want to see all data fields of a bug in the Emacs debbugs
list, select a bug and type "d".

Best regards, Michael.

Re: Bug statistics

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> Could you please give me an example bug where the web interface tells
> this? I didn't find any bug showing this information.

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18691

It has log_modified set to the archive time, and the close time at "03
Aug 2019 17:20:03" is nowhere to be seen in the SOAP data.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Bug statistics

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18691
>
> It has log_modified set to the archive time, and the close time at "03
> Aug 2019 17:20:03" is nowhere to be seen in the SOAP data.

As you see in that web page, it just reports actions at "Sat, 03 Aug
2019 17:20:03 GMT". This doesn't mean, that respective data fields are
added to the bug.

In debbugs.el, both found_date and fixed_date are documented. Plus the
comment "(empty for now)". I don't remember why I have written that, but
it looks like Debbugs::SOAP does not return these field values.

Would need to dig into the Perl sources. Let's see, whether I find
something.

Additionally, I will ask upstream.

Best regards, Michael.

Re: Bug statistics

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

> In debbugs.el, both found_date and fixed_date are documented. Plus the
> comment "(empty for now)". I don't remember why I have written that, but
> it looks like Debbugs::SOAP does not return these field values.

PS: Could you pls write bug about?

Best regards, Michael.

Re: Bug statistics

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> In debbugs.el, both found_date and fixed_date are documented. Plus the
> comment "(empty for now)". I don't remember why I have written that, but
> it looks like Debbugs::SOAP does not return these field values.
>
> Would need to dig into the Perl sources. Let's see, whether I find
> something.
>
> Additionally, I will ask upstream.

Great!

But I wonder...  fixed_date is one thing, but you can close a bug
without marking it as fixed.  Is there a separate date for the "close"
action somewhere in the database?

For instance,

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18688

was closed without being marked as "fixed".

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Bug statistics

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
>> In debbugs.el, both found_date and fixed_date are documented. Plus the
>> comment "(empty for now)". I don't remember why I have written that, but
>> it looks like Debbugs::SOAP does not return these field values.
>
> PS: Could you pls write bug about?

OK; done.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no