Re: Basic questions about the triage process

[Andrew Hyatt <ahyatt@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4713 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrew Hyatt <ahyatt@gmail.com>
>> Date: Tue, 29 Dec 2015 00:40:08 -0500
>> 
>> I've put together my notes into a file I stuck in the admin section.
>
> Thanks.  A few comments below.
>
>> +* The what and why of bug triage
>> +
>> +Bugs have to be regularly looked at and acted upon.  Not all bugs are
>> +critical, but at the least, each bug needs to be regularly re-reviewed
>> +to make sure it is still reproducible.  A large backlog of bugs is
>> +disheartening to the developers, and a culture of ignoring bugs is
>> +harmful to users, who expect software that works.
>
> This paragraph is probably better to move to CONTRIBUTE.  It should
> point to this file, which in turn should describe only the triage
> itself, not its importance.

OK, I've done so (see the modified patch I've attached).  This probably
needs a bit more work since we want to talk about more kinds of triage
as well (notably the triaging of new bugs), eventually.

>
>> +The goal of this triage is to prune down the list of old bugs, closing
>> +the ones that are not reproducible on the current release.
>
> I think triage is more than that: it should also strive to classify
> the bugs according to their importance.

Yes, but isn't that more about triaging new bugs?  I'm not writing about
that yet - but if someone tells me how to triage new bugs I'm happy to
write it up.

>
>> +  1. To start, enter debbugs mode (either debbugs-gnu or debbugs-org), and
>> +     accept the default list option of bugs that have severity serious,
>> +     important, or normal.  
>> +  2. This will also show closed bugs that have yet to be archived.  You can
>> +     filter these out in debbugs-gnu with "x" (debbugs-gnu-toggle-suppress).
>
> Triage can be done via a Web browser as well.  I suggest to mention
> debbugs-gnu as one possibility, perhaps the preferred one, but not the
> only one.

Done.

>
>> +  3. For each bug, do the following:
>> +     - Read the mail thread for the bug. Find out if anyone has been able to
>> +       reproduce this on the current release.
>> +     - If someone has been able to, then your work is finished for this bug.
>
> Again, having the bug reproducible is not the end of triage, at least
> not in general.  It is a good idea to use your judgment to decide
> whether the bug is really a bug (and if so, how important it is), a
> request for a new feature, or simply a rant.  debbugs.gnu.org supports
> tags for recording the results of this process; it would be good if at
> least some bugs got tagged accordingly as result of the triage.

Again, I think this right, but for new bug triage which I haven't
written up yet.  For the bug backlog, I'm assuming someone has already
taken a pass at each bug.

>
>> +     - If you can reproduce, then reply on the thread (either on the original
>> +       message, or anywhere you find appropriate) that you can reproduce this on
>> +       the current release.
>
> Here, I'd suggest to request adding relevant details.  Sometimes bug
> reports don't provide backtraces, or don't even describe the recipe in
> sufficient detail.  If the triage supplies these details, let alone if
> you can come up with a simpler reproducer, adding this information
> will be of great value to those who will come after you to try
> resolving the bug.
>
> Also, if the description isn't detailed enough, it might be a good
> idea to ask for more detailed description, because the stuff that was
> left out might be the reason for not being able to reproduce the bug
> in the first place.

Done (although I split this up into two parts - one new bullet point,
another as part of the text on what to do if you can reproduce the issue).

>
>> +     - If you can't reproduce, but the bug is newer than 2 years old, state that
>> +       you can't reproduce it on the current release, ask if they can try again
>> +       against the current release.
>
> There's a tag for that, I believe.

We can just use the "unreproducible" tag.  Sounds like a good idea.
I'll add that.

>
>> +  4. Your changes will take some time to take effect. After a period of minutes
>> +     to hours, you will get a mail telling you the control message has been
>> +     processed. At this point, you and everyone else can see your changes.
>
> That mail can also say there were errors, something to mention here, I
> think.

Mentioned, but I'm a bit at a loss on what to say if there were errors.

>
> Thanks again for working on this (and on the triage itself).

No problem.  I've also removed, by popular demand, the distinction
between pre-and-post 2 years old.  Now we'll always just ask first.
I've also fixed a few spelling and grammar errors.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Version 2 of triage notes patch --]
[-- Type: text/x-patch, Size: 3928 bytes --]

From 3e496267725a6a44e32439fe8934ebda5a33f150 Mon Sep 17 00:00:00 2001
From: Andrew Hyatt <ahyatt@gmail.com>
Date: Tue, 29 Dec 2015 00:36:09 -0500
Subject: [PATCH] Add a file detailing how to triage bugs.

Further changes in reponse to Eli's mail.
---
 CONTRIBUTE         |  9 +++++++++
 admin/notes/triage | 44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
 create mode 100644 admin/notes/triage

diff --git a/CONTRIBUTE b/CONTRIBUTE
index b385d68..6983df5 100644
--- a/CONTRIBUTE
+++ b/CONTRIBUTE
@@ -222,6 +222,15 @@ the tracker with the corresponding bugs/issues.
 GNU ELPA has a 'debbugs' package that allows accessing the tracker
 database from Emacs.
 
+A large backlog of bugs is disheartening to the developers, and a
+culture of ignoring bugs is harmful to users, who expect software that
+works.  Bugs have to be regularly looked at and acted upon.  Not all
+bugs are critical, but at the least, each bug needs to be regularly
+re-reviewed to make sure it is still reproducible.
+
+The process of going through open bugs and acting on them is called
+bug triage. This process is described in the file admin/notes/triage.  
+
 ** Document your changes.
 
 Any change that matters to end-users should have an entry in etc/NEWS.
diff --git a/admin/notes/triage b/admin/notes/triage
new file mode 100644
index 0000000..a2518ca
--- /dev/null
+++ b/admin/notes/triage
@@ -0,0 +1,44 @@
+HOW TO TRIAGE EMACS BUGS  -*- outline -*-
+
+This document just describes the procedure of triaging bugs, for
+information on how to work with the bug tracker, see the bugtracker
+file in this same directory.
+
+* Bug backlog triage procedure
+
+The goal of this triage is to prune down the list of old bugs, closing
+the ones that are not reproducible on the current release.
+
+  1. To start, enter debbugs mode (either debbugs-gnu, debbugs-org, or via the
+     web browser), and accept the default list option of bugs that have severity
+     serious, important, or normal.
+  2. This will also show closed bugs that have yet to be archived.  You can
+     filter these out in debbugs-gnu with "x" (debbugs-gnu-toggle-suppress).
+  3. For each bug, do the following:
+     - Read the mail thread for the bug. Find out if anyone has been able to
+       reproduce this on the current release.
+     - If someone has been able to, then your work is finished for this bug.
+     - Make sure there's enough information to reproduce the bug. It should be
+       very clear how to reproduce. If not, please ask for specific steps to
+       reproduce. If you don't get them, and you can't reproduce without them,
+       you can close as "doneunreproducible".
+     - If no one has mentioned being able to reproduce on the current release,
+       read the bug description and attempt to reproduce on an emacs started
+       with "emacs -Q" (the goal is to not let our personal configs interfere
+       with bug testing).
+     - If you can reproduce, then reply on the thread (either on the original
+       message, or anywhere you find appropriate) that you can reproduce this on
+       the current release. If your reproduction gives additional info (such as
+       a backtrace), then add that as well, since it will help whoever attempts
+       to fix it.
+     - If you can't reproduce, state that you can't reproduce it on the current
+       release, ask if they can try again against the current release. Tag the
+       bug as "unreproducable". Wait a few weeks for their reply - if they can
+       reproduce it, then that's great, otherwise close as "doneunreproducible".
+  4. Your changes will take some time to take effect. After a period of minutes
+     to hours, you will get a mail telling you the control message has been
+     processed. At this point, if there were no errors detected, you and
+     everyone else can see your changes.
+       
+
+  
-- 
2.4.9 (Apple Git-60)