|
Size: 6269
Comment:
|
← Revision 32 as of 2015-03-11 18:18:47 ⇥
Size: 6267
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 5: | Line 5: |
| Line 12: | Line 11: |
| == How to do triage == It's important to approach bug reports as a triage problem. Some reports are five-alarm fires, some are papercuts. We need to make sure that we don't miss the former while we're addressing the latter. And we need to be honest that we're probably not going to have time to address the latter. The most crucial step in this process is 'intake': dealing with newly-reported issues. We need to quickly and aggressively categorize these and then move them out of the intake stage. Bug reporters are likely to vanish within a couple days so we have only a narrow window to get any information we need from them. Every new bug you look at should be moved to one of these states: * security or data loss: CONFIRMED / CRITICAL / BUG * regression: CONFIRMED / URGENT / BUG * normal bug: CONFIRMED / NORMAL / BUG * feature request: CONFIRMED / FEATURE / WISH * duplicate: RESOLVED / DUPLICATE * not a bug: RESOLVED / INVALID or WONTFIX * not enough information: NEED_EXAMPLE First responders should: * try to identify regressions * cc: relevant developers * ask for essential missing information * update the state flags appropriately First responders should not: * suggest work-arounds before root causes are identified * leave the bug UNCONFIRMED |
|
| Line 16: | Line 40: |
| * '''NEED_EXAMPLE''' - we need more information to progress | * '''NEED_EXAMPLE''' - we need more information from the reporter |
| Line 27: | Line 51: |
| == Regressions, and why they're a priority == |
=== Regressions, and why they're a priority === |
| Line 33: | Line 56: |
| Note that "performance regressions" generally do not meet the above definition, and thus should not necessarily be promoted to urgent. | Note that "performance regressions" generally do not meet the above definition unless they're intolerably large, and thus should not necessarily be promoted to urgent. |
| Line 37: | Line 60: |
| == How to do triage == | == HG Bot and the BTS Reaper == There is a hook called !HgBot to update the BTS according to changelog commit messages. Summaries containing '(issueNNNN)' will automatically move issues to the testing state. References to `issueNNNN` in the body of the message will create references in that issue to the changeset. You can also explicitly write `closes issueNNN` or `fixes issueNNN` to move the bug into testing, or `see issueNNN` or `addressesNNN` to create a reference in the issue. |
| Line 39: | Line 63: |
| === Handling individual bugs === | There is also a nightly process called the BTS Reaper that automates some bug gardening: |
| Line 41: | Line 65: |
| * Get diagnostics first, give suggestions later (see [[Advice]]) * Respond to new bugs as quickly as possible, especially where more info would be useful * If it is a common issue, mark it as a duplicate * Make sure its priority is set appropriately * Make sure its title is useful (ie convert commands should mention the backend) * Add appropriate developers to CC list * Add appropriate topics * Bugs that have long been fixed in main should be marked RESOLVED/FIXED * Bugs that have been marked 'testing' can be marked RESOLVED/FIXED after two weeks if the user doesn't confirm earlier * Bugs that have recently accepted patches should be marked TESTING * Bugs with active work, including patches that aren't in main should be marked in IN_PROGRESS * Bugs that require more info from the user should be marked NEED_EXAMPLE * Bugs that remain in the NEED_EXAMPLE state for more than two weeks can be marked RESOLVED/ARCHIVED * Features and wishes that have been decided against should move to RESOLVED/WONTFIX * Issues that are unlikely to ever be addressed (eg MQ feature requests) should be moved to RESOLVED/ARCHIVED |
* close NEED_EXAMPLE reports with no response as INVALID * close TESTING reports as RESOLVED/FIXED * archive very old CONFIRMED bugs * move IN_PROGRESS bugs back to CONFIRMED * kick UNCONFIRMED reports (this should never happen) |
| Line 57: | Line 71: |
| {i} There is a hook installed on an hourly mirror of main that causes !HgBot to update the BugTracker according to changelog commit messages. Summaries containing '(issueNNNN)' will automatically move issues to the testing state. References to `issueNNNN` in the body of the message will create references in that issue to the changeset. You can also explicitly write `closes issueNNN` or `fixes issueNNN` to move the bug into testing, or `see issueNNN` or `addressesNNN` to create a reference in the issue. | == Manual bug gardening hints == In addition to the above, we should try to keep the BTS data high-quality: * adjust bug summary lines to be precise and specific * identify components correctly * actively discourage discussion of multiple topics in a single report * add keywords like EASY * try to link related bugs and mark duplicates * close bugs that are no longer relevant === Marking Duplicates === * choose whichever bug is resolved (or has a clear description) as the master * Set superseder on the duplicate to point to the master, add a note, and set the bug as resolved * If the master is not resolved, copy the nosy list from the duplicate and add a message pointing back to 'issueXXX' |
| Line 60: | Line 87: |
| Line 63: | Line 89: |
| * assign to project maintainer ``or`` add URL for proper BTS | * assign to project maintainer or add URL for proper BTS |
| Line 78: | Line 104: |
| When someone adds a patch to the BTS (which already contains a link to ContributingChanges next to the attach button!), do the following immediately so the patch doesn't sit in limbo for years: | |
| Line 79: | Line 106: |
| We don't accept patches on the BTS: * they are hard to view * they are hard to comment on * they are hard to apply * only people who are nosy on an issue see them * there is no easy way to search the BTS for unapplied patches so if a patch isn't immediately applied, it is forgotten So when someone adds a patch to the BTS (which already contains a link to ContributingChanges next to the attach button!), do the following immediately so the patch doesn't sit in limbo for years: * immediately assign the bug to submitter * mark the issue IN_PROGRESS |
* immediately assign the bug to the submitter * add the keyword "orphan" |
| Line 93: | Line 110: |
| === Marking Duplicates === * Choose whichever bug is resolved (or has a clear description) as the master * Set superseder on the duplicate to point to the master, add a note, and set the bug as resolved * If the master is not resolved, copy the nosy list from the duplicate and add a message pointing back to 'issueXXX' |
== Attacking the backlog == We will likely always have a backlog of issues that exceeds our development capacity, so it's tempting for developers to ignore the BTS and just work on features. |
| Line 98: | Line 113: |
| === Reducing Backlog === * Close out bugs that have been in '''UNCONFIRMED''', '''TESTING''', or '''NEED_EXAMPLE''' states too long * Similarly, check if bugs marked '''IN_PROGRESS''' are really in progress * Sort open issues by priority and activity - here is an ''old, invalid'' [[http://www.selenic.com/mercurial/bts/issue?@columns=title,id,activity,status,assignedto&@sort=status,activity&@group=priority&@filter=status,priority,status,priority&@pagesize=100&@startwith=0&status=-1,1,3,5&priority=1,2,3&@dispname=triage|triage]] and an ''old, invalid'' [[http://www.selenic.com/mercurial/bts/issue?@columns=title,id,activity,priority,status&@sort=activity&@filter=status&@pagesize=100&@startwith=0&status=-1,1,2,3,4,5,6,7&@dispname=unloved|unloved]] query * Degrade long-standing bugs to features and features to wishes where appropriate |
* please address URGENT bugs as soon as possible * please use the freeze window of the release cycle to lower the bug count |
| Line 105: | Line 117: |
Note:
This page is primarily intended for developers of Mercurial.
Managing Bugs
How to do bug triage on the BugTracker.
Contents
1. How to do triage
It's important to approach bug reports as a triage problem. Some reports are five-alarm fires, some are papercuts. We need to make sure that we don't miss the former while we're addressing the latter. And we need to be honest that we're probably not going to have time to address the latter.
The most crucial step in this process is 'intake': dealing with newly-reported issues. We need to quickly and aggressively categorize these and then move them out of the intake stage. Bug reporters are likely to vanish within a couple days so we have only a narrow window to get any information we need from them. Every new bug you look at should be moved to one of these states:
- security or data loss: CONFIRMED / CRITICAL / BUG
- regression: CONFIRMED / URGENT / BUG
- normal bug: CONFIRMED / NORMAL / BUG
- feature request: CONFIRMED / FEATURE / WISH
- duplicate: RESOLVED / DUPLICATE
- not a bug: RESOLVED / INVALID or WONTFIX
- not enough information: NEED_EXAMPLE
First responders should:
- try to identify regressions
- cc: relevant developers
- ask for essential missing information
- update the state flags appropriately
First responders should not:
- suggest work-arounds before root causes are identified
- leave the bug UNCONFIRMED
2. Definitions
2.1. Status levels:
UNCONFIRMED - no one's ever responded to this bug (bad)
CONFIRMED - we're discussing a solution
NEED_EXAMPLE - we need more information from the reporter
IN_PROGRESS - a partial fix exists or fix exists but isn't yet integrated
TESTING - a patch is merged, but poster has not confirmed it works, or it didn't reach the main repo yet
RESOLVED - problem is fixed or was not a bug
2.2. Priorities
critical - data loss or security issue
urgent - bug that's blocking development or is a regression
normal - bug that's not blocking development
wish - would be nice
2.3. Regressions, and why they're a priority
We pay special attention to regressions, which are defined as "a bug that breaks something that worked in earlier releases". These issues should immediately be set to urgent. We should attempt to fix these bugs before the next release.
Because regressions may break workflows or tools that users have come to depend on, they are more important than other bugs that simply impede new ways of trying to use Mercurial. Large numbers of unaddressed regressions will also make upgrading risky and unattractive for users, which is unhealthy for a project.
Note that "performance regressions" generally do not meet the above definition unless they're intolerably large, and thus should not necessarily be promoted to urgent.
![/!\](/moin-static/modernized/img/alert.png "/!\"){kind=small}
Because regressions are higher priority than other bugs, bug fixes in one area that introduce a regression elsewhere are considered worse than no action. This will prevent certain classes of bugs from being fixed.
3. HG Bot and the BTS Reaper
There is a hook called HgBot to update the BTS according to changelog commit messages. Summaries containing '(issueNNNN)' will automatically move issues to the testing state. References to issueNNNN in the body of the message will create references in that issue to the changeset. You can also explicitly write closes issueNNN or fixes issueNNN to move the bug into testing, or see issueNNN or addressesNNN to create a reference in the issue.
There is also a nightly process called the BTS Reaper that automates some bug gardening:
- close NEED_EXAMPLE reports with no response as INVALID
- close TESTING reports as RESOLVED/FIXED
- archive very old CONFIRMED bugs
- move IN_PROGRESS bugs back to CONFIRMED
- kick UNCONFIRMED reports (this should never happen)
4. Manual bug gardening hints
In addition to the above, we should try to keep the BTS data high-quality:
- adjust bug summary lines to be precise and specific
- identify components correctly
- actively discourage discussion of multiple topics in a single report
- add keywords like EASY
- try to link related bugs and mark duplicates
- close bugs that are no longer relevant
4.1. Marking Duplicates
- choose whichever bug is resolved (or has a clear description) as the master
- Set superseder on the duplicate to point to the master, add a note, and set the bug as resolved
- If the master is not resolved, copy the nosy list from the duplicate and add a message pointing back to 'issueXXX'
4.2. Dealing with third-party issues
If the third-party project is large enough to have its own BTS:
- assign to project maintainer or add URL for proper BTS
- mark the issue RESOLVED/INVALID
For smaller projects, we'll allow using our BTS to track issues if the maintainer is a known BTS user:
- assign to project maintainer
- reduce priority to bug or lower
mark RESOLVED/INVALID if bug remains inactive
For any other projects:
- link to project homepage or extension page on wiki
mark the issue RESOLVED/INVALID
4.3. Dealing with patches on the BTS
When someone adds a patch to the BTS (which already contains a link to ContributingChanges next to the attach button!), do the following immediately so the patch doesn't sit in limbo for years:
- immediately assign the bug to the submitter
- add the keyword "orphan"
add a comment that says "Read http://mercurial.selenic.com/wiki/ContributingChanges"
5. Attacking the backlog
We will likely always have a backlog of issues that exceeds our development capacity, so it's tempting for developers to ignore the BTS and just work on features.
- please address URGENT bugs as soon as possible
- please use the freeze window of the release cycle to lower the bug count