|
Size: 5079
Comment: Make important text stand out
|
← Revision 32 as of 2015-03-11 18:18:47 ⇥
Size: 6267
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 2: | Line 2: |
| <<Include(A:dev)>> |
|
| Line 3: | Line 5: |
| {{https://docs.google.com/spreadsheets/d/1lEvdGoN-jgLMZ_1eu8ceXDzEdWGWuDTzPSDVCz4jtxs/embed/oimg?id=1lEvdGoN-jgLMZ_1eu8ceXDzEdWGWuDTzPSDVCz4jtxs&oid=1929125339&zx=w96cwe5uoapn}} | |
| Line 6: | Line 9: |
| /!\ This page is intended for developers. | <<TableOfContents>> |
| Line 8: | Line 11: |
| <<TableOfContents>> | == How to do triage == It's important to approach bug reports as a triage problem. Some reports are five-alarm fires, some are papercuts. We need to make sure that we don't miss the former while we're addressing the latter. And we need to be honest that we're probably not going to have time to address the latter. The most crucial step in this process is 'intake': dealing with newly-reported issues. We need to quickly and aggressively categorize these and then move them out of the intake stage. Bug reporters are likely to vanish within a couple days so we have only a narrow window to get any information we need from them. Every new bug you look at should be moved to one of these states: * security or data loss: CONFIRMED / CRITICAL / BUG * regression: CONFIRMED / URGENT / BUG * normal bug: CONFIRMED / NORMAL / BUG * feature request: CONFIRMED / FEATURE / WISH * duplicate: RESOLVED / DUPLICATE * not a bug: RESOLVED / INVALID or WONTFIX * not enough information: NEED_EXAMPLE First responders should: * try to identify regressions * cc: relevant developers * ask for essential missing information * update the state flags appropriately First responders should not: * suggest work-arounds before root causes are identified * leave the bug UNCONFIRMED |
| Line 12: | Line 38: |
| * '''Unread''' - no one's ever responded to this bug (bad) * '''Deferred''' - we've decided not to deal with this problem now * '''Chatting''' - we're discussing a solution * '''Need-eg''' - we need more information to progress * '''In-progress''' - a partial fix exist or fix exists but isn't yet integrated * '''Testing''' - a patch is merged, but poster has not confirmed it works, or it didn't reach the main repo yet * '''Done-cbb''' - (could be better or couldn't be bothered) not perfect, but we've done as much as we're going to do * '''Resolved''' - problem is fixed or was not a bug |
* '''UNCONFIRMED''' - no one's ever responded to this bug (bad) * '''CONFIRMED''' - we're discussing a solution * '''NEED_EXAMPLE''' - we need more information from the reporter * '''IN_PROGRESS''' - a partial fix exists or fix exists but isn't yet integrated * '''TESTING''' - a patch is merged, but poster has not confirmed it works, or it didn't reach the main repo yet * '''RESOLVED''' - problem is fixed or was not a bug |
| Line 24: | Line 48: |
| * '''bug - bug''' that's not blocking development * '''feature''' - it's not a bug, it's a feature |
* '''normal''' - bug that's not blocking development |
| Line 28: | Line 51: |
| == How to do triage == | === Regressions, and why they're a priority === We pay special attention to '''regressions''', which are defined as "a bug that breaks something that worked in earlier releases". These issues should immediately be set to urgent. We should attempt to fix these bugs before the next release. |
| Line 30: | Line 54: |
| === Handling individual bugs === | Because regressions may break workflows or tools that users have come to depend on, they are more important than other bugs that simply impede new ways of trying to use Mercurial. Large numbers of unaddressed regressions will also make upgrading risky and unattractive for users, which is unhealthy for a project. |
| Line 32: | Line 56: |
| * Get diagnostics first, give suggestions later (see [[Advice]]) * Respond to new bugs as quickly as possible, especially where more info would be useful * Make sure its priority is set appropriately * Make sure its title is useful (ie convert commands should mention the backend) * Add appropriate developers to nosey * Add appropriate topics * If it is a common issue, mark it as a duplicate * Bugs that have long been fixed in main should be marked resolved * Bugs that have been marked 'testing' can be marked resolved after two weeks if the user doesn't confirm earlier * Bugs that have recently accepted patches should be marked testing * Bugs with active work, including patches that aren't in main should be marked in progress * Bugs that require more info from the user should be marked need-eg * Bugs that remain in the need-eg state for an extended period may be closed * Features and wishes that have been decided against should move to done-cbb |
Note that "performance regressions" generally do not meet the above definition unless they're intolerably large, and thus should not necessarily be promoted to urgent. |
| Line 47: | Line 58: |
| {i} There is a hook installed on an hourly mirror of main that causes HgBot to update the BugTracker according to changelog commit messages. Summaries containing '(issueNNNN)' will automatically move issues to the testing state. References to `issueNNNN` in the body of the message will create references in that issue to the changeset. You can also explicitly write `closes issueNNN` or `fixes issueNNN` to move the bug into testing, or `see issueNNN` or `addressesNNN` to create a reference in the issue. | /!\ Because regressions are higher priority than other bugs, bug fixes in one area that introduce a regression elsewhere are considered worse than no action. This will prevent certain classes of bugs from being fixed. == HG Bot and the BTS Reaper == There is a hook called !HgBot to update the BTS according to changelog commit messages. Summaries containing '(issueNNNN)' will automatically move issues to the testing state. References to `issueNNNN` in the body of the message will create references in that issue to the changeset. You can also explicitly write `closes issueNNN` or `fixes issueNNN` to move the bug into testing, or `see issueNNN` or `addressesNNN` to create a reference in the issue. There is also a nightly process called the BTS Reaper that automates some bug gardening: * close NEED_EXAMPLE reports with no response as INVALID * close TESTING reports as RESOLVED/FIXED * archive very old CONFIRMED bugs * move IN_PROGRESS bugs back to CONFIRMED * kick UNCONFIRMED reports (this should never happen) == Manual bug gardening hints == In addition to the above, we should try to keep the BTS data high-quality: * adjust bug summary lines to be precise and specific * identify components correctly * actively discourage discussion of multiple topics in a single report * add keywords like EASY * try to link related bugs and mark duplicates * close bugs that are no longer relevant === Marking Duplicates === * choose whichever bug is resolved (or has a clear description) as the master * Set superseder on the duplicate to point to the master, add a note, and set the bug as resolved * If the master is not resolved, copy the nosy list from the duplicate and add a message pointing back to 'issueXXX' |
| Line 50: | Line 87: |
| Line 53: | Line 89: |
| * assign to project maintainer ``or`` add URL for proper BTS * mark the issue done-cbb |
* assign to project maintainer or add URL for proper BTS * mark the issue RESOLVED/INVALID |
| Line 60: | Line 96: |
| * mark done-cbb if bug remains inactive | * mark '''RESOLVED/INVALID''' if bug remains inactive |
| Line 65: | Line 101: |
| * mark the issue done-cbb | * mark the issue '''RESOLVED/INVALID''' |
| Line 68: | Line 104: |
| When someone adds a patch to the BTS (which already contains a link to ContributingChanges next to the attach button!), do the following immediately so the patch doesn't sit in limbo for years: | |
| Line 69: | Line 106: |
| We don't accept patches on the BTS: * they are hard to view * they are hard to comment on * they are hard to apply * only people who are nosy on an issue see them * there is no easy way to search the BTS for unapplied patches so if a patch isn't immediately applied, it is forgotten So when someone adds a patch to the BTS (which already contains a link to ContributingChanges next to the attach button!), do the following immediately so the patch doesn't sit in limbo for years: * immediately assign the bug to them * mark the issue in-progress |
* immediately assign the bug to the submitter * add the keyword "orphan" |
| Line 83: | Line 110: |
| === Marking Duplicates === * Choose whichever bug is resolved (or has a clear description) as the master * Set superseder on the duplicate to point to the master, add a note, and set the bug as resolved * If the master is not resolved, copy the nosy list from the duplicate and add a message pointing back to 'issueXXX' |
== Attacking the backlog == We will likely always have a backlog of issues that exceeds our development capacity, so it's tempting for developers to ignore the BTS and just work on features. |
| Line 88: | Line 113: |
| === Reducing Backlog === * Close out bugs that have been in testing and need-eg state too long * Similarly, check if bugs marked in-progress are really in-progress * Sort open issues by priority and activity - here is a [[http://www.selenic.com/mercurial/bts/issue?@columns=title,id,activity,status,assignedto&@sort=status,activity&@group=priority&@filter=status,priority,status,priority&@pagesize=100&@startwith=0&status=-1,1,3,5&priority=1,2,3&@dispname=triage|triage]] and a [[http://www.selenic.com/mercurial/bts/issue?@columns=title,id,activity,priority,status&@sort=activity&@filter=status&@pagesize=100&@startwith=0&status=-1,1,2,3,4,5,6,7&@dispname=unloved|unloved]] query * Degrade long-standing bugs to features and features to wishes where appropriate |
* please address URGENT bugs as soon as possible * please use the freeze window of the release cycle to lower the bug count |
| Line 95: | Line 117: |
Note:
This page is primarily intended for developers of Mercurial.
Managing Bugs
How to do bug triage on the BugTracker.
Contents
1. How to do triage
It's important to approach bug reports as a triage problem. Some reports are five-alarm fires, some are papercuts. We need to make sure that we don't miss the former while we're addressing the latter. And we need to be honest that we're probably not going to have time to address the latter.
The most crucial step in this process is 'intake': dealing with newly-reported issues. We need to quickly and aggressively categorize these and then move them out of the intake stage. Bug reporters are likely to vanish within a couple days so we have only a narrow window to get any information we need from them. Every new bug you look at should be moved to one of these states:
- security or data loss: CONFIRMED / CRITICAL / BUG
- regression: CONFIRMED / URGENT / BUG
- normal bug: CONFIRMED / NORMAL / BUG
- feature request: CONFIRMED / FEATURE / WISH
- duplicate: RESOLVED / DUPLICATE
- not a bug: RESOLVED / INVALID or WONTFIX
- not enough information: NEED_EXAMPLE
First responders should:
- try to identify regressions
- cc: relevant developers
- ask for essential missing information
- update the state flags appropriately
First responders should not:
- suggest work-arounds before root causes are identified
- leave the bug UNCONFIRMED
2. Definitions
2.1. Status levels:
UNCONFIRMED - no one's ever responded to this bug (bad)
CONFIRMED - we're discussing a solution
NEED_EXAMPLE - we need more information from the reporter
IN_PROGRESS - a partial fix exists or fix exists but isn't yet integrated
TESTING - a patch is merged, but poster has not confirmed it works, or it didn't reach the main repo yet
RESOLVED - problem is fixed or was not a bug
2.2. Priorities
critical - data loss or security issue
urgent - bug that's blocking development or is a regression
normal - bug that's not blocking development
wish - would be nice
2.3. Regressions, and why they're a priority
We pay special attention to regressions, which are defined as "a bug that breaks something that worked in earlier releases". These issues should immediately be set to urgent. We should attempt to fix these bugs before the next release.
Because regressions may break workflows or tools that users have come to depend on, they are more important than other bugs that simply impede new ways of trying to use Mercurial. Large numbers of unaddressed regressions will also make upgrading risky and unattractive for users, which is unhealthy for a project.
Note that "performance regressions" generally do not meet the above definition unless they're intolerably large, and thus should not necessarily be promoted to urgent.
![/!\](/moin-static/modernized/img/alert.png "/!\"){kind=small}
Because regressions are higher priority than other bugs, bug fixes in one area that introduce a regression elsewhere are considered worse than no action. This will prevent certain classes of bugs from being fixed.
3. HG Bot and the BTS Reaper
There is a hook called HgBot to update the BTS according to changelog commit messages. Summaries containing '(issueNNNN)' will automatically move issues to the testing state. References to issueNNNN in the body of the message will create references in that issue to the changeset. You can also explicitly write closes issueNNN or fixes issueNNN to move the bug into testing, or see issueNNN or addressesNNN to create a reference in the issue.
There is also a nightly process called the BTS Reaper that automates some bug gardening:
- close NEED_EXAMPLE reports with no response as INVALID
- close TESTING reports as RESOLVED/FIXED
- archive very old CONFIRMED bugs
- move IN_PROGRESS bugs back to CONFIRMED
- kick UNCONFIRMED reports (this should never happen)
4. Manual bug gardening hints
In addition to the above, we should try to keep the BTS data high-quality:
- adjust bug summary lines to be precise and specific
- identify components correctly
- actively discourage discussion of multiple topics in a single report
- add keywords like EASY
- try to link related bugs and mark duplicates
- close bugs that are no longer relevant
4.1. Marking Duplicates
- choose whichever bug is resolved (or has a clear description) as the master
- Set superseder on the duplicate to point to the master, add a note, and set the bug as resolved
- If the master is not resolved, copy the nosy list from the duplicate and add a message pointing back to 'issueXXX'
4.2. Dealing with third-party issues
If the third-party project is large enough to have its own BTS:
- assign to project maintainer or add URL for proper BTS
- mark the issue RESOLVED/INVALID
For smaller projects, we'll allow using our BTS to track issues if the maintainer is a known BTS user:
- assign to project maintainer
- reduce priority to bug or lower
mark RESOLVED/INVALID if bug remains inactive
For any other projects:
- link to project homepage or extension page on wiki
mark the issue RESOLVED/INVALID
4.3. Dealing with patches on the BTS
When someone adds a patch to the BTS (which already contains a link to ContributingChanges next to the attach button!), do the following immediately so the patch doesn't sit in limbo for years:
- immediately assign the bug to the submitter
- add the keyword "orphan"
add a comment that says "Read http://mercurial.selenic.com/wiki/ContributingChanges"
5. Attacking the backlog
We will likely always have a backlog of issues that exceeds our development capacity, so it's tempting for developers to ignore the BTS and just work on features.
- please address URGENT bugs as soon as possible
- please use the freeze window of the release cycle to lower the bug count