Difference between revisions of "Bug Lists"

From Audacity Wiki
Jump to: navigation, search
(addded links to Manual P1, P2 bugs)
(attempt at writing up new bug definitions and quality standards)
Line 17: Line 17:
 
}}  
 
}}  
  
__NOTOC__
+
 
 +
__TOC__
 +
 
  
 
== Latest Changes for:==
 
== Latest Changes for:==
Line 29: Line 31:
 
* [http://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cassigned_to%2Cbug_status%2Cresolution%2Cshort_desc%2Cchangeddate&keywords=patch_ready&keywords_type=allwords&query_format=advanced&order=changeddate%20DESC%2Cbug_id&query_based_on= All open issues that have Patches attached, that are agreed on and ready for developer review]
 
* [http://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cassigned_to%2Cbug_status%2Cresolution%2Cshort_desc%2Cchangeddate&keywords=patch_ready&keywords_type=allwords&query_format=advanced&order=changeddate%20DESC%2Cbug_id&query_based_on= All open issues that have Patches attached, that are agreed on and ready for developer review]
  
* Manual bugs - '''Note:''' links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are NOT bugs.  
+
* Manual bugs (issues with the written documentation, not infrastructure or coding issues about serving it) - '''Note:''' links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are NOT bugs.  
 
** [http://manual.audacityteam.org/m/index.php?title=Special:WhatLinksHere/ToDo-1 Manual P1 bugs]  
 
** [http://manual.audacityteam.org/m/index.php?title=Special:WhatLinksHere/ToDo-1 Manual P1 bugs]  
 
** [http://manual.audacityteam.org/m/index.php?title=Special:WhatLinksHere/ToDo-2 Manual P2 bugs]
 
** [http://manual.audacityteam.org/m/index.php?title=Special:WhatLinksHere/ToDo-2 Manual P2 bugs]
Line 35: Line 37:
  
 
==Bugs by Priority==
 
==Bugs by Priority==
 +
  
 
{| width="90%" cellpadding="5px"
 
{| width="90%" cellpadding="5px"
Line 46: Line 49:
 
| width="20%" bgcolor="#F778A1"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P1 only]
 
| width="20%" bgcolor="#F778A1"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P1 only]
 
| width="5%"| 
 
| width="5%"| 
| width="50%"| '''P1''' bugs prevent any release, including beta.
+
| width="50%"| '''P1''' bugs prevent a new release, including most reproducible crashes and regressions.  
 
|-
 
|-
 
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P2]
 
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P2]
Line 52: Line 55:
 
|bgcolor="#EDE275"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P2 only]
 
|bgcolor="#EDE275"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P2 only]
 
|
 
|
| '''P2''' bugs prevent a stable release.
+
| '''P2''' bugs still open when the [[Release Process|release process]] starts can be fixed or not at the discretion of the Release Manager. Unfixed bugs must be release noted.  
 
|-
 
|-
 
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P3]
 
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P3]
Line 58: Line 61:
 
|bgcolor="#B5EAAA"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P3 only]  
 
|bgcolor="#B5EAAA"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P3 only]  
 
|
 
|
| '''P3''' bugs must be release noted.
+
| '''P3''' bugs are lower severity bugs which must still be release noted.
 
|-
 
|-
 
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P4]
 
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P4]
Line 64: Line 67:
 
|bgcolor="#CFECEC"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P4 only]  
 
|bgcolor="#CFECEC"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P4 only]  
 
|
 
|
| '''P4 and P5''' bugs are not release noted.
+
| '''P4 and P5''' bugs are not release noted. Many of these may be "enhancement issues".
 
|-
 
|-
 
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE All Open Bugs]
 
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE All Open Bugs]
Line 74: Line 77:
  
  
Deciding on bug priority is not an exact science, and some of the natural 'rules' conflict with each other.   
+
== Assessing bug priority to maintain Quality Requirement ==
* Reproducible crashes and regressions are generally '''P1'''.
+
 
* However, reproducible bugs which can't be progressed (e.g. because they are incompatibilities in a driver or operating system beyond our control) will usually be '''P3''' or lower, so they don't needlessly prevent stable releases.  
+
From the start of the [[Release Notes|2.0 series]], Audacity removed the former distinction between "stable" and "beta" releases in favour of single series of high-quality releases suitable for all users.
* Similarly where a bug lacks a ready way to reproduce it there may be pressures to demote it to '''P3''', even if it is a frequently reported crash or regression. One solution to handling a bug demoted for this reason may be to flag or keyword it, or otherwise encourage testers to discover how to reproduce it.  
+
 
 +
Accordingly, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" was discarded. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for single releases is as follows.   
 +
 
 +
* '''Never less good overall than the standard set by 2.0.0''' - only trivial regressions are likely to be tolerated in a release.  
 +
* '''Not necessarily laden with new or improved features''' - in a few releases the main focus may be on housekeeping and bug-fixing, while in most there will be a balance between features and fixes. Major new features will be isolated in plug-ins, and not included in a release until any major shortcomings have been ironed out.    
 +
* '''Not necessarily "fully stable"''' - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series, so it is possible that some crash bugs could be demoted from P1 release blocker status if their impact is minor, or if they only occur in rare or non-reproducible scenarios. Driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted.  
 +
  
Also a '''P2''' bug in a new feature we can turn off will generally be a reason to turn that feature off in stable releases so that we can get a stable release out.
+
{{hint|'''Bug fixing at release time:''' The Release Manager can decide to fix or not fix any bug for a release. Almost any P1 bug will be fixed, unless currently available fixes are deemed a stability risk, so requiring demotion to P2. Most P2 bugs open at release time will be accepted in a release, though the Release Manager may decide that some P2's or even P3's related to other improvements in a release should be fixed.}}
  
  

Revision as of 08:35, 25 March 2012

The Audacity Team track bugs and agreed enhancement issues on our Bugzilla. This page is a list of useful Bugzilla links for Developer/Quality Assurance use.
If you are a member of the public reporting a bug, please read Reporting Bugs and then send an e-mail to our feedback address. Suggestions for new or improved features are also welcome at that address.


Quick Links:


Tip: For easy access, bookmark any links in these bug lists!


Tip: If you go to our bugzilla, clicking on the Audacity Icon at top left takes you to a summary table of open bugs. Clicking on the numbers in that table takes you to those bugs.



Latest Changes for:

  • Manual bugs (issues with the written documentation, not infrastructure or coding issues about serving it) - Note: links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are NOT bugs.


Bugs by Priority

Open Bugs   Awaiting Developer Fix
PX   P1 only   P1 bugs prevent a new release, including most reproducible crashes and regressions.
P1 to P2   P2 only P2 bugs still open when the release process starts can be fixed or not at the discretion of the Release Manager. Unfixed bugs must be release noted.
P1 to P3   P3 only P3 bugs are lower severity bugs which must still be release noted.
P1 to P4   P4 only P4 and P5 bugs are not release noted. Many of these may be "enhancement issues".
All Open Bugs   P5 only


Assessing bug priority to maintain Quality Requirement

From the start of the 2.0 series, Audacity removed the former distinction between "stable" and "beta" releases in favour of single series of high-quality releases suitable for all users.

Accordingly, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" was discarded. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for single releases is as follows.

  • Never less good overall than the standard set by 2.0.0 - only trivial regressions are likely to be tolerated in a release.
  • Not necessarily laden with new or improved features - in a few releases the main focus may be on housekeeping and bug-fixing, while in most there will be a balance between features and fixes. Major new features will be isolated in plug-ins, and not included in a release until any major shortcomings have been ironed out.
  • Not necessarily "fully stable" - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series, so it is possible that some crash bugs could be demoted from P1 release blocker status if their impact is minor, or if they only occur in rare or non-reproducible scenarios. Driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted.


Bug fixing at release time: The Release Manager can decide to fix or not fix any bug for a release. Almost any P1 bug will be fixed, unless currently available fixes are deemed a stability risk, so requiring demotion to P2. Most P2 bugs open at release time will be accepted in a release, though the Release Manager may decide that some P2's or even P3's related to other improvements in a release should be fixed.


Workflow when resolving bugs

  • When a bug has a fix committed by a developer, the developer should move the bug status to DEVEL - FIX MADE. This means the bug is still OPEN. Then QA should test it, ideally on all three platforms. If QA are happy with the result, they move the bug status to RESOLVED - FIXED. That means the bug is closed. It will not be visible in searches except when searching for Fixed.
  • It is suggested the only case where a developer should change bug status to RESOLVED - FIXED should be for bugs P3 and below where a logic error was to blame which does not appear to have possible platform or machine dependencies, AND where this fix has been committed.


If you plan to use our bugtracker you may want to read about the advantages and hazards of bugtracking, along with a bit more terminology.


Some Categories


Reports

Not strictly bug lists...

One Level

Two Levels

  • Bug Type and Priority. This is showing open bugs. It is possibly the best table for getting an overview of the kinds of live bugs we have.
  • Status and Priority. This is showing both open and closed bugs. It shows, for example, that we tend to close a higher proportion of high priority bugs than lower priority bugs.
  • Status and Bug Type. This is showing open and closed bugs. It shows, for example, that summary issues tend to stay open.

Three Levels