Difference between revisions of "Bug Lists"

From Audacity Wiki
Jump to: navigation, search
(attempt at writing up new bug definitions and quality standards)
m (+deprecated)
 
(37 intermediate revisions by 4 users not shown)
Line 1: Line 1:
{{Intro|1=The Audacity Team track bugs and agreed enhancement issues on our [http://bugzilla.audacityteam.org Bugzilla]. This page is a list of useful Bugzilla links for Developer/Quality Assurance use.|2=If you are a member of the public reporting a bug, please read [[Reporting Bugs]] and then send an e-mail to our [http://audacity.sourceforge.net/contact/#feedback feedback address]. Suggestions for new or improved features are also welcome at that address.}}
+
{{deprecated|https://github.com/audacity/audacity/issues}}
 +
{{Intro|1=The Audacity Team track bugs and agreed enhancement issues on our [https://bugzilla.audacityteam.org Bugzilla]. This page is a list of useful Bugzilla links for Developer/Quality Assurance use.|2=If you are a member of the public reporting a bug, please read [[Reporting Bugs]] and then send an email to our [https://web.audacityteam.org/contact/#feedback feedback address]. Suggestions for new or improved features are also welcome at that address.}}
  
  
'''Quick Links:'''  
+
'''Top Tips:'''  
*[http://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Table of issues awaiting fix or review] by Priority and Bug Type (also available from the Audacity icon on Bugzilla)
 
* [[Nightly Builds|Latest alpha development builds of Audacity]]
 
* [mailto:[email protected]?subject=Bugzilla_account Ask for a Bugzilla account] (for experienced C++ developers or seasoned bug watchers who want to actively fix or test bugs)
 
* [[SubmittingPatches|How to submit patches]] to our feedback address or developers' mailing list
 
* [http://lists.sourceforge.net/lists/listinfo/audacity-devel audacity-devel] developers' mailing list information 
 
  
{{Hint|
+
{{tip|If you go to [https://bugzilla.audacityteam.org/ our bugzilla], clicking on the Audacity Icon at top left takes you to a summary table of open bugs.  Clicking on the numbers in that table takes you to those bugs.
'''Tip:''' For easy access, bookmark any links in these bug lists!
 
 
}}  
 
}}  
 +
{{tip|Audacity has some conventions and thinking of its own on [[Using Bugzilla]].}}
  
{{Hint|
+
'''See also:'''
'''Tip:''' If you go to [http://bugzilla.audacityteam.org/ our bugzilla], clicking on the Audacity Icon at top left takes you to a summary table of open bugs.  Clicking on the numbers in that table takes you to those bugs.
 
}}
 
  
 +
* [[Bug Counts]] - For historical record of bug levels.
  
 
__TOC__
 
__TOC__
  
 +
{{ednote|1=This does not work yet.
 +
==Bugzilla And The Bug Table==
  
== Latest Changes for:==
+
{{#widget:BugTable}}
 +
}}
  
* [http://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cbug_status%2Cshort_desc%2Cchangeddate&product=Audacity&query_format=advanced&order=changeddate%20DESC%2Cassigned_to%2Cbug_id&query_based_on= Open Audacity issues (excluding "Patches awaiting triage" product)] 
+
==Documentation Bugs/To-Dos for the Manual==
  
* [http://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cbug_status%2Cshort_desc%2Cchangeddate&product=Audacity&product=Audacity%20Patches&query_format=advanced&order=changeddate%20DESC%2Cassigned_to%2Cbug_id&query_based_on= All open issues (including "Patches awaiting triage" product)]
+
Sometimes mistakes in the manual are as serious as mistakes in Audacity code, and such mistakes block release of Audacity too, until they are sorted.
  
* [http://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cassigned_to%2Cbug_status%2Cresolution%2Cshort_desc%2Cchangeddate&keywords=patch&keywords_type=allwords&query_format=advanced&order=changeddate%20DESC%2Cbug_id&query_based_on= All open issues that have Patches attached]
+
''(The links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are not bugs.)''
 
+
* [https://alphamanual.audacityteam.org/index.php?title=Special:WhatLinksHere/ToDo-1 Manual P1 bugs]  
* [http://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cassigned_to%2Cbug_status%2Cresolution%2Cshort_desc%2Cchangeddate&keywords=patch_ready&keywords_type=allwords&query_format=advanced&order=changeddate%20DESC%2Cbug_id&query_based_on= All open issues that have Patches attached, that are agreed on and ready for developer review]
+
* [https://alphamanual.audacityteam.org/index.php?title=Special:WhatLinksHere/ToDo-2 Manual P2 bugs]
 
 
* Manual bugs (issues with the written documentation, not infrastructure or coding issues about serving it) - '''Note:''' links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are NOT bugs.  
 
** [http://manual.audacityteam.org/m/index.php?title=Special:WhatLinksHere/ToDo-1 Manual P1 bugs]  
 
** [http://manual.audacityteam.org/m/index.php?title=Special:WhatLinksHere/ToDo-2 Manual P2 bugs]
 
  
  
Line 45: Line 39:
 
|valign="top"|'''Awaiting Developer Fix'''
 
|valign="top"|'''Awaiting Developer Fix'''
 
|-
 
|-
| width="20%" bgcolor="#AFC7C7"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=PX&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE PX]
+
| width="20%" bgcolor="#AFC7C7"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=PX&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE PX]
 
| width="5%"| 
 
| width="5%"| 
| width="20%" bgcolor="#F778A1"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P1 only]
+
| width="20%" bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P0&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P0/P1]
 
| width="5%"| 
 
| width="5%"| 
| width="50%"| '''P1''' bugs prevent a new release, including most reproducible crashes and regressions.  
+
| width="50%"| '''P0''' bugs are bugs so bad they seriously impede development.  Preventing recording working, or disabling mouse drags or more prosaically a build breaker would be P0.  P0's affect the development team and impede testing by the QA team.  You'll almost never see them logged in bugzilla, because we fix them so quickly.
 
|-
 
|-
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P2]
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P0&priority=P1&priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P0 to P2]
 
| 
 
| 
|bgcolor="#EDE275"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P2 only]
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P2 only]
 
|
 
|
| '''P2''' bugs still open when the [[Release Process|release process]] starts can be fixed or not at the discretion of the Release Manager. Unfixed bugs must be release noted.  
+
| '''P1''' bugs prevent a new release. These will include most reproducible crashes, most things that cause data loss and most regressions.  P1's would affect large numbers of users, if released. QA is saying these bugs are not acceptable.  RM could in principle over-rule that and release without a proper tested fix, but almost never will, because then RM is taking sole responsibility for the consequences.
 
|-
 
|-
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P3]
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P3]
 
| 
 
| 
|bgcolor="#B5EAAA"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P3 only]  
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P3 only]  
 
|
 
|
| '''P3''' bugs are lower severity bugs which must still be release noted.
+
| '''P2''' bugs still open when the [[Release Process|release process]] starts, can be fixed or not at the discretion of the Release Manager. QA is saying that these bugs are serious, but they are OK with them going out into the  release.  Responsibility for the consequences is shared by RM and QA.  Unfixed P2 bugs must be release noted.  
 
|-
 
|-
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P4]
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P4]
 
| 
 
| 
|bgcolor="#CFECEC"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P4 only]  
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P4 only]  
 
|
 
|
| '''P4 and P5''' bugs are not release noted. Many of these may be "enhancement issues".
+
| '''P3''' bugs are lower severity bugs which must still be release noted.
 
|-
 
|-
|bgcolor="#ECE5B6"|[http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE All Open Bugs]
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE All Open Bugs]
 
| 
 
| 
|bgcolor="#CFECEC"|[http://bugzilla.audacityteam.org/buglist.cgi?priority=P5&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P5 only]  
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P5&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P5 only]  
|
 
 
|
 
|
 +
| '''P4 and P5''' bugs are not release noted. Many of these may be "enhancement issues".
 
|}
 
|}
  
  
== Assessing bug priority to maintain Quality Requirement ==  
+
== Our Quality Goals ==  
  
From the start of the [[Release Notes|2.0 series]], Audacity removed the former distinction between "stable" and "beta" releases in favour of single series of high-quality releases suitable for all users.  
+
We used to have "stable" releases and "beta" releases.  However, there is a social problem with that!  Developers like writing new features.  It isn't as much fun fixing bugs, especially other people's bugs.  So as developers made the releases, there was a huge temptation to make beta releases.  And then, after a bit, we get stuck in only making betas, because it is too much work to sort out the accumulated bugs. From the start of the [[Release Notes|2.0 series]], Audacity removed the former distinction between "stable" and "beta" releases in favor of single series of high-quality releases suitable for all users.  Bugs serious enough to block a stable release have to be fixed at the time, or the feature disabled.
  
Accordingly, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" was discarded. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for single releases is as follows.   
+
With that change, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" changed too. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for our releases is now as follows.   
  
* '''Never less good overall than the standard set by 2.0.0''' - only trivial regressions are likely to be tolerated in a release.  
+
* '''Never less good overall than the standard set by 2.2.0''' - only trivial regressions are likely to be tolerated in a release.  
* '''Not necessarily laden with new or improved features''' - in a few releases the main focus may be on housekeeping and bug-fixing, while in most there will be a balance between features and fixes. Major new features will be isolated in plug-ins, and not included in a release until any major shortcomings have been ironed out.   
+
* '''Not necessarily laden with new or improved features''' - in a few releases the main focus may be on housekeeping and bug-fixing.
* '''Not necessarily "fully stable"''' - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series, so it is possible that some crash bugs could be demoted from P1 release blocker status if their impact is minor, or if they only occur in rare or non-reproducible scenarios. Driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted.  
+
* '''Not necessarily "fully stable"''' - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series.  So in turn, driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted.  Releasing reasonably frequently is important.
 
   
 
   
 
+
We also track our [[Bug Counts|bug counts]] month by month and set yearly targets for reducing them.
{{hint|'''Bug fixing at release time:''' The Release Manager can decide to fix or not fix any bug for a release. Almost any P1 bug will be fixed, unless currently available fixes are deemed a stability risk, so requiring demotion to P2. Most P2 bugs open at release time will be accepted in a release, though the Release Manager may decide that some P2's or even P3's related to other improvements in a release should be fixed.}}
 
  
  
 
== Workflow when resolving bugs ==  
 
== Workflow when resolving bugs ==  
* When a bug has a fix committed by a developer, the developer should move the bug status to '''DEVEL - FIX MADE'''. This means the bug is still '''OPEN'''. Then QA should test it, ideally on all three platforms. If QA are happy with the result, they move the bug status to '''RESOLVED - FIXED'''. That means the bug is closed. It will not be visible in searches except when searching for        [http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=RESOLVED&resolution=FIXED Fixed].
 
  
* It is suggested the only case where a developer should change bug status to '''RESOLVED - FIXED''' should be for bugs '''P3 and below''' where a logic error was to blame which does not appear to have possible platform or machine dependencies, '''AND''' where this fix has been committed.  
+
* When a bug has a fix committed by a developer, the developer moves the bug status to '''DEVEL - FIX MADE'''. This means the bug is still '''OPEN'''. QA then test it, and they ALSO test things they think the fix may have broken. The developer may sometimes leave a bugzilla flag to indicate whether the bug fix requires testing on all three platforms.  When QA are happy with the result of testing, they move the bug status to '''RESOLVED - FIXED'''. That means the bug is closed. It will not be visible in searches except when searching for [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=RESOLVED&resolution=FIXED Fixed].
 +
* We aim to make sure that the same person doesn't both '''DEVEL - FIX MADE''' a bug and ''' RESOLVED - FIXED''' a bug.  It adds a bit more safety.
  
 
{{Hint|
 
{{Hint|
'''If you plan to use our bugtracker''' you may want to read about the [[Using_Bugzilla|advantages and hazards]] of bugtracking, along with a bit more terminology.   
+
'''If you plan to use our bugtracker''' you may want to read about the [[Using_Bugzilla|advantages and social hazards]] of bugtrackers, along with a bit more terminology.   
 
}}
 
}}
 +
 +
===Residuals===
 +
 +
Tracking "Residual" bugs after a fix can be tricky.
 +
 +
# If the residual is due to an '''incomplete fix of the code that causes the described bug''', then it should initially be treated as the same bug.
 +
# If the '''description''' of the residual '''differs markedly''' from the original bug description, then it should be treated as a new bug.
 +
# If the residual is due to an error in '''code that is unrelated to the original bug''', then it should be treated as a new bug, even if the symptoms have similarities.
 +
 +
===Chatty Bug Histories===
 +
 +
It is often NOT clear what to do about a bug.  We do strive not to clutter up the bug histories with chatter.  We try to use it for evidence, information, progress made.  Often we will discuss bugs, or groups of bugs on our quality list.  One trick to reduce the impact of long bug histories is this line:
 +
 +
*** STEPS UPDATED ***
 +
 +
This means that the steps to reproduce the bug have been updated, usually due to a partial fix that leaves some residual.  Sometimes though we just got a clearer understanding of the bug.  Generally when visiting a bug you can work back through the comments, and stop when you reached *** STEPS UPDATED ***.
  
  
 
==Some Categories==
 
==Some Categories==
* [http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=RESOLVED&resolution=FIXED Fixed ]
+
 
* [http://bugzilla.audacityteam.org/buglist.cgi?bug_status=DEVEL%20-%20FIX%20MADE&query_format=advanced&order=bug_id&query_based_on= Fix made by developer, not yet resolved by QA]
+
These are some links for some categories of bugs...
* [http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_severity=MoonPhase&bug_severity=HeisenBug&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED Moonphase or Heisenbug]
+
 
* [http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_severity=enhancement&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED Enhancement Requests]
+
* [https://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cbug_status%2Cshort_desc%2Cchangeddate&product=Audacity&product=Audacity%20Patches&query_format=advanced&order=changeddate%20DESC%2Cassigned_to%2Cbug_id&query_based_on= All open issues]
* [http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED Nyquist]
+
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&short_desc=crash&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&short_desc_type=allwordssubstr Bugs having  "Crash" in Summary]
* [http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&op_sys=Windows%20XP&op_sys=Windows%20Vista&op_sys=Windows%207&op_sys=All%20Windows&op_sys=Windows%20Vista%20%2F%207&op_sys=Mac%20OS&op_sys=Linux&op_sys=Linux%20and%20OS%20X&op_sys=Other Platform Specific]
+
* [https://bugzilla.audacityteam.org/buglist.cgi?keywords=wx3&resolution=--- Audacity wxWidgets 3.x bugs]
* [http://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED All awaiting fix or review by developer]
+
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=RESOLVED&resolution=FIXED Fixed ]
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?bug_status=DEVEL%20-%20FIX%20MADE&query_format=advanced&order=bug_id&query_based_on= Fix made by developer, not yet resolved by QA]
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_severity=MoonPhase&bug_severity=HeisenBug&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED Moonphase or Heisenbug]
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_severity=enhancement&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED Enhancement Requests]
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED Nyquist]
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&op_sys=Windows%20XP&op_sys=Windows%20Vista&op_sys=Windows%207&op_sys=All%20Windows&op_sys=Windows%20Vista%20%2F%207&op_sys=Mac%20OS&op_sys=Linux&op_sys=Linux%20and%20OS%20X&op_sys=Other Platform Specific]
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED All awaiting fix or review by developer]
  
  
Line 116: Line 132:
  
 
===One Level===
 
===One Level===
* [http://bugzilla.audacityteam.org/report.cgi?y_axis_field=priority&cumulate=1&z_axis_field=&format=bar&x_axis_field=&x_labels_vertical=1&query_format=report-graph&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Number of P1's to P5's]
+
* [https://bugzilla.audacityteam.org/report.cgi?y_axis_field=priority&cumulate=1&z_axis_field=&format=bar&x_axis_field=&x_labels_vertical=1&query_format=report-graph&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Number of P1's to P5's]
 
===Two Levels===
 
===Two Levels===
* [http://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Bug Type and Priority].  This is showing open bugs.  It is possibly the best table for getting an overview of the kinds of live bugs we have.
+
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Bug Type and Priority].  This is showing open bugs.  It is possibly the best table for getting an overview of the kinds of live bugs we have.
* [http://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_status&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Status and Priority].  This is showing both open and closed bugs.  It shows, for example, that we tend to close a higher proportion of high priority bugs than lower priority bugs.
+
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=Repeatable&bug_severity=RepeatableAll&bug_severity=Accessibility&bug_severity=Locale&bug_severity=MoonPhase&bug_severity=HeisenBug&bug_severity=CometReturns&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Bugs] and [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=Summary&bug_severity=Review&bug_severity=Enhancement&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Issues].  This pair of tables splits the actual real bugs from issues (enhancements, summary, review items).  We also have this table for [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&bug_severity=Repeatable&bug_severity=MoonPhase&bug_severity=HeisenBug&bug_severity=CometReturns&bug_severity=Summary&bug_severity=Review&bug_severity=Enhancement&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= closed bugs].
* [http://bugzilla.audacityteam.org/report.cgi?x_axis_field=bug_severity&y_axis_field=bug_status&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Status and Bug Type].  This is showing open and closed bugs.  It shows, for example, that summary issues tend to stay open.
+
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_status&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Status and Priority].  This is showing both open and closed bugs.  It shows, for example, that we tend to close a higher proportion of high priority bugs than lower priority bugs.
 +
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=bug_severity&y_axis_field=bug_status&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Status and Bug Type].  This is showing open and closed bugs.  It shows, for example, that summary issues tend to stay open.
 
===Three Levels===
 
===Three Levels===
* [http://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=component&z_axis_field=bug_severity&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= 3-Levels of Bug Counts]
+
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=component&z_axis_field=bug_severity&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= 3-Levels of Bug Counts]
  
  
 
[[Category:For Developers]][[Category:Quality]]
 
[[Category:For Developers]][[Category:Quality]]

Latest revision as of 11:59, 19 August 2021

Warning icon This page has been deprecated. Newer information can be found here: https://github.com/audacity/audacity/issues
The information on this page are likely out-of-date and will not be updated in the forseeable future. It may be removed at any time.
The Audacity Team track bugs and agreed enhancement issues on our Bugzilla. This page is a list of useful Bugzilla links for Developer/Quality Assurance use.
If you are a member of the public reporting a bug, please read Reporting Bugs and then send an email to our feedback address. Suggestions for new or improved features are also welcome at that address.


Top Tips:

Bulb icon If you go to our bugzilla, clicking on the Audacity Icon at top left takes you to a summary table of open bugs. Clicking on the numbers in that table takes you to those bugs.
Bulb icon Audacity has some conventions and thinking of its own on Using Bugzilla.

See also:

  • Bug Counts - For historical record of bug levels.
This does not work yet.

Bugzilla And The Bug Table

Documentation Bugs/To-Dos for the Manual

Sometimes mistakes in the manual are as serious as mistakes in Audacity code, and such mistakes block release of Audacity too, until they are sorted.

(The links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are not bugs.)


Bugs by Priority

Open Bugs   Awaiting Developer Fix
PX   P0/P1   P0 bugs are bugs so bad they seriously impede development. Preventing recording working, or disabling mouse drags or more prosaically a build breaker would be P0. P0's affect the development team and impede testing by the QA team. You'll almost never see them logged in bugzilla, because we fix them so quickly.
P0 to P2   P2 only P1 bugs prevent a new release. These will include most reproducible crashes, most things that cause data loss and most regressions. P1's would affect large numbers of users, if released. QA is saying these bugs are not acceptable. RM could in principle over-rule that and release without a proper tested fix, but almost never will, because then RM is taking sole responsibility for the consequences.
P1 to P3   P3 only P2 bugs still open when the release process starts, can be fixed or not at the discretion of the Release Manager. QA is saying that these bugs are serious, but they are OK with them going out into the release. Responsibility for the consequences is shared by RM and QA. Unfixed P2 bugs must be release noted.
P1 to P4   P4 only P3 bugs are lower severity bugs which must still be release noted.
All Open Bugs   P5 only P4 and P5 bugs are not release noted. Many of these may be "enhancement issues".


Our Quality Goals

We used to have "stable" releases and "beta" releases. However, there is a social problem with that! Developers like writing new features. It isn't as much fun fixing bugs, especially other people's bugs. So as developers made the releases, there was a huge temptation to make beta releases. And then, after a bit, we get stuck in only making betas, because it is too much work to sort out the accumulated bugs. From the start of the 2.0 series, Audacity removed the former distinction between "stable" and "beta" releases in favor of single series of high-quality releases suitable for all users. Bugs serious enough to block a stable release have to be fixed at the time, or the feature disabled.

With that change, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" changed too. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for our releases is now as follows.

  • Never less good overall than the standard set by 2.2.0 - only trivial regressions are likely to be tolerated in a release.
  • Not necessarily laden with new or improved features - in a few releases the main focus may be on housekeeping and bug-fixing.
  • Not necessarily "fully stable" - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series. So in turn, driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted. Releasing reasonably frequently is important.

We also track our bug counts month by month and set yearly targets for reducing them.


Workflow when resolving bugs

  • When a bug has a fix committed by a developer, the developer moves the bug status to DEVEL - FIX MADE. This means the bug is still OPEN. QA then test it, and they ALSO test things they think the fix may have broken. The developer may sometimes leave a bugzilla flag to indicate whether the bug fix requires testing on all three platforms. When QA are happy with the result of testing, they move the bug status to RESOLVED - FIXED. That means the bug is closed. It will not be visible in searches except when searching for Fixed.
  • We aim to make sure that the same person doesn't both DEVEL - FIX MADE a bug and RESOLVED - FIXED a bug. It adds a bit more safety.


If you plan to use our bugtracker you may want to read about the advantages and social hazards of bugtrackers, along with a bit more terminology.

Residuals

Tracking "Residual" bugs after a fix can be tricky.

  1. If the residual is due to an incomplete fix of the code that causes the described bug, then it should initially be treated as the same bug.
  2. If the description of the residual differs markedly from the original bug description, then it should be treated as a new bug.
  3. If the residual is due to an error in code that is unrelated to the original bug, then it should be treated as a new bug, even if the symptoms have similarities.

Chatty Bug Histories

It is often NOT clear what to do about a bug. We do strive not to clutter up the bug histories with chatter. We try to use it for evidence, information, progress made. Often we will discuss bugs, or groups of bugs on our quality list. One trick to reduce the impact of long bug histories is this line:

*** STEPS UPDATED ***

This means that the steps to reproduce the bug have been updated, usually due to a partial fix that leaves some residual. Sometimes though we just got a clearer understanding of the bug. Generally when visiting a bug you can work back through the comments, and stop when you reached *** STEPS UPDATED ***.


Some Categories

These are some links for some categories of bugs...


Reports

Not strictly bug lists...

One Level

Two Levels

  • Bug Type and Priority. This is showing open bugs. It is possibly the best table for getting an overview of the kinds of live bugs we have.
  • Bugs and Issues. This pair of tables splits the actual real bugs from issues (enhancements, summary, review items). We also have this table for closed bugs.
  • Status and Priority. This is showing both open and closed bugs. It shows, for example, that we tend to close a higher proportion of high priority bugs than lower priority bugs.
  • Status and Bug Type. This is showing open and closed bugs. It shows, for example, that summary issues tend to stay open.

Three Levels