Difference between revisions of "Bug Lists"

From Audacity Wiki
Jump to: navigation, search
(Updates.)
m (+deprecated)
 
(9 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
{{deprecated|https://github.com/audacity/audacity/issues}}
 
{{Intro|1=The Audacity Team track bugs and agreed enhancement issues on our [https://bugzilla.audacityteam.org Bugzilla]. This page is a list of useful Bugzilla links for Developer/Quality Assurance use.|2=If you are a member of the public reporting a bug, please read [[Reporting Bugs]] and then send an email to our [https://web.audacityteam.org/contact/#feedback feedback address]. Suggestions for new or improved features are also welcome at that address.}}
 
{{Intro|1=The Audacity Team track bugs and agreed enhancement issues on our [https://bugzilla.audacityteam.org Bugzilla]. This page is a list of useful Bugzilla links for Developer/Quality Assurance use.|2=If you are a member of the public reporting a bug, please read [[Reporting Bugs]] and then send an email to our [https://web.audacityteam.org/contact/#feedback feedback address]. Suggestions for new or improved features are also welcome at that address.}}
  
  
'''Quick Links:'''  
+
'''Top Tips:'''  
*[https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Table of issues awaiting fix or review] by Priority and Bug Type (also available from the Audacity icon on Bugzilla)
 
* [[Builds for alpha testing|Latest alpha development builds of Audacity]]
 
* [mailto:[email protected]?subject=Bugzilla_account Ask for a Bugzilla account] (for experienced C++ developers or seasoned bug watchers who want to actively fix or test bugs)
 
* [[SubmittingPatches|How to submit patches]] to our feedback address or developers' mailing list
 
* [http://lists.sourceforge.net/lists/listinfo/audacity-devel audacity-devel] developers' mailing list information 
 
  
 
{{tip|If you go to [https://bugzilla.audacityteam.org/ our bugzilla], clicking on the Audacity Icon at top left takes you to a summary table of open bugs.  Clicking on the numbers in that table takes you to those bugs.
 
{{tip|If you go to [https://bugzilla.audacityteam.org/ our bugzilla], clicking on the Audacity Icon at top left takes you to a summary table of open bugs.  Clicking on the numbers in that table takes you to those bugs.
 
}}  
 
}}  
 
{{tip|Audacity has some conventions and thinking of its own on [[Using Bugzilla]].}}  
 
{{tip|Audacity has some conventions and thinking of its own on [[Using Bugzilla]].}}  
 +
 +
'''See also:'''
 +
 +
* [[Bug Counts]] - For historical record of bug levels.
  
 
__TOC__
 
__TOC__
  
 
+
{{ednote|1=This does not work yet.
 
==Bugzilla And The Bug Table==
 
==Bugzilla And The Bug Table==
  
 
{{#widget:BugTable}}
 
{{#widget:BugTable}}
 +
}}
  
 +
==Documentation Bugs/To-Dos for the Manual==
  
== Latest Changes for:==
+
Sometimes mistakes in the manual are as serious as mistakes in Audacity code, and such mistakes block release of Audacity too, until they are sorted.
 
 
* [https://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cbug_status%2Cshort_desc%2Cchangeddate&product=Audacity&query_format=advanced&order=changeddate%20DESC%2Cassigned_to%2Cbug_id&query_based_on= Open Audacity issues (excluding "Patches awaiting triage" product)] 
 
 
 
* [https://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cbug_status%2Cshort_desc%2Cchangeddate&product=Audacity&product=Audacity%20Patches&query_format=advanced&order=changeddate%20DESC%2Cassigned_to%2Cbug_id&query_based_on= All open issues (including "Patches awaiting triage" product)]
 
  
* [https://bugzilla.audacityteam.org/buglist.cgi?f1=keywords&f2=OP&f3=keywords&f4=CP&j_top=OR&list_id=70&o1=equals&o3=equals&order=keywords%20DESC%2Cchangeddate%20DESC%2Cassigned_to%2Cbug_id&query_based_on=&query_format=advanced&resolution=---&v1=patch&v3=patch_ready All open issues that have Patches attached]
+
''(The links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are not bugs.)''
 +
* [https://alphamanual.audacityteam.org/index.php?title=Special:WhatLinksHere/ToDo-1 Manual P1 bugs]
 +
* [https://alphamanual.audacityteam.org/index.php?title=Special:WhatLinksHere/ToDo-2 Manual P2 bugs]
  
* [https://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cassigned_to%2Cbug_status%2Cresolution%2Cshort_desc%2Cchangeddate&keywords=patch_ready&keywords_type=allwords&query_format=advanced&order=changeddate%20DESC%2Cbug_id&query_based_on= All open issues that have Patches attached, that are agreed on and ready for developer review ("patch_ready")]
 
 
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&short_desc=crash&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&short_desc_type=allwordssubstr Bugs having  "Crash" in Summary]
 
 
* [https://bugzilla.audacityteam.org/buglist.cgi?keywords=wx3&resolution=--- Audacity wxWidgets 3.x bugs]
 
 
* Manual bugs (issues with the written documentation, not infrastructure or coding issues about serving it) - '''Note:''' links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are NOT bugs.
 
** [https://alphamanual.audacityteam.org/index.php?title=Special:WhatLinksHere/ToDo-1 Manual P1 bugs]
 
** [https://alphamanual.audacityteam.org/index.php?title=Special:WhatLinksHere/ToDo-2 Manual P2 bugs]
 
  
 
==Bugs by Priority==
 
==Bugs by Priority==
Line 50: Line 41:
 
| width="20%" bgcolor="#AFC7C7"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=PX&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE PX]
 
| width="20%" bgcolor="#AFC7C7"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=PX&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE PX]
 
| width="5%"| 
 
| width="5%"| 
| width="20%" bgcolor="#F778A1"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P0&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P0/P1]
+
| width="20%" bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P0&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P0/P1]
 
| width="5%"| 
 
| width="5%"| 
| width="50%"| '''P0''' bugs are bugs so bad they seriously impede development.  Preventing recording working, or disabling mouse drags or more prosaically a build breaker would be P0.  P0's affect the development team and impede testing by the QA team.
+
| width="50%"| '''P0''' bugs are bugs so bad they seriously impede development.  Preventing recording working, or disabling mouse drags or more prosaically a build breaker would be P0.  P0's affect the development team and impede testing by the QA team.  You'll almost never see them logged in bugzilla, because we fix them so quickly.
 
|-
 
|-
 
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P0&priority=P1&priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P0 to P2]
 
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P0&priority=P1&priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P0 to P2]
 
| 
 
| 
|bgcolor="#EDE275"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P2 only]
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P2&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P2 only]
 
|
 
|
| '''P1''' bugs prevent a new release. These will include most reproducible crashes, most things that cause data loss and most regressions.  P1's would affect large numbers of users, if released.
+
| '''P1''' bugs prevent a new release. These will include most reproducible crashes, most things that cause data loss and most regressions.  P1's would affect large numbers of users, if released.  QA is saying these bugs are not acceptable.  RM could in principle over-rule that and release without a proper tested fix, but almost never will, because then RM is taking sole responsibility for the consequences.
 
|-
 
|-
 
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P3]
 
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P3]
 
| 
 
| 
|bgcolor="#B5EAAA"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P3 only]  
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P3&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P3 only]  
 
|
 
|
| '''P2''' bugs still open when the [[Release Process|release process]] starts, can be fixed or not at the discretion of the Release Manager. Unfixed bugs must be release noted.  
+
| '''P2''' bugs still open when the [[Release Process|release process]] starts, can be fixed or not at the discretion of the Release Manager. QA is saying that these bugs are serious, but they are OK with them going out into the  release.  Responsibility for the consequences is shared by RM and QA.  Unfixed P2 bugs must be release noted.  
 
|-
 
|-
 
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P4]
 
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P1&priority=P2&priority=P3&priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE P1 to P4]
 
| 
 
| 
|bgcolor="#CFECEC"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P4 only]  
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P4&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P4 only]  
 
|
 
|
 
| '''P3''' bugs are lower severity bugs which must still be release noted.
 
| '''P3''' bugs are lower severity bugs which must still be release noted.
Line 74: Line 65:
 
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE All Open Bugs]
 
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE All Open Bugs]
 
| 
 
| 
|bgcolor="#CFECEC"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P5&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P5 only]  
+
|bgcolor="#ECE5B6"|[https://bugzilla.audacityteam.org/buglist.cgi?priority=P5&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED P5 only]  
 
|
 
|
 
| '''P4 and P5''' bugs are not release noted. Many of these may be "enhancement issues".
 
| '''P4 and P5''' bugs are not release noted. Many of these may be "enhancement issues".
 
|}
 
|}
  
== Assessing bug priority to maintain Quality Requirement ==
 
  
From the start of the [[Release Notes|2.0 series]], Audacity removed the former distinction between "stable" and "beta" releases in favor of single series of high-quality releases suitable for all users.
+
== Our Quality Goals ==
  
Accordingly, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" was discarded. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for single releases is as follows.   
+
We used to have "stable" releases and "beta" releases. However, there is a social problem with that!  Developers like writing new features.  It isn't as much fun fixing bugs, especially other people's bugs.  So as developers made the releases, there was a huge temptation to make beta releases. And then, after a bit, we get stuck in only making betas, because it is too much work to sort out the accumulated bugs. From the start of the [[Release Notes|2.0 series]], Audacity removed the former distinction between "stable" and "beta" releases in favor of single series of high-quality releases suitable for all usersBugs serious enough to block a stable release have to be fixed at the time, or the feature disabled.
  
* '''Never less good overall than the standard set by 2.0.0''' - only trivial regressions are likely to be tolerated in a release.  
+
With that change, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" changed too. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for our releases is now as follows. 
* '''Not necessarily laden with new or improved features''' - in a few releases the main focus may be on housekeeping and bug-fixing, while in most there will be a balance between features and fixes. Major new features will be isolated in plug-ins, and not included in a release until any major shortcomings have been ironed out.   
+
 
* '''Not necessarily "fully stable"''' - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series, so it is possible that some crash bugs could be demoted from P1 release blocker status if their impact is minor, or if they only occur in rare or non-reproducible scenarios. Driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted.  
+
* '''Never less good overall than the standard set by 2.2.0''' - only trivial regressions are likely to be tolerated in a release.  
 +
* '''Not necessarily laden with new or improved features''' - in a few releases the main focus may be on housekeeping and bug-fixing.
 +
* '''Not necessarily "fully stable"''' - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series.  So in turn, driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted.  Releasing reasonably frequently is important.
 
   
 
   
 
+
We also track our [[Bug Counts|bug counts]] month by month and set yearly targets for reducing them.
{{hint|'''Bug fixing at release time:''' The Release Manager can decide to fix or not fix any bug for a release. Almost any P1 bug will be fixed, unless currently available fixes are deemed a stability risk, so requiring demotion to P2. Most P2 bugs open at release time will be accepted in a release, though the Release Manager may decide that some P2's or even P3's related to other improvements in a release should be fixed.}}
 
  
  
 
== Workflow when resolving bugs ==  
 
== Workflow when resolving bugs ==  
* When a bug has a fix committed by a developer, the developer should move the bug status to '''DEVEL - FIX MADE'''. This means the bug is still '''OPEN'''. Then QA should test it, ideally on all three platforms. If QA are happy with the result, they move the bug status to '''RESOLVED - FIXED'''. That means the bug is closed. It will not be visible in searches except when searching for        [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=RESOLVED&resolution=FIXED Fixed].
 
  
* It is suggested the only case where a developer should change bug status to '''RESOLVED - FIXED''' should be for bugs '''P3 and below''' where a logic error was to blame which does not appear to have possible platform or machine dependencies, '''AND''' where this fix has been committed.  
+
* When a bug has a fix committed by a developer, the developer moves the bug status to '''DEVEL - FIX MADE'''. This means the bug is still '''OPEN'''. QA then test it, and they ALSO test things they think the fix may have broken. The developer may sometimes leave a bugzilla flag to indicate whether the bug fix requires testing on all three platforms.  When QA are happy with the result of testing, they move the bug status to '''RESOLVED - FIXED'''. That means the bug is closed. It will not be visible in searches except when searching for [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=RESOLVED&resolution=FIXED Fixed].
 +
* We aim to make sure that the same person doesn't both '''DEVEL - FIX MADE''' a bug and ''' RESOLVED - FIXED''' a bug.  It adds a bit more safety.
  
 
{{Hint|
 
{{Hint|
'''If you plan to use our bugtracker''' you may want to read about the [[Using_Bugzilla|advantages and hazards]] of bugtracking, along with a bit more terminology.   
+
'''If you plan to use our bugtracker''' you may want to read about the [[Using_Bugzilla|advantages and social hazards]] of bugtrackers, along with a bit more terminology.   
 
}}
 
}}
  
 
===Residuals===
 
===Residuals===
  
Tracking "Residual" bugs after a fix can be tricky, but I think that the guideline should be:
+
Tracking "Residual" bugs after a fix can be tricky.
  
# If the residual is '''part of the original bug''' description, then it should initially be treated as the same bug (subject to "3" and "4")
+
# If the residual is due to an '''incomplete fix of the code that causes the described bug''', then it should initially be treated as the same bug.
# If the residual is due to an '''incomplete fix of the code that causes the described bug''', then it should initially be treated as the same bug (subject to "3" and "4")
+
# If the '''description''' of the residual '''differs markedly''' from the original bug description, then it should be treated as a new bug.
# If the '''description''' of the residual '''differs''' from the original bug description, then it should be treated as a new bug.
 
 
# If the residual is due to an error in '''code that is unrelated to the original bug''', then it should be treated as a new bug, even if the symptoms have similarities.
 
# If the residual is due to an error in '''code that is unrelated to the original bug''', then it should be treated as a new bug, even if the symptoms have similarities.
  
Item 4 is likely to be difficult to determine for non-developers, but the developer that fixes the bug will normally be able to advise. In some cases, "4" may not initially be apparent to the developer, but if it becomes apparent at a later date, the residual may be migrated to a new bug.
+
===Chatty Bug Histories===
  
If there is doubt whether a "residual" is part of the original bug or not, it should be discussed on the QA list in the first instance. If it boils down to a question of code, then it may be migrated to the devel list.
+
It is often NOT clear what to do about a bug.  We do strive not to clutter up the bug histories with chatter.  We try to use it for evidence, information, progress made.  Often we will discuss bugs, or groups of bugs on our quality list. One trick to reduce the impact of long bug histories is this line:
  
This approach helps avoid long, meandering bug threads which can otherwise hinder fixes.
+
*** STEPS UPDATED ***
  
* As a rider, if we do choose to include a residual with new steps in the same bug, use
+
This means that the steps to reproduce the bug have been updated, usually due to a partial fix that leaves some residual.  Sometimes though we just got a clearer understanding of the bug.  Generally when visiting a bug you can work back through the comments, and stop when you reached *** STEPS UPDATED ***.
  
  *** STEPS UPDATED ***
 
  
In the bug comments.  This saves people new to the bug reading all the way back to the start and it makes it clear the bug is open for a residual.
+
==Some Categories==
  
==Some Categories==
+
These are some links for some categories of bugs...
 +
 
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&columnlist=bug_severity%2Cpriority%2Cop_sys%2Cbug_status%2Cshort_desc%2Cchangeddate&product=Audacity&product=Audacity%20Patches&query_format=advanced&order=changeddate%20DESC%2Cassigned_to%2Cbug_id&query_based_on= All open issues]
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&short_desc=crash&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&short_desc_type=allwordssubstr Bugs having  "Crash" in Summary]
 +
* [https://bugzilla.audacityteam.org/buglist.cgi?keywords=wx3&resolution=--- Audacity wxWidgets 3.x bugs]
 +
 
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=RESOLVED&resolution=FIXED Fixed ]
 
* [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&bug_status=RESOLVED&resolution=FIXED Fixed ]
 
* [https://bugzilla.audacityteam.org/buglist.cgi?bug_status=DEVEL%20-%20FIX%20MADE&query_format=advanced&order=bug_id&query_based_on= Fix made by developer, not yet resolved by QA]
 
* [https://bugzilla.audacityteam.org/buglist.cgi?bug_status=DEVEL%20-%20FIX%20MADE&query_format=advanced&order=bug_id&query_based_on= Fix made by developer, not yet resolved by QA]
Line 141: Line 135:
 
===Two Levels===
 
===Two Levels===
 
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Bug Type and Priority].  This is showing open bugs.  It is possibly the best table for getting an overview of the kinds of live bugs we have.
 
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Bug Type and Priority].  This is showing open bugs.  It is possibly the best table for getting an overview of the kinds of live bugs we have.
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=Repeatable&bug_severity=MoonPhase&bug_severity=HeisenBug&bug_severity=CometReturns&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Bugs] and [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=Summary&bug_severity=Review&bug_severity=Enhancement&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Issues].  This pair of tables splits the actual real bugs from issues (enhancements, summary, review items).  We also have this table for [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&bug_severity=Repeatable&bug_severity=MoonPhase&bug_severity=HeisenBug&bug_severity=CometReturns&bug_severity=Summary&bug_severity=Review&bug_severity=Enhancement&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= closed bugs].
+
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=Repeatable&bug_severity=RepeatableAll&bug_severity=Accessibility&bug_severity=Locale&bug_severity=MoonPhase&bug_severity=HeisenBug&bug_severity=CometReturns&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Bugs] and [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=Summary&bug_severity=Review&bug_severity=Enhancement&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Issues].  This pair of tables splits the actual real bugs from issues (enhancements, summary, review items).  We also have this table for [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_severity&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&bug_severity=Repeatable&bug_severity=MoonPhase&bug_severity=HeisenBug&bug_severity=CometReturns&bug_severity=Summary&bug_severity=Review&bug_severity=Enhancement&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= closed bugs].
 
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_status&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Status and Priority].  This is showing both open and closed bugs.  It shows, for example, that we tend to close a higher proportion of high priority bugs than lower priority bugs.
 
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=priority&y_axis_field=bug_status&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Status and Priority].  This is showing both open and closed bugs.  It shows, for example, that we tend to close a higher proportion of high priority bugs than lower priority bugs.
 
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=bug_severity&y_axis_field=bug_status&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Status and Bug Type].  This is showing open and closed bugs.  It shows, for example, that summary issues tend to stay open.
 
* [https://bugzilla.audacityteam.org/report.cgi?x_axis_field=bug_severity&y_axis_field=bug_status&z_axis_field=&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&bug_status=CLOSED&bug_status=DEVEL+-+FIX+MADE&bug_status=RESOLVED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= Status and Bug Type].  This is showing open and closed bugs.  It shows, for example, that summary issues tend to stay open.

Latest revision as of 11:59, 19 August 2021

Warning icon This page has been deprecated. Newer information can be found here: https://github.com/audacity/audacity/issues
The information on this page are likely out-of-date and will not be updated in the forseeable future. It may be removed at any time.
The Audacity Team track bugs and agreed enhancement issues on our Bugzilla. This page is a list of useful Bugzilla links for Developer/Quality Assurance use.
If you are a member of the public reporting a bug, please read Reporting Bugs and then send an email to our feedback address. Suggestions for new or improved features are also welcome at that address.


Top Tips:

Bulb icon If you go to our bugzilla, clicking on the Audacity Icon at top left takes you to a summary table of open bugs. Clicking on the numbers in that table takes you to those bugs.
Bulb icon Audacity has some conventions and thinking of its own on Using Bugzilla.

See also:

  • Bug Counts - For historical record of bug levels.
This does not work yet.

Bugzilla And The Bug Table

Documentation Bugs/To-Dos for the Manual

Sometimes mistakes in the manual are as serious as mistakes in Audacity code, and such mistakes block release of Audacity too, until they are sorted.

(The links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are not bugs.)


Bugs by Priority

Open Bugs   Awaiting Developer Fix
PX   P0/P1   P0 bugs are bugs so bad they seriously impede development. Preventing recording working, or disabling mouse drags or more prosaically a build breaker would be P0. P0's affect the development team and impede testing by the QA team. You'll almost never see them logged in bugzilla, because we fix them so quickly.
P0 to P2   P2 only P1 bugs prevent a new release. These will include most reproducible crashes, most things that cause data loss and most regressions. P1's would affect large numbers of users, if released. QA is saying these bugs are not acceptable. RM could in principle over-rule that and release without a proper tested fix, but almost never will, because then RM is taking sole responsibility for the consequences.
P1 to P3   P3 only P2 bugs still open when the release process starts, can be fixed or not at the discretion of the Release Manager. QA is saying that these bugs are serious, but they are OK with them going out into the release. Responsibility for the consequences is shared by RM and QA. Unfixed P2 bugs must be release noted.
P1 to P4   P4 only P3 bugs are lower severity bugs which must still be release noted.
All Open Bugs   P5 only P4 and P5 bugs are not release noted. Many of these may be "enhancement issues".


Our Quality Goals

We used to have "stable" releases and "beta" releases. However, there is a social problem with that! Developers like writing new features. It isn't as much fun fixing bugs, especially other people's bugs. So as developers made the releases, there was a huge temptation to make beta releases. And then, after a bit, we get stuck in only making betas, because it is too much work to sort out the accumulated bugs. From the start of the 2.0 series, Audacity removed the former distinction between "stable" and "beta" releases in favor of single series of high-quality releases suitable for all users. Bugs serious enough to block a stable release have to be fixed at the time, or the feature disabled.

With that change, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" changed too. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for our releases is now as follows.

  • Never less good overall than the standard set by 2.2.0 - only trivial regressions are likely to be tolerated in a release.
  • Not necessarily laden with new or improved features - in a few releases the main focus may be on housekeeping and bug-fixing.
  • Not necessarily "fully stable" - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series. So in turn, driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted. Releasing reasonably frequently is important.

We also track our bug counts month by month and set yearly targets for reducing them.


Workflow when resolving bugs

  • When a bug has a fix committed by a developer, the developer moves the bug status to DEVEL - FIX MADE. This means the bug is still OPEN. QA then test it, and they ALSO test things they think the fix may have broken. The developer may sometimes leave a bugzilla flag to indicate whether the bug fix requires testing on all three platforms. When QA are happy with the result of testing, they move the bug status to RESOLVED - FIXED. That means the bug is closed. It will not be visible in searches except when searching for Fixed.
  • We aim to make sure that the same person doesn't both DEVEL - FIX MADE a bug and RESOLVED - FIXED a bug. It adds a bit more safety.


If you plan to use our bugtracker you may want to read about the advantages and social hazards of bugtrackers, along with a bit more terminology.

Residuals

Tracking "Residual" bugs after a fix can be tricky.

  1. If the residual is due to an incomplete fix of the code that causes the described bug, then it should initially be treated as the same bug.
  2. If the description of the residual differs markedly from the original bug description, then it should be treated as a new bug.
  3. If the residual is due to an error in code that is unrelated to the original bug, then it should be treated as a new bug, even if the symptoms have similarities.

Chatty Bug Histories

It is often NOT clear what to do about a bug. We do strive not to clutter up the bug histories with chatter. We try to use it for evidence, information, progress made. Often we will discuss bugs, or groups of bugs on our quality list. One trick to reduce the impact of long bug histories is this line:

*** STEPS UPDATED ***

This means that the steps to reproduce the bug have been updated, usually due to a partial fix that leaves some residual. Sometimes though we just got a clearer understanding of the bug. Generally when visiting a bug you can work back through the comments, and stop when you reached *** STEPS UPDATED ***.


Some Categories

These are some links for some categories of bugs...


Reports

Not strictly bug lists...

One Level

Two Levels

  • Bug Type and Priority. This is showing open bugs. It is possibly the best table for getting an overview of the kinds of live bugs we have.
  • Bugs and Issues. This pair of tables splits the actual real bugs from issues (enhancements, summary, review items). We also have this table for closed bugs.
  • Status and Priority. This is showing both open and closed bugs. It shows, for example, that we tend to close a higher proportion of high priority bugs than lower priority bugs.
  • Status and Bug Type. This is showing open and closed bugs. It shows, for example, that summary issues tend to stay open.

Three Levels