|The Audacity Team track bugs and agreed enhancement issues on our Bugzilla. This page is a list of useful Bugzilla links for Developer/Quality Assurance use.
|If you go to our bugzilla, clicking on the Audacity Icon at top left takes you to a summary table of open bugs. Clicking on the numbers in that table takes you to those bugs.|
|Audacity has some conventions and thinking of its own on Using Bugzilla.|
Bugzilla And The Bug Table
Documentation Bugs/To-Dos for the Manual
Sometimes mistakes in the manual are as serious as mistakes in Audacity code, and such mistakes block release of Audacity too, until they are sorted.
(The links to MediaWiki:Sidebar, ToDo, Welcome For Editors and Help:Translating are not bugs.)
Bugs by Priority
|Open Bugs||Awaiting Developer Fix|
|PX||P0/P1||P0 bugs are bugs so bad they seriously impede development. Preventing recording working, or disabling mouse drags or more prosaically a build breaker would be P0. P0's affect the development team and impede testing by the QA team. You'll almost never see them logged in bugzilla, because we fix them so quickly.|
|P0 to P2||P2 only||P1 bugs prevent a new release. These will include most reproducible crashes, most things that cause data loss and most regressions. P1's would affect large numbers of users, if released. QA is saying these bugs are not acceptable. RM could in principle over-rule that and release without a proper tested fix, but almost never will, because then RM is taking sole responsibility for the consequences.|
|P1 to P3||P3 only||P2 bugs still open when the release process starts, can be fixed or not at the discretion of the Release Manager. QA is saying that these bugs are serious, but they are OK with them going out into the release. Responsibility for the consequences is shared by RM and QA. Unfixed P2 bugs must be release noted.|
|P1 to P4||P4 only||P3 bugs are lower severity bugs which must still be release noted.|
|All Open Bugs||P5 only||P4 and P5 bugs are not release noted. Many of these may be "enhancement issues".|
Our Quality Goals
We used to have "stable" releases and "beta" releases. However, there is a social problem with that! Developers like writing new features. It isn't as much fun fixing bugs, especially other people's bugs. So as developers made the releases, there was a huge temptation to make beta releases. And then, after a bit, we get stuck in only making betas, because it is too much work to sort out the accumulated bugs. From the start of the 2.0 series, Audacity removed the former distinction between "stable" and "beta" releases in favor of single series of high-quality releases suitable for all users. Bugs serious enough to block a stable release have to be fixed at the time, or the feature disabled.
With that change, the former meaning of a P2 bug as one that "blocks a stable but not a beta release" changed too. Bugs that would previously have been rated as P2 are now P1. Our Quality Requirement for our releases is now as follows.
- Never less good overall than the standard set by 2.2.0 - only trivial regressions are likely to be tolerated in a release.
- Not necessarily laden with new or improved features - in a few releases the main focus may be on housekeeping and bug-fixing.
- Not necessarily "fully stable" - perceived stability, like ratings of bugs, is not an exact science. We want to release more regularly than was the case during the 1.3 Beta series. So in turn, driver or operating system incompatibilities beyond our control will normally be "WONTFIX" P3 release-noted. Releasing reasonably frequently is important.
We also track our bug counts month by month and set yearly targets for reducing them.
Workflow when resolving bugs
- When a bug has a fix committed by a developer, the developer moves the bug status to DEVEL - FIX MADE. This means the bug is still OPEN. QA then test it, and they ALSO test things they think the fix may have broken. The developer may sometimes leave a bugzilla flag to indicate whether the bug fix requires testing on all three platforms. When QA are happy with the result of testing, they move the bug status to RESOLVED - FIXED. That means the bug is closed. It will not be visible in searches except when searching for Fixed.
- We aim to make sure that the same person doesn't both DEVEL - FIX MADE a bug and RESOLVED - FIXED a bug. It adds a bit more safety.
If you plan to use our bugtracker you may want to read about the advantages and social hazards of bugtrackers, along with a bit more terminology.
Tracking "Residual" bugs after a fix can be tricky.
- If the residual is due to an incomplete fix of the code that causes the described bug, then it should initially be treated as the same bug.
- If the description of the residual differs markedly from the original bug description, then it should be treated as a new bug.
- If the residual is due to an error in code that is unrelated to the original bug, then it should be treated as a new bug, even if the symptoms have similarities.
Chatty Bug Histories
It is often NOT clear what to do about a bug. We do strive not to clutter up the bug histories with chatter. We try to use it for evidence, information, progress made. Often we will discuss bugs, or groups of bugs on our quality list. One trick to reduce the impact of long bug histories is this line:
*** STEPS UPDATED ***
This means that the steps to reproduce the bug have been updated, usually due to a partial fix that leaves some residual. Sometimes though we just got a clearer understanding of the bug. Generally when visiting a bug you can work back through the comments, and stop when you reached *** STEPS UPDATED ***.
These are some links for some categories of bugs...
- Fix made by developer, not yet resolved by QA
- Moonphase or Heisenbug
- Enhancement Requests
- Platform Specific
- All awaiting fix or review by developer
Not strictly bug lists...
- Bug Type and Priority. This is showing open bugs. It is possibly the best table for getting an overview of the kinds of live bugs we have.
- Bugs and Issues. This pair of tables splits the actual real bugs from issues (enhancements, summary, review items). We also have this table for closed bugs.
- Status and Priority. This is showing both open and closed bugs. It shows, for example, that we tend to close a higher proportion of high priority bugs than lower priority bugs.
- Status and Bug Type. This is showing open and closed bugs. It shows, for example, that summary issues tend to stay open.