Ensuring high quality for open source software is a major challenge. Everyone contributing is a volunteer, so it is not possible to make people follow particular guidelines, only ask that they do.
The CodingStandards are a first line of defence.
Stable and Unstable Releases
An unstable 'kick the tyres' release can be produced easily. These will always have known problems and usually new features which are incomplete and under development. Stable releases require more work to release. We maintain a list of 'essentials', which are known problems which we all agree must be fixed before release. Stable releases are trialled on a smaller scale before being announced. In stable releases we generally disable features which are too incomplete, whereas unstable builds will have them present.
Plug-ins help to reduce the risk of destabilisation of Audacity when there are changes. It will usually be quickly apparent if a plug-in is at fault. Changes are more localised and hence problems easier to track down.
There is a very primitive 'benchmark' test in the help menu of debug builds. This allows us to perform a set number of edits automatically and verify the results. With batch chains and external scripting we are moving towards more automated testing than we previously had. In addition a new feature for capturing screenshots automatically will help with documentation, and when it is script enabled too will help us compare before-and-after results after significant changes.
One way we stress test is by running Audacity with a small blocksize. This forces Audacity to create many more small files than it would otherwise and helps to test the core system that writes and reads audio from disk.