Difference between revisions of "Quality"

From Audacity Wiki
Jump to: navigation, search
(new page)
 
(not volunteers any more)
 
(25 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Ensuring high quality for open source software is a major challenge. Everyone contributing is a volunteer, so it is not possible to make people follow particular guidelines, only ask that they do.
+
{{Introrel|Ensuring high quality for open source software is a major challenge. ||[[SubmittingPatches|Submitting Patches]] - If you've fixed a bug in the code, here's how to get your code changes to the developers.
 +
* [[Reporting Bugs]] - How to report a problem you are having with Audacity to the developers.
 +
* [[Feature Requests]] - Add or vote for your desired features here}}
  
=Coding Standards=
 
The [[CodingStandards]] are a first line of defence. 
 
  
=Stable and Unstable Releases=
+
==Coding Standards==
An unstable 'kick the tyres' release can be produced easily.  These will always have known problems and usually new features which are incomplete and under development.  Stable releases require more work to release.  We maintain a list of 'essentials', which are known problems which we all agree must be fixed before releaseStable releases are trialled on a smaller scale before being announced.  In stable releases we generally disable features which are too incomplete, whereas unstable builds will have them present.
+
The [[CodingStandards|Coding Standards]] are a first line of defence.   
  
=Plug-in Design=
+
==Comments==
Plug-ins help to reduce the risk of destabilisation of Audacity when there are changesIt will usually be quickly apparent if a plug-in is at faultChanges are more localised and hence problems easier to track down.
+
We aim to have doxygen comments at the class level so that someone new to the code can understand more rapidly what each class is forUnusual code is (or should be) commented to make it easier for people new to it to check that it is doing what it should.  We also particularly encourage comments where the reason for special case code is not obviousWe've had several cases where such code has been removed in 'tidying up', because it was not relevant to the developer who did so. Some examples of how this can happen:
  
=Self Test=
+
* Code is only relevant to users with localised versions.
There is a very primitive 'benchmark' test in the help menu of debug builds. This allows us to perform a set number of edits automatically and verify the results. With batch chains and external scripting we are moving towards more automated testing than we previously had. In addition a new feature for capturing screenshots automatically will help with documentation, and when it is script enabled too will help us compare before-and-after results after significant changes.
+
* Code is only relevant to people using a slow network drive.
 +
* Code is needed the first time Audacity runs, when there is no value in the stored config.
 +
* Code supports users who have files in non-standard places, e.g. a path with a space in the name.
 +
* Code addresses some security issue that is relevant when several people share the same machine.
  
=Stress Testing=
+
Such code should either be so clear as to why it is there that it doesn't need a comment, or it should have a comment.
One way we stress test is by running Audacity with a small blocksize. This forces Audacity to create many more small files than it would otherwise and helps to test the core system that writes and reads audio from disk.
+
 
 +
=== ANSWER-ME or FIX-ME comments ===
 +
These comments may be added to ask why the code is the way it is, or to indicate some issue that needs fixing. If the "ANSWER-ME" is later answered, we generally remove the question, leaving a comment that explains why the code is as it is. Similarly if the code is later fixed, we generally just modify the comment to indicate that the code is like this to fix a specific problem.
 +
 
 +
==Stable Releases==
 +
We now only officially release "stable" (non-alpha) builds with completed features. Incomplete or experimental features will have #defines in src/Experimental.h which will usually be commented out for releases.  The [[Bug Lists|Audacity Bugzilla]] maintains a list of issues that will or may block a release, plus lower priority bugs.
 +
 
 +
==Code Reviews==
 +
 
 +
[[Code Review Triage|Code reviews]] using Google's Rietveld interface started to be done in early 2010 after a move from CVS to SVN.
 +
 
 +
==Internal Releases and Test Builds==
 +
 
 +
Sometimes the extent and speed of changes being made to development code may mean that one or more release builds are made for internal and developer assessment, but are not officially advertised. Such was the case with the 1.3.6a series of releases during [[GSoC 2008 Projects|GSoC 2008]].
 +
 
 +
{{ednote|'''Peter 08Oct18:''' [[ToDo-2]] we have not posted nightlies for many moons now - so I'm commenting this out with an ednote:
 +
Additionally, regular [[Builds for alpha testing|alpha builds]] from HEAD (in release configuration) have been made available to the developer community since 2009. This encourages user feedback (with the obvious stability caveats). 
 +
}}     
 +
 
 +
Stable releases will always require at least one "release-candidate" build to be available for assessment in the developer community in the days before actual release. See [[Release Process]] for more details.
 +
 
 +
==Plug-in Design==
 +
Plugins help to reduce the risk of destabilization of Audacity when there are changes.  It will usually be quickly apparent if a plug-in is at fault.  Changes are more localized and hence problems easier to track down.  An initiative to speed up progress on the [[Modular Architecture Initiative|modular architecture]] was started in 2010.
 +
 
 +
==Scripted test==
 +
Using the mod-script-pipe plug-in we can now run a scripted test that exercises each of the effects.  We intend over time to extend the range of this test script. 
 +
 
 +
==Self Test==
 +
There is a very primitive 'benchmark' test in the Help menu of debug builds.  This allows us to perform a set number of edits automatically and verify the results.  With batch Macros and external scripting we are moving towards more automated testing than we previously had.  In addition a feature for capturing screenshots automatically is available in all builds to help with documentation. When it is script-enabled it will help us compare before-and-after results after significant changes.
 +
 
 +
==Stress Testing==
 +
One way we stress test is by running Audacity with a small blocksize. This forces Audacity to create many more small files than it would otherwise and helps to test the core system that writes and reads audio from disk.
 +
 
 +
 
 +
[[Category:For_Developers]][[Category:Quality]]

Latest revision as of 13:08, 30 July 2021

Ensuring high quality for open source software is a major challenge.
 
Related article(s):
  • Submitting Patches - If you've fixed a bug in the code, here's how to get your code changes to the developers.
  • Reporting Bugs - How to report a problem you are having with Audacity to the developers.
  • Feature Requests - Add or vote for your desired features here


Coding Standards

The Coding Standards are a first line of defence.

Comments

We aim to have doxygen comments at the class level so that someone new to the code can understand more rapidly what each class is for. Unusual code is (or should be) commented to make it easier for people new to it to check that it is doing what it should. We also particularly encourage comments where the reason for special case code is not obvious. We've had several cases where such code has been removed in 'tidying up', because it was not relevant to the developer who did so. Some examples of how this can happen:

  • Code is only relevant to users with localised versions.
  • Code is only relevant to people using a slow network drive.
  • Code is needed the first time Audacity runs, when there is no value in the stored config.
  • Code supports users who have files in non-standard places, e.g. a path with a space in the name.
  • Code addresses some security issue that is relevant when several people share the same machine.

Such code should either be so clear as to why it is there that it doesn't need a comment, or it should have a comment.

ANSWER-ME or FIX-ME comments

These comments may be added to ask why the code is the way it is, or to indicate some issue that needs fixing. If the "ANSWER-ME" is later answered, we generally remove the question, leaving a comment that explains why the code is as it is. Similarly if the code is later fixed, we generally just modify the comment to indicate that the code is like this to fix a specific problem.

Stable Releases

We now only officially release "stable" (non-alpha) builds with completed features. Incomplete or experimental features will have #defines in src/Experimental.h which will usually be commented out for releases. The Audacity Bugzilla maintains a list of issues that will or may block a release, plus lower priority bugs.

Code Reviews

Code reviews using Google's Rietveld interface started to be done in early 2010 after a move from CVS to SVN.

Internal Releases and Test Builds

Sometimes the extent and speed of changes being made to development code may mean that one or more release builds are made for internal and developer assessment, but are not officially advertised. Such was the case with the 1.3.6a series of releases during GSoC 2008.

Peter 08Oct18: ToDo-2 we have not posted nightlies for many moons now - so I'm commenting this out with an ednote:

Additionally, regular alpha builds from HEAD (in release configuration) have been made available to the developer community since 2009. This encourages user feedback (with the obvious stability caveats).

Stable releases will always require at least one "release-candidate" build to be available for assessment in the developer community in the days before actual release. See Release Process for more details.

Plug-in Design

Plugins help to reduce the risk of destabilization of Audacity when there are changes. It will usually be quickly apparent if a plug-in is at fault. Changes are more localized and hence problems easier to track down. An initiative to speed up progress on the modular architecture was started in 2010.

Scripted test

Using the mod-script-pipe plug-in we can now run a scripted test that exercises each of the effects. We intend over time to extend the range of this test script.

Self Test

There is a very primitive 'benchmark' test in the Help menu of debug builds. This allows us to perform a set number of edits automatically and verify the results. With batch Macros and external scripting we are moving towards more automated testing than we previously had. In addition a feature for capturing screenshots automatically is available in all builds to help with documentation. When it is script-enabled it will help us compare before-and-after results after significant changes.

Stress Testing

One way we stress test is by running Audacity with a small blocksize. This forces Audacity to create many more small files than it would otherwise and helps to test the core system that writes and reads audio from disk.