Testing: timing tests

From Audacity Wiki
Jump to: navigation, search
Warning icon Peter 31Mar15: This suite of tests is probably one that would be suitable for automated testing, but for now they can be performed manually.
The timings are based on a project with exactly 30 minutes of audio which you will need to create.

What you are looking for in this group of tests is an average time which gets significantly worse in the alpha or Release Candidate compared with the average for the current release. Significant improvements are worthy of reporting too.

Test methodology

  1. These tests cover much of the basic processing encountered in LP/tape conversion or webstream capture and output.
  2. The timings will vary depending on platform (hardware and OS) so are only relative and not absolute.
  3. When running these tests ensure that the computer is not running other compute-intensive tasks.
  4. Each test must be conducted 3 times.
  5. The tests should take 1-2 hours to complete.
  6. These timings give processing times - anything like Amplify that requires a re-draw can add another 10 to 30 seconds for the re-drawing.
  7. Record the processing times from the process dialogs.
  8. You may wish to create a spreadsheet to calculate the averages.
  9. Project rate & Sample format are Audacity defaults: 44100 Hz and 32-bit float.
  10. All down-sampling in these tests is done with dither set to "Shaped".
  11. Project should be exactly 30 minutes of audio.
  12. For tests no. 13 through 17, open a new, clean, project - i.e. one without editing and other tests. Then generate a 30 second tone/noise.
  13. For tests 18-19 open another new project
  14. Do all T1 tests then T2 and then T3 - I.e work vertically in the worksheet

Worksheet

Test 30 minute project Current Release T1 T2 T3 Average Alpha / RC T1 T2 T3 Average
1 Open Project Current Release Alpha or RC
2 Export 16-bit WAV Current Release Alpha or RC
3 Export 32-bit float WAV Current Release Alpha or RC
4 Export MP3 @192 JS CBR Current Release Alpha or RC
5 Export AAC Current Release Alpha or RC
6 Remove DC offset Current Release Alpha or RC
7 High Pass Filter (24dB & 30Hz) Current Release Alpha or RC
8 Click Removal (default) Current Release Alpha or RC
9 Noise Reduction (default) Current Release Alpha or RC
10 Amplify (-3dB ceiling) Current Release Alpha or RC
11 Normalize (to -1dB) Current Release Alpha or RC
12 Compressor Current Release Alpha or RC
New Project Current Release not applicable not applicable not applicable not applicable Alpha or RC not applicable not applicable not applicable not applicable
13 Import 30 minute 16-bit WAV Current Release Alpha or RC
14 Import 30 minute 32-bit float WAV Current Release Alpha or RC
15 Import 30 minute 192 MP3 Current Release Alpha or RC
16 Import 30 minute AAC Current Release Alpha or RC
17 Save 30 minute 4-track project Current Release Alpha or RC
- New project, add stereo track Current Release not applicable not applicable not applicable not applicable Alpha or RC not applicable not applicable not applicable not applicable
18 Generate 30 minutes brown noise Current Release Alpha or RC
19 Save 30 minute "noise" project Current Release Alpha or RC


|< Testing