Completed Proposal Spectral Selection

From Audacity Wiki
Jump to: navigation, search
Proposal pages help us get from feature requests into actual plans. This proposal page is about adding a way to select frequency ranges. It's particularly useful for cleaning certain kinds of noise/glitch and for changing the timbre of a sound.
Proposal pages are used on an ongoing basis by the Audacity development team and are open to edits from visitors to the wiki. They are a good way to get community feedback on a proposal.


  • Note: Proposals for Google Summer of Code projects are significantly different in structure, are submitted via Google's web app and may or may not have a corresponding proposal page.


Contents


Proposed Feature

On the spectrogram view, allow a region to be selected, not just a start and end time. Then do processing of only audio of that frequency range, for example making it quieter.

  • Frequency box that can be resized by edge and by centre line.
  • 3 x Nyquist effects that are frequency-selection aware (more can follow later).
  • New Toolbar (rather than part of selection toolbar)

Further Ideas

  • Hotkey bindable toggle for spectrogram/Waveform view.


Developer backing

  • James Crook
    • +1 for the feature. Really cool.
    • +1 for some experimentation with the colors. It is all a little dark at the moment.
    • +1 for textual tutorial in manual on using it to clean up sounds.
    • +1 for new Nyquist effects to be usable in time selection mode.
    • -1 for must use SHIFT to make a spectral selection, otherwise just a normal time selection. (If you're interested in frequencies you are probably interested in frequency selections.)
    • +1 for hotkey to toggle spectrogram/waveform.
    • +1 for 'play selected frequencies' at some point in the future.
  • Paul L
  • Gale
    • +1 for some refinement to default colors. Too little contrast at the moment. Perhaps inside the box does not need to be different?
    • +1 for (non-video) tutorial in manual on using it to clean up sound, e.g. mouth sounds. IMO there is not yet a good enough case for this feature being made on https://manual.audacityteam.org/man/Spectral_Selection
    • +1 for new Nyquist effects to be usable in time selection mode.
    • +1 for must use SHIFT to make a spectral selection, otherwise just a normal time selection. I would be persuaded to leave it as now (SHIFT required to make only a time selection) if the Status Bar message makes this distinction.
    • +1 for shortcut to toggle spectrogram/waveform.

Use Cases

  • Cleaning up unwanted sound (works best for sounds that are in a frequency band well separated from the signal; when used on longer spans, very similar to noise removal, but usable where you do not have an actual noise sample, but can see that noise on the spectrogram).
  • Enhancing certain resonances, and changing the quality of a voice. (can accentuate certain frequencies, e.g. lower frequencies by selecting them and boosting them. "how" should be mentioned in https://manual.audacityteam.org/man/Spectral_Selection and ideally a tutorial written.)
  • Removing mouth sounds from voice work.


Videos

Details

First Prototypes

Federico Miyara has a version of spectral editing in Audacity that can pick out arbitrary shapes:

FedericoSpectralEdit.JPG

Pual Licameli has this video of his modification to Audacity that can pick out rectangular regions.

RectangularSpecSelect.png


Current Status

The version of Spectral Selection with rectangular selection regions by Paul Licameli is now (Oct 2014) in SVN.

Subsequently:

  • Some tweaking of colors (more to do?)
  • Added Spectral selection toolbar (hidden by default).
  • Added q key as shortcut for time-selection only.

Issues and Enhancements

Would like:

  • Some clean up of code, specifically reduction of cut-and-paste repetition by using subroutines more.
  • A quick way to select one of the three Spectral Editing filters, e.g a context menu.
    • Gale 15Nov14: Why are clip lines and cut lines treated as spectral content? Does that have any benefit?
    • James 15Nov14: The clipping that caused the clip line leads to high frequencies. That is what the distortion you hear is. Ends of audio will always mess up frequency determination unless we take special measures. Where clips are right next to each other, we should take special measures, but we don't.
    • Gale 16Nov14: So is a bug report or enhancement needed? By "clip lines" I did not mean Show Clipping but Edit > Clip Boundaries > Split. I agree a split line or cut line in absolute silence has no spectral content. But a split made in audio shows as a bold glitch, and dragging the clips apart and rendering does not remove the "glitch".
    • James 17Nov14: Enhancement. It's also worth checking whether equalisation and plot spectrum behave correctly across clip boundaries. If they do, we can use the same mechanism. If they don't, we have a common problem. This is relevant to clips that are adjacent. Clips with a gap between will mean audio does not change smoothly. If the audio is not changing smoothly, then that means high frequencies in the spectrum. An actual gap is, and should be, equivalent to dropping to zero. You'll see the same thing if you silence part of a pure tone (without using snap to zero crossings). You'll hear clicks at the ends of the silence, and see a blip on the spectrum at each. The clicks can be a lot louder than the audio being silenced!
    • Bill 14Dec2014: It's a bug IMO. The spectrogram drawing code is apparently treating a clip boundary as a discontinuity. Plot Spectrum does not show this when you do a plot with a selection across a clip boundary.

Color comparison with 2.0.6

  • Gale 15Nov14: I find the spectral selection color rather too dark. Apart from the eye strain, it makes the center frequency line hard to see when over spectral content. Was it possible to use e.g. bright green or perhaps yellow for the center line or change its color dynamically according to the background? In fact, does the spectral box need to shaded at all - would it not be obvious if its four edges were bounded by a distinguishable color, top and bottom dotted as now?
  • Gale 16Nov14: A comparison of 2.0.6 (left) versus 2.0.7 (right) with all frequencies selected may be instructive:
Comparison of spectrogram in 2.0.6 (left) versus 2.0.7 (right) with all frequencies selected
For someone who wants to just change to spectrogram view to e.g. locate LP clicks, and not work with frequencies, the new selection color could be regarded as a regression. To me, the color for "all frequencies selected" in Spectrograms should be the same as the selection color in waveform view, and the frequency selection should be bounded only by lines within the selection color. This image in Spectro shows that a box border could work (leaving aside how to display the center line).
  • James 17Nov14: +1. Light, but different colors when selected would probably work best, and is easy to try.

case ColorGradientTimeSelected examples

   // partly dimmed
   r *= 0.75f;
   g *= 0.75f;
   b *= 0.75f;
   break;
case ColorGradientTimeAndFrequencySelected:
   // fully dimmed
   r *= 0.5f;
   g *= 0.5f;
   b *= 0.5f;
   break;

This would be a stop gap until we have theming of colors (but way too dark to use in released version).

Related

This segues into Proposal Source Separation.

Personal tools

Donate securely by PayPal, using your credit card or PayPal account!