GSoC Ideas 2021

From Audacity Wiki
Revision as of 09:36, 19 February 2021 by James (talk | contribs) (Add source separation; Edit down the explanations.)
Jump to: navigation, search
Google Summer of Code (GSoC) is Google's program for promoting Open Source Software development. Audacity was a mentoring organization for five students for Google Summer of Code 2008, and mentored two students in 2009. This page contains our ideas for suitable projects for 2021.
For information about our future plans and about Audacity software development, please join our developers mailing list
 
Related article(s):


Introduction

The Audacity audio editor is downloaded around 18 million times each year. It's a large, mature and clearly popular open source project. In 2008 and 2009 we participated in Google's Summer of Code, with 5 students in the first year and 2 in the second. We hope to mentor two students in 2021, so that we can give them enough attention that they have both challenges and every chance of success.


Project Ideas

These project ideas are deliberately just outlines. We are looking for more detail from you in your project proposals. For your project proposal, show us that:
  • You've looked into what would be involved. Have you tried making a small modification to the Audacity code base?
  • You plan for regular drops of code during the project. A project that only delivers code right at the end will not work and will not be accepted.
We expect incremental development, with near-daily check ins to GitHub of work in progress, and work that is ready enough that your mentor can easily try it out. In return, mentors will be available to guide and give feedback. We work by email, not IRC nor video chat. It's what works for us.


1. Spectral editing tools

For many years, Audacity has included visualization of the frequency components of sounds as spectrograms.

Since version 2.1.0, Audacity has also had a basic spectral editing capacity. This page demonstrates how a steady background whine is easily seen and selected with the mouse, then blotted out.

But not all extraneous background sounds in recordings fit neatly within rectangles. They may rise or fall in pitch.

The challenge is to develop

  • at least one "paintbrush" or "lasso" tool that can select and highlight an irregularly shaped portion of the spectrogram with the mouse
  • data structures to represent that selection as time and frequency bins
  • transformation of the sound wave to attenuate the selection, and only the selection.

Mathematical prerequisites

The required mathematical sophistication is not trivial, but not at an advanced post-graduate level. The challenge is in the novel reuse of existing routines with the data collected from the user's mouse gestures.
To apply spectral repairs, the general procedure will resemble that of the existing noise reduction effect, which transforms a sound to frequency domain, modifies coefficients, then makes an inverse transform, and blends the results of overlapping windows.
You need a general understanding of the discrete Fourier transform and its inverse, at least as far as the first four sections here.
You should understand qualitatively how window functions can be used to correct spectral leakage, and how time and frequency resolution trade off. Experiment in Audacity with rectangular versus Hann windows, and various window sizes, to see the differences in spectrograms of the same sound.


2. Source Separation

Audacity is interested in adding a process for automatic source separation (for example, separating voice from other music, or separating drums from other instruments in an audio mix). We are not aiming to contribute new research but rather to integrate a current state-of-the-art technique into Audacity so that ordinary users can take advantage of it. Depending on the form of the original implementation, this would could include creating an API to connect Audacity to source separation systems, some graphical interface work for interactive systems, and/or integration with a machine learning framework such as PyTorch, Tensorflow, or Keras.


More Ideas

Some of the best GSoC projects come from students own ideas. An ideal project proposal is presented as a collection of smaller features each of which is useful in itself.

  • We are looking at a range of Proposals for improving Audacity. If your idea is based on one of these, take care not to take on more than can reasonably be done in the time. Get some experience working with Audacity code, so that you know what is involved.
  • We have a developer section on this wiki with information to help you find your way around.
  • You will need to compile Audacity, and show that you can code.

We're looking forward to reading GSoC proposals from potential GSoC students.