GSoC Ideas 2021

From Audacity Wiki
Revision as of 20:35, 28 March 2021 by Paul L (talk | contribs) (Undo revision 43560 by James (talk))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Google Summer of Code (GSoC) is Google's program for promoting Open Source Software development. Audacity was a mentoring organization for five students for Google Summer of Code 2008, and mentored two students in 2009. This page contains our ideas for suitable projects for 2021.
For information about our future plans and about Audacity software development, please join our developers mailing list
 
Related article(s):


Introduction

The Audacity audio editor is downloaded around 18 million times each year. It's a large, mature and clearly popular open source project. We're returning to GSoC after a long break from it, trying out the shorter more focused 175-hour 10 week project format. We hope to mentor two students in 2021, so that we can give them enough attention that they have both challenges and every chance of success.


Discord Link

Our Discord Server for GSoC


Project Ideas

These project ideas are deliberately just outlines. We are looking for more detail from you in your project proposals. For your project proposal, show us that:
  • You've looked into what would be involved. Have you tried making a small modification to the Audacity code base?
  • You plan for regular drops of code during the project. A project that only delivers code right at the end will not work and will not be accepted.
We expect incremental development, with near-daily check ins to GitHub of work in progress, and work that is ready enough that your mentor can easily try it out. In return, mentors will be available to guide and give feedback.


1. Spectral editing tools

Mentor: Paul Licameli
Difficulty: Medium to Hard

For many years, Audacity has included visualization of the frequency components of sounds as spectrograms.

Since version 2.1.0, Audacity has also had a basic spectral editing capacity. This page demonstrates how a steady background whine is easily seen and selected with the mouse, then blotted out.

But not all extraneous background sounds in recordings fit neatly within rectangles. They may rise or fall in pitch.

The challenge is to develop

  • at least one "paintbrush" or "lasso" tool that can select and highlight an irregularly shaped portion of the spectrogram with the mouse
  • data structures to represent that selection as time and frequency bins
  • transformation of the sound wave to attenuate the selection, and only the selection.

Mathematical prerequisites

The required mathematical sophistication is not trivial, but not at an advanced post-graduate level. The challenge is in the novel reuse of existing routines with the data collected from the user's mouse gestures.
To apply spectral repairs, the general procedure will resemble that of the existing noise reduction effect, which transforms a sound to frequency domain, modifies coefficients, then makes an inverse transform, and blends the results of overlapping windows.
You need a general understanding of the discrete Fourier transform and its inverse, at least as far as the first four sections here.
You should understand qualitatively how window functions can be used to correct spectral leakage, and how time and frequency resolution trade off. Experiment in Audacity with rectangular versus Hann windows, and various window sizes, to see the differences in spectrograms of the same sound.

This project is medium difficulty, if confined to a paintbrush tool. It becomes hard if you do not already have the maths background and the transformation of audio is done too. It can be seen as an opportunity to learn that on an actual implementation. C++ essential.

  • Required: Paintbrush tool
  • Desired: A working effect, more tools, including lasso.

More detailed suggestions

See this page for pointers to parts of the source code to review, and possibilities for expanding the scope of the project with some "stretch" goals.

2. Source Separation

Mentor: Roger Dannenberg
Difficulty: Medium

Audacity is interested in adding a process for automatic source separation (for example, separating voice from other music, or separating drums from other instruments in an audio mix). We are not aiming to contribute new research but rather to integrate a current state-of-the-art technique into Audacity so that ordinary users can take advantage of it. Depending on the form of the original implementation, this work could include creating an API to connect Audacity to source separation systems, some graphical interface work for interactive systems, and/or integration with a machine learning framework such as PyTorch, Tensorflow, or Keras.

This project is medium difficulty, if confined to just the integration work, and we recommend doing that well rather than attempting a more ambitious project in the time. Python and C++ essential.

  • Required: Integration of one existing ML-based effect that source-separates a track into two tracks.
  • Desired: UI changes to make the interface nicer. Handling of source-separation on stereo tracks.


3. Mixing Desk Interface Improvements

Mentor: Jouni Helminen
Difficulty: Easy to Medium

Audacity has a mixing desk, but the look is dated. Also there is missing information. For each track, we want to show the most recently applied effect. Also we want to be able to cancel or redo the effect (using the existing undo mechanism) directly from the mixing desk. We would also like improvements that use the area in a more efficient way than now, so that more details can be put into the mixing desk without overcrowding it.

This project is medium difficulty, when integrating the undo mechanism properly. It becomes easy if just tweaks and appearance changes are made, e.g. using a dial rather than a slider and similar small changes. This is an opportunity to learn from a domain expert about user interface design. C++ essential. This project is primarily programming, but it is programming driven by UI considerations. We are interested in programming skills here rather than graphics design skills.

  • Required: Some coding change to support desired UX changes.
  • Desired: Several variants ready for formal UX testing.


4. Python Testing Coverage (Image Generation)

Mentor: James Crook
Difficulty: Easy to Medium

We have an existing python script that exercises Audacity. It currently needs fixing, because Audacity has moved on since it was originally written, several years ago. After that making it more flexible and more useful for generating images for our manual - its main purpose - would be good. There is plenty of scope to extend the project to increase the range of what is tested and improve on the image generation.

Requires C++ and some python. Project is easy if restricted to fixes and a few new tests of the same kind. It becomes medium if improved flexibility is added.

  • Required: Diagnose where/why the script fails and fix that. done.
  • Required: Some improvement to scripting.
  • Desired: Review what images in manual can't be produced automatically by the script yet, and address some of these.


More Ideas

Some of the best GSoC projects come from students' own ideas. An ideal project proposal is presented as a collection of smaller features each of which is useful in itself.

  • We are looking at a range of Proposals for improving Audacity. If your idea is based on one of these, take great care not to take on more than can reasonably be done in 175 hours of work. Get some experience working with Audacity code, so that you know what is involved.
  • We have a developer section on this wiki with information to help you find your way around.
  • You will need to compile Audacity, and show that you can code.

Do contact us to discuss ideas, whether your own or based on one of ours, before you submit a proposal to Google.

  • Required: A change you care about passionately, that is also wanted by us.
  • Desired: Piece that is complete in itself, and completely ready to go into our next release.

We're looking forward to reading GSoC proposals from potential GSoC students.


GSoC Participation

Some more information at this link for Hints on GSoC Participation 2021.