Difference between revisions of "GSoC Ideas 2021"

From Audacity Wiki
Jump to: navigation, search
(Idea Seeds: Added proposal for Spectral Editing Tools)
(Add source separation; Edit down the explanations.)
Line 1: Line 1:
<noinclude>{{GSoC_Header|This page contains our ideas for suitable projects for 2021}}
+
{{GSoC_Header|This page contains our ideas for suitable projects for 2021}}
  
  
 
__TOC__
 
__TOC__
 
==Executive Summary==
 
 
We have 3 seed ideas, for this years GSoC.
 
* These focus on the benefit we want from your work. 
 
* You will need to provide the detail, and convince us you have the skills to develop the chosen seed idea into a full completed tested project.
 
  
 
==Introduction==
 
==Introduction==
Line 15: Line 9:
  
  
:In many ways Audacity has 'grown up' between 2008 and now.  We've become more quality conscious.  We've become more aware of both the need to and how to bring on new developers.  Because of this we've changed what we ask for in project proposals.  Your project proposal *must* be presented as a collection of smaller features each of which is useful in itself.  It's no good proposing one single big change which can't be trialed by end users during the GSoC period.  A '''mix of smaller related features - along with related bug fixes''' is good.  If your proposal is different to that, if it can't be broken up into smaller useful pieces, if it does not include any bug fixing, or if the pieces are not related and working as a whole, you'll have more trouble convincing us that it will be successful.
+
==Project Ideas==
 +
 
 +
{{note||1=
  
:We *expect* incremental development, with near-daily check ins to GitHub of work in progress, and work that is ready enough that your mentor can easily try it outIn return, mentors will be available to guide and give feedback.  We work by email, not IRC nor video chat.  It's what works for us.
+
These project ideas are deliberately just outlines.  We are looking for more detail from you in your project proposalsFor your project proposal, show us that:
  
==Idea Seeds==
+
* You've looked into what would be involved.  ''Have you tried making a small modification to the Audacity code base?''
 +
* You plan for regular drops of code during the project.  A project that only delivers code right at the end will not work and will not be accepted.
  
{{note|In past years we provided pretty well fleshed out proposals as starting points.
+
We '''expect''' incremental development, with near-daily check ins to GitHub of work in progress, and work that is ready enough that your mentor can easily try it out. In return, mentors will be available to guide and give feedback. We work by email, not IRC nor video chat. It's what works for us.
* We expected applicants to provide more. 
 
* We found that some students just echoed our proposals back to us, making it hard for us to tell if the student had any knowledge or interest. 
 
This year we are providing seed project ideas.  They ARE fairly well worked out ''by us'', BUT to apply with one of these ideas you will need to do some work too.  That's the 'show us you can...', though you are welcome to find other ways to show us that you can do the project.
 
* To grow the seed you will need to hunt through our code and read [[Proposals]] on this wiki.
 
* We have a [[For Developers|developer]] section on this wiki with information to help you find your way around.
 
* You will need to compile Audacity, and show that you can code.
 
 
}}
 
}}
  
</noinclude><includeonly>
 
* These three tables come from [[GSoC Ideas]] where there is a discussion of how to interpret them.
 
</includeonly>
 
  
=== Spectral editing tools ===
+
== 1. Spectral editing tools ==
 
   
 
   
 
For many years, Audacity has included visualization of the frequency components of sounds as [https://manual.audacityteam.org/man/spectrogram_view.html spectrograms].
 
For many years, Audacity has included visualization of the frequency components of sounds as [https://manual.audacityteam.org/man/spectrogram_view.html spectrograms].
Line 48: Line 36:
 
* transformation of the sound wave to attenuate the selection, and only the selection.
 
* transformation of the sound wave to attenuate the selection, and only the selection.
  
==== Mathematical prerequisites ====
+
=== Mathematical prerequisites ===
 +
 
 +
:The required mathematical sophistication is not trivial, but not at an advanced post-graduate level.  The challenge is in the novel reuse of existing routines with the data collected from the user's mouse gestures.
 +
 
 +
:To apply spectral repairs, the general procedure will resemble that of the existing [https://manual.audacityteam.org/man/noise_reduction.html noise reduction] effect, which transforms a sound to frequency domain, modifies coefficients, then makes an inverse transform, and blends the results of overlapping windows.
  
The required mathematical sophistication is not trivial, but not at an advanced post-graduate level. The challenge is in the novel reuse of existing routines with the data collected from the user's mouse gestures.
+
:You need a general understanding of the discrete Fourier transform and its inverse, at least as far as the first four sections [https://en.wikipedia.org/wiki/Discrete_Fourier_transform here].
  
To apply spectral repairs, the general procedure will resemble that of the existing [https://manual.audacityteam.org/man/noise_reduction.html noise reduction] effect, which transforms a sound to frequency domain, modifies coefficients, then makes an inverse transform, and blends the results of overlapping windows.
+
:You should understand qualitatively how [https://en.wikipedia.org/wiki/Hann_function window functions] can be used to correct spectral [https://manual.audacityteam.org/man/spectrogram_view.html#types leakage], and how time and frequency resolution trade off.  Experiment in Audacity with rectangular versus Hann windows, and various window sizes, to see the differences in spectrograms of the same sound.
  
You need a general understanding of the discrete Fourier transform and its inverse, at least as far as the first four sections [https://en.wikipedia.org/wiki/Discrete_Fourier_transform here].
 
  
You should understand qualitatively how [https://en.wikipedia.org/wiki/Hann_function window functions] can be used to correct spectral [https://manual.audacityteam.org/man/spectrogram_view.html#types leakage], and how time and frequency resolution trade off.  Experiment in Audacity with rectangular versus Hann windows, and various window sizes, to see the differences in spectrograms of the same sound.
+
== 2. Source Separation==
 +
 
 +
Audacity is interested in adding a process for automatic source separation (for example, separating voice from other music, or separating drums from other instruments in an audio mix). We are not aiming to contribute new research but rather to integrate a current state-of-the-art technique into Audacity so that ordinary users can take advantage of it. Depending on the form of the original implementation, this would could include creating an API to connect Audacity to source separation systems, some graphical interface work for interactive systems, and/or integration with a machine learning framework such as PyTorch, Tensorflow, or Keras.
 +
 
  
 
== More Ideas ==  
 
== More Ideas ==  
  
See '''[[Release Checklist]]''' for an understanding of what we need to do...  
+
Some of the best GSoC projects come from students own ideas.  An ideal project proposal is presented as a collection of smaller features each of which is useful in itself.
 +
 
 +
* We are looking at a range of [[Proposals]] for improving Audacity.  If your idea is based on one of these, take care not to take on more than can reasonably be done in the time.  Get some experience working with Audacity code, so that you know what is involved.
 +
* We have a [[For Developers|developer]] section on this wiki with information to help you find your way around.
 +
* You will need to compile Audacity, and show that you can code.
 +
 
 +
We're looking forward to reading GSoC proposals from potential GSoC students.
 +
 
 +
 
  
 
[[Category:For Developers]][[Category:GSoC]][[Category:Feature Planning]]
 
[[Category:For Developers]][[Category:GSoC]][[Category:Feature Planning]]
 
</noinclude>
 
</noinclude>

Revision as of 09:36, 19 February 2021

Google Summer of Code (GSoC) is Google's program for promoting Open Source Software development. Audacity was a mentoring organization for five students for Google Summer of Code 2008, and mentored two students in 2009. This page contains our ideas for suitable projects for 2021.
For information about our future plans and about Audacity software development, please join our developers mailing list
 
Related article(s):


Introduction

The Audacity audio editor is downloaded around 18 million times each year. It's a large, mature and clearly popular open source project. In 2008 and 2009 we participated in Google's Summer of Code, with 5 students in the first year and 2 in the second. We hope to mentor two students in 2021, so that we can give them enough attention that they have both challenges and every chance of success.


Project Ideas

These project ideas are deliberately just outlines. We are looking for more detail from you in your project proposals. For your project proposal, show us that:
  • You've looked into what would be involved. Have you tried making a small modification to the Audacity code base?
  • You plan for regular drops of code during the project. A project that only delivers code right at the end will not work and will not be accepted.
We expect incremental development, with near-daily check ins to GitHub of work in progress, and work that is ready enough that your mentor can easily try it out. In return, mentors will be available to guide and give feedback. We work by email, not IRC nor video chat. It's what works for us.


1. Spectral editing tools

For many years, Audacity has included visualization of the frequency components of sounds as spectrograms.

Since version 2.1.0, Audacity has also had a basic spectral editing capacity. This page demonstrates how a steady background whine is easily seen and selected with the mouse, then blotted out.

But not all extraneous background sounds in recordings fit neatly within rectangles. They may rise or fall in pitch.

The challenge is to develop

  • at least one "paintbrush" or "lasso" tool that can select and highlight an irregularly shaped portion of the spectrogram with the mouse
  • data structures to represent that selection as time and frequency bins
  • transformation of the sound wave to attenuate the selection, and only the selection.

Mathematical prerequisites

The required mathematical sophistication is not trivial, but not at an advanced post-graduate level. The challenge is in the novel reuse of existing routines with the data collected from the user's mouse gestures.
To apply spectral repairs, the general procedure will resemble that of the existing noise reduction effect, which transforms a sound to frequency domain, modifies coefficients, then makes an inverse transform, and blends the results of overlapping windows.
You need a general understanding of the discrete Fourier transform and its inverse, at least as far as the first four sections here.
You should understand qualitatively how window functions can be used to correct spectral leakage, and how time and frequency resolution trade off. Experiment in Audacity with rectangular versus Hann windows, and various window sizes, to see the differences in spectrograms of the same sound.


2. Source Separation

Audacity is interested in adding a process for automatic source separation (for example, separating voice from other music, or separating drums from other instruments in an audio mix). We are not aiming to contribute new research but rather to integrate a current state-of-the-art technique into Audacity so that ordinary users can take advantage of it. Depending on the form of the original implementation, this would could include creating an API to connect Audacity to source separation systems, some graphical interface work for interactive systems, and/or integration with a machine learning framework such as PyTorch, Tensorflow, or Keras.


More Ideas

Some of the best GSoC projects come from students own ideas. An ideal project proposal is presented as a collection of smaller features each of which is useful in itself.

  • We are looking at a range of Proposals for improving Audacity. If your idea is based on one of these, take care not to take on more than can reasonably be done in the time. Get some experience working with Audacity code, so that you know what is involved.
  • We have a developer section on this wiki with information to help you find your way around.
  • You will need to compile Audacity, and show that you can code.

We're looking forward to reading GSoC proposals from potential GSoC students.