View source for GSoC 2008 - LV2 Support

{{IntroNoItal|1=LV2 is a more advanced evolution of the [http://www.ladspa.org/ LADSPA] plug-in architecture. It allows plug-ins to be categorized, and LV2 synths to be used as powerful new tone generators. 

This page is largely historical and discusses progress of the LV2 Support project during [[GSoC 2008 Projects|GSoC 2008]] (also see discussions about LV2 in the [http://lists.sourceforge.net/lists/listinfo/audacity-devel audacity-devel mailing list]). 

The LV2 code as it was at the end of this GSoC project was updated for 2.0.6 release to use the [http://drobilla.net/software/lilv/ Lilv] set of libraries instead of SLV2. This allowed LV2 support to compile on all platforms. Support is still incomplete in a few areas. For example there is only a textual interface, and some LV2 plug-ins such as the DISTRHO set for Windows do not yet load. See [https://bugzilla.audacityteam.org/buglist.cgi?query_format=advanced&short_desc=LV2&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=DEVEL%20-%20FIX%20MADE&short_desc_type=allwordssubstr LV2 bugs on Audacity Bugzilla.]}}
{{note|Effects Categorization remains experimental after being released then withdrawn. It could be further developed or could be dropped in favour of a user-invoked categorization that supports all plug-in formats including VST. See [[Proposal Effect Menu Categorization]] for discussion.}}

==Abstract==

[[Image:Sineshaper_in_audacity_crop.png|thumb|The LV2 synth Sineshaper loaded in Audacity with a generated note in the background]]

LV2 ("LADSPA Version 2") is an open plug-in architecture for signal processing plug-ins. It is designed to do everything that its predecessor LADSPA does, while being much easier for anyone to extend with new features.

Adding LV2 support to Audacity will let users apply LV2 effects to audio clips and, with support for the right extensions, use LV2 synthesizer plug-ins as powerful tone generators. LV2 also allows for categorization of plug-ins in a hierarchy - this could be extended to the built-in effects in Audacity and used to display them in a more user-friendly way.

See http://lv2plug.in for more information on LV2.

Cross-platform LV2 support is experimentally available in 2.0.6-alpha by [[CompilingAudacityForBeginners|compiling Audacity HEAD]] or downloading the [[Builds for alpha testing]]. See [https://manual.audacityteam.org/man/Effect_Menu#LV2_Effects LV2 Effects in the Development Manual] for paths where Audacity looks for LV2 effects.  

==Project documentation==
For anyone who wants to look at and test the code there is some documentation here: [[LV2 Support/Project Docs]]

==Progress==
This pretty diagram shows how I'm using my time for the project. Labels on the horizontal axis are weeks, vertical are the tasks listed below. Grey <b><font color="gray">X</font></b>s are the plan, colored <b><font color="red">X</font><font color="orange">X</font><font color="green">X</font></b>s are actual progress. Red means that I'm just getting started on that task, orange means that I'm in the middle of it, and green means that it's pretty much done, just cleaning up. This diagram will be updated a couple of times every week.

 <b>               |  22  |  23  |  24  |  25  |  26  |  27  |  28  |  29  |  30  |  31  |  32  |  33  |
 _______________|===================================================================================|
 <font color="green">1</font>              |<font color="gray">XXXXXX</font>|      |      |      |      |      |      |      |      |      |      |      |
 _______________|<font color="red">XX</font><font color="orange">XXXX</font>|<font color="green">XX X</font>  | <font color="green">X</font> <font color="green">X</font>  |      |      |      |      |      |      |      |      |   <font color="green">X</font>  |
 <font color="green">2</font>              |      |<font color="gray">XXXXXX</font>|      |      |      |      |      |      |      |      |      |      |
 _______________|    <font color="red">X</font><font color="green">X</font>|<font color="green">X</font>     | <font color="green">X</font>    |  <font color="green">X  X</font>|      |      |      |      |      |      |      |      |
 <font color="green">3</font>              |      |      |<font color="gray">XXXXXX</font>|<font color="gray">XXXXXX</font>|      |      |      |      |      |      |      |      |
 _______________|      |  <font color="red">X</font><font color="orange">XX</font> |<font color="orange">X</font>  <font color="orange">XX</font> |<font color="orange">XXX</font> <font color="green">X</font> |  <font color="green">X</font>   | <font color="green">X</font>    |<font color="orange">X</font>     |      |      |      |      |      |
 <font color="green">4</font>              |      |      |      |      |<font color="gray">XXXXXX</font>|<font color="gray">XXXXXX</font>|      |      |      |      |      |      |
 _______________|      |      |      |    <font color="red">XX</font>|<font color="red">XX XX</font> |<font color="orange">X XXX </font>| <font color="orange">XXXX</font> | <font color="orange">X</font>    |      |      | <font color="green">X</font>    |      |
 <font color="green">5</font>              |      |      |      |      |      |      |<font color="gray">XXXXXX</font>|<font color="gray">XXXXXX</font>|      |      |      |   <font color="green">X</font>  |
 _______________|      |      |      |      |      |      |      |  <font color="red">X</font>   |      | <font color="red">X</font><font color="orange">XX</font>  |<font color="green">X</font>     |      |
 <font color="green">6</font>              |      |      |      |      |      |      |      |      |<font color="gray">XXXXXX</font>|      |      |      |
 _______________|      |      |      |      |      |      |      |      |      |      |   <font color="red">XX</font><font color="green">X</font>|<font color="green">X</font>     |
 <font color="green">7</font>              |      |      |      |      |      |      |      |      |      |<font color="gray">XXXXXX</font>|      |      |
 _______________|      |      |      |      |      |      |      |      |      |      |<font color="red">X</font><font color="orange">X</font>   <font color="orange">X</font>| <font color="green">X</font>    |
 <font color="green">8</font>              |      |      |      |      |      |      |      |      |      |      |<font color="gray">XXXXXX</font>|      |
 _______________|      |      |      |      |      |      |      |      |      |      |  <font color="red">X</font><font color="orange">X</font><font color="green">X</font> |      |
 <font color="green">Finishing up</font>   |      |      |      |      |      |      |      |      |      |      |      |<font color="gray">XXXXXX</font>|
 _______________|      |      |      |      |      |      |      |      |</b>      |      |      |      |
                |===================================================================================|
 Notes          |[[#2008-05-27:_Modus_operandi|1]]     |[[#2008-06-04:_Category_support_working|1]] [[#2008-06-05:_LRDF_categories_sort_of_working|2]]   |[[#2008-06-12:_LRDF_categories_committed|1]]     |      |      |[[#2008-07-04:_LV2_support_committed|1]]     |[[#2008-07-09:_LV2_libraries_added_to_lib-src|1]]     |      |      |      |      |      |

==Plan==
This is how I'm planning to break down my work. The tasks below will be done in order, though some overlap may occur. Numbers 1, 2 and 3 form the early spinoff for this project and my first focus will be on getting those parts usable.


[[Image:plug-incategories.png|thumb|Effects grouped into categories, one effect both a standalone entry and in a category]]

# '''Effect category support'''<br>Add infrastructure in Audacity to allow categorisation of plug-ins and built-in effects. The user-visible part of this would be that the "Effects" menu would have submenus for the different categories instead of the current submenus named "plug-ins 1 - 15", "plug-ins 2 - 30" etc, making it easier to find plug-ins and figure out what they do. Since there may still be plug-ins that are not categorized there would also have to be an "Unsorted" submenu that could have the same submenu division as the current "Effects" menu.
# '''Categorize bundled effects'''<br>Add support for categories to Nyquist and built-in effects and categorize all of them.
# '''LADSPA category support'''<br>Add partial support for LRDF, a system of metadata for LADSPA plug-ins that includes plug-in categories. Not all LADSPA plug-in authors have written LRDF data files for their plug-ins, but most of the large packages have categorized plug-ins (SWH, TAP, CAPS).
# '''Basic LV2 support'''<br>Add basic support for LV2 effect plug-ins, i.e. loading them, presenting them in the Effects menu (using their categories), generating control dialogs for them on request and applying them to audio regions. Once this is done users would be able to use the CALF LV2 effects, my Math plug-ins, and other LV2 effect plug-ins that generates audio output from audio input.
# '''LV2 synth support'''<br>Add support for using LV2 synthesiser plug-ins, that generate audio output from MIDI input, as tone generators. This would include implementing partial support for the Event Port extension (for input ports only) and for sending MIDI events to event ports. The synthesiser plug-ins would be presented in the "Generate" menu, and the user would be able to set all their control parameters and the MIDI key, MIDI velocity, time between Note On and Note Off, and total time for the generated signal (since most synths don't go silent right away when receiving a Note Off but rather have a gradual release). Once this is done users would be able to use the CALF Monosynth, the CALF Organ, Sineshaper, Rudolf 556, Nekobee and other LV2 synth plug-ins as tone generators.
# '''Scale point support'''<br>Add support for LV2 scalePoints, which are a part of the LV2 core specification and allows a plug-in author to label certain values for certain plug-in parameters, e.g. "Left" and "Right" for the values -1 and 1 for a stereo balance parameter. These labels would be displayed attached the widget for that parameter in the control dialog for that LV2 plug-in, making it easier for users to see what a particular parameter does.
# '''LV2 I18N support'''<br>Add support for internationalisation for LV2 plug-in and parameter names (and scalePoint labels and category names). This would display these strings in the currently selected language, assuming that the plug-in author has added translations for that language. This would make the plug-in control dialogs and the plug-in names in the "Effects" and "Generate" menus blend in better with the rest of the UI, as well as making them easier to understand for non-English speakers.
# '''LV2 port group support'''<br>Add support for the LV2 Port Group extension for input control ports. This allows plug-in authors to group related parameters in a plug-in together and add labels to the groups, for example the group "LFO 1" could contain amplitude and frequency parameters for the first LFO in the plug-in and the group "LFO 2" could contain the same for the second LFO. This would be presented in the plug-in control dialog by putting parameter controls belonging to the same group in wxStaticBox frames with the group name showing. This would make the plug-in control dialogs more structured and easier to read when there are lots of parameters.

==Other==
Some earlier LV2 work by me, unrelated to Audacity:
* [http://ll-plugins.nongnu.org ll-plug-ins] - a bunch of plug-ins and an experimental host
* [http://ll-plugins.nongnu.org/hacking.html lv2-c++-tools] - a set of C++ libraries and tools for easy plug-in hacking

==Notes==
===2008-05-27: Modus operandi===
Just a few notes about my setup, in case anyone would be interested.

Audacity uses CVS for revision control. I've decided to use my own local revision control so I can do small commits, create local branches, backtrack, cherry-pick patches etc without spamming the CVS repository. I use Git since I've recently used it for other things and really like it. Unfortunately there are no perfect Git <-> CVS sync tools - there is <tt>git-cvsimport</tt> which imports the history of a CVS repository into Git, from scratch or incrementally, and <tt>git-cvsexportcommit</tt> which generates a patch for a single Git commit, applies it to a separate CVS working directory, and optionally commits it.

So what I've done is to run <tt>git-cvsimport</tt> to retrieve the entire history of the <tt>audacity</tt> CVS module from 2001 to 2008. I ran into a problem - the <tt>audacity</tt> module is actually a ''virtual'' CVS module that redirects to <tt>audacity-src</tt> with <tt>lib-src</tt> as a subdirectory, which <tt>git-cvsimport</tt> couldn't handle. I solved that by first importing <tt>audacity-src</tt> and then <tt>lib-src</tt> separately, in a subdirectory of my Git repository for <tt>audacity-src</tt>. I can then run <tt>git-cvsimport</tt> in the repositories to fetch all changes from the CVS server into Git remote branches, see what everyone else is doing, and merge or cherry-pick into my working branches whenever I want.

For the other direction (Git -> CVS) I have a separate CVS checkout that I can run <tt>git-cvsexportcommit</tt> on when I have a sequence of local Git commits that I want to send to the CVS server, either as one CVS commit per Git commit or in batches.

Syncing Git repositories against the main Audacity repository will probably be much easier in the future if the code moves to SVN, the Git <-> SVN sync tools look much nicer. But for now we're stuck with CVS.

===2008-06-04: Category support working===
Sent the first patch for review to Chris yesterday, it implements support for effect categories and adds categories to the built-in effects and the bundled Nyquist plug-ins. I decided to use the LV2 standard categories as a model for Audacity and map other plug-in systems' categories to that (e.g. LRDF for LADSPA), but that can easily be changed in the future. The standard LV2 categories look like this:

 plug-in
 |
 |\_Utility
 |  |
 |  |\_Converter
 |  |
 |  |\_Analyser
 |  |
 |   \_Mixer
 |
 |\_Generator
 |  |
 |  |\_Instrument
 |  |
 |   \_Oscillator
 |
 |\_Simulator
 |  |
 |   \_Reverb (also subcategory of Delay)
 |
 |\_Delay
 |  |
 |   \_Reverb (also subcategory of Simulator)
 |
 |\_Modulator
 |  |
 |  |\_Phaser
 |  |
 |  |\_Flanger
 |  |
 |   \_Chorus
 |
 |\_Filter
 |  |
 |  |\_Lowpass
 |  |
 |  |\_Highpass
 |  |
 |  |\_Bandpass
 |  |
 |  |\_Comb
 |  |
 |  |\_Allpass
 |  |
 |   \_Equaliser
 |     |
 |     |\_Parametric
 |     |
 |      \_Multiband
 |
 |\_Spectral Processor
 |  |
 |   \_Pitch Shifter
 |
 |\_Amplifier
 |
 |\_Distortion
 |  |
 |   \_Waveshaper
 |
  \_Dynamics Processor
    |
    |\_Compressor
    |
    |\_Expander
    |
    |\_Limiter
    |
     \_Gate

The top LV2 category, "plug-in", is not used in Audacity - all the categories that are direct subcategories of "plug-in" in LV2 are considered "root categories". 

LADSPA plug-ins (through LRDF) and LV2 plug-ins can define their own categories in their RDF data as subcategories of these, and it's up to the LADSPA/LRDF and LV2 loaders (which I will write) to add those categories to the EffectManager.

Categories will only show up in the Effect menu if they or any of their subcategories contain more than one effect. If they contain exactly one effect the category submenu will be replaced with a menu entry for that effect. There is some room for improvement here, for example it would be good to automatically collapse linear subgraphs to reduce the number of submenus (e.g. Dynamics Processor -> Compressor -> [ Compressor, Leveller ] to Dynamics Processor -> [ Compressor, Leveller ] ).

===2008-06-05: LRDF categories sort of working===
At least they are working on Debian, using system libraries, on my laptop: [http://image.bayimg.com/najikaabn.jpg Screenshot]

Some LADSPA plug-ins do not have category metadata installed (32 out of 254 on my system). I may write up some simple RDF files for those and send to their authors if I have time.

===2008-06-12: LRDF categories committed===
Still Linux only. I'll start adding libraries to lib-src/ after the sunday alpha.

You need to have headers and libraries for liblrdf installed, on Debian they are in the package liblrdf0-dev. If you for some reason would want to turn off LRDF support you can do that with the --without-liblrdf parameter to the configure script.

===2008-07-04: LV2 support committed===
Linux only, system libraries only - you need libslv2 installed. Quite a lot of bugs so far, hopefully they won't be that hard to squash. 

=== 2008-07-09: LV2 libraries added to lib-src===

LV2 code added to lib-src (directories redland, liblrdf, slv2). The libraries build on Linux and Mac, but slv2 does not build on Windows yet (code is actually C99 which is not supported in Visual Studio). 

====How to test support on Mac and Windows====

(copied from {{external|[http://n2.nabble.com/Request-for-Redland-LRDF-SLV2-compilation-on-Mac-and-Windows-td471031.html this thread]}})  

For liblrdf, you should define the preprocessor symbol USE_LIBLRDF in your config*.h and any installed LADSPA plug-ins that have associated RDF data will show up in the correct category submenu instead of under "Unsorted"...

..except they won't, because I'm only loading LADSPA RDF files by default on Linux so far - I didn't know the installation path for RDF data on other systems. You can set the environment variable LADSPA_RDF_PATH to point to the path(s) where the RDF files are installed, or you can add default paths in the code - the relevant part is src/effects/ladspa/LoadLadspa.cpp:184-193:

<pre>
 rdfPathVar = wxGetenv(wxT("LADSPA_RDF_PATH"));
 if (rdfPathVar != wxT(""))
  wxGetApp().AddMultiPathsToPathList(rdfPathVar, rdfPathList);

#ifdef __WXGTK__
 wxGetApp().AddUniquePathToPathList(wxT("/usr/share/ladspa/rdf"), 
                                    rdfPathList);
 wxGetApp().AddUniquePathToPathList(wxT("/usr/local/share/ladspa/rdf"), 
                                    rdfPathList);
#endif
</pre>

For SLV2, you should define the preprocessor symbol USE_SLV2 and compile and link src/effects/LoadLV2.cpp and src/effects/LV2Effect.cpp. This is still very buggy, but if you can compile the example plug-in at {{external|http://lv2plug.in/plug-ins/Amp-example.lv2/}} and put it in a LV2 plug-in directory ($(HOME)/Library/Audio/Plug-Ins/LV2 on Mac, anywhere that you add in the environment variable LV2_PATH on Windows) it should appear under "Unsorted" as "Simple amplifier" and there should be a line printed on startup that says that it was found.

SLV2 doesn't have any default search path on Windows, I'll write a patch for that. What would be a good set of system-wide and user paths for LV2 plug-ins? The LV2 docs only mention Linux and Mac, so we would be setting a precedent. "C:\Program Files\LV2 Plug-ins" ; "$(HOME)\My LV2 Plug-ins" ?

 


----
[[Category:GSoC]]