GSoC 2008 - Label Track Enhancements
|Label Track Enhancements removed a long-standing limitation that Audacity's labels did not stick to the audio track and move and edit with them.|
- 1 Abstract
- 2 User Documentation
- 3 Progress
- 4 Initial Work
- 5 Comments
User documentation was produced by the student and has been incorporated into the Audacity Manual at Linking labels with audio tracks. Note that details of the documentation or of the project implementation itself may change over time.
18 Aug 2008
- GSoC finished, project completed
4/6/11 Aug 2008
- More changes to how pastes behave with multiple selection
29 July 2008
- Fixed ChangeSpeed bugs
- Disallowed pasting from one type of track to another
25 July 2008
- Labels now shift properly on label pastes
23 July 2008
- Added support for pasting labels. You can copy/paste them by themselves, or wave tracks and labels at the same time (by selecting both tracks).
- Pasting into multiple tracks at the same time no longer adds silence in inappropriate tracks.
- Added Change Tempo/Pitch support for label shifting. Built into SoundTouchEffect so that future effects added into SoundTouch will be supported.
- NOTE: A bug currently exists in SoundTouch that can cause the labels to seem like they're shifted incorrectly. See this discussion.
- Speeding up a section (with any Change ... effect) will no longer insert silence into the track if it wouldn't be preferred. Ie, silence is not inserted if all the tracks in a grouping are being sped up. Previous behaviour is retained if only a subset of the tracks in a group are being sped up.
- Fixed corner case HandlePaste() bug.
7 July 2008
- Generate... no longer inserts even though there's a selection, if linking is on. Regular behaviour is back.
- Menu item and button for enabling linking are now sync'd.
- LabelTrack::Delete now uses my label shifting code (if linking is off) instead of the old stuff.
4 July 2008
- Change Speed support is now sample accurate.
2 July 2008
- Added support for Change Speed.
- Slowing down a section in a wave track adds silence to linked wave tracks.
- Speeding up a section adds silence to that wave track.
- Labels shift accordingly.
- Added support for Generate functions. Chirp, DTMF, Noise, Tone and Silence all behave as Pastes.
- Fixed linking button not toggling states (Push down/pop-up).
- Turned linking on by default.
- Renamed menu item from "Link Wave and Label Tracks" to "Link Audio and Label Tracks".
- Fixed bug related to cuts/deletes from label tracks. Selecting only a region in a label track and deleting it will delete that region in linked wave tracks. Selecting a label track in addition to a wave track is now the same as selecting only the wave track. If linking is off, the previous behaviour is used.
- Fixed Cut behaviour when selecting multiple tracks. It previously copied one track, then ran the group clear, shifting the track. The rest of the copy calls would be copying the wrong selection. OnCut() no longer calls the wave track's cut functions (which call copy->clear), but instead calls the copy and clear functions directly.
26 Jun 2008
- Added paste functionality.
- I wanted to build it in a similar style as clears, so I rearranged the paste functions a bit, to more closely mirror how clears are handled (with main Clear() functions that call HandleClear() where the work is actually done).
- Pasting into a wave track adds silence in other linked wave tracks.
- Fixed bug where clearing from multiple tracks at the same time would shift the labels multiple times.
- Added check mark to toggle menu item.
23 Jun 2008
- New version that uses a different linking style finished. Instead of explicitly selecting which wave track a label track is linked to, a wave track is implicitly linked to a contiguous block of label tracks underneath it.
- So, if a project was set up to look like this:
- WaveTrack 1
- WaveTrack 2
- WaveTrack 3
- LabelTrack 1
- LabelTrack 2
- WaveTrack 4
- LabelTrack 3
- WaveTrack 5
The following links would be established: Label tracks one and two are linked to wave tracks one through three, label track three with wave track four, and wave track five is not linked to any label track.
- To ensure that labels are not deleted/shifted when the user doesn't want them to be, the functionality must be enabled. A menu item has been tentatively placed in the "Tracks" menu to enable/disable linking for the entire project.
- Committed to CVS.
14 Jun 2008
- Early version of Sticky Labels checked into CVS. To use it, create a label or label track.
- What works:
- Grouping a label track with any number of audio tracks. Do this by selecting the pulldown menu on the label track's Track Control Panel, and use the submenu to pair the label track to your chosen audio track.
- When a label track is grouped with a wave track, any clears or pastes into that wave track will affect the label track, by shifting the labels left and right.
- Problem compiling on Windows fixed.
- Crash on Windows/delinking on Ubuntu after using "Undo" under investigation.
TrackPanel vs LabelTrack: Where to Build?
One question that I haven't answered well enough is where to do the heft of the work, with respect to grouping. I originally did the menu building in TrackPanel, but I couldn't come up with a good way to handle enabling/disabling different selections. This is necessary because a wave track only holds a single pointer to its grouped label track, so you can't have two label tracks grouped with the same wave track. I pulled out a lot of the work into the LabelTrack class, and while it would work, I'm still not exactly happy with my implementation. There's a lot of glue and finicky things in my current revision. The good thing is that I've got some ideas about how to handle what was tripping me up before, when building in TrackPanel. The plan is now to go back to a TrackPanel based system, with the extra things I've picked up, and eliminate most (hopefully all) of the oddities.
Just-in-Time Menu Building
I've been wrestling with whether or not to have a wxMenu built for each label track and simply update it when necessary, or build this menu when needed. From the start, I've been using a menu that I create when the label track is created, and then update it. Enabling/disabling selections played a role in this decision also. One thing that forced my hand was running into this bug with wxWidgets. This problem cropped up when I removed the sticky label submenu from the TrackPanel menu, and tried reinserting it. Leland gave me a few tips, and I stuck with his idea of rebuilding the entire menu. Now, I still wasn't rebuilding the sticky label menu, I was using the same method as before for that.
The thing with this plan is that there's a lot of little things that need to be taken care of to ensure the menus didn't go out of sync. Updating the names, reordering the menus, reordering arrays to handle the reordering of the menus. Again, not a very nice solution. More good advice from Leland, and I'm planning on scrapping my current method in favour of building the menus up when requested. This eliminates a lot of the patch code that's been necessary, and really tidies things up (at least in my head). Google sent quite an appropriate book as the gift to the students this year. It's definitely made me unhappy with some of my ideas.
Insertion into Wave Track
Since the WaveTrack::Paste function is reused for effects such as Noise, Silence and Repeat, as well as a traditional paste usage, it makes it easy to find when the labels need to be shifted to the right. Though there are separate execution paths for single- and multi-clip paste, I don't think this entails extra work where labels are concerned, and we can ignore the number of clips, focusing solely on the length of the track to be inserted, which is already available for us in the function as insertDuration. There are only three places where a track can be inserted. In the first case, we shift the label right by insertDuration. The second case shifts the closing bracket of the label right by insertDuration. We can ignore the third case, since it isn't affected by insertions.
Clearing a Part of a Wave Track
Again, we save a lot of work because WaveTrack::HandleClear is used for many different purposes. There's an interesting case we need to deal with here, and that's the issue of a split delete. Personally, I feel that it would be best to not do anything to the labels in this instance, even if they are labelling part of the section that is to be deleted. However, we definitely need to do work if the user is performing a regular delete. Again, it's easier to demonstrate the different actions we need to take with a set of six cases. Case one requires no work. Case two should delete the label, and case three should shift the label left by the size of the region to be deleted. Case four and five have multiple ways of dealing with them. I think that in case four, t1 should be made to equal b, the start of our cut, thereby shortening the label. Case five should behave similarly, though with t moving closer to t1. The sixth case should be dealt with by having t1 move left by the size of the cut.
A sample interface for sticking wave tracks to a label tracks. This was my original plan, and it seems like I'll be sticking with this. I tried out using wxComboBox, but the problem with that was that it was just too big to fit comfortably on the track panels, and shrinking it made it look quite ugly.
- James 12:16, 7 June 2008 (PDT): I'd like to see this in the 15th June Alpha. However, there needs to be a way to enable/disable it so that people can try it out but are not forced to use it. Can I suggest a menu item (or a button on the toolbar) that enables/disables whatever the current linking is with one click? This would be enough to make it suitable for a first-look in the alpha. We'd be reasonably sure that we're addressing issues of building on all three platforms, and it would give a chance for some feedback and opinions from the whole team. Please also get Leland to review the source code so that you can do your first check-in.
- Doytch 17:43, 7 June 2008 (PDT): Well, I've already got it all sectioned off with Experimental.h, but I could also throw something in that disables all links. It'd still build menus and such though.
- James 01:44, 8 June 2008 (PDT): Please do. That sounds about right.
- James 14:12, 9 June 2008 (PDT): Is the from uenz, posted to Audacity-Devel 07-Jun-2008 any use in this project?
- James 15:27, 29 June 2008 (PDT) Hi Mark. I've been re-reading the change-speed part of your project proposal, and I think you're making things harder for yourself by looking at 'lots of separate cases' for what to do with a label when there's a speed change. How about stop thinking in terms of the length of the label, but instead think of its end points? You need a function which given an original time tells you what time it ends up at. Use the same function for updating the start time and the end time of each label. That way you treat all labels exactly the same? Easier? This also (for later, and quite probably that's for us to do) generalises to effects which 'change time' in a more complex way.
- Doytch 15:38, 29 June 2008 (PDT) I'm not sure if I understand. The way I've been thinking, I use the two endpoints of the label, the two endpoints of the selection, and the percent change to calculate the shift/stretch. The length of a label doesn't affect how I handle it, only where its endpoints lie in relation to the selection. I end up with six cases, like Clear. Those of course, only happen if we're slowing down the audio. If we speed it up, we don't do anything with the labels and just open a gap (per Richard's comments on the mailing list).
- James The way I suggest it, you apply an effect, then you go through all labels, doing the same thing to each. Change start point, change end point. Use the same function to do each endpoint. Isn't that easier, surely?