Proposal Structured Audio

Proposal pages help us get from feature requests into actual plans. This page is a proposal for a structured audio format, primarily for language learners.

Proposal pages are used on an ongoing basis by the Audacity development team and are open to edits from visitors to the wiki. They are a good way to get community feedback on a proposal.

• Note: Proposals for Google Summer of Code projects are significantly different in structure, are submitted via Google's web app and may or may not have a corresponding proposal page.

Proposed Feature

It's a standard for marking up audio or video streams for structured playback. Most likely we want the full flexibility of an existing scripting language, so the delivery format will often be a zip consisting of javascript and the stream, with the stream containing standard tags. There will be alternative delivery formats too. I chose this one as a first target as it is suitable with slight adaptation for web embedding. Initially the whole thing will be darned clunky. To make it smooth we will need to:

• Make structured playback editing tools (for simple scenarios) within Audacity.
• Publish the standard so that other apps start using it and shaping it.
• Integrate with open source closed-captioning solutions.

Developer Backing

• James Crook - Keen to see this.

Use Cases

From some discussion:

On your mobile phone, load up a language learning lesson, click play, click a button to go back by a logical unit (a whole sentence) another to extend the pause at the end of each sentence, press to get a new word or phrase explained in simpler terms (but still in the foreign language), reach a multiple choice and what you choose affects where you go next.

Random Thoughts

Rockbox players, Android, MIDI controllers, voice-command input. Use at Khan Academy for educational videos.

AAC format supports chapter markers, so we should probably support that in Import/Export.

• Gale: I guess a minor addiional consideration is that if you use .M4B extension that also supports bookmarking in the player app (playback resumes from where left off).

---