Difference between revisions of "Proposal Label Enhancements"

From Audacity Wiki
Jump to: navigation, search
(Dragging)
(Added section on making label tracks accessible)
Line 12: Line 12:
 
* RTL support
 
* RTL support
 
* New Preferences
 
* New Preferences
 +
* Making label tracks accessible
  
 
'''Dragging:''' Instead of pushing edges when they collide and becoming a point label, we swap edge being dragged similar to how audio selection drags work.  The (undotted) circles would move entire labels.  For joined labels the circle additionally gets a 'dot' in it to indicate that it is dragging edges, not moving the entire label.  We also show that dot when hovering over the arrow part of an edge.  We lose the ability to move two labels at the same time, which is a very minor loss.  We gain the ability to move labels without a modifier key.  We retain the ability to convert a point label to a region and a region to a point and to drag a boundary between joined labels without a modifier key.  We also gain an additional visual cue that labels are joined.  The way we explain it in the manual is that 'Empty circle drags entire label.  Circle with a dot drag just that edge.'
 
'''Dragging:''' Instead of pushing edges when they collide and becoming a point label, we swap edge being dragged similar to how audio selection drags work.  The (undotted) circles would move entire labels.  For joined labels the circle additionally gets a 'dot' in it to indicate that it is dragging edges, not moving the entire label.  We also show that dot when hovering over the arrow part of an edge.  We lose the ability to move two labels at the same time, which is a very minor loss.  We gain the ability to move labels without a modifier key.  We retain the ability to convert a point label to a region and a region to a point and to drag a boundary between joined labels without a modifier key.  We also gain an additional visual cue that labels are joined.  The way we explain it in the manual is that 'Empty circle drags entire label.  Circle with a dot drag just that edge.'
Line 107: Line 108:
  
 
* Add a Preference for "Label typing:" "Standard" (CTRL + B only required for typing in a label track if cursor is at same place as an existing label - this is current behaviour) or "Manual" (CTRL + B always required).  Use case: unmodified shortcuts can be used even if focus is in label track.
 
* Add a Preference for "Label typing:" "Standard" (CTRL + B only required for typing in a label track if cursor is at same place as an existing label - this is current behaviour) or "Manual" (CTRL + B always required).  Use case: unmodified shortcuts can be used even if focus is in label track.
 +
** '''DRB:''' Also extemely helpful for users of screen readers.
 
*  Every new audio track creates its own label track underneath - requirement for DAISY Books
 
*  Every new audio track creates its own label track underneath - requirement for DAISY Books
 
* New label track(s) placed underneath selected track(s) - not at bottom of project (5 votes on Feature Requests)   
 
* New label track(s) placed underneath selected track(s) - not at bottom of project (5 votes on Feature Requests)   
** '''GA:''' This applies both to Tracks > Add New > Label Track and where CTRL + B adds a new label track. Best default behaviour needs to be considered, as well as if a Preference is required. We made behavioural changes here when linking was still available, which should be checked over. I had suggestions at the time for improvements when linking is on. I'll try and find those again from -devel.  
+
** '''GA:''' This applies both to Tracks > Add New > Label Track and where CTRL + B adds a new label track. Best default behaviour needs to be considered, as well as if a Preference is required. We made behavioural changes here when linking was still available, which should be checked over. I had suggestions at the time for improvements when linking is on. I'll try and find those again from -devel.
 +
 
 +
=== Making Label Tracks Accessible ===
 +
 
 +
Currently, label tracks are almost completely inaccessible to users of screen readers. They can create labels, which involves the focus moving to a label track, but then have to move away immediately so that they don't acccidentally create labels etc. Some of the funtionalilty of label tracks is accessible using the labels editor, but for some tasks this involves many more keystrokes than would be required if the label tracks were fully accessible.
 +
 
 +
The main problems and suggested solutions:
 +
 
 +
* When you navigate to a label (tab), then screen readers don't read the label. They should!
 +
 
 +
* It's currently too easy to accidentally create spurious labels. This could be fixed by having a preference for label typing, as proposed in the New Preferences section. When set to manual, you can't create a label by just typing the label, you have to do something first, eg press ctrl + b.
 +
 
 +
* It's currently too easy to accidentally edit the text of a label, since when you navigate to a label, you're automatically in edit mode. Possible solution:
 +
** when you tab to a label you're not automatically in edit mode.
 +
** to enter edit mode, press f2. (mouse users can also just click in the label )
 +
** to exit edit mode, press enter.
 +
** pressing tab to move to the next label doesn't take you out of edit mode if you're already in edit mode.
 +
 +
Other suggestions:
 +
 
 +
* A selected label could have a context menu, so that you could delete it, etc.
  
  

Revision as of 14:36, 1 March 2010

Proposal pages help us get from feature requests into actual plans. This page has multiple proposals for enhancements to label tracks.
Proposal pages are used on an ongoing basis by the Audacity development team and are open to edits from visitors to the wiki. They are a good way to get community feedback on a proposal.


  • Note: Proposals for Google Summer of Code projects are significantly different in structure, are submitted via Google's web app and may or may not have a corresponding proposal page.

Proposed Features

  • Better dragging of labels, so point and region labels can be dragged unchanged, while region labels can be dragged backwards on themselves like audio regions. Bugzilla:109.
Improved 31Jan10, with SHIFT + drag always moving point or region labels. See discussion
  • Labels in the waveform, not just on the label track
  • Multiple Selection of Labels
  • Better transcription support
  • RTL support
  • New Preferences
  • Making label tracks accessible

Dragging: Instead of pushing edges when they collide and becoming a point label, we swap edge being dragged similar to how audio selection drags work. The (undotted) circles would move entire labels. For joined labels the circle additionally gets a 'dot' in it to indicate that it is dragging edges, not moving the entire label. We also show that dot when hovering over the arrow part of an edge. We lose the ability to move two labels at the same time, which is a very minor loss. We gain the ability to move labels without a modifier key. We retain the ability to convert a point label to a region and a region to a point and to drag a boundary between joined labels without a modifier key. We also gain an additional visual cue that labels are joined. The way we explain it in the manual is that 'Empty circle drags entire label. Circle with a dot drag just that edge.'


Developer/QA Backing

  • Better dragging: Bill, James, Edgar, Gale.
Improved 31Jan10, with SHIFT + drag always moving point or region labels. See discussion
  • Better transcription support: Gale, James
  • Multiple Selection of Labels: Gale, James


N.B: James has a longer term interest in making labels good for 'structured audio' for language learners. That for example means ability to mark phrases and complete sentences for easy replay of 'last phrase' or to cause longer pauses between sentences. We'd also have different colour labels for different languages. Also ability to hyperlink the labels, so that on a smart player you could click a button to hear 'more' e.g. express the same idea with simpler words for a less experienced speaker. Converting labelled audio into a quiz would be another option. In the shorter term I'm interested by label tracks, but not seeing the bigger enhancements like labels on the waves themselves as a priority.


Motivation / Use Cases

Dragging

BW:

  • Point labels should have a circle handle and both triangle handles. Dragging the circle handle moves the point label. Dragging a triangle handle expands it to a region label. Done
    • GA: Tested on Ubuntu. However you can only expand a point label to a region label by grabbing the opposite arrow to the intended drag direction. So to drag a point label out rightwards, grab the left (and left-pointing!) arrow and drag rightwards. Grabbing the right arrow and dragging rightwards (which you would expect) drags the point label rightwards as is. I think this is unintuitive and unlikely to be a long-term solution.
  • Dragging the circle handle always moves the label. Region labels would retain their length. Use SHIFT and drag to shift the label
    • GA: That still leaves no way to move a region label "as is" with an unmodified drag. I don't feel that's acceptable longer term either.
  • After collapsing a region label, a continued drag should flip the label rather than moving the temporarily-created point label. You would have to change handles too?
    • GA: That depends how powerful the handles can be made. I think the handles are sub-optimal now (both point "inwards only" which suggests the user can't e.g. drag the left handle leftwards). Can the handles always display as they do for a point label (pointing both ways), and so each half is replaced when it over-rides as necessary? Functionally the handles can always be dragged in either direction, so the current idea makes no sense.
  • When a region label is collapsed to a point label it should always "light up" to let the user know this, similar to the snap-to-edges feedback.
    • GA: Currently, the label edge turns white when it snaps to the cursor in the track above (which happens whether or not the label is selected, and whether a point or region label). I previously thought this white line was what Bill was suggesting. I'm just about +1 on Bill's idea, though I think the colour would have to be neither white nor yellow so it was clear it had a separate meaning.
  • Alt-click (or option-click) on label text to move the audio selection to correspond to the label without opening the label for editing.
  • When moving or resizing a label, the cursor or audio selection should always change to match. Users will likely be trying to adjust the label to match something in the audio and this feedback would be very helpful. To have the cursor or region move, the label must be selected, which then requires ENTER or arrow to play the region.
    • The user may not always want this to happen, so perhaps alt-click on a handle moves the cursor/selection to that point.
      • Alt-click on a circle handle moves the cursor to that point
      • Alt-click on a triangle handle in a region label selects the region
      • Alt-click on a triangle handle in a point labels behaves like a click-and-drag in the waveform
        • GA: Having to click in the label to make cursor or region move with the label does however provide alternative behaviour, and ALT + Click isn't very discoverable. I think I'm -1, but I would support a *temporary* region or cursor in selected tracks while you hold the mouse, whether you click in the label or not. Is it too advanced for Widgets?

GA:

  • Where labels are joined, the circle has a special function (as now) of adjusting the labels within the joined area.
  • JC had suggested a workround within the current method: add an option where you can shift-drag or control-drag a label edge and preserve the label size. This would also give a one-drag way to drag a 'point' label left - by shift-dragging the left edge. Because drag modifying isn't always discoverable, I prefer arrows drag, circle moves.

James:

  • I don't want to lose the special function of the circle. I very much like that the same draggable label widgets can be used for regions, for points and for boundaries between regions without the user having to insert/delete/change three different kinds of 'thing'.
    • GA: We still have a (to me) unsatisfactory situation where circle does different things according to context. So I probably ought to contradict myself and suggest that circle should always move labels (including joined ones) as is. Consistency is needed here for the majority cases. I think doing things with joined labels may be a fringe case, so we just find some other way to adjust labels within the joined area. If we can make the handle to always have both-directions-arrows, which I think is desirable, then the arrows could be surrounded by a circular widget when handles are at a label join. Drag that widget which only appears when labels are joined to adjust labels within the joined area.
      • James: It is already consistent. Dragging a circle drags multiple edges, if they are co-incident. For a point label the left and right edges are co-incident. Updated proposal moved into proposal section.
      • James: Joined labels is not a fringe case. In labelling vocals you want point markers, but the text is for the regions in between these markers not for the points themselves.
      • James: +1 on auto-swap over of edge being dragged. The pushing behaviour was good for shifting labels, and good for dragging a region down to a point, but with other ways of moving pushing becomes less valuable. Getting auto-swap right so there is no jump at the swap-over is a little fiddly, but should be quite doable.
      • James: -1 on making double headed <0> draggers at both ends of a range label. It would not work well for joined labels. More direction neutral draggers like |0| for a point label would be possible, with a range label being 0|----Label----|0. The '|' could be a somewhat compacted <> I guess.
  • I don't mind that shift+drag is less discoverable. We should add it to the mouse preferences list because that is one way we have alerted users to what different kinds of dragging can do.
    • GA: I agree with adding that to Mouse preferences, but contend that most users won't find it, so there should always be a consistent way of moving "as is" with unmodified drag, as proposed here. Label handles are not really a common piece of interface like a slider where a user may figure to modify the drag.
  • Having labels 'cross over' as selections do could be good, and helps improve consistency. It does make point labels a little less discoverable and harder to create. I would like to make this a user preference something like 'Label boundaries can swap when dragging' (default off).
    • BW: So what happens when a user collapses a region label to a point and continues to drag? I don't understand how this could make point labels harder to create or less discoverable. A point label is automatically created when the audio selection is a cursor and the user choose Add Label at Selection. If, during collapse, the point label lights up (and perhaps has a snap-to "slop" setting (not a preference!)), point labels would be easy to create from region labels. But I really have no idea how often someone would want to do this. I just found it odd that after collapsing a region label to a point that the resulting point label would be moved, especially since I'm "holding" the triangle handle, not the circle handle.
    • GA: I can see what James means (collapsing to a point then moving that point would require an extra step of stopping when the point lights up, then switching to dragging the circle). But the confusion with the mixed messages the current widgets give you - plus the fact that if you select a region label then drag the region you can flip it - makes me convinced that Bill's approach is better.
  • I am considering an option that allows users to choose either <0> point labels or >0< point labels. It is really just 'graphic design'. Or that could perhaps be an outcome of the theme preferences.
    • BW: In terms graphic design, in the current version the triangles hug the circle quite nicely. But in terms of reflecting what the handles do, the inward-pointing arrows on region labels infer that dragging them will moves the edges "inward".
    • GA: -1 on options for handle direction at a point label. It's correct now (two handles, pointing outwards). It's wrong, as soon as you drag either label, and I know this confuses users.
    • SD: +1 for 'flip' labels rather than 'push' label when left edge moves past right edge and vice-versa. The handle would have to also flip, which would create a small discrepancy between the mouse pointer position and the handle position but I think that is fairly minor. More importantly it would solve the issue of having to grab the left handle on a point marker to extend the marker to the right as a region, which is totally counter-intuitive.

Labels in the Waveform

  • (from Feature Requests - 29 votes: "Ability to drop vertical marks that are 'glued' to the waveform on the main track (not on a label track). They should move when when track is moved, and also be adjustable and label-able if required. Should be possible to snap a region to the markers. This is an extremely useful feature available in Sound Forge and most other editing programs. (For most editing purposes, the new split function now works in lieu of markers on waveform - thanks - but this would still be nice.)"
    • BW:
      • Would this replace linking?
      • I use the split function for this purpose, however it seems from comments on my "splitting recordings" tutorial that many users do not understand that splits can be used for this purpose.
    • GA: I think split lines are not adequate replacements for markers in the waveform (if we think we need those) because they have a merge function and disappear if you click too close to them.
    • JC: The 'marks glued on the waveform' I see as just a display mode for labels. It is, if we get linking 'right'. So I would see users as being able to toggle the label display between showing in tracks and showing as an overlay.
      • GA: +1, though I suspect some of these votes may be actually for a mark-in/out feature which is fairly incompatible with labels...
    • BW: This has given me a totally new idea for indicating linking. When linking is on, the label lines extend up into the linked track(s).
      • GA: But only when user has this feature enabled, and if they have a label in a label track?
      • BW: Yes. Like this:
        LinkedLabelledTrack.png

Multi-select

  • 6 votes on Feature Requests
  • Allowing users to select many labels at a time would allow them to apply effects, copy or delete several regions very quickly. Users could control-click to add labels to a selection, or shift-click to select everything from the start of one label to the end of another.
    • GA: We must remember to move SHIFT + click for adding to selected tracks to CONTROL + click.
    • GA: Would "Play" play all of the selected areas in the audio file, skipping the non-selected regions, as asked for on Feature Requests? "This feature would be useful for comparing two regions of an audio file without delay between the playback of the two (or more) labeled regions."
      • James: If/When we do it, it should.
  • Also there could be commands to select multiple label regions, to invert the selection, or to ignore specified labels. See extra detail here.
  • Should work on point labels, too, thereby selecting regions between labels. (3 votes on Feature Requests)
  • GA: Also a couple of votes on Feature Requests for selecting multiple (by implication, unlabelled) regions.

Better Transcription support

GA:

  • Labels won't accept any more text input once two factors are true: (1) the text reaches the right-hand end of the screen (2) the audio track is not sufficiently zoomed in, leading to there being no scrolling region remaining. This is the same whether a track is playing or not. This limits use of labels for transcription. There are 5 votes on Feature Requests for finding a solution
    • JC: (from Feature Requests) Correct this by adding multi-line support and linking to text editor?
    • GA: This situation is slightly better now we allow more space after the end of the track, but you can still be limited to only 20 characters or so in a label right at the end of a zoomed in track. I suppose it means setting a maximum supported characters per line, then allowing end of project to move if the end of the label line requires it.

RTL support

  • Support for right-to-left languages, selected in label drop-down menu

New Preferences

  • Add a Preference for "Label typing:" "Standard" (CTRL + B only required for typing in a label track if cursor is at same place as an existing label - this is current behaviour) or "Manual" (CTRL + B always required). Use case: unmodified shortcuts can be used even if focus is in label track.
    • DRB: Also extemely helpful for users of screen readers.
  • Every new audio track creates its own label track underneath - requirement for DAISY Books
  • New label track(s) placed underneath selected track(s) - not at bottom of project (5 votes on Feature Requests)
    • GA: This applies both to Tracks > Add New > Label Track and where CTRL + B adds a new label track. Best default behaviour needs to be considered, as well as if a Preference is required. We made behavioural changes here when linking was still available, which should be checked over. I had suggestions at the time for improvements when linking is on. I'll try and find those again from -devel.

Making Label Tracks Accessible

Currently, label tracks are almost completely inaccessible to users of screen readers. They can create labels, which involves the focus moving to a label track, but then have to move away immediately so that they don't acccidentally create labels etc. Some of the funtionalilty of label tracks is accessible using the labels editor, but for some tasks this involves many more keystrokes than would be required if the label tracks were fully accessible.

The main problems and suggested solutions:

  • When you navigate to a label (tab), then screen readers don't read the label. They should!
  • It's currently too easy to accidentally create spurious labels. This could be fixed by having a preference for label typing, as proposed in the New Preferences section. When set to manual, you can't create a label by just typing the label, you have to do something first, eg press ctrl + b.
  • It's currently too easy to accidentally edit the text of a label, since when you navigate to a label, you're automatically in edit mode. Possible solution:
    • when you tab to a label you're not automatically in edit mode.
    • to enter edit mode, press f2. (mouse users can also just click in the label )
    • to exit edit mode, press enter.
    • pressing tab to move to the next label doesn't take you out of edit mode if you're already in edit mode.

Other suggestions:

  • A selected label could have a context menu, so that you could delete it, etc.


Some links

BW:

  • The screenshot of Audition is hard to read, Ardour is easier. I've not used either program but it appears that the "labels" in these programs are naming what we would call "clips".
  • The GarageBand labels seem to merely carry the name of the track.

Past Discussion

There were some inconclusive discussion of this on the page, deleted in Jan 2010. Some issues touched on:


  • Where to add Label Tracks With the current way of defining track groups, the 'where' of where to add new label tracks becomes important.
  • How to show what linking will do There was some discussion of using two-tone selections to show what would be affected by a linked-operation. One possibility is to always select the same on all tracks in a group when in linked mode.


Other possibilities from Feature Requests page

Edit Hint: This section is 'indicative' of feature requests and voting and provided for convenience. The definitive version, particularly of current voting, is on the Feature Requests page. Don't expect this section to stay up-to-date.

Feature Requests as at 31 Jan 10 also show the following:

  • 8 votes for Mark-in and mark-out points: As per video editors, "I" sets Edit In Point and "O" sets Edit Out Point. Should use zero-crossing accuracy. Doesn't matter what you do between those two actions; once you press "O", everything between "I" and "O" is selected.
  • 7 votes for "Ability to import/export cue sheets for CD burning (and/or text file compatible with shntool) to and from labels"
  • 4 votes for "Split to multiple projects: Split the project into smaller multiple projects by labels, similar to the "Export Multiple" command. Each project retains the per-track labels of the original single project."
  • 3 votes for XML/SMIL Label Export for using Audacity to analyze interviews or to generate Audio-Picture-Slideshows e.g. for educational use). Nodes: "trackname" with Nodes "start, end, label" or using SMIL.
  • 2 votes for "Add Next and Previous buttons for navigation between labels to the panel of the Label Track."
  • 2 votes for "Automatic snap-to CDDA boundaries when exporting multiple with labels: The FLAC encode actually has an option to do this."
  • 1 vote for ODF Label Export with the timecode of each label in an ODF document. JC: We already have CSV. Already easy to get ODF.