Talk:Completed Proposal: Improvements to Scrubbing - Phase-2

From Audacity Wiki
Revision as of 17:28, 13 August 2016 by James (talk | contribs) (Clarified, and answered Gale.)
Jump to: navigation, search

Bug 1423 - a familiar scrub

I think there are better solutions to the problem of doing a single scrub or single seek than the solution proposed in Bug 1423. Swapping the gestures will just mean that seeking (rather than scrubbing) now has to be explicitly cancelled, e.g. with ESC.

Peter 12Aug16: or more properly stopped by use of: Stop button, Transport>Stop command, Spacebar shortcut or pressing the active Seek key.

The design problem is that there are FOUR modes:

  1. - A single seek session which ends.
  2. - A single scrub session which ends.
  3. - Alternating seek-and-scrub sessions, which can end.
  4. - Alternating scrub-and-seek sessions, which can end.

Instead of having both a seek button and a scrub button we could have a single button that is either seek or scrub. A drag that starts in the scrub ruler (i.e the strip under the QP timeline) initiates scrubbing or seeking depending on the state of that button. When we mouse up, the scrubbing or seeking ends. This gives us 1 and 2.

Peter 12Aug16: I'm liking this single button idea. But it does mean that the Scrub Bar must be present for Scrubbing or Seeking to be available. Personally I like that - but Gale was adamant that he wanted to be able to scrub without the scrub bar, having the scrub widget run in the Timeline. I much prefer the total separation of the Scrub Bar and the Timeline. But remember you still need a mechanism to hide/display the scrub bar - unless that is you always have the Scrub Bar and it is not dismissable.
  • Peter 13Aug16: Overnight thinking:
    1. The logical place for such a single button would be the Transport Toolbar (or later as one of your multi-buttons)
    2. We could dispense with the Scrub Bar altogether - when Scrubbing/Seeking is active then the Timeline belongs to the Scrubbing and Quick-Play is inhibited. When Scrubbing/Seeking is inactive then the Timeline belongs to Quick-Play.
  • James 13Aug16: There are many possible choices. I don't think that button merits being Transport Toolbar button in size. I'd have preferences "enable scrubbing", "enable seeking". If either are enabled, the scrub ruler (i.e. the strip under the QP timeline) shows. If both are enabled, the scrub/seek button shows too. If the scrub/seek button is present, it's on the timeline, beside the pinned/unpinned button. There is room. The scrub toolbar (the one that held three buttons) is gone, and there's no little ruler button anymore as the scrub ruler presence is controlled by the "enable scrubbing" and "enable seeking" preferences.
    • Peter 13Aug16: Hmmm, preferences like that would work a treat for me. Would you have the default for both be "off" or "on" - I would choose "off" - but probably we want "on" for discoverabilty. It would be good to have a right click menu on the buttons to be able to turn them off too I'm thinking. I like the idea of putting the button in the Timeline above the TCP alongside the pinned/unpinned head button. I'm not clear what you mean by "scrub time-strip shows" is that just the Timeline (and in which case with scrub or seek enabled then Quick-Play would be inaccessible) or is is something new? Or is it the Scrub Bar by a different name just managed by the presence or absence of the Scrub/Seek buttons, with no separate Scrub Bar button (I like that idea).
    • Gale 13Aug16: 1423 is P4, so not mandatory to address it now, though the reason it's there is that some (many?) users will never buy into our scrubbing unless a technique familiar from other apps is available. The suggested changes seem quite major. If no Scrub Bar, how do we give people something tactile to drag? I understood from Paul that people may want to switch on the fly between seeking and scrubbing, so that might be a consideration. I'm really asking if this is for 2.1.3 now, in which case I'll have to read and think about this properly, and probably bug 1423 should be P3 and invite some sort of user feedback in the release note?
    • James 13Aug16 Gale: These ideas (single scrub/seek button etc) are NOT for 2.1.3. Peter: I've clarified wording.

A shortcut key can change the state of the button, and we can do that in-flight. This gives us 3 and 4. We also have a preference option that allows mouse-up-and-down in the scrub ruler to change the button state from seek-to-scrub or vice versa. That gives another way to switch modes in flight using mouse, which might be more convenient. When we eventually finish such a sequence (e.g. by mouse up outside the scrub ruler) the scrub/seek button reverts to our preferred starting state (seek or scrub).


With this scheme (as described so far) we never have to ESC out of a scrub or seek. In my own flow I would probably be only using seeking, and it would work well for that. The design does also support those users who want to alternate scrubbing and seeking. I think this is enough. I don't think we need more (the RSI free variant).

Peter 12Aug16: the ESC is explicitly provided as the one way for the user to retain their previous cursor position or selection (Gale explicitly wanted to be able to do that) otherewise stopping scrubbing/seeking leaves th cursor at the current scrub position (many of us wanted that).


Gale has in the past stressed the importance (to RSI) of scrub (item 3) being possible without mouse down. I personally disagree. I think the mouse up modes are unhelpful, and they run the risk of trapping users in a state which they might not know to SPACE or ESC out of (though the stop button will help). However we can do the RSI free version too:

For RSI free operation, we have a preference. If set, you have to click and release in the scrub bar to initiate scrubbing or seeking. Now scrubbing or seeking will continue until you explicitly end it, e.g by ESC or a click outside the scrubbing bar. You can switch from RSI free seeking to RSI free scrubbing (or vice versa) using the state shortcut, or by clicking-releasing in the scrubbing bar.

This scheme puts scrubbing and seeking on the same footing. It is easy to do one-off seeking or scrubbing. It is easy to chain them, starting from either. RSI free operation (if it is wanted) is available for both seeking and scrubbing, not just for one or the other.

Peter 12Aug16: Steve wrote last year (see below) "Moving the mouse pointer stretches a "rubber band" between the indicator position and the current mouse position, The length of the rubber band indicates the playback speed, When short, playback is slow. When long, playback is fast."
  • Extending that idea we could combine scrub and seek into a single gesture whereby when short playback is slow when long the scrub changes to seek rather than just a pure speed change. You can't make sense of scrubbed audio when going faster that 2-2.5x at best.
  • And I do like the idea of having to hold down the left mouse fror scrub/seek and that relealsing it then stops the scrub/seek - nice and simple, no chance of "getting stuck".

Peter's email of 29Sep15 (an excerpt)

Paul wrote:

>If it is not then obvious that you can stop scrub play just as you stop other play, I don't know what would make it more obvious.

There are a couple of things that would make it "more obvious" and they are not mutually exclusive

1) You ask: "do we need some other interface for initiating scrub that you are less likely to do by accident?" My answer is "Oh yes we do" - and we on QA have been asking you for such for quite a while now.

As we already know from a posting on the Forum we have already had one poor benighted user who stumbled into scrubbing mode by mistake and then couldn't find his /her way out.

The problem is that the interface for moving into scrubbing is not GUI, i.e. it is not graphical, so there is nothing to see. What is needed is some form of graphical interface for moving into scrubbing.

This could either be an additional Tool in the Tools Toolbar for scrubbing which would place you in Scrubbing mode, Steve has suggested this approach I seem to recall. For me this is sub-optimal as I strongly think that scrubbing should be independent of the currently selected tool (just as Play and Record are). Also it would imply that to escape from scrubbing the user would need to choose and alternative tool from the Tools Toolbar. And fundamentally Scrubbing is not a tool, rather it is a form of playing. So I do not support this approach.

So what I would really like to see is an additional button on the Transport Toolbar for Scrub-play (scrubbing brush icon? maybe superimposed on a green Play icon?). This should make it clearer that the user should consider using the Stop button in the same Transport Toolbar to stop scrubbing. And btw you still need to fix the problem that the Transport Toolbar's Pause button and the Transport>Pause menu command are not grayed out as unavailable while scrubbing is active.

And if we do add such a button then we should probably also add a "Scrubbing" or "Scrub-Play" command to the Transport menu.

Apart from any other consideration I suspect that you will find more folk using scrubbing if there was a more visual method of entering scrub-play (and I'm sure that given all your work on scrubbing you would like to see that).

Steve's email of 29Sep - how other audio editors do scrubbing

In Protools, scrubbing is switched on by selecting the "Scrubbing" tool. It has a loudspeaker icon. Scrubbing is then performed by clicking on a track and dragging left (scrub forward) or right (scrub backward). Scrub play stops when the mouse button is released. When the scrubbing tool is selected, scrubbing may also be controlled with the "J" and "K" keys. "Jog wheel" control is supported when using an Avid control surface (nice, but expensive).

Soundforge uses a slightly different approach, but with similarities. In Soundforge, each audio track has its own window, and at the bottom of each window is a mini-transport toolbar. The mini-transport toolbar has an additional button as the final item, which is the "scrub play" button. When the scrub play button is pressed, scrubbing is performed by clicking on a track and dragging left (scrub forward) or right (scrub backward). Scrub play stops when the mouse button is released. When the scrubbing tool is selected, scrubbing may also be controlled with the "J" and "K" keys.

Sweep on Linux has another variation. Sweep has a scrub tool button. When selected, the cursor changes to the scrub icon when over the track panel. The cursor has the same design as the icon on the scrub tool button. Scrub play starts by clicking on a track. The Play button immediately changes to "button down". The difference between the Sweep behaviour and the other two examples above is that releasing the mouse button "lets go" and the track plays from the current position at normal speed. To stop playback, either click the stop button or press spacebar to pause, or the return key to stop. In each case the playback cursor stops at the current playback position. Sweep also has a "playback" menu in addition to the transport toolbar.

Ardour 3 has yet another variation. In Ardour 3 there are also many "advanced" options for scrubbing, but the basic operation is as follows:

Select the "scrub" tool to enter scrubbing mode. The cursor changes to a loudspeaker icon when over a track. Click and drag on the track to start scrub playback. A bar under the transport toolbar shows the current playback speed and direction. The direction and speed of playback may be controlled either by dragging in the required direction on the track, or by dragging the pointer in the bar under the transport toolbar, or from the keyboard. When dragging either on the track, or in the bar below the transport toolbar, playback stops when the mouse button is released.

Of all the methods described above, Ardour is my favourite. It is very clear what is happening, highly intuitive, provides immediate visual feedback and has "advanced" options for the more experienced user. Someone has clearly spent a lot of time and effort getting this right. It also supports "jog wheel" if the user has a compatible control surface. The only thing that I'm not so keen on, is that keyboard control is less easy than some of the alternatives.

Steve's email 02Oct15

As I understand it, the purpose of scrubbing is to enable the user to locate a specific point in time for editing. Perhaps I'm not using it the way it is intended, but for me, I find the exact spot that I want to edit, the playback cursor is exactly in the right place, but as soon as I stop scrub playback the cursor position is no longer there - it has jumped back to the previous playback position.

OK, so "I" know that there are some special key bindings available, but I've been using Audacity for over a decade.

Shouldn't the default behaviour be that when the user stops scrubbing, the playback cursor remains at its current position?

What should happen if there was a Selection when scrub mode was started?

Peter 02Oct15: To be addressed in Phase 2.2

Paul's email of 29Sep15

I am trying to assimilate the criticisms. I don't feel a consensus yet about what the better interface would be.

I) The non-immediacy of appearance change when you enter scrubbing is a problem but it is not the "stuck in a mode" problem. It is the easier thing to fix than the user interface redesign. II) James' "stuck in a mode" perception results from (a) selection change not working during scrub and (b) stopping when the play head catches up to the cursor.

How much of this perception results from each? I feel more concerned to fix (a) so that scrub play is no more restrictive of other actions, than is ordinary play. But (b) seems inherent to scrubbing, if the purpose of scrubbing is to find some precise place so that you can stop and put the cursor there. Unless you would rather have a play-head with "inertia" when you do not move the mouse. I don't know whether that will feel better in practice.

Meanwhile let's remember the scrubbing features we have. Binding each to some gesture crowds out the use of that gesture for some other purpose and that means tradeoffs in designing the user interface. Are the features all important enough to keep? Do we sacrifice any of them?

1) We have the scrolling version of display in which mouse position relative to the center affects the speed of scrubbing or the length of skips for seeking. You choose that variation or not when scrubbing begins. 2) We can switch between a stuttering seek and variable speed play during scrubbing. Variable speed simulates tape head scrubbing and can even go backwards. Seeking plays always at normal speed but with frequent skips. 3) If not seeking, then we can accelerate or decelerate the maximum speed during the scrub.

And to remember the user interface choices for the above.

1) ctrl-double-click in track panel to start scrolling scrub, ctrl-single-click for the ordinary scrub. 2) Hold left mouse button for seeking, release it for scrubbing. 3) Roll the mouse wheel.

Now let me remind you of tradeoffs that led to these decisions.

If mouse wheel is best for number 3, that is one reason why WE DO NOT WANT SCRUB TO BE A DRAG WITH A BUTTON DOWN. (An additional reason might be, we want to allow selection changes during scrub.) I think left button while rolling the wheel is just too awkward (two fingers at once, rather than just using the index finger for one or the other), and remember that when I used middle drag, Steve did not like rolling the wheel while also pressing it.

If seeking happens with left button down, that is all right because the speed determined by the wheel is irrelevant, so there is no conflict. But it is agreed that this interface is not yet the best. THIS IS ALSO WHY YOU CAN'T SELECT DURING SCRUBBING, part of what James says he means by "stuck in a mode," because certain familiar actions (selection changes) are excluded. Other alternatives were considered and rejected for their other problems. Making a shift or ctrl modifier key do it would interfere with the use of certain useful keyboard commands during scrubbing, like ctrl-m to drop a label or shift-a to stop and set cursor.

So please suggest an alternative to the left button. Perhaps we should require you to click and drag the green triangle in the time ruler (which is not part of the track panel proper). But then clicks and drags in the tracks would be free to change selection during scrubbing.

  • James:. Yes, why not? What's wrong with doing that?

Another problem is that SCRUBBING WORKS ONLY IN SELECT TOOL. I am not pleased with that. It should work at least in multi tool too. Why not indeed in all tools? But the reason for this was choice 1. Scrubbing must be started by some mouse action. The choice of ctrl-click conflicts with things like time shifting in the multi tool.

What is a better choice for starting a scrub then? A click in the time ruler instead? There are already several mouse actions with meaning in the time ruler. Is anything available for scrubbing to use?

Finally Steve likes the idea of a button for a scrubbing mode. But James dislikes the multiplicity of tool buttons that we have already. He would rather have graphical handles to click on to access different functions. Also in another project, spectral selection, I deliberately avoided making a new button and a new mode. I still think it is better to accept scrub as "another kind of play" that is stopped in the same ways play is stopped, and you don't need finger pressure anywhere to continue in it. But during that play, as with ordinary play, certain other mouse actions and keyboard commands remain useful.

And don't forget ctrl-mousewheel to magnify -- I like it too that this remains available during scrubbing. But again it would be awkward if you had to keep the left button down.


James (talk) 17:00, 2 October 2015 (EDT)

Phase-1 is mostly about the 'mode' and not being stuck in it. Phase-2 suggestions here may improve discoverability and ease of use.

Possible Phase 2 changes.

  • Scrubbing and seeking treated like play, loop-play, play-excluding are. The mode is initiated by toolbar button or bound keyboard shortcut.
  • Ctrl, shift click etc become user bindable, in same way that keyboard shortcuts are. Possibly unbound by default.
  • Scrubbing/Seeking could be started from a multi-button
  • Radical change where:
    • Play no longer resets cursor to start
    • Scrubbing is done by dragging the play cursor. It only happens whilst the mouse button is down and dragging. Play cursor WILL lag behind desired drag target.
    • Single scrub-seek mode rather than a scrub mode and a seek mode. In scrub-seek you scrub, if not too big a change, and seek if distance is above some threshold.
    • Can scrub-seek when playing or when not. You could start playing at x1, then scrub-seek by dragging and continue playing at x1 after you stop dragging.
  • Fixed-cursor scrolling-wave becomes an independent feature. One of the tick boxes in the multi-buttons.

The above Phase 2 ideas are NOT prescriptive and some are mutually exclusive.

James (talk) 08:08, 5 October 2015 (EDT)

If we are dragging something tangible rather than just moving the mouse the need for ctrl (or shift or double click) modifier goes away. Then the argument for restricting scrubbing to the selection-cursor mode, which Paul has done but is not keen on, goes away. The main remaining interface problem from Paul's perspective is then, I think, that he really wants to at least have the option to use the mouse wheel to set a speed. Since mouse-wheel and drag has been found to be troublesome when used together, that seems to require that scrubbing and seeking can operate with mouse up.

My picture is that the mouse x-y and the mouse wheel-roll are valuable resources. The tools-buttons are a clunky way of sharing that resource. Draggable tangible things are a better way of sharing. We need a way to share wheel-roll too. I don't have a precise proposal here. The outline idea is that mouse-wheel clicking on a slider binds the wheel-roll to that slider. ESC (or another mouse-wheel click) cancels the binding. So we could bind to the scrub-speed slider, the play-at-speed slider, or the pan slider, or volume slider, if we wanted to and vary those whilst playing.

James (talk)

If, as now, moving the mouse without any dragging can change the playback speed, then in my view we have a mode. It is possible to be stuck in that mode and not know what is going on. I thought we had agreed that we do not want scrubbing to be a new mode?

  • Gale 08Oct15: I see in "2.1)" that "It should be possible to enter scrubbing mode via GUI." So if does not look to me it is agreed we do not want scrubbing to be a mode. There are benefits in reduced RSI in not having to drag and hold, but those benefits are IMO thrown away by the current implementation.

So I wrote:

"2.2) IF mouse position is used, THEN there should be some tangible widget to be dragged."

(note the CAPS). This has been changed to:

  • Gale 08Oct15: James changed my "Should?" question to a statement that I could not understand out of its context. I did not want to be represented as making that statement.

"2.2) If mouse position is used to drag playback, there could be some tangible widget to be dragged. However this goes against 2.5) which seeks to make the drag less physical effort. Therefore an alternative could be to have the playhead widget not as the dragger but only as the up (off) /down (on) control to initiate scrubbing. If there is no drag widget then some option is needed to keep scrubbing alive when the playhead reaches the pointer."

Surely the alternative is then a mode? The only way it is better than the existing stuck-in-mode problem is that there is better signalling of the mode, which might give more clue to a new user as to how to turn it off again!

  • Gale 08Oct15: I think being stuck by accident in scrub in much less issue than our showing how to scrub for those that want to use it, and IF we have a scrub mode, giving a clear indication that it is on.
    • Peter 08Oct15: I think both are important - but I think if we fix the UI properly "showing how to scrub for those that want to use it" then that will of itself lead to fewer folk getting stuck in scrubbing without realizing it - and provide them with a clearer, more discoverable, method of escape. Those are the key challenges for this proposed redesign of the scrubbing UI: discoverability, usability, escapabilty.

The 2.5 interpretation seems to imply that moving the mouse without dragging can change the playback speed, and so if we accept 2.5 it seems we must have a mode.

  • Gale 08Oct15: The standard single-click scrub (the only one the user is likely to discover) does not really change playback speed even when dragging. The scroll wheel usage to actually change playback speed doesn't even work when dragging. All drag does is to jump standard playback like holding down period or comma (optionally with SHIFT) do. Double-click scroll scrub does change playback speed when merely moving the mouse but that seems unintuitive to me.

    The scroll wheel speed change is unexpected and undiscoverable, though *very* nice if you do discover it. It is more flexible than a dragged widget that changed playback speed would be. If we kept scroll wheel speed change, I would like to see at least a temporary mouse pointer when entering scrub mode that showed a scroll wheel with + above it and - below it.

    Question: Is the Audacity audio engine good enough to really change playback speed when dragging a widget, so that playback speeded up and slowed down according to the speed of dragging the slider, as opposed to the on/off "stutter play" which drag does now?

I am rather strongly against modes of this kind and think avoiding having a mode is more important than the extra effort of holding a mouse button down to drag something tangible. IF we go for no mouse down THEN we need a very good way to signal how to get out of the mode. I'm -1 on the mode. If we did go for a mode a line from the mouse cursor to the widget that allows us to turn off the mode would help users get out of the mode.

  • Gale 08Oct15: Again, users getting trapped in the mode is the lesser of scrub's issues. I thought no-one wanted a draggable widget except James, as per "2.1".

    The draggable widget would be the most intuitive solution for users who don't use pro DAW's (i.e. I guess, most of our users). I would support a drag widget but we have lots of great, undiscoverable functionality in the current implementation. My ideal would be a simple, drag-hold, no-mode widget, but also add some other control on the play cursor to enable more advanced "mode" features that perhaps hide the drag widget. Perhaps that control might open a dropdown menu where I could choose a mode that does not stop when the playhead reaches the pointer, to replace and improve on Timeline Quick-Play.

    All that said, should we be looking at what the big boy DAW's do? Is what we have now really like the big boys? What does their "jog scrub" do?


James' More detailed Proposal

Here is what I think my ideal would be:

  • Scrubbing: We can drag the play cursor triangle (using left mouse) at any time, whether we are playing or not. If we are playing, Audacity continues playing when we release. I regard this as sufficiently modeless, in that users are only 'stuck-in-the-mode' whilst the mouse is down.
    • Details: The cursor triangle changes to a green double-headed arrow, with a tooltip "Drag to Scrub" when we hover over it. The fact we can still drag whilst playing isn't very discoverable, but I think people who want to may actually try it. If the drag distance is small enough the speed changes (up to + or - x3). If the drag distance is too large we jump/stutter-play. There is some smoothing of the cursor speed, so the mouse movement can be a bit jerky but the playback is still smooth. That means we're dragging a 'ghost' cursor which the real cursor is trying to keep up with.
  • Right-Only Scrubbing: We can drag the left edge of the selection. When it bumps the play cursor, this behaves very like dragging the cursor triangle does, except that the cursor only bumps left to right.
  • Left-Only Scrubbing: We can drag the right edge of the selection. When it bumps the play cursor, this behaves very like dragging the cursor triangle does, except the cursor only moves right to left.
    • Details: In loop-play mode we loop rather than bump, if dragging the right edge would bump. In normal play mode, if we catch up with a right selection edge that is being dragged, but not actually being moved, we don't exit play mode. That only happens when we release the mouse. We need to do this or Left-Only scrubbing would often unexpectedly end playback. There are also some tweaks needed to how the cursor behaves at other times. In current Audacity we reset the cursor to start of selection when making a new selection and at the end of play. I'd like to change that, if we support Left-Only Scrubbing.
  • Vari-Speed Play: The variable speed slider now also supports negative speeds and is real time. Clicking on it binds the middle mouse wheel to it. ESC or clicking on some other slider, scroll bar or magnify-icon, ends that binding.
  • Seeking: We can hop the play cursor to a new point, even whilst playing, by clicking the new position (on the timeline). If we're playing when we do this, we continue playing, even if we are now outside the selection region.
  • Continuous Scrolling Wave: An option for 'play' in which the wave scrolls past continuously as it plays and (if not dragged) the cursor stays in a fixed position.
    • Details: To be discoverable and avoid stuck-in-a-mode this requires multibuttons. To move the cursor and wave together we can scroll, as now. The cursor could also drift back to the screen position it was dragged from over time. When dragging the cursor there could/should be some gearing, so that as I drag the cursor right the wave also moves left so we can scrub more than a screen's width. More important at high zoom.


Claims:

  • Scrubbing is more discoverable, mainly due to the cursor change and tooltip.
  • No stuck in mode. We exit scrubbing on mouse up. Mouse-wheel users could be stuck in a binding, but they will be out of it with just about the first thing they try.
  • Functionality. What you can do as an expert user is at least as good as the existing scrubbing for an expert user. Partly this is because the features combine. You can now do scrubbing with/without play. Play (at vari-speed), scrubbing and seeking are now orthogonal features rather than mutually exclusive.
  • RSI: The vari-speed mode is as RSI free as before.


  • Gale 08Oct15:
    • I was not able to understand what the right- and left-only scrubbing does. Perhaps this is because I missed AU14. This is not about left and right mouse button, is it?
      • James: not about mouse buttons. Right-Only-Scrubbing is a bit like scrubbing without reverse play. Instead of dragging the cursor you drag the left edge of the selection (you need to be playing a selection). Only when the left edge catches up with the cursor is the cursor pushed right by it.
    • Are we going to implement centered play cursor? If not, a possible benefit of scroll-scrub will be lost, and I don't see how we could grab the playhead if we are zoomed in such that the playhead is moving rapidly.
      • James: Added. Grabbing the moving playhead at high zoom is indeed impractical. But in such a circumstance scrubbing would only make tiny differences to the play speed anyway.
    • Dragging the playhead without modifying its appearance after pressing Play or SPACE may not be that discoverable. Can we also click and drag the green vertical line? Discoverability is further reduced if the playhead is moving fast. Are you sure we don't want a conventional drag bar whose handle is fairly static irrespective of the zoom level?
      • James: As soon as you hover over the playhead it changes appearance, whether or not we are playing. (But not if recording). I was not going to have the green vertical line draggable, but could do. A conventional drag bar is more discoverable. If someone else makes a detailed proposal with a conventional drag bar, then I might be just as happy with their detailed proposal.
      • Gale: If the aim is discovery of a dragged widget, then a conventional drag bar seems best. For example after importing a file, so fully selected, but not playing it, where do we see the green triangle that we drag to start scrub playback? A drag bar lets us "leave Quick-Play alone" as Steve requests. I think your proposal is currently vague about the impact on Quick-Play. How do we do the current drag of a Quick Play region - drag below the playhead?

        I think instead we could have a widget to drag on the green vertical play line where the widget appears when you hover the mouse over that line. We can't start a scrub without playing first but that is a standard limitation for a player app.

        I think the mistake is rolling all features into a draggable no-mode widget especially if you put it in the Timeline. I think there is room for a modal full-featured scrub that works in all tools as well as a simple and conventional drag-widget. My only proviso is that you can see what to click to turn the modal scrub on and you can see you are in that mode.

      • James: I am not so keen on something as wispy as a cursor line being draggable, though if an icon appears when you hover over it as you suggest it could be OK. A solution that avoids being in the timeline is to add a second inverted triangle at the foot of the play cursor line, and use that for the new functionality.
      • Gale: If so, then for projects taller than the vertical scroll, the second triangle would have to be e.g. at the bottom of the last fully visible track. It might also seem a bit counter-intuitive that you can drag one inverted triangle but not the other, so the one you can drag would have to look different before you hover over it, to encourage dragging it.

        If we are moving towards the thing to be dragged only being visible when playing, a drag bar would equally suit the purpose. I'm not able to make a convincing case for a drag bar yet, given we have three other sliders already, one of which is a play-at-speed slider. I don't know if the thing to be dragged could be superimposed on or placed inside the horizontal scrollbar, but that kind of looks a "logical" place a user might look for it. If the drag bar was there, its length would represent the length of the entire project, and its handle would show the play position relative to that entire length.

    • Playhead dragging seems to imply Timeline Quick-Play "click to play" is removed, yes? Otherwise I can envisage lots of confusion when the smaller Timeline Quick-Play cursor appears when moving the mouse into the Timeline while playing.
      • James: UI for Quick Play would need to change. Possibly 'click to play' is now the same thing as seeking. (later) or maybe not, if we use second triangle at cursor line foot. I do believe it is possible to make quick-play an integral part of the scrub/seek/varispeed collection, but am not yet presenting a detailed proposal for that, and given Steve and Pete not wanting it to be touched second triangle is my preferred route.
    • Does playback stop when I drag the playhead then stop dragging and still hold the mouse button down?
      • James: If the playhead has caught up with dragging it stops moving (and you hear silence) until you release the mouse.
    • I just noticed Steve's list of scrubbing in four DAW's. All seem to have something to press on to start scrub mode. I think Paul's concern about not being able to do anything else with the mouse while dragging have validity. This is why I suggested retaining advanced scrubbing "modes" (enabled by a clearly marked control with clear signage you are in that mode).
      • James: IF mouse position is controlling scrubbing THEN it shouldn't be controlling something else at the same time! So if I understand the point correctly I disagree. There is no point having a non-dragging (i.e. non mouse down) way of having mouse position control scrubbing. Of course we can use keyboard scrubbing or mouse wheel vari-speed and have the mouse position free for other roles.
      • Gale: If your objection is to using the scroll wheel to change playback speed while dragging a playhead or drag bar, I don't understand the reason for the objection, beyond it's physically hard to do. I think the point is that we if have a scrubbing mode that does not depend on holding/dragging something then we have the possibility to draw a selection, change speed or do other things while scrubbing without need of octopus fingers.
      • James: I am happy enough about the mouse wheel. Varispeed play is different to scrubbing. Varisepeed play based on play-at-speed slider that is in turn controlled by mouse wheel is fine. No keyboard or mouse button needs to be down. My quarrel is with scrubbing (not varispeed play) which in my terms means dragging the play cursor - and I quarrel with the idea of doing that with the mouse button up. That is creating an unnecessary mode the user could be stuck in. There are two arguments in favour of mouse-button up scrubbing, one being that I can segue from scrubbing to selecting by releasing the mouse button, the other is that it reduces RSI. I don't buy that RSI argument. If mouse down dragging is such a problem, we need to be tackling it for all cases of dragging. The seguing is more interesting. If the seguing is important we can find another way to segue. There is only one mouse position, so we can't independently use the mouse to select a region and at the same time use the mouse to control the scrubbing at a different position. I am fine with scrubbing and varispeed play at almost the same time. Scrubbing sets the target position in the audio, and Audacity deduces what speed to play at (or skip) to get there promptly. Varispeed sets the speed we resume at when we exit scrubbing mode.
      • Gale: Mouse down dragging is much more significant for RSI when scrubbing. The user will rarely be dragging selections back and forth over a period of minutes to find the correct places.

        A scrubbing mode is more flexible, rather than un-necessary. I don't buy the argument about a user getting trapped in a mode, as long as a widget rather than waveform click enables the mode and you can see when you are in the mode and how to escape it.

        If we have a simple drag bar/widget plus one or more scrubbing modes, or a scrubbing mode with preferences, then (one of) the mode(s) can allow left-click and drag to draw selections while scrubbing, and seek can be done in some other way (as per the very first implementation of scrubbing).

        We seem to be accepting Timeline Quick-Play should stay, given it has been developed and has further potential that may make it hard to integrate non-confusingly with scrubbing. Similarly, we already have two scrub modes that have been developed. Rather than throw the modal idea out, we could retain and refine scrub modes so they are easier to use. If we have the simple drag bar/widget as well, we can afford to make it harder to turn the modes on, and those that do so accept that it is an advanced mode, just as a DAW with scrub mode is "advanced".

        • Peter 11Oct15: "I don't buy the argument about a user getting trapped in a mode, as long as a widget rather than waveform click enables the mode and you can see when you are in the mode and how to escape it." Precisely, this is what Phase-2 is all about: making it discoverable, obvious and escapable. And us a result of that we should find that we've thereby fixed the "user getting trapped in a <scrubbing> mode". It was precisely that user getting stuck in scrubbing, reported on the Forum, that provided me the impetus for me to develop these two linked proposals.
    • Real Time Vari-Speed with the Play-at-Speed slider would be very nice on its own terms. Clicking it to bind scroll wheel to it seems like a hard to discover extra step, needed most times you start scrub, so not an improvement on that point. I would rather see some option on by default that entering any type of playback always bound scroll wheel to the play at speed slider, and stopping playback unbinds.
      • James: Well, scroll wheel doing anything other than scroll is hard to discover too. You click on the varispeed slider to drag it, so if you start changing speed then you can continue doing so by mouse wheel. We should show that the slider has focus as an indication of the binding. We could also give it focus (and bind to it) when you click on play-at-speed.
      • Gale: Scroll wheel varispeed could be easy to discover with the temporary mouse pointer I suggested. You could put the option in the Transport Menu. I am not understanding what the varispeed slider is. I took that as a synonym for the Play-at-Speed slider but the above suggests it is something else. The playhead is not a true varispeed slider, but is that what you meant?
      • James: Varispeed slider is synonym for play-at-speed slider. I think it is hard to discover that the scroll wheel does anything other than scroll, unless you've read the mouse preferences. I think it would be easier if there was a pattern of it binding to the last slider or scrollbar that you have clicked. We could though just treat the scroll wheel as an advanced user feature, and just make sure that all the functionality we want to expose is exposed in a more discoverable way. I am not so fussed if there is a discoverable way into a feature and a (less) discoverable route too. Varispeed and scrubbing are two different things. Varispeed from the play-at-speed slider, scrubbing from the play cursor (or your temporary mouse pointers). If you try to do two different things from the same widget you are back to ctrl-shift combinations.
      • Gale: Any binding should not prevent modified uses of the scroll wheel. For example the current standard scrub still supports horizontal-zoom-and-scroll with modified scroll wheel. Standard scrub prevents scrolling up and down the project, but if we prevent that in scrub play we may as well prevent it in all play, to my mind. These problems would be lessened if we had configurable mouse bindings and a pair of shortcuts to scroll up and down a project.

---

Reduced physical effort to use scrubbing - Phase-2.5

Gale: No-one else seems to care about this, but I want to go back to a previous place and hear from there with less physical effort than now. I want an option to never stop when the playhead reaches the pointer, or to be able to click to resume play when that happens. An option for forwards play only is one possible solution. There is a discussion of this at http://forum.audacityteam.org/viewtopic.php?p=286530#p286530.

Comments based on 2.1.3 alpha as of 11Aug16:

1) Gale wrote above: "I want to go back to a previous place and hear from there with less physical effort than now."

  • Peter 11Aug16: this is relatively straightforward but only if you are using the Scrub Bar. With the Scrub Bar yon easily select where to start scrubbing from and then you can slide the widget along the Scrub Bar, click again and Scrubbing moves to start from there.

2) Gale wrote above: "I want an option to never stop when the playhead reaches the pointer, or to be able to click to resume play when that happens."

  • Peter 11Aug16: This is relatively easy to achieve now that the cursor position moves to the current scrun point when scrubbing is stopped. Simply make a single click on the Play button in the transport toolbar - or make two presses on the spacebar (one to stop scrubbing, the other to start Play).

Earlier discussion - from before Paul's new 2.1.3 scrubbing

  • Peter 08Oct15: But Gale that is precisely what Quick-Play on the Timeline does for us right now with a simple single click of the left mouse button (I use it all the time to do just that).
  • Gale 08Oct15: In the link to the topic above I say why I believe Timeline Quick-Play is inferior to scrubbing in the waves. In short, Timeline Quick-Play offers no speed change on the fly, no drag-seek (which is less RSI than continual left-click), and makes it easier to see "what you are doing" - you can focus on the waveform while listening, without distractions of Timeline numbers, an extra playback cursor and a vertical white line.
    • Steve 09Oct15: I must disagree with Gale most strongly on this point. Timeline Quick Play was carefully and thoughtfully designed with the needs of users in mind. I am very pleased with the outcome, I use this feature all the time, and have received many thanks and compliments from users about this feature. I will strongly contest any proposal that will mess up Timeline Quick Play. I agree that scrubbing needs radical improvement to make it useful, but please leave Timeline Quick Play out of this. There are plenty of other ways to improve scrubbing without messing with the highly successful Timeline Quick Play.
    • Gale 09Oct15: Timeline Quick-Play's (undoubted) improvements removed a feature I used, to lock the play cursor. I think we have agreement to eventually reinstate that in an expanded feature that has "recording points and regions" and "playback points and regions".

      In any case, nothing I wrote necessarily advocated removal of Quick-Play - indeed I queried James over how his "drag the playhead" proposal would impact Timeline Quick-Play and questioned if doing dragged scrubbing in the Timeline is the best place to do it. Unlike Steve I don't see it as impossible that dragged scrubbing could replace Timeline Quuick-Play.

      But from my personal perspective, Timeline Quick-Play remains more awkward than the now removed CTRL + Click in the waveform Quick-Play. Timeline Quick-Play is not a complete substitute for a vari-speed scrubbing feature that has "minimal RSI" ability, but we don't have that yet. Example: Drag in the Timeline and release the mouse. While the audio is still playing, press the mouse button and drag again. Audio stops at the end of the previous play region. Yes I know you can release the mouse again...

    • Peter 10Oct15: But James was writing that Timeline Quick-Play could be removed and replaced by some form of scrubbing gesture, enabling the Timeline to be used to control scrubbing. Which is why Steve added his proposal for a separate scrub control bar. I would be totally against the removal of the Timeline Quick-Play, it is a nice simple easy to use feature that does the job nicely and straightforwardly. I use it all the time for production work, it is one of the most useful tools in my personal Audacity toolbox. And btw if we did implement the Scrubbing control bar that Steve proposes - or some other form of scrubbing initiation as James proposes then this opens up the possibility of you getting back your preferred Ctrl+click in the waveform for a "poor-man's quick-play". I never liked that for quick-play as it is a compound gesture - I much prefer the single click in the Timeline (aided by the white line on the waveform for positioning).
    • Gale 11Oct15: Waveform Quick-Play was obviously limited to click play and did not do not drag play. All I am saying is that personally I found that much more pleasant to do click play than Timeline Quick-Play. I don't see a lot of point in bringing back waveform Quick-Play as long as we retain waveform scrubbing. The current waveform scrubbing is much more functional than waveform Quick-Play but at the moment, quite hard to use.

Steve's Alternative Solution: Scrub Toolbar (scrub ruler)

  • An additional "toolbar" (Scrubbing toolbar), similar to the Timeline, but dockable above or below the track window. Docking the toolbar above the track window allows quick and convenient switching between "Quick Play" and "Scrubbing", while docking at the bottom allows quick and convenient switching between scrubbing and scrolling.
  • There is no option for this toolbar to "float". It may be above the tracks window, below the tracks window, or hidden. Suggested default; above the tracks window.
  • Mouse down in the scrubbing toolbar initiates scrubbing, regardless of which tool is currently selected.
  • Mouse up stops scrubbing.
  • Moving the mouse out of the scrubbing toolbar stops scrubbing.
  • The scrubbing toolbar has an indicator that may be dragged left or right, either by mouse or keyboard. This controls the speed and direction of scrubbing.
  • Moving the mouse pointer stretches a "rubber band" between the indicator position and the current mouse position, The length of the rubber band indicates the playback speed, When short, playback is slow. When long, playback is fast.
  • The default position of the scrub speed indicator is the middle of the toolbar.
  • The position of the indicator is overridden by clicking on the scrubbing toolbar. On click, the indicator moves to the clicked position,
  • Scrubbing commences from the time position that is clicked on the scrubbing toolbar.
  • When scrubbing stops, the playback cursor stops.
  • The playback cursor is disentangled from the selection, so that (when stopped), the cursor may be at a different location to the selection.
  • Adjusting the selection has "snapping" to the cursor position for easy adjustment of the selection to the current playback cursor position.
  • Clicking on the waveform does not remove the current selection.
  • Click and drag on the waveform creates a selection.
  • When focus is in the scrubbing toolbar, the number keys control playback speed, 1 slowest, 9 fastest, 5 normal speed, 0 stop. A modifier key controls playback direction, Modifier down for reverse, modifier key up for forward.

Alternative Solution: Scrub tool

This is much like other audio software.

  • F7 selects the scrub tool.
  • When the mouse is over a track, the pointer becomes a "loudspeaker" or some other icon that indicates "play".
  • Mouse down initiates scrubbing.
  • Mouse up stops playback
  • Moving the mouse left / right controls the speed and direction of playback.
  • Additional features and refinements are possible.