Talk:Completed Proposal: Improvements to Scrubbing - Phase-2
- 1 James' email of 29Sep15
- 2 Peter's email of 29Sep15 (an excerpt)
- 3 Steve's email of 29Sep - how other audio editors do scrubbing
- 4 Steve's email 02Oct15
- 5 Paul's email of 29Sep15
- 6 James (talk) 17:00, 2 October 2015 (EDT)
James' email of 29Sep15
James wrote in an email 29Sep15:
Play and record are indeed modes too, just ones users are much more familiar with. As we improve Audacity I/we would like to be able to do more things we currently can't do in those modes, but that is nice to have, not an urgent problem. The problem with scrubbing being a mode is more serious.
With play and record the cursor is moving and either sound is coming out or new waveform is being drawn. With scrubbing, if the cursor has caught up with the mouse, it isn't as clear that the user is in a mode. Sure, the play button is down, but that does look a bit as if play is active but for some reason stuck. Attempting to select a region won't work, whereas it does work in play. To a user not familiar with scrubbing it looks as if Audacity is broken.
Some random clicking around won't get the user out of the mode that they don't really know they are in.
A solution that would work for my use of scrubbing is that we go out of scrubbing as soon as we release the left mouse, (or release middle wheel - and middle click as well as ctrl-left-click activates scrubbing). That might defeat some of how you use scrubbing. It would though completely satisfy me that scrubbing was no longer 'a mode a user could be trapped in'.
Other alternative design features that could make this mode more OK:
- Change the green triangle on the cursor, so indicating that it is not recording, nor playing, but some other mode.
- Change the green triangle on the play button to indicate scrubbing-play (in both cases possibly a double headed triangle would work, indicating bi-directional). Combined these give a clue that clicking on transport buttons could help.
- Make selecting a region of more than 10 pixels take us out of scrubbing (selecting is probably one of the first things a stuck user would do)
- Make ESC take us out of scrubbing.
- Make clicking outside the track take us out of scrubbing.
These are not prescriptive, they are just possible approaches to the problem. Hopefully there is something in the suggestions that from your perspective does not damage scrubbing in any way, and that I and Steve would be happy with as solving the problems for the stuck-in-scrubbing users.
A clarification: I am not stating a requirement that selecting be possible whilst scrubbing! For example if scrubbing only happens with mouse down and dragging something, that is fine. Rather, with the currently implemented scrubbing mode, the mode-stuck user can think selection is broken. It is one of the first broken things they will find (and different to playing mode too). Making it throw them out of the mode and actually select is just one approach to get them out of the mode.
Peter 02Oct15: I have incorporated most of James' bulleted points in the proposal as points 1.1 through 1.4. The "Make selecting a region of more than 10 pixels ... " was not incorporated as you cannot make a selection while you are scrubbing. His suggestion that "we go out of scrubbing as soon as we release the left mouse" should be addressed later in Phase 2.1.
Peter's email of 29Sep15 (an excerpt)
Paul wrote: >If it is not then obvious that you can stop scrub play just as you stop other play, I don't know what would make it more obvious.
There are a couple of things that would make it "more obvious" and they are not mutually exclusive
1) You ask: "do we need some other interface for initiating scrub that you are less likely to do by accident?" My answer is "Oh yes we do" - and we on QA have been asking you for such for quite a while now.
As we already know from a posting on the Forum we have already had one poor benighted user who stumbled into scrubbing mode by mistake and then couldn't find his /her way out.
The problem is that the interface for moving into scrubbing is not GUI, i.e. it is not graphical, so there is nothing to see. What is needed is some form of graphical interface for moving into scrubbing.
This could either be an additional Tool in the Tools Toolbar for scrubbing which would place you in Scrubbing mode, Steve has suggested this approach I seem to recall. For me this is sub-optimal as I strongly think that scrubbing should be independent of the currently selected tool (just as Play and Record are). Also it would imply that to escape from scrubbing the user would need to choose and alternative tool from the Tools Toolbar. And fundamentally Scrubbing is not a tool, rather it is a form of playing. So I do not support this approach.
So what I would really like to see is an additional button on the Transport Toolbar for Scrub-play (scrubbing brush icon? maybe superimposed on a green Play icon?). This should make it clearer that the user should consider using the Stop button in the same Transport Toolbar to stop scrubbing. And btw you still need to fix the problem that the Transport Toolbar's Pause button and the Transport>Pause menu command are not grayed out as unavailable while scrubbing is active.
And if we do add such a button then we should probably also add a "Scrubbing" or "Scrub-Play" command to the Transport menu.
Apart from any other consideration I suspect that you will find more folk using scrubbing if there was a more visual method of entering scrub-play (and I'm sure that given all your work on scrubbing you would like to see that).
Steve's email of 29Sep - how other audio editors do scrubbing
In Protools, scrubbing is switched on by selecting the "Scrubbing" tool. It has a loudspeaker icon. Scrubbing is then performed by clicking on a track and dragging left (scrub forward) or right (scrub backward). Scrub play stops when the mouse button is released. When the scrubbing tool is selected, scrubbing may also be controlled with the "J" and "K" keys. "Jog wheel" control is supported when using an Avid control surface (nice, but expensive).
Soundforge uses a slightly different approach, but with similarities. In Soundforge, each audio track has its own window, and at the bottom of each window is a mini-transport toolbar. The mini-transport toolbar has an additional button as the final item, which is the "scrub play" button. When the scrub play button is pressed, scrubbing is performed by clicking on a track and dragging left (scrub forward) or right (scrub backward). Scrub play stops when the mouse button is released. When the scrubbing tool is selected, scrubbing may also be controlled with the "J" and "K" keys.
Sweep on Linux has another variation. Sweep has a scrub tool button. When selected, the cursor changes to the scrub icon when over the track panel. The cursor has the same design as the icon on the scrub tool button. Scrub play starts by clicking on a track. The Play button immediately changes to "button down". The difference between the Sweep behaviour and the other two examples above is that releasing the mouse button "lets go" and the track plays from the current position at normal speed. To stop playback, either click the stop button or press spacebar to pause, or the return key to stop. In each case the playback cursor stops at the current playback position. Sweep also has a "playback" menu in addition to the transport toolbar.
Ardour 3 has yet another variation. In Ardour 3 there are also many "advanced" options for scrubbing, but the basic operation is as follows:
Select the "scrub" tool to enter scrubbing mode. The cursor changes to a loudspeaker icon when over a track. Click and drag on the track to start scrub playback. A bar under the transport toolbar shows the current playback speed and direction. The direction and speed of playback may be controlled either by dragging in the required direction on the track, or by dragging the pointer in the bar under the transport toolbar, or from the keyboard. When dragging either on the track, or in the bar below the transport toolbar, playback stops when the mouse button is released.
Of all the methods described above, Ardour is my favourite. It is very clear what is happening, highly intuitive, provides immediate visual feedback and has "advanced" options for the more experienced user. Someone has clearly spent a lot of time and effort getting this right. It also supports "jog wheel" if the user has a compatible control surface. The only thing that I'm not so keen on, is that keyboard control is less easy than some of the alternatives.
Steve's email 02Oct15
As I understand it, the purpose of scrubbing is to enable the user to locate a specific point in time for editing. Perhaps I'm not using it the way it is intended, but for me, I find the exact spot that I want to edit, the playback cursor is exactly in the right place, but as soon as I stop scrub playback the cursor position is no longer there - it has jumped back to the previous playback position.
OK, so "I" know that there are some special key bindings available, but I've been using Audacity for over a decade.
Shouldn't the default behaviour be that when the user stops scrubbing, the playback cursor remains at its current position?
What should happen if there was a Selection when scrub mode was started?
Peter 02Oct15: To be addressed in Phase 2.2
Paul's email of 29Sep15
I am trying to assimilate the criticisms. I don't feel a consensus yet about what the better interface would be.
I) The non-immediacy of appearance change when you enter scrubbing is a problem but it is not the "stuck in a mode" problem. It is the easier thing to fix than the user interface redesign. II) James' "stuck in a mode" perception results from (a) selection change not working during scrub and (b) stopping when the play head catches up to the cursor.
How much of this perception results from each? I feel more concerned to fix (a) so that scrub play is no more restrictive of other actions, than is ordinary play. But (b) seems inherent to scrubbing, if the purpose of scrubbing is to find some precise place so that you can stop and put the cursor there. Unless you would rather have a play-head with "inertia" when you do not move the mouse. I don't know whether that will feel better in practice.
Meanwhile let's remember the scrubbing features we have. Binding each to some gesture crowds out the use of that gesture for some other purpose and that means tradeoffs in designing the user interface. Are the features all important enough to keep? Do we sacrifice any of them?
1) We have the scrolling version of display in which mouse position relative to the center affects the speed of scrubbing or the length of skips for seeking. You choose that variation or not when scrubbing begins. 2) We can switch between a stuttering seek and variable speed play during scrubbing. Variable speed simulates tape head scrubbing and can even go backwards. Seeking plays always at normal speed but with frequent skips. 3) If not seeking, then we can accelerate or decelerate the maximum speed during the scrub.
And to remember the user interface choices for the above.
1) ctrl-double-click in track panel to start scrolling scrub, ctrl-single-click for the ordinary scrub. 2) Hold left mouse button for seeking, release it for scrubbing. 3) Roll the mouse wheel.
Now let me remind you of tradeoffs that led to these decisions.
If mouse wheel is best for number 3, that is one reason why WE DO NOT WANT SCRUB TO BE A DRAG WITH A BUTTON DOWN. (An additional reason might be, we want to allow selection changes during scrub.) I think left button while rolling the wheel is just too awkward (two fingers at once, rather than just using the index finger for one or the other), and remember that when I used middle drag, Steve did not like rolling the wheel while also pressing it.
If seeking happens with left button down, that is all right because the speed determined by the wheel is irrelevant, so there is no conflict. But it is agreed that this interface is not yet the best. THIS IS ALSO WHY YOU CAN'T SELECT DURING SCRUBBING, part of what James says he means by "stuck in a mode," because certain familiar actions (selection changes) are excluded. Other alternatives were considered and rejected for their other problems. Making a shift or ctrl modifier key do it would interfere with the use of certain useful keyboard commands during scrubbing, like ctrl-m to drop a label or shift-a to stop and set cursor.
So please suggest an alternative to the left button. Perhaps we should require you to click and drag the green triangle in the time ruler (which is not part of the track panel proper). But then clicks and drags in the tracks would be free to change selection during scrubbing.
- James:. Yes, why not? What's wrong with doing that?
Another problem is that SCRUBBING WORKS ONLY IN SELECT TOOL. I am not pleased with that. It should work at least in multi tool too. Why not indeed in all tools? But the reason for this was choice 1. Scrubbing must be started by some mouse action. The choice of ctrl-click conflicts with things like time shifting in the multi tool.
What is a better choice for starting a scrub then? A click in the time ruler instead? There are already several mouse actions with meaning in the time ruler. Is anything available for scrubbing to use?
Finally Steve likes the idea of a button for a scrubbing mode. But James dislikes the multiplicity of tool buttons that we have already. He would rather have graphical handles to click on to access different functions. Also in another project, spectral selection, I deliberately avoided making a new button and a new mode. I still think it is better to accept scrub as "another kind of play" that is stopped in the same ways play is stopped, and you don't need finger pressure anywhere to continue in it. But during that play, as with ordinary play, certain other mouse actions and keyboard commands remain useful.
And don't forget ctrl-mousewheel to magnify -- I like it too that this remains available during scrubbing. But again it would be awkward if you had to keep the left button down.
Phase-1 is mostly about the 'mode' and not being stuck in it. Phase-2 suggestions here may improve discoverability and ease of use.
Possible Phase 2 changes.
- Scrubbing and seeking treated like play, loop-play, play-excluding are. The mode is initiated by toolbar button or bound keyboard shortcut.
- Ctrl, shift click etc become user bindable, in same way that keyboard shortcuts are. Possibly unbound by default.
- Scrubbing/Seeking could be started from a multi-button
- Radical change where:
- Play no longer resets cursor to start
- Scrubbing is done by dragging the play cursor. It only happens whilst the mouse button is down and dragging. Play cursor WILL lag behind desired drag target.
- Single scrub-seek mode rather than a scrub mode and a seek mode. In scrub-seek you scrub, if not too big a change, and seek if distance is above some threshold.
- Can scrub-seek when playing or when not. You could start playing at x1, then scrub-seek by dragging and continue playing at x1 after you stop dragging.
- Fixed-cursor scrolling-wave becomes an independent feature. One of the tick boxes in the multi-buttons.
The above Phase 2 ideas are NOT prescriptive and some are mutually exclusive.
If we are dragging something tangible rather than just moving the mouse the need for ctrl (or shift or double click) modifier goes away. Then the argument for restricting scrubbing to the selection-cursor mode, which Paul has done but is not keen on, goes away. The main remaining interface problem from Paul's perspective is then, I think, that he really wants to at least have the option to use the mouse wheel to set a speed. Since mouse-wheel and drag has been found to be troublesome when used together, that seems to require that scrubbing and seeking can operate with mouse up.
My picture is that the mouse x-y and the mouse wheel-roll are valuable resources. The tools-buttons are a clunky way of sharing that resource. Draggable tangible things are a better way of sharing. We need a way to share wheel-roll too. I don't have a precise proposal here. The outline idea is that mouse-wheel clicking on a slider binds the wheel-roll to that slider. ESC (or another mouse-wheel click) cancels the binding. So we could bind to the scrub-speed slider, the play-at-speed slider, or the pan slider, or volume slider, if we wanted to and vary those whilst playing.
If, as now, moving the mouse without any dragging can change the playback speed, then in my view we have a mode. It is possible to be stuck in that mode and not know what is going on. I thought we had agreed that we do not want scrubbing to be a new mode?
- Gale 08Oct15: I see in "2.1)" that "It should be possible to enter scrubbing mode via GUI." So if does not look to me it is agreed we do not want scrubbing to be a mode. There are benefits in reduced RSI in not having to drag and hold, but those benefits are IMO thrown away by the current implementation.
So I wrote:
"2.2) IF mouse position is used, THEN there should be some tangible widget to be dragged."
(note the CAPS). This has been changed to:
- Gale 08Oct15: James changed my "Should?" question to a statement that I could not understand out of its context. I did not want to be represented as making that statement.
"2.2) If mouse position is used to drag playback, there could be some tangible widget to be dragged. However this goes against 2.5) which seeks to make the drag less physical effort. Therefore an alternative could be to have the playhead widget not as the dragger but only as the up (off) /down (on) control to initiate scrubbing. If there is no drag widget then some option is needed to keep scrubbing alive when the playhead reaches the pointer."
Surely the alternative is then a mode? The only way it is better than the existing stuck-in-mode problem is that there is better signalling of the mode, which might give more clue to a new user as to how to turn it off again!
- Gale 08Oct15: I think being stuck by accident in scrub in much less issue than our showing how to scrub for those that want to use it, and IF we have a scrub mode, giving a clear indication that it is on.
The 2.5 interpretation seems to imply that moving the mouse without dragging can change the playback speed, and so if we accept 2.5 it seems we must have a mode.
- Gale 08Oct15: The standard single-click scrub (the only one the user is likely to discover) does not really change playback speed even when dragging. The scroll wheel usage to actually change playback speed doesn't even work when dragging. All drag does is to jump standard playback like holding down period or comma (optionally with SHIFT) do. Double-click scroll scrub does change playback speed when merely moving the mouse but that seems unintuitive to me.
The scroll wheel speed change is unexpected and undiscoverable, though *very* nice if you do discover it. It is more flexible than a dragged widget that changed playback speed would be. If we kept scroll wheel speed change, I would like to see at least a temporary mouse pointer when entering scrub mode that showed a scroll wheel with + above it and - below it.
Question: Is the Audacity audio engine good enough to really change playback speed when dragging a widget, so that playback speeded up and slowed down according to the speed of dragging the slider, as opposed to the on/off "stutter play" which drag does now?
I am rather strongly against modes of this kind and think avoiding having a mode is more important than the extra effort of holding a mouse button down to drag something tangible. IF we go for no mouse down THEN we need a very good way to signal how to get out of the mode. I'm -1 on the mode. If we did go for a mode a line from the mouse cursor to the widget that allows us to turn off the mode would help users get out of the mode.
- Gale 08Oct15: Again, users getting trapped in the mode is the lesser of scrub's issues. I thought no-one wanted a draggable widget except James, as per "2.1".
The draggable widget would be the most intuitive solution for users who don't use pro DAW's (i.e. I guess, most of our users). I would support a drag widget but we have lots of great, undiscoverable functionality in the current implementation. My ideal would be a simple, drag-hold, no-mode widget, but also add some other control on the play cursor to enable more advanced "mode" features that perhaps hide the drag widget. Perhaps that control might open a dropdown menu where I could choose a mode that does not stop when the playhead reaches the pointer, to replace and improve on Timeline Quick-Play.
All that said, should we be looking at what the big boy DAW's do? Is what we have now really like the big boys? What does their "jog scrub" do?