The Essence of Magic
|This page is about user interface design because, actually, user interface design is closely related to magic
- See also GUI Design Guidelines which is more prescriptive, more in the form of rules to be obeyed.
Where the title comes from
A magician may have a magic trick in which they can make a blue ball disappear in one place, and a red ball appear in another.
- That is NOT what a good magician would do.
A good magician would either:
- Make the blue ball appear to travel from one place to another, OR
- Make the blue ball appear to change colour.
The point here is that magic has rules. Magic defies the normal rules of physics, yet for us to engage with it, it must still have a logic of its own. Magic does not just 'do' strange things. If it does, it ceases to make (magic) sense. Just so with user interface design.
Designers of user interfaces are, like magicians, working with expectations. We, watching, consciously or unconsciously, are always looking for patterns. It is how we understand the world. The magician and user interface designer operate outside the normal constraints. They have to artificially create the rules and patterns for us to find. They HAVE to do this, to create expectations.
A user interface designer can make almost anything happen on screen. Widgets can slide, change colour, size, transmute into other widgets. From those possibilities the designer must craft a user interface that makes (user interface) sense. The interface can't just 'do' strange things. The designer chooses to impose a logic on their interface that limits what the interface does. You do not click a button and have a blue widget disappear and a red widget appear somewhere else.
The dynamic is the same in both magic and user interface design. The magician works the mechanics of the trick AND the logic of the apparent magical world. It is a mistake in magic to show the full range of what the underlying trick can do. By constraining the magic, the magician's magic becomes more magical, more engaging, more real. The user interface developer works the machine logic of the user interface AND then constrains it. They design rules and a consistency to how it operates. Those additional rules make the user interface less arbitrary, more engaging, more real.
Many of the rules are not written down. Some though are codified as Human-Computer-Interface (HCI) guidelines:
- OK buttons on the right, Cancel buttons on the left, always at the bottom of a dialog.
- Menu items that open a dialog end with '...'. Menu items that act immediately don't.
- ESC cancels things consistently.
- Never 'warp' the cursor to a new place, unless the user is moving it.
The rules may be somewhat arbitrary, but they add consistency, they create expectations for the user - even if the user is not consciously aware of the consistency. That is exactly what magicians do. They create and use expectations all the time. Without that, you are never delighted by the magic.
The Little-Big Rule
In graphic design, it's easy to find oneself with multiple graphics competing for attention on a page. A good strategy is to choose one of those graphics and make it big, the others small. The big graphic makes it easy to determine where attention should be first directed. This is called the 'Little-Big' rule. One big, others small.
Our code gives us the flexibility to make any button big and any button small. We need to formulate a rule to decide what to make big and what small. The lazy approach is to make all buttons small. That leads to a messy confusing user interface. The button size rule is completely unnecessary from a coding perspective, a tiny detail, but it is important none the less.
One approach is to enlarge a button whenever a user hovers over it. That uses size for focus. The close spacing of receptors in our fovea in effect gives us more size on what we look at, so this approach amplifies that. Another possible rule is to size buttons by importance.
- In Audacity, we make the control tool bar buttons bigger than other buttons such as the edit buttons. It's not necessary from a purely utilitarian perspective, but it is from user interface considerations. It directs the users attention. It shows them that those buttons are important.
One point about the little-big rule is that you cannot consider each piece of the user interface in total isolation. Many small buttons look bad, if that is all you have. Once you have a few larger buttons to focus attention, the visual clutter from the smaller buttons is no longer the problem that it was.
Orthogonality + Rules
A recurring principle in user interface design is to have engines and code that are highly orthogonal, giving a great deal of flexibility in what the user interface can achieve. And then, on top of that, we layer guidelines, that make an intelligible user interface. The guidelines create expectations rather than surprises.
Orthogonality must be preserved at the code level. It is ludicrous to force the underlying code to be different if an action is being presented via a menu item, a small button or a large button. The underlying code should be able to play audio forwards or backwards, in a loop or not, at x1 speed or x3, skipping over a region or not, with a click filter or without. It's only when you get to the user interface that you decide to give play forward at x1 speed more prominence than other options.
It is tempting to codify the user interface guidelines in actual code. For example, a subroutine to place buttons could always place 'OK' buttons on the right, always place 'checkbox menu items' at the bottom of menus. That kind of coding works for simple rules. It does not work so well when statistics and probability of importance come into play. We don't typically codify that certain functionality is related, that record, play, stop, pause, rewind, belong together. Without information about what features work together and frequency of use, an algorithm to choose how to give prominence has nothing to chew on. We instead rely on the user interface designer and their intuition about what is important and how to group things.
If we had more information about how things are used, could we do better? If we had more information about how frequently menu items are used, could we design more pleasing, more efficient, more logical menus?
A problem we have in Audacity is that developers tend to look at features one at a time rather than as a whole. In particular they tend to look at the features they have added themselves. They want these features to be readily accessible. The temptation is to put those features onto buttons, or in the top levels of menus. Multi-level menus take more steps to navigate than single level ones. So why not put everything at the top level? The answer, of course, is that some menu items are used more than others. Those items should be given prominence at the expense of less used items.
Coding theory gives us some numerical guidance as to how probability of use of a menu item relates to the level it should be at. The steps in navigating a menu can be thought of as akin to choices of letters in a word. Coding theory tells us that frequently used words should be short with few letters. Rarely used words should be longer. The numerical guideline is to use the logarithm of the probability to set the length. An item one level down in a menu might be, say, one twelfth as likely to be used as items in the top level. Two levels down, and it's now one in 144. It's a mistake to put rarely used items high up in the menus, where they crowd out and slow down use of the most frequently used items.
In Audacity we have a hierarchy of importance with the most important commands getting more real estate and more customisation.
Wave-form -> Large Buttons -> Smaller Buttons -> Top Level Menu Items -> Second Level Menu items.
It's interesting that the highest level items can afford to have the most baroque and unusual user interface. It is the more rarely used items, of which there are far more, that MUST be handled in a consistent, uniform, more pedestrian way.
Stuck in a Mode
Time and again we hear of new users stuck in a mode in Audacity. That is, they have clicked a button to do something, and now they are stuck doing that and cannot get back to 'normal' Audacity. Examples include 'sound activated recording' where from the user perspective Audacity unexpectedly starts and stops recording at random, sync-lock, where clips are inexplicably locked together and move together, pause-mode, where many menu items are inexplicably greyed out. Sometimes we can avoid modes completely. We planned that for the tools, but have only partial success in combining Audacity's tool modes into just one mode. Some modes, such as recording and playback are inevitable. But we should still avoid unnecessary proliferation of modes, and find ways for users to easily escape from being trapped in a mode.
- Mostly ESC should get users out of a mode.
- Often connecting a 'mode' to a mouse down action is a way to prevent the mode being too sticky.
We should look at what users attempt to do to get out of a mode. Stuck in Pause? Many actions taken from the menu that are disallowed in pause should simply pop the pause up, ending it.
Besides the dexterity problems, we can't rely on users using or remembering compound key combinations. We're not catering primarily for users who will use Audacity every day, and who will learn the shortcuts. Primarily our users are occasional users who just want to get the job done. So in the main we should have draggable widgety things on screen that do things, and that light up when you hover over them to indicate that they are active controls not just passive decoration.
Initiatives to save space in the Audacity user interface can lead to better user interface. For example, information about sample rate in the track control panel is of lower priority than the mute button, and can come later. It can disappear when the height of the track control panel is reduced. Combining related things together can also both reduce space and make for a more intelligible user interface. We have the speaker and mic icons three times over in the toolbars - in the meters, the volume setting sliders and the device settings bar. They can all be combined into one reducing space and clutter. The volume slider can be on top of the meter, which has a logic to it, as the meter responds to a change in volume setting. The icon for speaker and mic on that toolbar can become active buttons giving access to the device settings.