Using the Hand as a Mouse

As you watch this demonstration, you’ll notice that the user interface relies heavily on the position of the user’s hand. What that means is that the user has to rely on visual feedback from the screen to know if they have their hand in the right position.

In other words, the user interface treats the operator’s hand like a mouse pointer.

The problem with this is that it requires the operator’s attention – they need to monitor the display to place their hand correctly. Now, this is not a great idea in general because our hands and arms are fairly imprecise – especially when dangling in mid-air – but you can see that this is even more of a problem when you are driving and supposed to be focused on the road.

Some Alternatives

A couple of alternative approaches come to mind:

1. Real Gestures
2. Physical Buttons (gasp!)

Real Gestures

Rather than relying on hand position, interpreting actual gestures would be a more effective approach. For example, different hand gestures (palm, fist, various numbers of fingers) could be used to represent different commands. (Maybe the system being used here just doesn’t have that level of fidelity.)

The advantage of using gestures is that they are independent of hand position. That means the operator can keep their eyes on the road, while simultaneously making gestures. The disadvantage of gestures is that they require the user to memorise them (except for maybe a few, more intuitive gestures – like palm for stop in many cultures).

Physical Buttons

Are physical buttons so bad in en environment like a car cockpit? Physical buttons have some particular advantages:

• You get physical feedback through your sense of touch to confirm you’ve selected the command.
• Once spotted visually, an operator can rely on spatial memory and proprioception (are sense of how our limbs are physically deployed) to select a button – they do not have to maintain attention.
• Once used a few times, muscle memory can kick in, thus negating the need for visual target acquisition (yes human factors folk refer to it as target acquisition).
• Most of the time a physical button has only one function which never changes. This allows for effective mental mapping.

When is a gesture not a gesture

To my mind, a gesture’s meaning is determined by the relative position or movement of limb and digits, not merely a position in space.

At the end of the day both approaches have their role in gesture-based products. Hand positions are best for new users, so long as they can be fully attentive. Gestures are better for situations where the operator’s attention is elsewhere – but they need to be learned.

In this image it says it is “Ready to perform category backup”. Good, coz I’m ready too. Now what? In fact, it will sit there a long time being “Ready to perform category backup”, and then, suddenly, it will perform category backup. (Turns out it’s waiting for me to go to bed.)

The Importance of Good Communication

Good software keeps you up to date with what’s going on, what’s about to happen and what you need to do (or, it reassures you that you don’t need to do anything).

We’ll be talking more about our experience designing hand-held devices for use by cochlear implant recipients and their carers.

More info here: Switching On My Ears

Thanks to Cochlear for allowing us to share our experiences. If you’ll be in Dublin for Interaction 12, get in touch!

If you look carefully you’ll see I’m wearing a shirt from Stamford Interactive. Way to promote the opposition Shane!

In the full talk I describe 5 benefits of prototyping throughout the software development lifecycle. Prototyping isn’t just for UX!

The full session is here: Prototypes, Prototypes and Prototypes, and the Difference Between Them

Shane.

The challenge was to design a navigation paradigm that streamlined the tuning process while still allowing audiologists the freedom to choose the most appropriate approach for any situation.

Working in an agile team over 10 months, activities included:

• Persona creation for primary and secondary user groups.
• Navigation design to support a more streamlined tuning flow.
• Interaction design allowing clinicians to focus on the task at hand while still having access to a range of other tools and options.
• Usability testing involving existing and new user groups.

UX Australia 2010 Melbourne: Audio and Slides from our presentation describing this project can be found at the UX Australia website: Defining the recipient journey: The role of software to support hearing restoration at Cochlear

Collaborators: Yas Nakano – Visual and Interaction Design

Credits: This work built on from foundation work by Different, and has been subsequently refined by Matt Balara.

Technology: Microsoft Windows, Windows Presentation Foundation (WPF), Expression Blend

The application included a sophisticated multi-touch ‘attractor’ based on ANZ’s new branding, as well as a multi-player ‘beat the clock’ game and an interactive environment for exploring tennis memoriabilia and other information. The latter application made use of “cards” placed around the pavillion which unlocked content when placed on one of the Microsoft Surface devices.

Collaborators:

• Object Consulting - Implementation
• M&C Saatchi - Concept, Design production
• Information and Design - Usability testing
• Jack Ukleja - Development
• Alex Knight - Visual design

Technology: Microsoft Surface