In the 1980s a simple blind test sent the world’s largest soft drinks producer into a panic, when subjects were blindfolded and had to decide which type of cola tasted best. Its outcome has gone down is history, as the majority chose Pepsi cola. Spurred into action, Coca-Cola invented ‘New Coke’, a type of cola with a new recipe that it felt was bound to prevail over Pepsi in blind tests. ‘New Coke’ went down in business history as one of the biggest marketing flops of all time. Consumers rejected the lavishly promoted drink and demanded the old recipe back. Shortly afterwards, the company brought out their ‘Classic Coke’.
How did a multi-billion-dollar company manage to make such a blunder over some blind tests? The answer is simple: their assumption that blind tests provide a representation of customers’ real decision-making behaviour was false.
Cutting costs
Let’s move on to acceptance and usability testing. When developing products, services or applications, idea generators, designers and developers make myriad assumptions about the usefulness, functionality, learning capacity and enthusiasm of (potential) users. They can draw upon experience from previous projects, recognised best practice and knowledge that stems from research.
However, it is impossible to understand the intentions, needs, reservations and proclivities of consumers in any detail without engaging with them intensively. Without consumer research and ongoing user testing, pushing forward with a new development means flying blind. The longer this goes on, the more difficult and expensive it becomes to get back on track. Tweaks to the concept are and always have been more cost-effective than adapting already developed solutions. As such, it is worth checking that assumptions are correct at the earliest possible stage.
Figure 1: Cutting Costs
Small steps first
Based on our experience, we recommend conducting acceptance and usability tests as early as possible. Lots of small-scale rounds of testing provide greater insight than a few larger tests. Just five to eight trial users allow any major stumbling blocks and missing functions or information to be identified (Source: Nielson Norman Group). Depending on the hypothesis, those devising the test can adjust the level of detail of the prototype. This allows them to test out their ideas and processes even with paper and cardboard models. As a rule, only detailed optimisations such as form entry and microinteractions require more advanced prototypes.
Do your planning at the very start
Acceptance and usability tests provide optimal results if they are conducted in a systematic way. Having a structured procedure increases the level of quality and the comparability of the tests. This has a bearing on the formulation of hypotheses and the definition of objectives, as well as tasks for test users, such as: ‘Find and purchase an XY-branded vacuum cleaner in the online shop’.
After the tests, the collected data are analysed and the results summarised in reports. The findings are fed into the next iteration of the design or stage in the development process. As we see it, there’s no need to fear processes and procedures. An unstructured test always provides more insight than no test at all.
Figure 2: Usability Optimization
The ball’s in your court
Early and regular acceptance and usability tests reduce the costs of late changes and maximise customer value. Discuss it with us via our Pipeline – and make sure that you don’t fall into the ‘New Coke’ trap’!
