Power to the people test

With the cost of game development rising, studios are under increasing pressure to ensure that their game achieves both critical and financial success. The risk of a title receiving unfavourable review scores – and therefore quite likely poor sales figures – may result in the studio’s brand being damaged, or worse, studio closure.

To minimise these risks, developers are increasingly keen on investigating new approaches that can help deliver a high quality game on budget. One approach which has steadily been gaining momentum in the video gaming industry is that of user research.

User research (sometimes called playtesting or user testing) focuses on understanding the player; in particular, how they perceive, interact with, and experience a game. It’s an umbrella term for the many methods that can be used to understand player behaviour, but broadly speaking, these methods can be divided into two groups: usability and user experience.

Usability comes from the discipline of Human-Computer Interaction and refers to the set of techniques that have aims such as making software easier and more efficient to use. Some of these aims would clearly apply to game development – after all, who doesn’t want to make their game more accessible and sell it to a larger audience?

However, traditional usability doesn’t analyse the enjoyment of playing a game and that’s what it’s all about: the player experience. This is where User Experience, a group of methods designed to explore the human emotions such as frustration, enjoyment or excitement, comes in.

So why should you integrate user research into your development process? When done correctly, these methods offer a remarkable insight into how a game is going to be played by the buyer, and even how it’s likely to be reviewed.

In a study we recently completed, 171 games reviews from August 2008 to July 2009 were analysed with the aim of understanding which aspects of a game were considered worthy of praise or criticism by professional game reviewers. It became clear that the vast majority of issues mentioned were relating to the player experience, not technical issues. The reviewer comments could be divided into four groups:

1. Usability – is it easy to understand, menu design, does it behave as expected?

2. User Interface (UI) – problems with the HUD, icons, failure to read important text.

3. Interaction Design (IxD)/Controls – players cannot issue commands, can’t remember them, physically difficult to use or be precise.

4. User Experience (UX) – how does it make you feel, signs of frustration, losing track of time, shock, pleasure.

A fifth group could be included which deals with the technical issues (bugs). It’s probably becoming clear that user research is not QA – in fact, it’s an ideal complement to it. Whereas QA is focused on identifying issues within the game itself, user research aims to identify issues that are between the game and the player. This could be called cognition, experience or interaction, but it’s really the mapping between what a game is communicating to the player, and how the player interprets that message and feeds back a response.

Despite the dedication and talent of the development team, it is difficult to predict how the general game-buying public will respond to a game. If there ever was such a thing as the ‘typical’ gamer, then they have changed drastically in the last five years – casual gaming being one reason – and they’re possibly about to change again with new gestural-based controllers such as Natal.

If used iteratively throughout development and not just at the end, user research should provide the studio with the means to correct any unintended player issues, delivering a game which is as true to the original design as possible.

Now that you’re hopefully more receptive to the benefits UX testing can bring to development, the next thing to tackle is how you actually go about performing this user research.

Unfortunately, there is no standard process which we can just simply apply. It will mainly depend on firstly identifying which questions you want to answer about the game, then choosing the right methods to help get you those answers.

The very first thing that you have to decide is exactly what you want to know; you need to figure out what the aim of your research is. Are you interested in testing the player’s reaction to a new feature or control technique? Perhaps you want to compare different versions of the build and get data to help provide a clear path forwards? It may even be the case that you don’t know, you just want to keep an open mind and see what a player’s first reaction is to your game.

The next thing you have to do is fully design the study in advance to ensure you capture useful and valid data during your tests – otherwise you’ll not be able to answer those original questions. You also need to decide how this data will be captured. These are big topics, and in next month’s article we’ll discuss the pros and cons of six different playtesting techniques, plus more advice for making the most from your tests.
www.verticalslice.co.uk

Dr Graham McAllister is the director of Vertical Slice, the first UK company to focus specifically on video game usability. He is a senior lecturer in human-computer interaction at the University of Sussex and is also an Apple Distinguished Educator.

Part 2 of this feature can be found here.