Games Usability Testing is not QA!

As an emerging field, the aims and characteristics of usability and player experience testing can often be unknown or unrecognised within game companies, outside of a few key industry leaders, such as Valve and Bungie. This can often lead to comparisons being made with QA testing, or confusing usability testing as an element of QA. As an already established area of game development, it’s a common misconception that they are the similar, or the same fields. This is not the case

There are similar elements, as both involve considering the end user’s experience, and involve getting players to physically play the game. However, they have different goals, and this is what I will be covering today, by looking briefly at the aims of QA, the aims of usability testing and how they differ.

What is QA?

Quality Assurance (or Assessment) is an established field within game development. Often performed at both a developer and publisher level, it typically involves a room full of underpaid gamers endlessly playing a game in every conceivable way. They’re looking for bugs, which will be documented and then passed on to the coding or art department.

Typical bugs would be “the hitbox on that model is wrong”, or “when you shoot the tires on that Jaguar E Type, it makes a metal ricochet sound, not a rubber one”. So the task is largely monotonous, and will involve running into every wall in the game, and testing every dialogue choice, to find one which gives an unexpected result. Moving beyond bugs, QA often includes other areas such as localisation, compliance, and compatibility with a range of devices.

...and taste

Throughout the process, QA is trying to make sure the player can’t get themselves stuck, and no bugs in the game prevent them from completing, or enjoying the game.

What is Usability and Player Experience testing?

Similar to QA, usability and player experience testing involves playing the game. However, this typically wouldn’t be in a similar ‘farm’ setting. Instead, usability tests often attempt to recreate a typical playing environment, to emulate how a player would typically play the game (required: a sofa, and a 2 litre bottle of pepsi .)

Player experience is focused on whether a player enjoys what they do in the game, and whether they understand their goals. This can encompass many sub-categories of player experience, such as how is challenge and interest is maintained, why and where players give up, and how they understand the game. Essentially, the aim of player experience testing is to optimise the game so that people want to play the game.

Meanwhile usability testing blurs slightly more with QA, yet has some fundamental differences. Usability testing is focused on whether the game allows the player to achieve their goals, for example do they notice when something in game changes (such as picking up a new item), or do they understand where they have to go to complete the level. This is different to the ‘bugs’ that QA discovers, since these are game features, not ‘mistakes’.

Why do they differ?

Essentially this is the key difference between the two. QA focuses on the unintentional problems with a game, and aims to make the final product as close to the design features documented. A QA success would be a game with no bugs, and with an implementation that matches the designer’s vision.

My vision is money... lots of money

In contrast, usability and player experience testing aims to influence and guide the designer’s vision. By involving players, they gain an insight into how the game will be received by its audience, and help the designer create the reaction they’re after. As such, the focus is on the game’s intentional aspects, not its unintentional bugs.

This leads to different characteristics in the problems found. QA works largely in the edge cases, and tests every hit box, every wall, every enemy and every dialogue choice. Player experience and usability is much more interested in the average experience, to ensure that players ‘get’ it.

We can see a shift in the gaming industries perceptions of player experience and usability testing, having been championed by leaders such as Microsoft and Valve, more companies are starting to recruit user experience professionals. With the rise of free social games which require the player to ‘get’ games quickly, we can see that the field is becoming a key requirement for success, and is becoming a focal point of how games are made. But it’s not QA!

If you found this interesting, you may like:

1. How real-world game usability testing is changing
2. Why you should always video players when testing games
3. Some great articles on games usability and player experience
4. The importance of usability in Mobile Geolocation games.
5. Ian Marsh, creator of Tiny Tower, on user research, play-testing and F2P