Getting the whole team involved should be a priority even with the smallest scale tests. Not only is it a vital opportunity to sell the UX process, but it’ll give an undeniably clear example of the benefits that UX testing can bring, and help secure funding to ensure the next round of usability and user experience testing will not be on such a small scale.

Phase 2 of Testing

How to Do it:

So how can we get the whole team involved when user testing is taking place? This can be divided by time frame into 3 key areas.

Before a user testing session:

• Inform everyone that the tests will be happening – send a group email, including details on what will be happening, and when, to all interested parties. This could be sent a week in advance, and on the day of the tests, which will increase awareness of user testing and allow you to….
• Invite people to spectate – Let people know that they can watch the user testing, and that their input would be valued. Perhaps doughnuts will prove a big enough incentive to get people to give up some of their time to spectate.
• Ask priorities and ‘goals’ for investigation – Encourage people to spectate and be invested in the process by asking what they’re interested in finding out, and incorporating this into the user tests. This will be followed up by a debrief, described later.

Getting the preparation before a user test correct will help increase awareness of what you do, and how you can help people, break down misconceptions about user testing, and get the team invested in the process. If a team feel like their priorities matter to you, and that they can help shapre the process to ensure it will help them, they’ll become proponents of user experience and usability testing.

During a session

• Set up a remote viewing session – This should ideally be in a shared conference room, which allows people to spectate the tests in progress. This can be done using cheap/free equipment such as webcams, team viewer, and IM clients. Encourage attendance by advertising free food, and letting people know their priorities will be incorporated.
• Make space in the test for questions from the team – At the end of the session, check with the team if there is anything they want to be asked, for example did they want to know more about a subject’s thoughts when displaying behaviour the team found interesting. Note that it’s important that these questions go through you, as this’ll prevent potentially leading questions getting through the net, and increase the validity of the participant’s responses.

After a session

• Run a group debrief after each session – After the user testing session, meet with all the spectators, and give them freedom to discuss what they saw, and their conclusions from it. This allows the team to share their findings, and will encourage them to get involved again, especially since you will…
• Incorporate their findings into your final report – and give credit!

You don't want to be accused of stealing credit...

Advantages

So why go to all the effort of getting people involved with user testing?

The first advantage is that it will give credibility to your findings – not only will the team have helped shaped the conclusions, but they will have seen firsthand the evidence of that behaviour. It’s pretty hard to deny that, for example, the player doesn’t notice when they pick up a power up, having seen them miss it all day.

A secondary advantage through involving them with the process is that they’re likely to be more invested and enthused about the process – the sort of buzz that can help you gain an investment of more time or money in user testing.

The most useful advantage of getting the wider team involved is that they’ll add their own expertise and critical eye, highlighting areas of importance to them, and helping make observations in areas that you may miss. This can only help your final report, and gives a much wider legitimacy to the user testing.

Making the whole team aware, and involved in the user testing process therefore provides advantages to everyone – not only yourself, but to the whole team. It’ll help promote the process, and sell further user testing, and therefore is a key aspect of selling UX testing to games companies.

Related posts:

1. Selling UX in games – everyone else is doing it.
2. Test with real users – not your team
3. Why you should always video players when testing games

I’d be interested in hearing your thoughts on this – please use the comments to add critiques or alternative viewpoints, as I’m sure there will be many! Anyhow, onwards with the 30 minute facebook game usability test.
I’ve been thinking about the social games found on facebook, and I think the most important part is the first 15 minutes of a new player’s experience, e.g. what happens in those fifteen minutes, do they understand the game, and do they want to come back and play again.

This means the most important aspect of user testing is understanding and improving the ‘new user’ experience. For this you’d need some people who’ve never played the game before (and who are close to the target audience of players you want to attract), and simply get them to play the game from the start (without special instructions, just how they would if it was on facebook). Maybe a small incentive (like pizza!) would be enough to recruit people for these short sessions.

a small incentive

Explain to the player that you want them to just play the game as they would at home, and that you are testing the game – not them. Pre planning some notes on this introduction speech will make it easier. It’s probably a legal requirement to let them know if the session is being recorded.

You should have interested developers watch them play through a remote desktop tool (but they shouldn’t interrupt the player!), or record the session to review later. Free tools, such as team viewer, are available to do this. This will be invaluable for getting their buy-in for further user testing, and will prove the benefit of user testing to an often dubious audience.

It’s common to use a ‘think aloud’ methodology, where you ask the player to speak their thoughts aloud as they play. It’s not perfect, but it’ll give some insight into what they’re thinking. You could prompt them by asking non-leading questions such as “what are you doing now” or “what are you thinking” if they’re silent for too long!

what are you thinking?

Then after the fifteen minutes (or completing the tutorial), you can ask them questions to gauge how they understood the game – did they know what they were doing, were they confused by the game, did they know what to do next, would they like to continue playing, etc. Maybe you saw some interesting behaviours while they were playing that you’d like to ask about. Often people don’t remember what they did, and so you’d have to prompt them, or maybe the developers watching have some questions to ask.

Hopefully this quick methodology will show where the user’s are getting confused, or not understanding what to do next, or getting bored, or any other factors that turn player’s away from the game in that crucial first play.

Jakob Nielsen said that most major usability problems can be found by 3-5 users, so you wouldn’t need to run this test many times per iteration of the game. As to when this should be done, its best to get started as soon as there is something playable, as changes will be easier to make the earlier they are found, and then a similar test can be repeated with the next version of the game. Many social games go through an agile design process, with many iterations, and so this process will fit in well.

What do you think? Does this sound helpful? Or do you think that the ‘new user’ experience is not the most important part of a facebook game – maybe converting players to paying players is instead… let me know your thoughts in the comments here, and we can improve this 30 minute facebook game test.

Related posts:

1. Test with real users – not your team
2. How real-world game usability testing is changing
3. Game Usability: Advancing the Player Experience Book Review

This inaccurate view is set to change, with the rise of in-house and independent usability and user experience labs, and the breadth of academic research increasing our understanding of user experience. Hence I’m planning a series of articles about my thoughts on selling UX to game companies, helping you to communicate why game companies should be thinking about UX with their games. New posts in this series will come under the category ‘Selling UX in Games’

Everyone else is doing it

So why should your company be interested in games usability and user experience? Well I’m glad you asked…

• Many top selling games have gone through user experience testing
• The companies who have tried it recognise the benefit, and go on to do it again. In fact, one developer said…
• “It adds 10% to the metacritic score”

So let’s look in more depth at some games that have been reaping the benefits of user experience testing. These popular games all found success through user experience testing, and can serve as helpful examples when communicating why your company should be looking into user experience.

oh... well, thats easy to understand

Halo 3

Bungie recognised the value of the player experience when developing their multi-million selling Halo 3, and hence used user experience testing to create a game that was consistently fun, ensuring the player was never confused or lost. As documented in Isbister’s book Game Usability Microsoft’s usability labs were brought on board to ensure that Halo 3 was fun throughout.

Each area was thoroughly tested with real users, using techniques such as video and ‘heatmapping’ to log where the choke points were, where people got lost or stuck, and where they died. This could then be used to easily find problems in the game, and fix these areas. Fixes are typically simple, either by signposting the way to go, altering the enemies or adding points of interest to draw players (like a sparking light to encourage players to look up). This technique could also be applied to multiplayer maps, to balance the game’s flow, and make the game fair.

The success of this technique in Halo 3 has been extended to Halo Reach, where telemetric data was gained from a long public beta testing period. This gave real player information on how to balance and improve the game, and will undoubtedly be a large factor in its success.

To read more about the Halo 3 user testing experience, see the coverage in Wired.

Split Second

Brighton based Black Rock Studios explored user experience testing in their successful racing games ‘Pure’ and ‘Split Second’.  With consultation from the usability lab Vertical Slice they tested over 100 players, getting new user’s experiences to fine tune aspects of the game, ensuring the pacing was correct, and that the game was delivering key ‘fun’ moments, which they denoted ‘woah moments’.

They built on this successful experience from Pure in developing Split Second, where they combined in house talent, who worked on testing users, with assistance from a local university, who performed an expert evaluation of the usability and user experience issues.

As documented by Graham McAllister, the developer believed that the process “added 10% to their metacritic score”. More information about how user experience testing helped Pure become a success can be found in the book Evaluating User Experience in Games.

Farmville

It’s not just the expensive AAA games which benefit from user testing.  Zynga’s Farmville is everyone’s favourite facebook success story, and has gone from 0 t o 63 million active users in just a few months. This success is largely down to logging the user behaviour, and using this data to improve the player experience.

The online nature of facebook games allow user testing in a way not possible for traditional console games. Metrics can be gathered while people play to log behaviours, such as discovering how long a player spends on each task, which tasks are repeated, and where players get bored and drop off (i.e. only 85% of people make it past the tutorial).

An iterative design process, where the game is evaluated and improved every few days allow Zynga to take advantage of these metrics, and try to improve player behaviour. For example, by running two competing versions of the tutorial, they can judge which one retains more players, and implement that version fully. Repeating this process over every aspect of the game helps make Farmville the addictive success it has become.

Farmville distilled

Conclusion

So, want to help your game be more like these top titles? They all have user testing in common, and have explored them in very different ways, including in house testing, and recording player metrics. It’s likely one of these models will be applicable to your company. These games are all well respected, and most companies want to replicate these success stories.

Not enough to help you convince your company that games usability and user experience is worth pursuing? In the next ‘Selling UX in games’, I’ll be looking at getting team members involved in the UX process.

Related posts:

1. Selling UX in Games – Get everyone involved
2. Games Usability Testing is not QA!
3. Selling Usability by John Rhodes Book Review

Game development teams need a constant supply of fresh users to test the ‘new user’ experience with. I’ve seen teams keep their project secret from their colleagues, not for official reasons, but so their colleagues can be tested as ‘new users’. Other teams test their games with their HR and secretarial staff, since they are unlikely to have had much exposure to the game.

However it’s a good guideline to never test with your team (unless of course you are building something for them). It’s understandable why this situation arises – often budgets are too tight for intensive user testing, forcing teams to perform ad-hoc tests with their colleagues; however this often causes problems further down the line:

such as ...accidents

Dont test with your team

• Your team are not your users – Unless you are in a very specialist field, or are developing an internal project like an intranet, it’s unlikely that your team are the same people as your users. And they are very unlikely to act in the same way a typical user would.
• Your team know things users wouldn’t – It’s likely your team will have had prior exposure to your game or application that a new user wouldn’t, and will be bringing prior knowledge to the testing session. This also applies to people who do not work directly on your team. To get a true outside perspective, you need to seek outside users.
• Your team know you – Unlike a stranger, your team are already know you, and (hopefully) like you. Their answers, and interactions will be biased to please you, and tell you what you want to hear based on what they know about your job, the project your working on, or your beliefs (for example, attempting to validate your design choices).

Advantages of testing with real users.

How people act can often be surprising. If this wasn’t the case, there would be little point in user testing. That’s why it’s extremely important to gather real data, from the people who will actually be using your product. Only real users will approach your product from an authentic ‘new user’ angle, and give an insight into how your product will be perceived and used.

Getting real users involved with product development will get them engaged with the product. Asking their opinions, and being interested in their experience will make the user feel positive about you, and your product, and will mean they will be more likely to purchase it when it’s ready. In newsrooms, this has been widely known for years – hence the proliferation of lists of names in local papers.

Most importantly, involving users will get them talking about your product, generating true grass roots ‘word of mouth’ promotion (hopefully without breaking any NDA’s!). Giving customers an early exposure to your product can build excitement, and market your product for free!

saving millions on megaphones

Conclusion

Finding real users can be cheaper than you think. Not only is it possible to pull people off the street, using the methods suggested above, but new usability testing methods such as remote user testing allow you to find and test real users from the comfort of your office, for very low cost. These days there’s almost no excuse not to test with real users, and it can be just as easy as testing with your team, with much more rewarding returns.

Related posts:

1. The thirty minute facebook game usability test
2. The right environment for user testing
3. How real-world game usability testing is changing

Early chapters in the book try to define what the field of user experience is, and what it means in relation to games. There is a degree of confliction here, with each chapter giving a slightly different interpretation, but they often settle on themes such as immersion, fun, presence, involvement, engagement, flow and playability.

Also wind, fire and heart!

The book gets more exciting when it presents a range of methods for evaluating user experience in games, with a variety of models appropriate for various stages of game development, from prototyping to post-production. This encompasses many custom models for different situations, such as a model for inexperienced gamers, or one for fitness games.  The book also presents studies of the usability of game controllers, and the development of heuristics, which is particularly interesting in the last chapter which aims to collate and amalgamate previously created gaming heuristic.

All this content is interesting; however, a liberal spreading of maths means it often comes across as extremely academic (particularly in comparison to Isbister’s book). This can largely be accounted to the background of the various authors, largely coming from academic institutions, compared to the real world perspective of Game Usability.

Where the book excels is the divergence from this academic interpretation, notably in the papers by Vertical Slice and Emily Brown. Vertical Slice cover the current state of user experience evaluation across three Brighton game companies, and give an insight into the methods used through case studies, from the expert evaluation found in the earliest stages of production, through to the user testing close to the end of a project.

Brown gives a comprehensive overview of the range of tools currently in use, and shows optimism for the future, since she recognises only a “lack of knowledge” as a hindrance to the extension of user testing into gaming, rather than opposition to the methods. This conclusion is reinforced by the case studies by McCallister and White, who show game developers are looking to extend their application of user experience testing in the future.

Which will be the same as today, but with more robots

Unlike Remote Research or Game Usability, this book is not a practical how-to guide. Instead it presents the state of user experience in games, and where the cutting edge of research is. Personally I have found it very useful for developing my own research.  However the book would be unlikely to be a ‘one stop shop’ for a developer looking to start user experience research at their company.

It will be useful to see how the wide range of interesting ideas found within this book can be integrated into practical solutions for companies to use when producing games. This move from the theoretical to the practical will greatly assist the field of user experience in games, and will truly see the ‘coming of age’ of video game usability.

Related posts:

1. Game Usability: Advancing the Player Experience Book Review
2. Remote Research – Book Review
3. A Theory of Fun for Game Design by Ralph Koster – Book Review

Since my recent post on using mechanics from games to make dull tasks fun, a couple of upcoming applications have taken the step of applying some of the lessons from gaming, to make work fun. Epic Win is a ‘to-do’ list disguised as an RPG, and Dev Hub is a website creation tool with points. I’ll look at what these applications are, how they apply game mechanics, and how successful we can expect them to be at making dull tasks fun.

Epic Win

The new iPhone ‘game’ Epic Win is a productivity application (essentially a to-do list?) mixed with an RPG. After inputting your list of tasks (such as wash the dishes, or write a blog post), the game will reward you with XP for completing each task, allowing you to level up your character, as well as granting special items and other rewards.

Achievement Unlocked: Made Dog Useful

There is a trailer for Epic Win here, which may explain the theory better.

Essentially, through giving you points for achieving tasks in the real world, the game aims to incentivise the player to perform the real world tasks, by applying a common game mechanic.

DevHub

DevHub is a website creation tool, focusing on automating the process for creating blogs for niche topics (like this one?) and allowing authors to monetize their site (maybe I should be interested…)

Me, writing this blog

In the initial implementation, they found they had a problem. People were making only the simplest sites, using a small range of DevHub’s features. So to incentivise people to use a fuller range of features, they added game mechanics.

Now tasks like blogging, or linking your site to your facebook profile accrue points, which can then be spent on improvements to the site, such as new templates or widgets. This gives a gradual reveal of the site’s features, and gives the owner (player?) a sense of progression.

Thus game mechanics help DevHub’s users discover, and utilize a wider range of features than before.

Are they fun?

The game mechanics in both of these new applications seem to be simpler than those outlined as successful in the ‘Just Add Points’ presentation I covered recently. Both use points as a mechanic to incentivize players to repeat actions, or go further than they normally would. Points allow the clear construction of goals, and for progress to be measured. I imagine competition will become a key part of these two applications, as social media will allow players to compare and compete on points.

However many of the problems outlined in Sebastian Deterding’s presentation still apply.

• Epic Win doesn’t change the player’s goal (wash the dishes), it just monetises it, meaning the ‘fun’ derives from gaining points
• I cannot see how the validity of the points value can be enforced. Since the goals are self-assigned, and self-reviewed, players who want points will just set tasks such as ‘sit down’, and reward themselves (or just lie altogether).
• This makes points valueless, and hence points don’t help the game add ‘fun’ to achieving tasks.
• Also, this makes social comparison, a key factor in the success of points, flawed or impossible (so no high score tables)

DevHub may also run into trouble, since withholding features that can be found on other competing sites for no effort will only work if your site has a strong unique selling point. We’ll have to see what other monetizing blog hosts do.

It’ll be interesting to see how these applications do over the next few months. Deterding’s presentation implies that neither have a successful model for applying fun to dull tasks. I’m looking forward to seeing what the players think.

Related posts:

1. Applying Games UX lessons makes dull tasks fun!
2. Selling UX in games – everyone else is doing it.
3. Get Lost! – Improving player experience through signposting and map design in games.

The presentation first covers ways in which the UX lessons learnt from games have, or can, be applied to dull tasks to incentivise people to do them. Some examples of this can be found on Volkswagen’s thefuntheory.com website, such as turning a staircase into a piano to encourage people to take the stairs, or turning a bottle bank into a game to encourage people to recycle.

all the encouragement I need...

Deterding does provide some critical analysis of this model – what happens on day 2, for example? Is it still fun to recycle? I also question the justification provided by Volkswagen that the bottle bank performed better than a standard one. Although it did in the example, when the user was provided with a choice between two geographically-close bottle banks, this fails to be a conclusive proof of the fun bottle bank being more effective at encouraging recycling. (would the ‘dull’ one receive an equal amount of recycling to the fun one if there was no alternative – what about over a number of months?)

The typical theory of fun is that ‘adding points’ will magically make dull activities fun, because of It adds competition, re-playability, and a new ‘meta-game’ to the activity taking place. However, Deterding’s presentation challenges this, and says that ‘just adding points’ is a too simplistic understanding of the application of fun to menial tasks. Instead, games present an optimized version of many positive psychological features of real life, and through the recognition of this, real life can be optimised.

As I discussed in my review of Ralph Koster’s book, ‘fun’ is the act of learning and successfully applying, and adapting the knowledge learnt, and typically games present an adaption of this. Games optimise ‘fun’ because:

• They allow the construction of clear, realistic goals, with measurable progress
• The goals are presented in a manageable manner, with a clear ‘call to action’, indicating what is to be done, and when it has been achieved
• The player’s current status is clear, and their progress towards the end goal is indicated
• New tasks are built upon knowledge already gained
• Social comparison can be made with your friends to compare progress

So the obvious way of making dull tasks fun would presumably be to integrate these principles from games? However this conflicts with software, and menial tasks, core goals of efficiency. As I’ve noted before, ‘press a button to win’ is effective, but not fun.

Unlike games, software (and menial tasks) doesn’t give designers full control over the environment – instead the user defines the goal (such as ‘write a letter to the TV Licensing people’). This makes direct application of the features from games difficult.

We have taken re _ _ _ sses_ _ _ of your _ a _ s _ _ r

Instead, Deterding presents us with a list of ‘patterns, models and words for emotion and rule design’, that he has derived from games. Unfortunately, they are not as simple as ‘just add points’!.

I highly recommend watching Deterding’s presentation, it is an effective synopsis of a debate that is very much still in progress, and shows us why a simplified or direct application of Ralph Koster’s rules doesn’t work with non-games, despite what Volkswagen have been showing us. Instead, Deterding presents his own models, which are not as simple, or easy, and yet may turn out to be a more practical lesson for how we can apply knowledge from games to improve the user experience of mundane tasks.

Related posts:

1. Make work fun – examples of applying the UX of games
2. Get Lost! – Improving player experience through signposting and map design in games.
3. Games Usability Testing is not QA!

Why ‘User Experience’

‘Experience’ is simple enough – the object is to design how the person who uses your product feels when they use it – essentially, how they experience it. Simples.

My kind of experience

The aspect of the title that I feel isn’t accurate, is ‘User’. The ‘User’ in User Experience is rooted in traditional computing terminology, derived from authentication requirements, like a log-in, to access directories and applications. The computer’s ability to have multiple people access the same system, and hence create a multi-user environment solidified the term, and the effects of the internet, and hence a worldwide network of authentication has made the term common place.

As such, you can see why the generic term to call someone who uses a computer has become ‘user’. And, even when playing games, this term is still currently used. But maybe it shouldn’t…

Why not?

Why don’t I like the term ‘user experience’ for games? More so than with the web, or applications, gaming suffers from the association with the other main connotation of ‘user’ – drug use, and addiction.

Unfairly, computer games have often been compared to addictions and drug use (and not the nice drugs either!).  The press often cover stories such as “Gaming as addictive as cocaine”, “Addicted to Xbox”, or Gaming Addiction Clinics.

I hated family holidays...

And everyone’s heard about the people who live in World of Warcraft, ordering food through its pizza delivery service, and eternally grinding, letting their ‘real’ life fall to pieces. Social games in particular, through excessive positive reinforcement and social competition, aim to get player’s ‘hooked’, and keep on playing. So, you can see why gaming comes off unfairly when compared to drug use, through the term ‘user’.

I say unfairly as, like most things, gaming is psychologically addictive. In that, if you like doing it, you’ll do it again. But then, so are all the fun things you enjoy, like petting the cat, or reading a book. Addiction is doing these things to excess, and it’s the excess that’s bad, not the activity. Hard drugs on the other hand are chemically addictive. Which is completely different and creates dependence.

Instead…

So, to avoid these comparisons with the other types of users, I prefer the term ‘Player Experience’. Not only does this remove the unsightly comparison, but it’s more accurate than the term user for playing games.

Computer Gaming has little to do with authentication, and exists independent from the platform – as an activity it is closer to games than computing. The designer is crafting the game to change the player’s experience, rather than crafting the computer, and hence the term should reflect gaming’s prominence in this relationship.

Hence I believe that ‘Player Experience’ is a more accurate, and nice, term for describing what is being designed within games.

(I also have a vested interest in the term as I’m on the first page of Google for ‘player experience’, and miles down the list for ‘User Experience’ – that said, more people come to this blog having searched for User Experience.)

However, for the web, and applications, I’m not so sure. Obviously ‘Player Experience’ doesn’t fit. And does computing in general suffer from the negative public image with regards to addiction that gaming does? If not, maybe the term shouldn’t be changed.

Other alternatives, for software, could be ‘Customer Experience’ – however this is rather corporate, and not universally applicable, or simply using ‘Experience Design’, and dropping the user. I would be interested in hearing your thoughts, or alternative terms, or if you prefer ‘User Experience’ as a term. Comment, or add me on Twitter, and let me know what you think!

Related posts:

1. Improving the Player Experience – How to make great loading screens
2. Get Lost! – Improving player experience through signposting and map design in games.
3. Game Usability: Advancing the Player Experience Book Review

There are similar elements, as both involve considering the end user’s experience, and involve getting players to physically play the game. However, they have different goals, and this is what I will be covering today, by looking briefly at the aims of QA, the aims of usability testing and how they differ.

What is QA?

Quality Assurance (or Assessment) is an established field within game development. Often performed at both a developer and publisher level, it typically involves a room full of underpaid gamers endlessly playing a game in every conceivable way. They’re looking for bugs, which will be documented and then passed on to the coding or art department.

Typical bugs would be “the hitbox on that model is wrong”, or “when you shoot the tires on that Jaguar E Type, it makes a metal ricochet sound, not a rubber one”. So the task is largely monotonous, and will involve running into every wall in the game, and testing every dialogue choice, to find one which gives an unexpected result. Moving beyond bugs, QA often includes other areas such as localisation, compliance, and compatibility with a range of devices.

...and taste

Throughout the process, QA is trying to make sure the player can’t get themselves stuck, and no bugs in the game prevent them from completing, or enjoying the game.

What is Usability and Player Experience testing?

Similar to QA, usability and player experience testing involves playing the game. However, this typically wouldn’t be in a similar ‘farm’ setting. Instead, usability tests often attempt to recreate a typical playing environment, to emulate how a player would typically play the game (required: a sofa, and a 2 litre bottle of pepsi .)

Player experience is focused on whether a player enjoys what they do in the game, and whether they understand their goals. This can encompass many sub-categories of player experience, such as how is challenge and interest is maintained, why and where players give up, and how they understand the game. Essentially, the aim of player experience testing is to optimise the game so that people want to play the game.

Meanwhile usability testing blurs slightly more with QA, yet has some fundamental differences. Usability testing is focused on whether the game allows the player to achieve their goals, for example do they notice when something in game changes (such as picking up a new item), or do they understand where they have to go to complete the level. This is different to the ‘bugs’ that QA discovers, since these are game features, not ‘mistakes’.

Why do they differ?

Essentially this is the key difference between the two. QA focuses on the unintentional problems with a game, and aims to make the final product as close to the design features documented. A QA success would be a game with no bugs, and with an implementation that matches the designer’s vision.

My vision is money... lots of money

In contrast, usability and player experience testing aims to influence and guide the designer’s vision. By involving players, they gain an insight into how the game will be received by its audience, and help the designer create the reaction they’re after. As such, the focus is on the game’s intentional aspects, not its unintentional bugs.

This leads to different characteristics in the problems found. QA works largely in the edge cases, and tests every hit box, every wall, every enemy and every dialogue choice. Player experience and usability is much more interested in the average experience, to ensure that players ‘get’ it.

We can see a shift in the gaming industries perceptions of player experience and usability testing, having been championed by leaders such as Microsoft and Valve, more companies are starting to recruit user experience professionals. With the rise of free social games which require the player to ‘get’ games quickly, we can see that the field is becoming a key requirement for success, and is becoming a focal point of how games are made. But it’s not QA!

Related posts:

1. Why you should always video players when testing games
2. How real-world game usability testing is changing
3. The importance of usability in Mobile Geolocation games.

This review aims to evaluate the playability of the game, including pertinent usability issues, and the effect this has on player experience. This is especially important, given 100 Rogues’ mission of making a difficult genre accessible. I’ve based the review on the heuristic points identified by Heather Desurvire’s paper ‘Using Heuristics to evaluate the playability of games’

What this review doesn’t cover is non-usability or player experience issues. Hence, it’s not a review of the game itself (which I gather is a pretty standard Rogue-like). It’s also not QA, and so doesn’t cover bugs in the game. This is especially important as 100 Rogues has infamously been plagued with a number of bugs since its release last week. The first version would crash if the player equipped a shield. The fix for this introduced several new bugs. And I still haven’t been able to get defeat the first boss without the game killing my player after I’ve defeated the boss. I gather the developers are working on this though.

100 Rogues

Game Play

Game play covers the game’s goals, and player’s involvement in achieving these.  100 Rogues succeeds in many areas here. Crucially, it guides the player through their first steps, and introduces them to the range of actions available to them, by immediately giving them the ability to level up their player, which is reinforced by the low cap for the second level up, allowing the player to practise this mechanic twice within the first 2 minutes of the game.

More complex actions are also introduced to the player, through the game’s challenge mode – a series of short scenarios where only the use of an advanced technique (such as ‘tele-stabbing’) will succeed. These introduce the player to some of the more complex moves available to them, in an intuitive way (rather than just… telling them)

The game handles the inevitable death of your character well, also. As mentioned previously, when playing a Rogue-like, you only have one life, no reprieves, and you will die. They explain this by likening the game to Tetris – the goal with your one life is to see how far you can get, not to reach the ‘end’. However death in any specific situation is never inevitable and the game always seems fair. This isn’t Mario Kart, where the CPU will always cheat at the last moment. Instead, after every death you’ll always believe that you could have done something differently and survived, and hence the game encourages a ‘one more go’ attitude to playing, and challenge comes off as a positive experience.

The only game play element which seems poorly balanced, and takes some of the control away from the player, is the ‘food’ mechanic. If you don’t eat food, your character dies. Makes sense. And sometimes you cannot find anything but rotten food, which will fill you up, but reduce your maximum HP. That’s fine too, if it’s a choice between being weakened, and death.  But sometimes the game will spawn no food at all. And then you’re stuck, and nothing you can do will save your character from death. This looks like it needs to be balanced in future games, so the game at least spawns some food (rotten or not) when the player is critically hungry. As it stands, the player doesn’t feel in control of their destiny, and has no ability to prevent their character from dying.

Pictured: 100 Rouges

Story

Story defines how your characters’ actions fit into the world you are presented with and the feeling that the game-space exists as a real world, which you can affect, but which exists without you.

The story in 100 Rogues is simple. Satan is at the bottom of the dungeons, and you must kill him. Why? Because he’s Satan.

This story is introduced by a cut scene at the start of the game, and loading text gives character-related background, however this is where the player’s involvement in the story ends. The in game content, and enemies don’t reflect this final goal (aside from the end boss), and the character development isn’t plot related.  Occasional infighting among the enemies gives the potential for further depth within the story, however this is not explored further. Ultimately, like Tetris, the story of ‘why’ you are doing this is not a focal point of the game.

Mechanics

Mechanics covers consistency in how the game reacts, how the AI acts, and how the player controls their character. The AI in the game is a success, with the NPC’s acting consistently, and yet occasionally surprising the player – such as when an archer’s miss-fired arrow will hit another enemy, and they will start battling it out.  Hence the game balances allowing the player to understand how NPC’s react, without making them entirely predictable.

The game allows the player to track their own progress, through in game score/stats, and a global high-score table. Although implementation of this isn’t perfect, as I’m not convinced scores are being posted to the internet, the bug has been noted and is being ‘worked on’.

An area where the mechanics of 100 Rogues fails is with its controls. The character is controlled by touching the screen – touch the screen at the top to make the player go up, or touch an enemy to attack them. However, with no confirmation before an action is taken, and no indication of the active areas for each target, I found myself ‘miss-clicking’ numerous times, with often deadly consequences. Hence, when trying to click on an enemy for a ranged attack, I’d instead walk towards them, putting my character in danger.  Restricting the range of actions available on a single click, or making ‘attack’ a double click, may help alleviate some of these issues.

Must be my sausage fingers...

Usability

Usability concerns how the game gives feedback for inputting actions, and whether they can achieve their goals. A success for 100 Rogues is how it saves the game state on quit, allowing the game to continue from the last point, as I’ve discussed before

As I discussed above, the game also gives direction to a first time player, by giving them an introduction to levelling up on game start. However, it hinders play the second time you start by… giving the same introduction. Since the game demands multiple play-throughs, I feel that I have grasped this mechanic the 20^th time it has been introduced to me.

Conclusion

As has been made clear by a heuristic evaluation of 100 Rogues, the game has a high degree of playability, and provides an accessible entry point to a traditionally difficult genre. As noted, there is room for improvement, yet the game offers significant advances on other games in this genre.

However, I’d be hesitant to recommend the game, as it stands, as a positive player experience. Although, playability shouldn’t include bugs, bugs will undoubtedly have an effect on player experience. Hence, as the game stands, the unexpected crashes and deaths will detract from player’s opinions of the game. What incentive do players have to give the time and effort of playing, when their character could be taken away from them through no fault of their own?

That said, the development team have been dedicated to fixing bugs – having released two patches in the week after the game was released, and are promising up to two-three times more content released periodically, which is an advantage of the iPhone as a platform. Within a few more iterations, I can see this game being the definitive introduction to the Rogue-like genre.

Related posts:

1. Conducting an Expert Review
2. Game Usability: Advancing the Player Experience Book Review
3. iPhone Game Design Heuristics