I’d be interested in hearing your thoughts on this – please use the comments to add critiques or alternative viewpoints, as I’m sure there will be many! Anyhow, onwards with the 30 minute facebook game usability test.
I’ve been thinking about the social games found on facebook, and I think the most important part is the first 15 minutes of a new player’s experience, e.g. what happens in those fifteen minutes, do they understand the game, and do they want to come back and play again.

This means the most important aspect of user testing is understanding and improving the ‘new user’ experience. For this you’d need some people who’ve never played the game before (and who are close to the target audience of players you want to attract), and simply get them to play the game from the start (without special instructions, just how they would if it was on facebook). Maybe a small incentive (like pizza!) would be enough to recruit people for these short sessions.

a small incentive

Explain to the player that you want them to just play the game as they would at home, and that you are testing the game – not them. Pre planning some notes on this introduction speech will make it easier. It’s probably a legal requirement to let them know if the session is being recorded.

You should have interested developers watch them play through a remote desktop tool (but they shouldn’t interrupt the player!), or record the session to review later. Free tools, such as team viewer, are available to do this. This will be invaluable for getting their buy-in for further user testing, and will prove the benefit of user testing to an often dubious audience.

It’s common to use a ‘think aloud’ methodology, where you ask the player to speak their thoughts aloud as they play. It’s not perfect, but it’ll give some insight into what they’re thinking. You could prompt them by asking non-leading questions such as “what are you doing now” or “what are you thinking” if they’re silent for too long!

what are you thinking?

Then after the fifteen minutes (or completing the tutorial), you can ask them questions to gauge how they understood the game – did they know what they were doing, were they confused by the game, did they know what to do next, would they like to continue playing, etc. Maybe you saw some interesting behaviours while they were playing that you’d like to ask about. Often people don’t remember what they did, and so you’d have to prompt them, or maybe the developers watching have some questions to ask.

Hopefully this quick methodology will show where the user’s are getting confused, or not understanding what to do next, or getting bored, or any other factors that turn player’s away from the game in that crucial first play.

Jakob Nielsen said that most major usability problems can be found by 3-5 users, so you wouldn’t need to run this test many times per iteration of the game. As to when this should be done, its best to get started as soon as there is something playable, as changes will be easier to make the earlier they are found, and then a similar test can be repeated with the next version of the game. Many social games go through an agile design process, with many iterations, and so this process will fit in well.

What do you think? Does this sound helpful? Or do you think that the ‘new user’ experience is not the most important part of a facebook game – maybe converting players to paying players is instead… let me know your thoughts in the comments here, and we can improve this 30 minute facebook game test.

Related posts:

1. Test with real users – not your team
2. How real-world game usability testing is changing
3. Game Usability: Advancing the Player Experience Book Review

What is the environment?

With user testing, the environment comes down to the context in which the software or product is used. This includes where the user is, when they are using something, and who they are using it with.  It’s important to not forget the factor that the environment plays when testing a product, otherwise your final results, conclusions and design decisions may end up way off the mark.

Considering the environment for testing is important for both usability and user experience.  For usability testing, a realistic setting is needed to understand how a system will perform in that environment when out in the field. User experience focuses on how a system makes the user feel, and to truly understand this, you need to test users in an equivalent setting. This is why you’ll find comfy sofas wherever games are tested, since games are typically played at home in front of the TV.

pizza boxes are optional

Getting it wrong

Common mistakes with user testing involve inaccurately representing the environment in which a product is to be used (poor “ecological validity”).

This could include an unfair representation of the conditions in which the product would be used – for example testing a car radio without having the user drive would not give a true understanding of how the radio would function when the user’s attention is diverted.

Another common mistake is failing to test things in a time sensitive manner. For example, ticket machines are often used in high stress situations, minutes before the train is going to leave. Giving the user ample time in a comfortable lab setting does not recreate a typical interaction with the ticket machine, and would give an unrealistic impression of its usability.

User tests should also be separated, though concurrent, to the design process, and the development team should not be allowed to run the user sessions (though it’s a good idea to have them spectate from afar). Having the team in the same room as the player can intimidate them into praising the game or product, and reduce the validity of the results.

you liked it... got that?

Getting it right

So how can you be sure to get the environment right for testing?  The following things must be kept in mind:

• The right place: The ideal place would be the actual environment in which the game or product would be used. However this isn’t always possible, and so it’s important to recreate the environment as closely as possible. It’s also a good tip with games to test somewhere away from where the games are made – a typical player would be excited to be at a game studio, and their opinions would be nicer, and more eager than a typical home experience.
• The right mood: The sofa, and snacks mentioned in the introduction are all part of making the environment comfortable for the user, so that they feel at ease when testing the product or game. Another advantage to the previously mentioned idea of running these tests off-site, away from the people who made the product, is that the user won’t feel intimidated into praising the product, and they’ll be free to give a true opinion.
• The right people: It’s important to recruit the realistically, and test with real users. I’ve written before about how you should test with real users, not your team, and it is one small step which can increase the the validity of your results massively.
• The right friends: Remember when testing multiplayer games that a new environment will initially make the users uncomfortable, and this is magnified if they are playing with strangers. If you want to see how your game is received when played among friends, or with families, there is no shortcut – you have to test with groups of friends to get a true representation.

Taking care when selecting the right environment is important whether you are making a game, a ticket machine, or a washing machine for your cat.  A suitable environment for user tests will increase the validity of your results, and ultimately help make informed design decisions.

Related posts:

1. Test with real users – not your team
2. No user testing? Oops! – The Digiscent iSmell
3. Why you should always video players when testing games

Game development teams need a constant supply of fresh users to test the ‘new user’ experience with. I’ve seen teams keep their project secret from their colleagues, not for official reasons, but so their colleagues can be tested as ‘new users’. Other teams test their games with their HR and secretarial staff, since they are unlikely to have had much exposure to the game.

However it’s a good guideline to never test with your team (unless of course you are building something for them). It’s understandable why this situation arises – often budgets are too tight for intensive user testing, forcing teams to perform ad-hoc tests with their colleagues; however this often causes problems further down the line:

such as ...accidents

Dont test with your team

• Your team are not your users – Unless you are in a very specialist field, or are developing an internal project like an intranet, it’s unlikely that your team are the same people as your users. And they are very unlikely to act in the same way a typical user would.
• Your team know things users wouldn’t – It’s likely your team will have had prior exposure to your game or application that a new user wouldn’t, and will be bringing prior knowledge to the testing session. This also applies to people who do not work directly on your team. To get a true outside perspective, you need to seek outside users.
• Your team know you – Unlike a stranger, your team are already know you, and (hopefully) like you. Their answers, and interactions will be biased to please you, and tell you what you want to hear based on what they know about your job, the project your working on, or your beliefs (for example, attempting to validate your design choices).

Advantages of testing with real users.

How people act can often be surprising. If this wasn’t the case, there would be little point in user testing. That’s why it’s extremely important to gather real data, from the people who will actually be using your product. Only real users will approach your product from an authentic ‘new user’ angle, and give an insight into how your product will be perceived and used.

Getting real users involved with product development will get them engaged with the product. Asking their opinions, and being interested in their experience will make the user feel positive about you, and your product, and will mean they will be more likely to purchase it when it’s ready. In newsrooms, this has been widely known for years – hence the proliferation of lists of names in local papers.

Most importantly, involving users will get them talking about your product, generating true grass roots ‘word of mouth’ promotion (hopefully without breaking any NDA’s!). Giving customers an early exposure to your product can build excitement, and market your product for free!

saving millions on megaphones

Conclusion

Finding real users can be cheaper than you think. Not only is it possible to pull people off the street, using the methods suggested above, but new usability testing methods such as remote user testing allow you to find and test real users from the comfort of your office, for very low cost. These days there’s almost no excuse not to test with real users, and it can be just as easy as testing with your team, with much more rewarding returns.

Related posts:

1. The thirty minute facebook game usability test
2. The right environment for user testing
3. How real-world game usability testing is changing

Surveys are a common tool used to evaluate a participant’s opinions of the user experience, and usability of a system. I’ve written about how to make good questionnaires before, and have often seen them used as a tool when analysing a large group of participants. However, as a method of understanding users, they are imperfect, and not just because they are poorly designed – instead it’s a fundamental problem with surveys. Let’s look at why this is the case, and why people are tempted to use surveys despite this.

Where are surveys used?

When I’ve been involved with user tests for games, I’ve often seen surveys used as a way of recording the player’s experience. For example, after completing a level, or game mode, they would be asked to rate their experience on a Likert scale (1-10), on categories such as how difficult they found the level, how fun it was, how it compared to other levels. This is often complemented by text notes, where the participant can write in things they particularly liked or disliked.

Outside of gaming, surveys can often be found on the internet – such as website’s satisfaction surveys, or on professional survey sites, like Survey Monkey.

Survey Monkey in action

Why are surveys used?

It’s easy to understand why surveys are often used when testing user experience. Most obvious is that they are easy to quantify, since the scores are given as a numeric value, which can then be averaged, and given an overall ‘score’. This can then be stuck on a graph, to impress people too busy and important to be involved with the testing itself. Compared to moderated testing, simple analysis is easy, and ‘results’ can be gained with little effort – particularly if an online survey tool is used.

Similarly, with surveys it’s easy to get a large number of opinions quickly, and in a largely un-moderated setting. Hence, 10 (or 10,000) people can test a game at the same time, with only light moderation, and fill out a survey after to record their views. Surveys also don’t require a large degree of specialist equipment – just a printer, and a pen (or they can be done online). This makes them cheaper than many moderated settings, which require a lab decked out with recording equipment.

Problem with surveys

Surveys sound great, don’t they. Cheap, Easy, and give some hard numbers. However, there are a number of problems with surveys, and one key issue that prevent them being suitable for user experience analysis.

First of all, it’s easy for the data from surveys to be misrepresented (either unintentionally or to further a top secret agenda!). Without hard evidence, such as watching (and recording) an individual player of the game, the analysis becomes reduced to which level ‘scores better’, regardless of the intricacies of the play test. Minor issues become lost within the overarching ‘score’.

Much more importantly, the fundamental problem with attempting to understand user experience with a survey is that they log opinions, and not behaviour. People are (sometimes?) stupid, and don’t know what they think. So a player who has had a positive experience throughout a level, and got stuck near the end, will often be left thinking poorly of the entire level. And without an independent observer to monitor, their in-game opinions are lost, or forgotten. Just like I cannot tell how bad my singing is, a player is too close to the subject matter to gain a full understanding of it.

Its pretty bad...

Essentially, surveys introduce a layer of abstraction from the game that is difficult for a player to follow. It is difficult for them to recognise what parts of a game made it fun, and which parts frustrated them, and it often takes someone else to spot these patterns.

Pride, and psychology can also be a contributing factor – players who have needed 10 attempts to complete a section will still say it was “easy” after finally completing it – psychologically they will often believe it as well, since they have felt the satisfaction of completing the task. Other times they will be too proud to say the section was too difficult, and lie. Again, this rich data is lost through a survey.

What should be used instead?

To gain a truer understanding of the user experience (or player experience) of participants when testing a system, or a game, surveys are therefore inadequate. Instead, a moderated task based analysis session, which is recorded for later analysis, will give a truer understanding of how the participant found the system, and their true experience, unaltered by their own perceptions. I’ve written about recording these sessions before, and will discussed them further in the future.

As we have seen, surveys are cheap and easy, and hence should not be disregarded entirely. However they should not be used exclusively, as they can miss key user experience findings, and require users to know themselves, and their feelings, extensively.

Related posts:

1. Why you should always video players when testing games
2. A Terrible User Experience & how to fix it– Zoomerang.com
3. The right environment for user testing

Why do teams need to understand UX?

When working within a team, whether it’s a small web development company, or a large multi-national company, it’s impossible for one person to oversee every aspect of development. That is why it’s important for everyone to understand the principles of user experience, due to the large variety, and span of tasks that it encompasses. A good user experience designer is not the one who works longer hours, or has more creative control over the final product, instead they are the one who makes sure everyone understands UX  goals, and works towards them.

It is this reason that books such as Krug’s Rocket Surgery Made Easy heavily promote the idea of getting everyone, from the CEO to a junior developer, involved when testing with real user’s, as it is these people who will be able to implement user experience ‘fixes’. The solution Krug suggests involves inviting these people to view the user testing sessions, and bribing them with snacks.

Because encouraging successful user experiences requires everyone’s involvement, it’s important that everyone understands the goals of user experience, and can incorporate them into their work. The problem is making them understand what those goals are.

Pictured: not a goal

What are stories good for?

Homer’s Odyssey was composed in the 8^th century BC, and yet a canonical written version wasn’t produced for another 300 years. Before this, the tale was remembered, and shared orally, through song. Considering that the Odyssey contains over 12,000 lines, this is no small feat. That’d be like remembering, and reciting on demand, every blog post I’ve done so far… So how was it possible?

The answer lies in the power of stories. Stories, like Homer’s Odyssey, are memorable, due to the plot and the events contained within, and hence easily form visual images within the mind, and aid memory. Plots make stories interesting, and hence easier to remember than non-fictional texts (such as remembering all of my blog posts).

Stories can also be used to impart useful information. These be obvious life lessons (The Three Little Pigs teaching the benefit of putting adequate effort into your tasks), or life saving, such as Ring a Ring o’ Roses imparting symptoms of the Black Death (although this may not be true)

or Cinderella teaching us the dangers of stepmothers

Using stories for UX

So, stories are memorable, can be used to impart useful information, and are easily transferred, as made evident by the 300 years that Homer’s Odyssey lasted without being written down. Hence they seem ideal for the promotion of UX within teams, as they can be used to quickly impart the benefits of UX, and what each individual needs to do to aid a user centred design process, in a memorable and transferable fashion. By bringing the whole team onboard with the principles and goals of UX, a greater degree of coherence can be achieved across the team. Therefore to get better results without manually verifying everything the team does, a User Experience Designer needs to champion user experience principles with all members of the team, and stories are an easy way to do this.

Some examples of UX stories

Many authors have understood how stories help to impart information in a memorable fashion.

• In The Design of Every Day Things, Don Norman uses stories of real world examples to emphasise the dangers of ignoring user centred design, with examples including nuclear accidents and plane crashes.
• Stories based on metaphors are also used by Alan Cooper in The Inmates Are Running the Asylum to emphasise the importance of designing for the end user, by comparing the software design process to film making.

Also of interest, Sam Nixon shared a short story by David Travis called “The Fable of User Centered Design”, which aims to bring clients and team members on board with the benefits of a user centred design approach. Definitely worth the read, and should be shared. The book is available to download from his website.

If anyone has other examples of stories that help define the importance of user experience, remember to leave a comment below!

Related posts:

1. The user experience of Ticketmaster ruins Christmas
2. Evaluating existing technologies, paper prototypes in action, Windows 7 and the disappointing user experience of my DVD player!
3. The right environment for user testing

The UXBrighton event was presented in a new format as a series of short talks, from Harry Brignull’s tips on time stamping notes, to Danny Hope’s templates for understanding user roles. Also interesting was a talk on using google analytics, although the length of the talk meant that topic could only be skimmed, dissapointing as I’m an analytics fan. The most interesting idea presented was iCrossing’s presentation on “The iCrossing Connected Brand index: how to measure a brand’s effectiveness online”, given by Ifraz Mughal.

Expert Evaluation

As I’ve mentioned before an expert evaluation is a useful tool for getting an insight into potential usability and user experience issues on a website, or game, with limited resources. Although it can never replace running tests with real users, it can provide a quick approximation, and help highlight the biggest issues.

The ‘method’ for an expert evaluation is simple. Get an expert to look at the site, or game, and tell the client what they think. Job done.

My expert eye tells me you need smarter users...

However an expert evaluation can only ever be subjective, and this is it’s biggest weakness. A client can look at your page full of recommendations, and dismiss it as the opinion of one person. There’s no easy way to see progress with changes, and a comparison with other sites can only ever be abstract.

Quantifying an Expert Evaluation

iCrossing’s solution is to quantify their expert evaluation. As part of their ‘Connected Brand Index’ idea, they rate their clients sites (and competitors), on UX-centric areas such as “usefulness”, “usability” and “desirability”.

A traditional expert evaluation would give a qualitative rating, and give examples to back this up, i.e. “Poor – little emphasis, and diffused call to actions”. Instead iCrossing will give the site a score, on a scale of -2 to 2 (2 being very good). This of course can be backed up with examples in a more in depth report.

after the first few pages, the report can just be pictures of kittens. No-one reads that far.

The advantages:

There are numerous reasons why a client would prefer a scored ‘rating’, rather than comments.

• A ‘score’ makes it easy to benchmark, and compare your own scores against competitors. By dividing the expert evaluation into separate topics, and scoring each, a finely grained comparison can be made, and communicated
• Similarly, a score makes it easy for a client to see progress. If they scored -1 before hiring you, and 1 after, your work can be justified (as long as no-one questions who is doing the scoring!)
• Because this produces a concrete score, clients will be able to handle and communicate the data. Graphs can be made, which wouldn’t be possible for subjective comments. These can be invaluable for justifying and communicating with managers and project sponsors, who do not need to see the details, just get a high-level overview.
• This expert evaluation can be encompassed as one aspect of a larger ‘score’ given to websites, or games. This is the idea behind iCrossing’s connected brands index.

Conclusions:

There is an argument this can be seen as a bit of a scam. Giving arbitrary numbers to your opinions doesn’t make them any less subjective. This method of presenting the data could be misleading if presented incorrectly, and the client should be made aware of the method behind the score system. This could become an issue when running comparative studies before and after your work, since you’d be biased towards giving the site a better score after you’ve worked on it.

The point of this method is to aid communication with the client, and give them data in a format that is useful to them. As I discussed in the review of Selling Usability, management and non-technical people would typically much rather see pretty graphs, and statistics, than a list of comments. This method helps manage client expectations, and gives them what they want.

To make the method more valid, it would be useful to perform a study to ensure the method is sound. Perhaps get a wide range of experts to independently rate a wide range of websites on this scale, and note the correlations between the scores. It’d be first step in countering complaints that this method is still inherently subjective, and help make an art into a science.

Related posts:

1. Conducting an Expert Review
2. How to present a UX Report
3. Understanding players through biometrics

The uncanny valley was originally discovered in the field of robotics, but also frequently applies to video games. It describes a phenomenon with replications of humans, whether they are life-like androids or avatars on a computer game. Initially, as the reproduction of a human and its movement becomes more lifelike we react more positively towards the object, so we’d like Lara Croft more than Leisure Suit Larry. However a point is reached, when the reproduction becomes too life-like, and the emotional response drops rapidly, meaning we feel repulsed from the object. Consider Keanu Reeves’ acting. Almost human, but utterly repulsive!

The term ‘uncanny valley’ therefore comes from plotting a graph showing our emotional response against how lifelike the reproduction is, with a sharp ‘valley’ appearing in the emotional response between a very lifelike reproduction, and the real thing.

Got that?

A similar phenomena can be seen in the production of wireframes, and hence is of critical importance to UX designers. After spending hours producing beautiful wireframes in Omnigraffle, I presented them to a client, to show how their ‘event registration’ pages would function. They came back and said “yeah it looks good, but we need to change that label text… and we need to make the dropdown arrow bolder… and can we make the heading font bigger”. This was their first view of the wireframes, to approve whether they functioned correctly, and it’s obvious what I’d done wrong.

The time and effort I’d put into making the wireframes look good, and look like a real website weren’t just wasted, they were actually hindering the process – since the mock up looked like a real webpage, the client was focusing on the small presentational details, and not the functionality itself. They expected it to look and function like the final product. If instead, I’d done a rough sketch on paper to demonstrate how the registration process should work, the client would have focused on the functionality instead. A design that looks to be in the early stages will encourage more far-reaching comments and criticism, rather than ‘fine-tuning’.

This is most important when you’re trying to focus the user experience when performing tests, especially with people not overly familiar with your site or game. Performing tests to ascertain the correct information architecture, or user’s experiences with a website’s functionality would be useless if all your comments ended up being about the site’s colour scheme. To make it clear that the designs are rough, and the presentation is not the focus, it is important not to create overly realistic designs.

Similarly care should be taken to pick an appropriate prototyping method. Paper isn’t used just because it’s quick and easy, but it also helps manage the client’s expectations. If you spent the time making the webpage on a computer, they’d be expecting it to work like a real product. On paper, this isn’t the case. Like the uncanny valley, getting too close to the real thing will be detrimental to the client’s perception of your work.

And you wouldn't want your work to look like this...

So, what steps do we need to take to ensure that the client, or user will focus on the right areas of your wireframes and designs?

• When performing initial designs, use lo-fi methods, like paper, post-stick notes, and whiteboards, where possible
• If using design software, like Pidoco or Omnigraffle, use a ‘sketches’ template, which renders your design in a pseudo-drawn method.
• Avoid drawing/designing unnecessary parts of the design – focus only on the essential
• Use filler text, and don’t work on the copy until later.
• Make it clear to the user/client that these are rough, disposable prototypes.

So, whether you’re working with a client to design their site, or conducting user testing, take care not to over-present the design, in order to manage expectations, and prevent unnecessary complaints!

What is the Likert scale?

Lets start with an example.

Most people have seen a Likert scale before. Do you agree with this statement?

• Strongly agree
• Agree
• Neither agree or disagree
• Disagree
• Strongly disagree

And the responses should be balanced... unless you have an agenda

Often used to gauge opinions, they are especially important for people involved with measuring usability or player experience, as they can help quantify subjective things like a user’s experiences. They are usually in the form of a statement, followed by a selection of statements, to indicate how far someone agrees with the statement. They can often be used to quantify things like ease-of-use, or fun, which would be impossible to quantify through other methods. Hence they are of particularly important for us, since user experience is essentially abstract.

Different kinds of Likert scales.

The essential question when it comes to implementing a Likert scale, is how many responses to offer.

‘Forced Choice’ scales are those which have an even number of options. Essentially this means missing out the ‘neither agree or disagree’ option, and forcing the participant to make a selection (see what they did with the name? very clever!). This would be done to force participants to show an opinion, but there are dangers inherent with this. Forcing a response may give a larger degree of ‘static’ in the responses, reducing their accuracy, since the responses may not map their opinions. People who don’t agree or disagree may not be happy about being forced to give an opinion, reducing their likelihood to answer later questions accurately. However if your aim is to support a conclusion that people do/do-not like a system, you may be willing to risk these to prove your point when designing the survey.

Forced choice means its hard to tell who is neutral, and who doesn’t want to participate

If you select to use a scale with an odd number of options, there are a few issues that should be kept in mind when deciding between a five or seven point scale. The most obvious difference is that a finer grain of responses can be analysed from a seven point scale, as it can represent a wider range of views. Also, take into account that it’s been shown participants shy away from the ‘edges’, the extreme like and dislike options offered. This means a five point scale will likely only get responses in the ‘slightly’ columns from all except the most ardent fanatics. Again, you have to consider whether a wider range of responses is useful to the topic you are exploring.

Should you use a Likert Scale

Ultimately if you are trying to track opinions, a Likert scale is a good method of accessing this data. There is no all-encompassing correct answer for which scale is appropriate, the context of use and what you want to find out will all affect this. As long as you keep in mind that not only the phrasing of the question, but the range and number of responses you offer will affect the results, and anticipate this affect, you can’t really go wrong. Happy surveying!

Related posts:

1. 7 aspects of successful usability questionnaires
2. The Problems with Surveys for User Experience Tests
3. Five essential iPhone apps for usability professionals

How to make a Persona

            We’re going to assume you’ve already take some steps towards understanding the users. You could have designed a survey, or performed one on one interviews, and now you have a pretty good understanding of who your users are, and what they think about your product.

            First we need to divide the people into demographics. Did all the women over 50 find your game idea too violent? Did most of the children think it sounded like it would get dull?  Try to focus on the key areas where people picked up on things, and ignore the outliers for now – you want to turn all your results into 3-5 personae, of which only one or two will be the ‘key’ users you are designing for.

            Now you want to write about these groups you’ve divided the people into. Make them real people; give them names, occupations and a back story. Make their name an in-joke, that’ll make you laugh, but no-one else will ever get (Hello Bibi Andersson). Include some text about how they’d use the product, and what they’d expect from it. Also include some concerns that they may have. You want these people to be treated as real people, and so they need to be complete! Include a picture, so that people can see the people they are designing for.

pictures make everything more fun

Why you should be making personae

            There are many advantages to making usability and user experience personae. As a usability expert, it can help you by:

• Communicating your findings in a way that will be read (unlike that big 50 page report)
• Can be stuck on the wall, keeping your role in the team prominent
• Allowing you to make a definitive impact on the design of the product
• Incorporating usability and user experience principles from the start of the design process, promoting the use of your field as a holistic practice, rather than a ‘last stage’ of product development.

not that sort of persona

Why they need you to make personae

            You obviously don’t just make personae for your own benefit. The design team benefits in many ways from your deliverable as well. Not only does it give them an easy way to understand your findings, and focuses on the aspects that are important in their roles, but it gives them a concrete person, or group of people, to design for.

A persona gives an important baseline for future developments by the design team. By understanding who they are designing for, it reduces coder’s natural urge to spend too long designing for the ‘edge case’ users, and focuses their attention on the core functionality required (perhaps an example of how to maximise with the 80/20 rule ). This reduces feature creep, as the team can consider whether the personae identified would actually use the new suggested features, and as such helps deliver games and projects on time, and to budget. I’m guessing that 3D realms did not use personae.

Related posts:

1. The right environment for user testing
2. No user testing? Oops! – The Digiscent iSmell
3. Effective UI by the EffectiveUI team – Book Review

Recently I’ve been involved with the testing of the user experience for a number of high profile upcoming games. The results of these sessions have always been very interesting, and undeniably shown that running tests with real players gives a true understanding of the issues in the games, finding issues that often cannot be seen by the developers. However, all the places I’ve worked at have all been making the same, easily fixable mistake of not videoing the player’s reactions while playing.

a mistake never made by this man

There are many possible excuses given why people running player tests on games fail to record the player. In a hectic working environment, it can be difficult to find time to review the video with tight deadlines hanging overhead. Similarly, the initial set up costs, of a camera or software that will record the player seems like an area where cutbacks can be made with a tight budget. Often tester’s will try to justify this – surely they can get the same information, and cheaper, by using surveys and interviews after the event?

Here is just of few of the things missed when you fail to video a player’s experience

• A timeline of the player experience, not just their end of play feelings
• The ability to pinpoint the ‘key’ fun moments
• The ability to pinpoint the dullest moments.
• The ability to verify the opinions given to you by a post-game survey
• A key ‘visual’ way of showing the stakeholders real player’s experiences (useful for presentations)

Would this experience be the same without video?

Having this data leads to the production of better games – the incredibly dull level, that is only redeemed by an exciting boss battle need not be an incredibly dull level, and videoing the players can mean the difference between picking up on these moments and missing them. If you miss them, you’ll never fix these problem moments. It is well worth the time to review what aspects of the game the player especially enjoyed, or hated, without the ‘bias’ introduced by the time delay before getting their opinions. If you are asking them to ‘think aloud’ (and you should be!), a video will also keep notes of the opinions given throughout the session. The problems identified can then be fixed.

User (or player) experience is of utmost importance for game designers – unlike software, which businesses can force upon their employees, everyone who plays games does so because they enjoy doing so. Videoing their reactions can lead to a more enjoyable game, and makes your game more consistently fun, and how can that be a bad thing?

Related posts:

1. Games Usability Testing is not QA!
2. Understanding players through biometrics
3. The right environment for user testing