Agile is a software development philosophy which encompasses a number of ideals, most interestingly the idea of delivering working software over excessive documentation and planning. Distilled to it’s core essence, it divides project development into short (bi-weekly) ‘sprints’, where an agreed amount of prioritised software is constructed, tested and delivered.

Something like this...

Because of Agile’s emphasis on delivering working software as soon as possible, it gives plenty of opportunities to integrate user insight into the process, and user insight in turn gives a wide range of benefits to software development. In games, it can mean the difference between addictive game-play, or a dull experience.

So, how should user insight be integrated in order to create a successful game?

1. Gather user insight before you start.

Before the project begins, it is typical to have a ‘sprint zero’, to allow necessary preparation/evaluation of the problem to be performed. This is an ideal time to start on user research, since the cost of integrating any findings would be minimal

A variety of techniques could be used to understand users before production has begun.  Paper prototypes, or evaluating competitors products will give insight into what aspects of the game would be desirable, and which are unnecessary. Evaluating competitor’s games is practically free, and will prevent you from making the mistakes that they made!

2. Test at each sprint

Agile delivers working software at the end of each sprint. This gives an opportunity not possible in traditional (waterfall) software delivery – to test an element of your final product with real players. Ignoring the programmer-art graphics, or game-stopping bugs, the core game mechanics can be tested, and refined from a very early stage.

A technique often used to achieve this is ‘staggered sprints’.  This is the idea of running user testing in parallel with development, testing what was created in the last sprint, and feeding back refinements based on user testing during the next development sprint. This will give the team the ability to integrate the feedback from users consistently, and improve the next iteration. It will also settle arguments about features!

3. Release early and iterate

Starting with the proliferation of ‘patches’ for games, updating games is now easier than ever due to automated updating platforms such as the iOS App-store, or Xbox Live.  This means it is now possible to release a game at the earliest possible stage, and then iterate upon it to update it.

This technique has been used to enormous success with facebook games such as CityVille. By releasing a game early, either to beta testers, or the general public, and improving it based on player feedback, it allows you direct insight into what players want, what they don’t want, and prevents you wasting time on unpopular or unnecessary features.

Plus you start getting revenue from a very early stage – just look at Minecraft’s millions!

pictured: notch

4. Prioritise

A key aspect of Agile is the idea of prioritising the workload. Before each sprint, the most important features are decided upon, and only these are worked on. This means that each iteration will produce the ‘best’ version of the game possible at that stage, and any delays/cuts will only affect the least important aspects of the game.

The advantage of integrating user insight into the game development process, as outlined above, is that it can help inform this prioritisation, and ensure that the features being developed are the ones most critical to the players.

5. Evaluate

By far the most important aspect of developing games with agile is understanding what aspects are going well, and what needs revision. By working in short sprints with user insight as a ‘review process’, it is easy to judge what areas of the game are working, and which are not.

If this is contrasted to a closed development process, where the players do not see the game until it’s complete, it is often not possible to understand how features will be received, or whether the game is ‘fun’ until much too late! And if you do decide to make changes? It’ll be a lot more expensive at the end than if you’d caught it at the end of a two week sprint – this limits the rework you’d have to do.

In summary, developing games with an agile methodology produces better quality products, and at a faster pace. Integrating user insight into games development is not only possible with agile, but brings massive boosts to the quality of the final game.

Related posts:

1. UX needs an Agile environment
2. Why you should always video players when testing games
3. Understanding players through biometrics

There are a number of games which aim to disrupt the completion of tasks, with amazing success in controlling the psychology of the player. Here are just a few of them:

Game Dev Story

In Game Dev Story, the player’s job is to run a game studio and create hit games. The game takes the player through a ‘flow’ of designing and building your game, naming it, and releasing it.

Such as monkey tennis

This sequence seems to flow logically, with each section following the previous, and leads to a natural ‘conclusion’ point after a game launches. However, Game Dev Story then takes an important step which helps create it’s addictive nature.

To see how well the game did, the player have to wait weeks for the sales data to come in. However, while waiting, the player’s development team is sitting idle. So, they may as well start building another game, to avoid wasting any precious development time. Essentially, before the player’s felt the first task is ‘complete’, they start another cycle, disrupting the game’s natural ‘end point’. Hence, without a natural place to finish playing, and with an incentive to continue, the player doesn’t feel like they want to, or have the opportunity to, stop playing.

Civilisation Revolution

World-Domination-Simulator-Lite, Civilisation Revolution, allows the player  to embody a famous ruler from history, and take over the world. A key part of this is building cities, exploring the globe, and attacking rivals.

Much like in Game Dev Story, to build things in cities (such as armies, or defences), the player is required to start construction then wait for construction to complete. For things like city walls, this can take up to ten turns. Ten turns in which the player is left hanging, with nothing to do but use their existing armies to attack other states, or get themselves into trouble.

Hence, by the time construction is complete, it is likely that the player will suddenly have a much more interesting scenario to deal with, which will inevitably require more construction, and your attention elsewhere. The ‘downtime’ created by waiting for construction encourages the player to start a new endeavour, and hence keeps their engagement levels up.

Farmville

Farmville, and similar successful ‘social’ games have utilised aspects from the games seen above as a core mechanic.  As discussed in my social games blog post, a key aspect of these games is the delay between planting a crop, and harvesting it. This long wait not only gives players opportunity to explore other tasks and journeys within the game while waiting, but gives the player an incentive to come back to the game later when their crops are done – particularly since Farmville allows the player to choose which crops they plant based on the time to harvest, letting the game fit into their social life

Pictured: My Social Life

How can your game replicate this?

So how can you apply these lessons to your game? Here are the key elements distilled into knowledge-flakes:

Create a flow in your game

• These games all take advantage of a clear task flow, where the player knows what series of actions they need to take do to progress.
• This creates familiarity with the mechanics, and lets the player know how their actions will affect the game world.
• It also gives us the opportunity to….

Disrupt the flow near the end to avoid ‘down-points’

• As was particularly evident in Game Dev Story, which disrupted the player as the flow ends, and gives them a new task or area to explore.
• This removes natural ‘end-points’ for games, and will keep the player active in the game – making sure they don’t want to stop playing!
• This also prevents the player feeling like they are just continually following repetitive actions.

Keep them coming back

• Through careful use of longer-term flows, games like Farmville create reasons for the player to keep coming back.

Giving players the ability to decide how long they need to wait before acting again allows the game to work around their real-world commitments, without being obnoxious!

Related posts:

1. How to make an addictive social game
2. None More Black (My First Flash Game!)
3. The thirty minute facebook game usability test

Today I’ll be looking at the first 5 minutes of a successful social game, and highlighting which elements have made this game successful. In the future we’ll look at some games which have missed the point, and hence lost a large proportion of their customer base. From this, we can learn how to make an addictive social game.

The Game

The game we were originally going to study was intended to be Farmville, since Farmville is famously the ‘poster-child’ of social gaming success. However it wanted me to connect with my facebook account, and wouldn’t let me play without doing so. Game over!  (Presumably this is just an annoyance to me however, and hasn’t stopped the game’s success)

Instead, I looked at the ‘Top Grossing’ iPhone apps, and found ‘Smurf’s Village’. Like farmville, it’s a landforming game, where the player has to create a new village for the Smurfs, and look after their crops. Most importantly, it’s free, yet still a top-grossing app – clearly a sign of it’s continued success. (or was it all the work of this kid charging $1400 to their parents credit card) So, what happens in the first 5 minutes of Smurf’s Village?

The Experience

The first thing that happens when running the game is that it asks if I want to hook it up to facebook. This is a key business goal for the app developer, since it increases awareness of the game and is a key element to how social games work. Importantly for the user experience however, it is optional (unlike Farmville). I declined at this time:

Linking with facebook

After a very short introductory story, explaining why the Smurfs are moving, the game drops you straight into making a new village. With clear directions, the game tells me to click on the sign and lay a plot of land:

Planting a field

And with that, I get an immediate positive reaction “Quest complete”, and reinforcement of the lesson I just learnt:

Quest Complete

Now the same happens again with ‘growing blueberrys’, where an arrow clearly indicates what I need to do, and I’m given positive reinforcement having completed this task:

Planting berries...

And with that, a level up is gained:

Level up!

The game goes on, with the next level teaching you how to build a house, and crucially linking speeded up progress with using smurfberrys. Since smurfberries are the games in-app-purchase, and hence the primary method the developer makes money, this is an important link to make early on:

In app purchases

This earns me another level up. The game then goes on to repeat this to teach me how to build grass, before letting me play a mini-game to break up the monotony. If I were to continue playing beyond the first few minutes, the game would go on to ‘set me free’ and let me use the lesson’s I’ve learnt to continue to build the village unguided.

The Rules

So, what magic rules did we find that led to a successful first 5 minutes with this game?

1. Explain actions one at a time, let the player do them

• The game takes care to only show the player one action at a time, and first describes the task, before guiding them through performing the task themselves, to ensure that players know how to do it. The game then confirms the correct action has taken place after it has been completed, and tells the player what they did.
• By repeating the action numerous times, the game hence reinforces to the player how the action is achieved.

2. Reward the player for performing an action correctly

• When an action has been performed, there are multiple layers of success shown. Not only are there visible indications that the task has taken place, and the player is rewarded with both experience points and a ‘quest complete’ dialog, but the game also levels up the player’s village.
• Hence there is positive reinforcement that the correct action has taken place, causing the player to want to perform it again, much like Peggle’s Ode to Joy

3. Show the player the potential of what they can achieve

• By setting the game after the destruction of the original village, and allowing them to build a new one from the ground-up, this game highlights what can be done by the player if they were to continue playing.
• This message would potentially be reinforced by introducing the social element of the game, for it will be possible to see other player’s complete villages.

4. Let them take control

• After the tutorial period (admittedly longer than the 5 minutes I played for today), the game will hand over the reins to the player, having shown them what’s possible, and give the ability to create solutions in a manner of their own choosing.
• Although there will be continued goals to achieve, the method to achieve these goals becomes more free-form after the tutorial allowing the player to introduce creativity into their gameplay.

All these factors have been an important part of the continued success of this social game. Soon I will look at an alternative, unsuccessful, social game, and note what factors lead to its failure.

Related posts:

1. How to make an addictive strategy game
2. All Change – Apple’s new social gaming network
3. The usability of iPhone’s Game Center

As promoters of user insight in design, it’s therefore unfortunate that recording people secretly is unworkable in most real world situations. Much more importantly, as an aspect of scientific research, recording people in this manner is unethical. Today I’ll be considering why this is so, and look at how to do ethics correctly.

Ethics is an important element of user research, because scientific research without restraint can, and has, harmed people. Consider the experimentation that Dr Mengele performed on humans in concentration camps – his position was that it was serious work and furthering our knowledge scientific and biological knowledge for the benefit of humanity. So does the scientific merit justify these experiments? Consider also the research into Shock Therapy, as documented by Naomi  Klein – where subjects were subjected to seizure-inducing electric shocks, in order to ‘cure’ conditions such as bipolar disorder or catatonia. Subjects were often not in a position to give informed consent, which is an aspect of ethics we’ll be looking at.

It’s highly unlikely our research into user-insight will touch these morally-dubious boundaries, yet scientific research has since formalised many rules in order to prevent research becoming ethically wrong. Hence it’s important, when working directly with users, to be aware of ethical considerations.

So what ethical considerations do we need to follow when working with users?

Primarily there is the issue of informed consent. Consent is the idea of giving permission for research to take place, and the results be presented in a manner defined in the agreement. For consent to be informed, the participant has to have a clear understanding of the facts, implications and consequences of the research. Hence, we can see in the electroshock example given above that subjects, often in catatonic states, were unable to give their informed consent for being participants in research. As researchers, informed consent often has to be given before the research/observation begins, and again after to ensure that the participant is fully aware of what they are agreeing to.

We can see that informed consent becomes difficult to apply when working with special groups, such as children or people with certain disabilities. However it is these cases where the correct application of consent is particularly important, due to the vulnerability of the groups!

A second aspect of correct ethical procedure is allowing users to change their mind and leave during a study, without necessarily giving a reason. Although this has the potential to introduce a bias into the results (since your user group will be restricted to only willing participants), this has benefits in the validity of the data received – an unwilling and unhappy participant may intentionally disrupt an experiment.

Changing your mind...

As a part of this, the participant also has to have free choice over being involved in the experiment – it’s important to consider factors such as social pressure when understanding whether a user is truly free to participate.

Last of all, a guideline when working with real people is ‘Don’t be evil’. Google’s motto, though not always met by their actions, should be a guiding principle in all endeavours – the people you are working with are real people, and their wellbeing is morally more important than your research. Hence we can understand why the Stanford Prison Experiment, which looked into the psychological effects of becoming a prisoner or a guard yet descended into abuse, was prematurely stopped.  Research should be terminated

Yes, even you Doctor Badman

This issue becomes less clear-cut in certain cases. What if your research would harm the participants, but make life better for millions of others? Do the needs of the many outweigh the needs of the few? Spock thought so. However, for usability experts, a good rule of thumb is to consider the experiment from the participant’s viewpoint, and only involve participants in ways you yourself would be happy to perform.

Related posts:

1. How presentation affects perception when working with users
2. Test with real users – not your team
3. Watching ‘average users’: Word

It doesn't mean these guys

User Experience

User experience is a difficult subject to define, particularly in the field of video games and contains many competing elements. At its simplest, user experience is “how does a system (or game) make the user feel?”

As a design philosophy it includes integrating user insight into all stages of a design process (UCD), measuring the user’s experience (what the feel about it), and then using this insight to inform the development. This makes it more than just a ‘step’ in the design process, and instead it becomes an overarching philosophy informing development. Of course, it’s difficult to understand what a user is feeling – although there are many common methods, such as ‘think-aloud’, biometrics or interviews, no method is perfect, and each has its own idiosyncrasies.

User experience is not usability

User experience is more than just usability, although their can often be a direct link between the two.

Usability focuses on the ‘ease of use’, ensuring that the task at hand can be understood, and that a system allows people to achieve their goals. This encompasses factors such as visibility, response times, and learn-ability.

Usability and User Experience link when usability affects how a user feels about the product, whether this is through an easy working process making the user smile, or (and much more likely), a poor workflow making them angry. Consider how annoyed people get when their computer isn’t responsive, or a button causes the wrong action to be performed. Poor usability can therefore influence the user experience of a product or system.

and thats how you get angry birds

User experience and games.

Within games, user experience becomes a more complicated field. In traditional software development, the goals of the user are clear, typically “perform a task quickly and easily”, and emotion should not play a major part in the interaction (beyond ‘don’t annoy the user’). In games, the goal is not so simple, as neither speed nor ease are the primary objectives.

Instead user experience becomes about defining the emotions or feelings your game should create, and ensuring that the game makes this happen. For example, chart favourite Just Dance will have some key emotions and responses it wants to create in its players, such as ‘group enjoyment’, ‘laughter’ and ‘make the players actually get off the sofa’, and so will have been designed to elicit these responses.

Some elements are not unique to each game, for example its common that game’s don’t want to cause unnecessary frustration, however it is important when designing games to declare measurable goals for the user experience, and regularly ensure that these are being met.

User Experience in games is an emerging field, with lots of academic interest in its progress. It will be interesting to see what the future holds, as techniques such as biometrics and the application of user experience methodologies from other fields increase our understanding, and ability to measure, people’s feelings around games.

Related posts:

1. Telling Tales – Stories for promoting user experience.
2. Evaluating User Experience in Games – Book Review
3. The right environment for user testing

I’d be interested in hearing your thoughts on this – please use the comments to add critiques or alternative viewpoints, as I’m sure there will be many! Anyhow, onwards with the 30 minute facebook game usability test.
I’ve been thinking about the social games found on facebook, and I think the most important part is the first 15 minutes of a new player’s experience, e.g. what happens in those fifteen minutes, do they understand the game, and do they want to come back and play again.

This means the most important aspect of user testing is understanding and improving the ‘new user’ experience. For this you’d need some people who’ve never played the game before (and who are close to the target audience of players you want to attract), and simply get them to play the game from the start (without special instructions, just how they would if it was on facebook). Maybe a small incentive (like pizza!) would be enough to recruit people for these short sessions.

a small incentive

Explain to the player that you want them to just play the game as they would at home, and that you are testing the game – not them. Pre planning some notes on this introduction speech will make it easier. It’s probably a legal requirement to let them know if the session is being recorded.

You should have interested developers watch them play through a remote desktop tool (but they shouldn’t interrupt the player!), or record the session to review later. Free tools, such as team viewer, are available to do this. This will be invaluable for getting their buy-in for further user testing, and will prove the benefit of user testing to an often dubious audience.

It’s common to use a ‘think aloud’ methodology, where you ask the player to speak their thoughts aloud as they play. It’s not perfect, but it’ll give some insight into what they’re thinking. You could prompt them by asking non-leading questions such as “what are you doing now” or “what are you thinking” if they’re silent for too long!

what are you thinking?

Then after the fifteen minutes (or completing the tutorial), you can ask them questions to gauge how they understood the game – did they know what they were doing, were they confused by the game, did they know what to do next, would they like to continue playing, etc. Maybe you saw some interesting behaviours while they were playing that you’d like to ask about. Often people don’t remember what they did, and so you’d have to prompt them, or maybe the developers watching have some questions to ask.

Hopefully this quick methodology will show where the user’s are getting confused, or not understanding what to do next, or getting bored, or any other factors that turn player’s away from the game in that crucial first play.

Jakob Nielsen said that most major usability problems can be found by 3-5 users, so you wouldn’t need to run this test many times per iteration of the game. As to when this should be done, its best to get started as soon as there is something playable, as changes will be easier to make the earlier they are found, and then a similar test can be repeated with the next version of the game. Many social games go through an agile design process, with many iterations, and so this process will fit in well.

What do you think? Does this sound helpful? Or do you think that the ‘new user’ experience is not the most important part of a facebook game – maybe converting players to paying players is instead… let me know your thoughts in the comments here, and we can improve this 30 minute facebook game test.

Related posts:

1. Test with real users – not your team
2. How to make an addictive social game
3. Perfect Dark – Game Usability from the 90’s

What is the environment?

With user testing, the environment comes down to the context in which the software or product is used. This includes where the user is, when they are using something, and who they are using it with.  It’s important to not forget the factor that the environment plays when testing a product, otherwise your final results, conclusions and design decisions may end up way off the mark.

Considering the environment for testing is important for both usability and user experience.  For usability testing, a realistic setting is needed to understand how a system will perform in that environment when out in the field. User experience focuses on how a system makes the user feel, and to truly understand this, you need to test users in an equivalent setting. This is why you’ll find comfy sofas wherever games are tested, since games are typically played at home in front of the TV.

pizza boxes are optional

Getting it wrong

Common mistakes with user testing involve inaccurately representing the environment in which a product is to be used (poor “ecological validity”).

This could include an unfair representation of the conditions in which the product would be used – for example testing a car radio without having the user drive would not give a true understanding of how the radio would function when the user’s attention is diverted.

Another common mistake is failing to test things in a time sensitive manner. For example, ticket machines are often used in high stress situations, minutes before the train is going to leave. Giving the user ample time in a comfortable lab setting does not recreate a typical interaction with the ticket machine, and would give an unrealistic impression of its usability.

User tests should also be separated, though concurrent, to the design process, and the development team should not be allowed to run the user sessions (though it’s a good idea to have them spectate from afar). Having the team in the same room as the player can intimidate them into praising the game or product, and reduce the validity of the results.

you liked it... got that?

Getting it right

So how can you be sure to get the environment right for testing?  The following things must be kept in mind:

• The right place: The ideal place would be the actual environment in which the game or product would be used. However this isn’t always possible, and so it’s important to recreate the environment as closely as possible. It’s also a good tip with games to test somewhere away from where the games are made – a typical player would be excited to be at a game studio, and their opinions would be nicer, and more eager than a typical home experience.
• The right mood: The sofa, and snacks mentioned in the introduction are all part of making the environment comfortable for the user, so that they feel at ease when testing the product or game. Another advantage to the previously mentioned idea of running these tests off-site, away from the people who made the product, is that the user won’t feel intimidated into praising the product, and they’ll be free to give a true opinion.
• The right people: It’s important to recruit the realistically, and test with real users. I’ve written before about how you should test with real users, not your team, and it is one small step which can increase the the validity of your results massively.
• The right friends: Remember when testing multiplayer games that a new environment will initially make the users uncomfortable, and this is magnified if they are playing with strangers. If you want to see how your game is received when played among friends, or with families, there is no shortcut – you have to test with groups of friends to get a true representation.

Taking care when selecting the right environment is important whether you are making a game, a ticket machine, or a washing machine for your cat.  A suitable environment for user tests will increase the validity of your results, and ultimately help make informed design decisions.

Related posts:

1. Test with real users – not your team
2. No user testing? Oops! – The Digiscent iSmell
3. Why you should always video players when testing games

Game development teams need a constant supply of fresh users to test the ‘new user’ experience with. I’ve seen teams keep their project secret from their colleagues, not for official reasons, but so their colleagues can be tested as ‘new users’. Other teams test their games with their HR and secretarial staff, since they are unlikely to have had much exposure to the game.

However it’s a good guideline to never test with your team (unless of course you are building something for them). It’s understandable why this situation arises – often budgets are too tight for intensive user testing, forcing teams to perform ad-hoc tests with their colleagues; however this often causes problems further down the line:

such as ...accidents

Dont test with your team

• Your team are not your users – Unless you are in a very specialist field, or are developing an internal project like an intranet, it’s unlikely that your team are the same people as your users. And they are very unlikely to act in the same way a typical user would.
• Your team know things users wouldn’t – It’s likely your team will have had prior exposure to your game or application that a new user wouldn’t, and will be bringing prior knowledge to the testing session. This also applies to people who do not work directly on your team. To get a true outside perspective, you need to seek outside users.
• Your team know you – Unlike a stranger, your team are already know you, and (hopefully) like you. Their answers, and interactions will be biased to please you, and tell you what you want to hear based on what they know about your job, the project your working on, or your beliefs (for example, attempting to validate your design choices).

Advantages of testing with real users.

How people act can often be surprising. If this wasn’t the case, there would be little point in user testing. That’s why it’s extremely important to gather real data, from the people who will actually be using your product. Only real users will approach your product from an authentic ‘new user’ angle, and give an insight into how your product will be perceived and used.

Getting real users involved with product development will get them engaged with the product. Asking their opinions, and being interested in their experience will make the user feel positive about you, and your product, and will mean they will be more likely to purchase it when it’s ready. In newsrooms, this has been widely known for years – hence the proliferation of lists of names in local papers.

Most importantly, involving users will get them talking about your product, generating true grass roots ‘word of mouth’ promotion (hopefully without breaking any NDA’s!). Giving customers an early exposure to your product can build excitement, and market your product for free!

saving millions on megaphones

Conclusion

Finding real users can be cheaper than you think. Not only is it possible to pull people off the street, using the methods suggested above, but new usability testing methods such as remote user testing allow you to find and test real users from the comfort of your office, for very low cost. These days there’s almost no excuse not to test with real users, and it can be just as easy as testing with your team, with much more rewarding returns.

Related posts:

1. The thirty minute facebook game usability test
2. The right environment for user testing
3. How real-world game usability testing is changing

Surveys are a common tool used to evaluate a participant’s opinions of the user experience, and usability of a system. I’ve written about how to make good questionnaires before, and have often seen them used as a tool when analysing a large group of participants. However, as a method of understanding users, they are imperfect, and not just because they are poorly designed – instead it’s a fundamental problem with surveys. Let’s look at why this is the case, and why people are tempted to use surveys despite this.

Where are surveys used?

When I’ve been involved with user tests for games, I’ve often seen surveys used as a way of recording the player’s experience. For example, after completing a level, or game mode, they would be asked to rate their experience on a Likert scale (1-10), on categories such as how difficult they found the level, how fun it was, how it compared to other levels. This is often complemented by text notes, where the participant can write in things they particularly liked or disliked.

Outside of gaming, surveys can often be found on the internet – such as website’s satisfaction surveys, or on professional survey sites, like Survey Monkey.

Survey Monkey in action

Why are surveys used?

It’s easy to understand why surveys are often used when testing user experience. Most obvious is that they are easy to quantify, since the scores are given as a numeric value, which can then be averaged, and given an overall ‘score’. This can then be stuck on a graph, to impress people too busy and important to be involved with the testing itself. Compared to moderated testing, simple analysis is easy, and ‘results’ can be gained with little effort – particularly if an online survey tool is used.

Similarly, with surveys it’s easy to get a large number of opinions quickly, and in a largely un-moderated setting. Hence, 10 (or 10,000) people can test a game at the same time, with only light moderation, and fill out a survey after to record their views. Surveys also don’t require a large degree of specialist equipment – just a printer, and a pen (or they can be done online). This makes them cheaper than many moderated settings, which require a lab decked out with recording equipment.

Problem with surveys

Surveys sound great, don’t they. Cheap, Easy, and give some hard numbers. However, there are a number of problems with surveys, and one key issue that prevent them being suitable for user experience analysis.

First of all, it’s easy for the data from surveys to be misrepresented (either unintentionally or to further a top secret agenda!). Without hard evidence, such as watching (and recording) an individual player of the game, the analysis becomes reduced to which level ‘scores better’, regardless of the intricacies of the play test. Minor issues become lost within the overarching ‘score’.

Much more importantly, the fundamental problem with attempting to understand user experience with a survey is that they log opinions, and not behaviour. People are (sometimes?) stupid, and don’t know what they think. So a player who has had a positive experience throughout a level, and got stuck near the end, will often be left thinking poorly of the entire level. And without an independent observer to monitor, their in-game opinions are lost, or forgotten. Just like I cannot tell how bad my singing is, a player is too close to the subject matter to gain a full understanding of it.

Its pretty bad...

Essentially, surveys introduce a layer of abstraction from the game that is difficult for a player to follow. It is difficult for them to recognise what parts of a game made it fun, and which parts frustrated them, and it often takes someone else to spot these patterns.

Pride, and psychology can also be a contributing factor – players who have needed 10 attempts to complete a section will still say it was “easy” after finally completing it – psychologically they will often believe it as well, since they have felt the satisfaction of completing the task. Other times they will be too proud to say the section was too difficult, and lie. Again, this rich data is lost through a survey.

What should be used instead?

To gain a truer understanding of the user experience (or player experience) of participants when testing a system, or a game, surveys are therefore inadequate. Instead, a moderated task based analysis session, which is recorded for later analysis, will give a truer understanding of how the participant found the system, and their true experience, unaltered by their own perceptions. I’ve written about recording these sessions before, and will discussed them further in the future.

As we have seen, surveys are cheap and easy, and hence should not be disregarded entirely. However they should not be used exclusively, as they can miss key user experience findings, and require users to know themselves, and their feelings, extensively.

Related posts:

1. Why you should always video players when testing games
2. A Terrible User Experience & how to fix it– Zoomerang.com
3. What is user experience?

Why do teams need to understand UX?

When working within a team, whether it’s a small web development company, or a large multi-national company, it’s impossible for one person to oversee every aspect of development. That is why it’s important for everyone to understand the principles of user experience, due to the large variety, and span of tasks that it encompasses. A good user experience designer is not the one who works longer hours, or has more creative control over the final product, instead they are the one who makes sure everyone understands UX  goals, and works towards them.

It is this reason that books such as Krug’s Rocket Surgery Made Easy heavily promote the idea of getting everyone, from the CEO to a junior developer, involved when testing with real user’s, as it is these people who will be able to implement user experience ‘fixes’. The solution Krug suggests involves inviting these people to view the user testing sessions, and bribing them with snacks.

Because encouraging successful user experiences requires everyone’s involvement, it’s important that everyone understands the goals of user experience, and can incorporate them into their work. The problem is making them understand what those goals are.

Pictured: not a goal

What are stories good for?

Homer’s Odyssey was composed in the 8^th century BC, and yet a canonical written version wasn’t produced for another 300 years. Before this, the tale was remembered, and shared orally, through song. Considering that the Odyssey contains over 12,000 lines, this is no small feat. That’d be like remembering, and reciting on demand, every blog post I’ve done so far… So how was it possible?

The answer lies in the power of stories. Stories, like Homer’s Odyssey, are memorable, due to the plot and the events contained within, and hence easily form visual images within the mind, and aid memory. Plots make stories interesting, and hence easier to remember than non-fictional texts (such as remembering all of my blog posts).

Stories can also be used to impart useful information. These be obvious life lessons (The Three Little Pigs teaching the benefit of putting adequate effort into your tasks), or life saving, such as Ring a Ring o’ Roses imparting symptoms of the Black Death (although this may not be true)

or Cinderella teaching us the dangers of stepmothers

Using stories for UX

So, stories are memorable, can be used to impart useful information, and are easily transferred, as made evident by the 300 years that Homer’s Odyssey lasted without being written down. Hence they seem ideal for the promotion of UX within teams, as they can be used to quickly impart the benefits of UX, and what each individual needs to do to aid a user centred design process, in a memorable and transferable fashion. By bringing the whole team onboard with the principles and goals of UX, a greater degree of coherence can be achieved across the team. Therefore to get better results without manually verifying everything the team does, a User Experience Designer needs to champion user experience principles with all members of the team, and stories are an easy way to do this.

Some examples of UX stories

Many authors have understood how stories help to impart information in a memorable fashion.

• In The Design of Every Day Things, Don Norman uses stories of real world examples to emphasise the dangers of ignoring user centred design, with examples including nuclear accidents and plane crashes.
• Stories based on metaphors are also used by Alan Cooper in The Inmates Are Running the Asylum to emphasise the importance of designing for the end user, by comparing the software design process to film making.

Also of interest, Sam Nixon shared a short story by David Travis called “The Fable of User Centered Design”, which aims to bring clients and team members on board with the benefits of a user centred design approach. Definitely worth the read, and should be shared. The book is available to download from his website.

If anyone has other examples of stories that help define the importance of user experience, remember to leave a comment below!

Related posts:

1. What is user experience?
2. The user experience of Ticketmaster ruins Christmas
3. Evaluating existing technologies, paper prototypes in action, Windows 7 and the disappointing user experience of my DVD player!