Getting the whole team involved should be a priority even with the smallest scale tests. Not only is it a vital opportunity to sell the UX process, but it’ll give an undeniably clear example of the benefits that UX testing can bring, and help secure funding to ensure the next round of usability and user experience testing will not be on such a small scale.

Phase 2 of Testing

How to Do it:

So how can we get the whole team involved when user testing is taking place? This can be divided by time frame into 3 key areas.

Before a user testing session:

• Inform everyone that the tests will be happening – send a group email, including details on what will be happening, and when, to all interested parties. This could be sent a week in advance, and on the day of the tests, which will increase awareness of user testing and allow you to….
• Invite people to spectate – Let people know that they can watch the user testing, and that their input would be valued. Perhaps doughnuts will prove a big enough incentive to get people to give up some of their time to spectate.
• Ask priorities and ‘goals’ for investigation – Encourage people to spectate and be invested in the process by asking what they’re interested in finding out, and incorporating this into the user tests. This will be followed up by a debrief, described later.

Getting the preparation before a user test correct will help increase awareness of what you do, and how you can help people, break down misconceptions about user testing, and get the team invested in the process. If a team feel like their priorities matter to you, and that they can help shapre the process to ensure it will help them, they’ll become proponents of user experience and usability testing.

During a session

• Set up a remote viewing session – This should ideally be in a shared conference room, which allows people to spectate the tests in progress. This can be done using cheap/free equipment such as webcams, team viewer, and IM clients. Encourage attendance by advertising free food, and letting people know their priorities will be incorporated.
• Make space in the test for questions from the team – At the end of the session, check with the team if there is anything they want to be asked, for example did they want to know more about a subject’s thoughts when displaying behaviour the team found interesting. Note that it’s important that these questions go through you, as this’ll prevent potentially leading questions getting through the net, and increase the validity of the participant’s responses.

After a session

• Run a group debrief after each session – After the user testing session, meet with all the spectators, and give them freedom to discuss what they saw, and their conclusions from it. This allows the team to share their findings, and will encourage them to get involved again, especially since you will…
• Incorporate their findings into your final report – and give credit!

You don't want to be accused of stealing credit...

Advantages

So why go to all the effort of getting people involved with user testing?

The first advantage is that it will give credibility to your findings – not only will the team have helped shaped the conclusions, but they will have seen firsthand the evidence of that behaviour. It’s pretty hard to deny that, for example, the player doesn’t notice when they pick up a power up, having seen them miss it all day.

A secondary advantage through involving them with the process is that they’re likely to be more invested and enthused about the process – the sort of buzz that can help you gain an investment of more time or money in user testing.

The most useful advantage of getting the wider team involved is that they’ll add their own expertise and critical eye, highlighting areas of importance to them, and helping make observations in areas that you may miss. This can only help your final report, and gives a much wider legitimacy to the user testing.

Making the whole team aware, and involved in the user testing process therefore provides advantages to everyone – not only yourself, but to the whole team. It’ll help promote the process, and sell further user testing, and therefore is a key aspect of selling UX testing to games companies.

Related posts:

1. Selling UX in games – everyone else is doing it.
2. Test with real users – not your team
3. Why you should always video players when testing games

This inaccurate view is set to change, with the rise of in-house and independent usability and user experience labs, and the breadth of academic research increasing our understanding of user experience. Hence I’m planning a series of articles about my thoughts on selling UX to game companies, helping you to communicate why game companies should be thinking about UX with their games. New posts in this series will come under the category ‘Selling UX in Games’

Everyone else is doing it

So why should your company be interested in games usability and user experience? Well I’m glad you asked…

• Many top selling games have gone through user experience testing
• The companies who have tried it recognise the benefit, and go on to do it again. In fact, one developer said…
• “It adds 10% to the metacritic score”

So let’s look in more depth at some games that have been reaping the benefits of user experience testing. These popular games all found success through user experience testing, and can serve as helpful examples when communicating why your company should be looking into user experience.

oh... well, thats easy to understand

Halo 3

Bungie recognised the value of the player experience when developing their multi-million selling Halo 3, and hence used user experience testing to create a game that was consistently fun, ensuring the player was never confused or lost. As documented in Isbister’s book Game Usability Microsoft’s usability labs were brought on board to ensure that Halo 3 was fun throughout.

Each area was thoroughly tested with real users, using techniques such as video and ‘heatmapping’ to log where the choke points were, where people got lost or stuck, and where they died. This could then be used to easily find problems in the game, and fix these areas. Fixes are typically simple, either by signposting the way to go, altering the enemies or adding points of interest to draw players (like a sparking light to encourage players to look up). This technique could also be applied to multiplayer maps, to balance the game’s flow, and make the game fair.

The success of this technique in Halo 3 has been extended to Halo Reach, where telemetric data was gained from a long public beta testing period. This gave real player information on how to balance and improve the game, and will undoubtedly be a large factor in its success.

To read more about the Halo 3 user testing experience, see the coverage in Wired.

Split Second

Brighton based Black Rock Studios explored user experience testing in their successful racing games ‘Pure’ and ‘Split Second’.  With consultation from the usability lab Vertical Slice they tested over 100 players, getting new user’s experiences to fine tune aspects of the game, ensuring the pacing was correct, and that the game was delivering key ‘fun’ moments, which they denoted ‘woah moments’.

They built on this successful experience from Pure in developing Split Second, where they combined in house talent, who worked on testing users, with assistance from a local university, who performed an expert evaluation of the usability and user experience issues.

As documented by Graham McAllister, the developer believed that the process “added 10% to their metacritic score”. More information about how user experience testing helped Pure become a success can be found in the book Evaluating User Experience in Games.

Farmville

It’s not just the expensive AAA games which benefit from user testing.  Zynga’s Farmville is everyone’s favourite facebook success story, and has gone from 0 t o 63 million active users in just a few months. This success is largely down to logging the user behaviour, and using this data to improve the player experience.

The online nature of facebook games allow user testing in a way not possible for traditional console games. Metrics can be gathered while people play to log behaviours, such as discovering how long a player spends on each task, which tasks are repeated, and where players get bored and drop off (i.e. only 85% of people make it past the tutorial).

An iterative design process, where the game is evaluated and improved every few days allow Zynga to take advantage of these metrics, and try to improve player behaviour. For example, by running two competing versions of the tutorial, they can judge which one retains more players, and implement that version fully. Repeating this process over every aspect of the game helps make Farmville the addictive success it has become.

Farmville distilled

Conclusion

So, want to help your game be more like these top titles? They all have user testing in common, and have explored them in very different ways, including in house testing, and recording player metrics. It’s likely one of these models will be applicable to your company. These games are all well respected, and most companies want to replicate these success stories.

Not enough to help you convince your company that games usability and user experience is worth pursuing? In the next ‘Selling UX in games’, I’ll be looking at getting team members involved in the UX process.

Related posts:

1. Selling UX in Games – Get everyone involved
2. Games Usability Testing is not QA!
3. Selling Usability by John Rhodes Book Review

What is the environment?

With user testing, the environment comes down to the context in which the software or product is used. This includes where the user is, when they are using something, and who they are using it with.  It’s important to not forget the factor that the environment plays when testing a product, otherwise your final results, conclusions and design decisions may end up way off the mark.

Considering the environment for testing is important for both usability and user experience.  For usability testing, a realistic setting is needed to understand how a system will perform in that environment when out in the field. User experience focuses on how a system makes the user feel, and to truly understand this, you need to test users in an equivalent setting. This is why you’ll find comfy sofas wherever games are tested, since games are typically played at home in front of the TV.

pizza boxes are optional

Getting it wrong

Common mistakes with user testing involve inaccurately representing the environment in which a product is to be used (poor “ecological validity”).

This could include an unfair representation of the conditions in which the product would be used – for example testing a car radio without having the user drive would not give a true understanding of how the radio would function when the user’s attention is diverted.

Another common mistake is failing to test things in a time sensitive manner. For example, ticket machines are often used in high stress situations, minutes before the train is going to leave. Giving the user ample time in a comfortable lab setting does not recreate a typical interaction with the ticket machine, and would give an unrealistic impression of its usability.

User tests should also be separated, though concurrent, to the design process, and the development team should not be allowed to run the user sessions (though it’s a good idea to have them spectate from afar). Having the team in the same room as the player can intimidate them into praising the game or product, and reduce the validity of the results.

you liked it... got that?

Getting it right

So how can you be sure to get the environment right for testing?  The following things must be kept in mind:

• The right place: The ideal place would be the actual environment in which the game or product would be used. However this isn’t always possible, and so it’s important to recreate the environment as closely as possible. It’s also a good tip with games to test somewhere away from where the games are made – a typical player would be excited to be at a game studio, and their opinions would be nicer, and more eager than a typical home experience.
• The right mood: The sofa, and snacks mentioned in the introduction are all part of making the environment comfortable for the user, so that they feel at ease when testing the product or game. Another advantage to the previously mentioned idea of running these tests off-site, away from the people who made the product, is that the user won’t feel intimidated into praising the product, and they’ll be free to give a true opinion.
• The right people: It’s important to recruit the realistically, and test with real users. I’ve written before about how you should test with real users, not your team, and it is one small step which can increase the the validity of your results massively.
• The right friends: Remember when testing multiplayer games that a new environment will initially make the users uncomfortable, and this is magnified if they are playing with strangers. If you want to see how your game is received when played among friends, or with families, there is no shortcut – you have to test with groups of friends to get a true representation.

Taking care when selecting the right environment is important whether you are making a game, a ticket machine, or a washing machine for your cat.  A suitable environment for user tests will increase the validity of your results, and ultimately help make informed design decisions.

Related posts:

1. Test with real users – not your team
2. No user testing? Oops! – The Digiscent iSmell
3. Why you should always video players when testing games

Game development teams need a constant supply of fresh users to test the ‘new user’ experience with. I’ve seen teams keep their project secret from their colleagues, not for official reasons, but so their colleagues can be tested as ‘new users’. Other teams test their games with their HR and secretarial staff, since they are unlikely to have had much exposure to the game.

However it’s a good guideline to never test with your team (unless of course you are building something for them). It’s understandable why this situation arises – often budgets are too tight for intensive user testing, forcing teams to perform ad-hoc tests with their colleagues; however this often causes problems further down the line:

such as ...accidents

Dont test with your team

• Your team are not your users – Unless you are in a very specialist field, or are developing an internal project like an intranet, it’s unlikely that your team are the same people as your users. And they are very unlikely to act in the same way a typical user would.
• Your team know things users wouldn’t – It’s likely your team will have had prior exposure to your game or application that a new user wouldn’t, and will be bringing prior knowledge to the testing session. This also applies to people who do not work directly on your team. To get a true outside perspective, you need to seek outside users.
• Your team know you – Unlike a stranger, your team are already know you, and (hopefully) like you. Their answers, and interactions will be biased to please you, and tell you what you want to hear based on what they know about your job, the project your working on, or your beliefs (for example, attempting to validate your design choices).

Advantages of testing with real users.

How people act can often be surprising. If this wasn’t the case, there would be little point in user testing. That’s why it’s extremely important to gather real data, from the people who will actually be using your product. Only real users will approach your product from an authentic ‘new user’ angle, and give an insight into how your product will be perceived and used.

Getting real users involved with product development will get them engaged with the product. Asking their opinions, and being interested in their experience will make the user feel positive about you, and your product, and will mean they will be more likely to purchase it when it’s ready. In newsrooms, this has been widely known for years – hence the proliferation of lists of names in local papers.

Most importantly, involving users will get them talking about your product, generating true grass roots ‘word of mouth’ promotion (hopefully without breaking any NDA’s!). Giving customers an early exposure to your product can build excitement, and market your product for free!

saving millions on megaphones

Conclusion

Finding real users can be cheaper than you think. Not only is it possible to pull people off the street, using the methods suggested above, but new usability testing methods such as remote user testing allow you to find and test real users from the comfort of your office, for very low cost. These days there’s almost no excuse not to test with real users, and it can be just as easy as testing with your team, with much more rewarding returns.

Related posts:

1. The thirty minute facebook game usability test
2. The right environment for user testing
3. How real-world game usability testing is changing

Early chapters in the book try to define what the field of user experience is, and what it means in relation to games. There is a degree of confliction here, with each chapter giving a slightly different interpretation, but they often settle on themes such as immersion, fun, presence, involvement, engagement, flow and playability.

Also wind, fire and heart!

The book gets more exciting when it presents a range of methods for evaluating user experience in games, with a variety of models appropriate for various stages of game development, from prototyping to post-production. This encompasses many custom models for different situations, such as a model for inexperienced gamers, or one for fitness games.  The book also presents studies of the usability of game controllers, and the development of heuristics, which is particularly interesting in the last chapter which aims to collate and amalgamate previously created gaming heuristic.

All this content is interesting; however, a liberal spreading of maths means it often comes across as extremely academic (particularly in comparison to Isbister’s book). This can largely be accounted to the background of the various authors, largely coming from academic institutions, compared to the real world perspective of Game Usability.

Where the book excels is the divergence from this academic interpretation, notably in the papers by Vertical Slice and Emily Brown. Vertical Slice cover the current state of user experience evaluation across three Brighton game companies, and give an insight into the methods used through case studies, from the expert evaluation found in the earliest stages of production, through to the user testing close to the end of a project.

Brown gives a comprehensive overview of the range of tools currently in use, and shows optimism for the future, since she recognises only a “lack of knowledge” as a hindrance to the extension of user testing into gaming, rather than opposition to the methods. This conclusion is reinforced by the case studies by McCallister and White, who show game developers are looking to extend their application of user experience testing in the future.

Which will be the same as today, but with more robots

Unlike Remote Research or Game Usability, this book is not a practical how-to guide. Instead it presents the state of user experience in games, and where the cutting edge of research is. Personally I have found it very useful for developing my own research.  However the book would be unlikely to be a ‘one stop shop’ for a developer looking to start user experience research at their company.

It will be useful to see how the wide range of interesting ideas found within this book can be integrated into practical solutions for companies to use when producing games. This move from the theoretical to the practical will greatly assist the field of user experience in games, and will truly see the ‘coming of age’ of video game usability.

Related posts:

1. Game Usability: Advancing the Player Experience Book Review
2. Remote Research – Book Review
3. A Theory of Fun for Game Design by Ralph Koster – Book Review

Since my recent post on using mechanics from games to make dull tasks fun, a couple of upcoming applications have taken the step of applying some of the lessons from gaming, to make work fun. Epic Win is a ‘to-do’ list disguised as an RPG, and Dev Hub is a website creation tool with points. I’ll look at what these applications are, how they apply game mechanics, and how successful we can expect them to be at making dull tasks fun.

Epic Win

The new iPhone ‘game’ Epic Win is a productivity application (essentially a to-do list?) mixed with an RPG. After inputting your list of tasks (such as wash the dishes, or write a blog post), the game will reward you with XP for completing each task, allowing you to level up your character, as well as granting special items and other rewards.

Achievement Unlocked: Made Dog Useful

There is a trailer for Epic Win here, which may explain the theory better.

Essentially, through giving you points for achieving tasks in the real world, the game aims to incentivise the player to perform the real world tasks, by applying a common game mechanic.

DevHub

DevHub is a website creation tool, focusing on automating the process for creating blogs for niche topics (like this one?) and allowing authors to monetize their site (maybe I should be interested…)

Me, writing this blog

In the initial implementation, they found they had a problem. People were making only the simplest sites, using a small range of DevHub’s features. So to incentivise people to use a fuller range of features, they added game mechanics.

Now tasks like blogging, or linking your site to your facebook profile accrue points, which can then be spent on improvements to the site, such as new templates or widgets. This gives a gradual reveal of the site’s features, and gives the owner (player?) a sense of progression.

Thus game mechanics help DevHub’s users discover, and utilize a wider range of features than before.

Are they fun?

The game mechanics in both of these new applications seem to be simpler than those outlined as successful in the ‘Just Add Points’ presentation I covered recently. Both use points as a mechanic to incentivize players to repeat actions, or go further than they normally would. Points allow the clear construction of goals, and for progress to be measured. I imagine competition will become a key part of these two applications, as social media will allow players to compare and compete on points.

However many of the problems outlined in Sebastian Deterding’s presentation still apply.

• Epic Win doesn’t change the player’s goal (wash the dishes), it just monetises it, meaning the ‘fun’ derives from gaining points
• I cannot see how the validity of the points value can be enforced. Since the goals are self-assigned, and self-reviewed, players who want points will just set tasks such as ‘sit down’, and reward themselves (or just lie altogether).
• This makes points valueless, and hence points don’t help the game add ‘fun’ to achieving tasks.
• Also, this makes social comparison, a key factor in the success of points, flawed or impossible (so no high score tables)

DevHub may also run into trouble, since withholding features that can be found on other competing sites for no effort will only work if your site has a strong unique selling point. We’ll have to see what other monetizing blog hosts do.

It’ll be interesting to see how these applications do over the next few months. Deterding’s presentation implies that neither have a successful model for applying fun to dull tasks. I’m looking forward to seeing what the players think.

Related posts:

1. Applying Games UX lessons makes dull tasks fun!
2. Selling UX in games – everyone else is doing it.
3. Get Lost! – Improving player experience through signposting and map design in games.

Surveys are a common tool used to evaluate a participant’s opinions of the user experience, and usability of a system. I’ve written about how to make good questionnaires before, and have often seen them used as a tool when analysing a large group of participants. However, as a method of understanding users, they are imperfect, and not just because they are poorly designed – instead it’s a fundamental problem with surveys. Let’s look at why this is the case, and why people are tempted to use surveys despite this.

Where are surveys used?

When I’ve been involved with user tests for games, I’ve often seen surveys used as a way of recording the player’s experience. For example, after completing a level, or game mode, they would be asked to rate their experience on a Likert scale (1-10), on categories such as how difficult they found the level, how fun it was, how it compared to other levels. This is often complemented by text notes, where the participant can write in things they particularly liked or disliked.

Outside of gaming, surveys can often be found on the internet – such as website’s satisfaction surveys, or on professional survey sites, like Survey Monkey.

Survey Monkey in action

Why are surveys used?

It’s easy to understand why surveys are often used when testing user experience. Most obvious is that they are easy to quantify, since the scores are given as a numeric value, which can then be averaged, and given an overall ‘score’. This can then be stuck on a graph, to impress people too busy and important to be involved with the testing itself. Compared to moderated testing, simple analysis is easy, and ‘results’ can be gained with little effort – particularly if an online survey tool is used.

Similarly, with surveys it’s easy to get a large number of opinions quickly, and in a largely un-moderated setting. Hence, 10 (or 10,000) people can test a game at the same time, with only light moderation, and fill out a survey after to record their views. Surveys also don’t require a large degree of specialist equipment – just a printer, and a pen (or they can be done online). This makes them cheaper than many moderated settings, which require a lab decked out with recording equipment.

Problem with surveys

Surveys sound great, don’t they. Cheap, Easy, and give some hard numbers. However, there are a number of problems with surveys, and one key issue that prevent them being suitable for user experience analysis.

First of all, it’s easy for the data from surveys to be misrepresented (either unintentionally or to further a top secret agenda!). Without hard evidence, such as watching (and recording) an individual player of the game, the analysis becomes reduced to which level ‘scores better’, regardless of the intricacies of the play test. Minor issues become lost within the overarching ‘score’.

Much more importantly, the fundamental problem with attempting to understand user experience with a survey is that they log opinions, and not behaviour. People are (sometimes?) stupid, and don’t know what they think. So a player who has had a positive experience throughout a level, and got stuck near the end, will often be left thinking poorly of the entire level. And without an independent observer to monitor, their in-game opinions are lost, or forgotten. Just like I cannot tell how bad my singing is, a player is too close to the subject matter to gain a full understanding of it.

Its pretty bad...

Essentially, surveys introduce a layer of abstraction from the game that is difficult for a player to follow. It is difficult for them to recognise what parts of a game made it fun, and which parts frustrated them, and it often takes someone else to spot these patterns.

Pride, and psychology can also be a contributing factor – players who have needed 10 attempts to complete a section will still say it was “easy” after finally completing it – psychologically they will often believe it as well, since they have felt the satisfaction of completing the task. Other times they will be too proud to say the section was too difficult, and lie. Again, this rich data is lost through a survey.

What should be used instead?

To gain a truer understanding of the user experience (or player experience) of participants when testing a system, or a game, surveys are therefore inadequate. Instead, a moderated task based analysis session, which is recorded for later analysis, will give a truer understanding of how the participant found the system, and their true experience, unaltered by their own perceptions. I’ve written about recording these sessions before, and will discussed them further in the future.

As we have seen, surveys are cheap and easy, and hence should not be disregarded entirely. However they should not be used exclusively, as they can miss key user experience findings, and require users to know themselves, and their feelings, extensively.

Related posts:

1. Why you should always video players when testing games
2. A Terrible User Experience & how to fix it– Zoomerang.com
3. The right environment for user testing

The presentation first covers ways in which the UX lessons learnt from games have, or can, be applied to dull tasks to incentivise people to do them. Some examples of this can be found on Volkswagen’s thefuntheory.com website, such as turning a staircase into a piano to encourage people to take the stairs, or turning a bottle bank into a game to encourage people to recycle.

all the encouragement I need...

Deterding does provide some critical analysis of this model – what happens on day 2, for example? Is it still fun to recycle? I also question the justification provided by Volkswagen that the bottle bank performed better than a standard one. Although it did in the example, when the user was provided with a choice between two geographically-close bottle banks, this fails to be a conclusive proof of the fun bottle bank being more effective at encouraging recycling. (would the ‘dull’ one receive an equal amount of recycling to the fun one if there was no alternative – what about over a number of months?)

The typical theory of fun is that ‘adding points’ will magically make dull activities fun, because of It adds competition, re-playability, and a new ‘meta-game’ to the activity taking place. However, Deterding’s presentation challenges this, and says that ‘just adding points’ is a too simplistic understanding of the application of fun to menial tasks. Instead, games present an optimized version of many positive psychological features of real life, and through the recognition of this, real life can be optimised.

As I discussed in my review of Ralph Koster’s book, ‘fun’ is the act of learning and successfully applying, and adapting the knowledge learnt, and typically games present an adaption of this. Games optimise ‘fun’ because:

• They allow the construction of clear, realistic goals, with measurable progress
• The goals are presented in a manageable manner, with a clear ‘call to action’, indicating what is to be done, and when it has been achieved
• The player’s current status is clear, and their progress towards the end goal is indicated
• New tasks are built upon knowledge already gained
• Social comparison can be made with your friends to compare progress

So the obvious way of making dull tasks fun would presumably be to integrate these principles from games? However this conflicts with software, and menial tasks, core goals of efficiency. As I’ve noted before, ‘press a button to win’ is effective, but not fun.

Unlike games, software (and menial tasks) doesn’t give designers full control over the environment – instead the user defines the goal (such as ‘write a letter to the TV Licensing people’). This makes direct application of the features from games difficult.

We have taken re _ _ _ sses_ _ _ of your _ a _ s _ _ r

Instead, Deterding presents us with a list of ‘patterns, models and words for emotion and rule design’, that he has derived from games. Unfortunately, they are not as simple as ‘just add points’!.

I highly recommend watching Deterding’s presentation, it is an effective synopsis of a debate that is very much still in progress, and shows us why a simplified or direct application of Ralph Koster’s rules doesn’t work with non-games, despite what Volkswagen have been showing us. Instead, Deterding presents his own models, which are not as simple, or easy, and yet may turn out to be a more practical lesson for how we can apply knowledge from games to improve the user experience of mundane tasks.

Related posts:

1. Make work fun – examples of applying the UX of games
2. Get Lost! – Improving player experience through signposting and map design in games.
3. Games Usability Testing is not QA!

Don't judge it by it's cover...

Why would you consider remote research?

Written by advocates of remote research, the book highlights many of the potential advantages that remote research gives compared to a more traditional lab based study. These advantages are fleshed out throughout the book through testimonies of experts who have experience in this field, who offer real world examples to emphasise these points.

Some key advantages are:

• Access to a geographically diverse user base. Unlike traditional research, where a moderator would have to be in the same physical location as the subjects, remote research allows a study to be run with anyone who has a high speed internet connection, widely expanding the potential study-group.
• Easy to let stakeholders get involved. Because the research session is being broadcast over the internet, it’s possible to allow stakeholders (i.e. executives and designers) to view the session, and give (moderated) input. This of course increases their engagement with the process, and will be the ‘evidence’ for any conclusions derived from the research.
• Natural browsing environment. The validity of the research can be improved, not only because you are allowing the user to perform the task in a familiar environment (their own home computer), but also some recruitment methods allow you to capture a user performing a task they have selected. For example, recruiting a user who came to the site to buy trousers, for a task based on buying trousers, would provide more accurate results than asking someone to pretend to buy trousers…
• Cheaper (debatably). Not having to pay for travel can keep costs down, however other costs, such as incentives, will still be required, as well as paying for the software.

The remote research book doesn’t advocate killing off lab tests though – instead, it recognises that there are cases when the lab is still appropriate, such as when privacy is a concern. The book also features Andy Budd’s defence of the lab, which argues that remote research fails to pick up aspects of non-verbal behaviour, as well as arguing that remote research doesn’t just remove a selection bias (geography), since it also adds another (internet speed and technical ability). It’s brave of the book to include the case against remote research, and helps project a more trustworthy and reliable image for the book itself.

How to do remote research

The ‘meat’ of the book are the sections dedicated to how-to guides on the different forms of remote research. The book contains step by step instructions on performing moderated or un-moderated research, and includes key topics such as recruitment (and live recruiting), card sorts, and lots of handy hints – such as using IM clients as a chat room for multiple observers to automatically share and timestamp notes.

The book doesn’t just cover basic topics – it goes on to develop novel approaches to user research, such as using ‘reverse screen sharing’ to protect confidential software or data, and using mobile web to gain a new understanding of time-dependant information, outside of the traditional moderated setting.

It also extends the remits of remote research – it doesn’t have to just be websites, but can include doodles or sketches, as well as developing ideas for automatic research with analytics.

Another sort of remote research?

Conclusion

Remote Research is one of the easiest to read UX books I’ve reviewed. Like many Rosenfeld publications, it is laid out well, without appearing dense with text, and has a friendly tone throughout. The book can be likened to Krug’s writing in its style, and presentation.

The book is also practical and realistic, and deals with real world issues, like ‘fakers’ (who can be outed by using open ended questions to discover motives), legal issues, and common challenges such as reluctant stakeholders.

Most importantly for the practical UX practitioner, the book is not dogmatic. This is especially evident in the last chapter which admits that usability shouldn’t be the exclusive goal of product design, and needs to be coupled with initiative, and innovation to develop great things.

Overall this book is a great introduction, and how-to guide to the growing field of remote research, and will be an important tool for anyone trying to keep up to date with the latest research methods.

Related posts:

1. Rocket Surgery Made Easy by Steve Krug – Book Review
2. Evaluating User Experience in Games – Book Review
3. Selling Usability by John Rhodes Book Review

Why ‘User Experience’

‘Experience’ is simple enough – the object is to design how the person who uses your product feels when they use it – essentially, how they experience it. Simples.

My kind of experience

The aspect of the title that I feel isn’t accurate, is ‘User’. The ‘User’ in User Experience is rooted in traditional computing terminology, derived from authentication requirements, like a log-in, to access directories and applications. The computer’s ability to have multiple people access the same system, and hence create a multi-user environment solidified the term, and the effects of the internet, and hence a worldwide network of authentication has made the term common place.

As such, you can see why the generic term to call someone who uses a computer has become ‘user’. And, even when playing games, this term is still currently used. But maybe it shouldn’t…

Why not?

Why don’t I like the term ‘user experience’ for games? More so than with the web, or applications, gaming suffers from the association with the other main connotation of ‘user’ – drug use, and addiction.

Unfairly, computer games have often been compared to addictions and drug use (and not the nice drugs either!).  The press often cover stories such as “Gaming as addictive as cocaine”, “Addicted to Xbox”, or Gaming Addiction Clinics.

I hated family holidays...

And everyone’s heard about the people who live in World of Warcraft, ordering food through its pizza delivery service, and eternally grinding, letting their ‘real’ life fall to pieces. Social games in particular, through excessive positive reinforcement and social competition, aim to get player’s ‘hooked’, and keep on playing. So, you can see why gaming comes off unfairly when compared to drug use, through the term ‘user’.

I say unfairly as, like most things, gaming is psychologically addictive. In that, if you like doing it, you’ll do it again. But then, so are all the fun things you enjoy, like petting the cat, or reading a book. Addiction is doing these things to excess, and it’s the excess that’s bad, not the activity. Hard drugs on the other hand are chemically addictive. Which is completely different and creates dependence.

Instead…

So, to avoid these comparisons with the other types of users, I prefer the term ‘Player Experience’. Not only does this remove the unsightly comparison, but it’s more accurate than the term user for playing games.

Computer Gaming has little to do with authentication, and exists independent from the platform – as an activity it is closer to games than computing. The designer is crafting the game to change the player’s experience, rather than crafting the computer, and hence the term should reflect gaming’s prominence in this relationship.

Hence I believe that ‘Player Experience’ is a more accurate, and nice, term for describing what is being designed within games.

(I also have a vested interest in the term as I’m on the first page of Google for ‘player experience’, and miles down the list for ‘User Experience’ – that said, more people come to this blog having searched for User Experience.)

However, for the web, and applications, I’m not so sure. Obviously ‘Player Experience’ doesn’t fit. And does computing in general suffer from the negative public image with regards to addiction that gaming does? If not, maybe the term shouldn’t be changed.

Other alternatives, for software, could be ‘Customer Experience’ – however this is rather corporate, and not universally applicable, or simply using ‘Experience Design’, and dropping the user. I would be interested in hearing your thoughts, or alternative terms, or if you prefer ‘User Experience’ as a term. Comment, or add me on Twitter, and let me know what you think!

Related posts:

1. Improving the Player Experience – How to make great loading screens
2. Get Lost! – Improving player experience through signposting and map design in games.
3. Game Usability: Advancing the Player Experience Book Review