Posts Tagged ‘qualitative’
multi.player Conference Review – Social Interaction, Bartle and Orcs!
I’ve just got back from a few days in Hohenheim, at the multi.player conference, and wanted to share my experiences. The conference was on “The social aspects of digital gaming”, and was a fantastic chance to meet people working in a similar field to myself.
The talk Pejman and I gave, on measuring social interaction in collocated gaming sessions, seemed to be really well received, and has received some insightful comments. I was a bit worried before about presenting an application of the Bartle Test to the man himself, however he didn’t seem to complain (much…). The research gave a better insight into how different player types react in collocated sessions, and hence was distinct from the other presentations – we also had a strong link to industry!
User Centered Design vs. Genius Method – Which Approach Is Best for you?
This is a guest blog post by Devin Jordan of IdentityMine.com. Devin discusses some of the benefits of using an expert-evaluation based model, rather than a pure user-centred method. Read on to see his argument, and comment on what you think!
Ethnography as an application of third space theory
After many competing companies had failed, IBM were tasked with creating an air traffic control system. As you can imagine in this setting a correct solution was crucial – lives were at risk if anything went wrong. The first thing IBM’s designers did was go to the air traffic control tower for a few weeks, and watch how they worked. But why? And how did this influence the design process? Today, we look at ethnographic research, and how it is based in the theory of the ‘third space’.
Selling UX in Games – Get everyone involved
This post forms part 2 of the series of selling UX to games companies, focusing on how you, as a proponent of user testing can overcome the major obstacles stopping game companies from investing in this emerging field. This week I’m focusing on visibility and reveal how getting people involved is key to selling UX testing. Specifically, I cover how to get people involved with usability and user experience testing, and the many advantages this will bring to both them and you.
Getting the whole team involved should be a priority even with the smallest scale tests. Not only is it a vital opportunity to sell the UX process, but it’ll give an undeniably clear example of the benefits that UX testing can bring, and help secure funding to ensure the next round of usability and user experience testing will not be on such a small scale.
Phase 2 of Testing
How to Do it:
So how can we get the whole team involved when user testing is taking place? This can be divided by time frame into 3 key areas.
Before a user testing session:
- Inform everyone that the tests will be happening – send a group email, including details on what will be happening, and when, to all interested parties. This could be sent a week in advance, and on the day of the tests, which will increase awareness of user testing and allow you to….
- Invite people to spectate – Let people know that they can watch the user testing, and that their input would be valued. Perhaps doughnuts will prove a big enough incentive to get people to give up some of their time to spectate.
- Ask priorities and ‘goals’ for investigation – Encourage people to spectate and be invested in the process by asking what they’re interested in finding out, and incorporating this into the user tests. This will be followed up by a debrief, described later.
Getting the preparation before a user test correct will help increase awareness of what you do, and how you can help people, break down misconceptions about user testing, and get the team invested in the process. If a team feel like their priorities matter to you, and that they can help shapre the process to ensure it will help them, they’ll become proponents of user experience and usability testing.
During a session
- Set up a remote viewing session – This should ideally be in a shared conference room, which allows people to spectate the tests in progress. This can be done using cheap/free equipment such as webcams, team viewer, and IM clients. Encourage attendance by advertising free food, and letting people know their priorities will be incorporated.
- Make space in the test for questions from the team – At the end of the session, check with the team if there is anything they want to be asked, for example did they want to know more about a subject’s thoughts when displaying behaviour the team found interesting. Note that it’s important that these questions go through you, as this’ll prevent potentially leading questions getting through the net, and increase the validity of the participant’s responses.
After a session
- Run a group debrief after each session – After the user testing session, meet with all the spectators, and give them freedom to discuss what they saw, and their conclusions from it. This allows the team to share their findings, and will encourage them to get involved again, especially since you will…
- Incorporate their findings into your final report – and give credit!
You don't want to be accused of stealing credit...
Advantages
So why go to all the effort of getting people involved with user testing?
The first advantage is that it will give credibility to your findings – not only will the team have helped shaped the conclusions, but they will have seen firsthand the evidence of that behaviour. It’s pretty hard to deny that, for example, the player doesn’t notice when they pick up a power up, having seen them miss it all day.
A secondary advantage through involving them with the process is that they’re likely to be more invested and enthused about the process – the sort of buzz that can help you gain an investment of more time or money in user testing.
The most useful advantage of getting the wider team involved is that they’ll add their own expertise and critical eye, highlighting areas of importance to them, and helping make observations in areas that you may miss. This can only help your final report, and gives a much wider legitimacy to the user testing.
Making the whole team aware, and involved in the user testing process therefore provides advantages to everyone – not only yourself, but to the whole team. It’ll help promote the process, and sell further user testing, and therefore is a key aspect of selling UX testing to games companies.
The thirty minute facebook game usability test
Bit of a cheat this week, since this is an adoption of a recent email conversation I had, but I think it may be of interest to a wider audience. The idea is a proposed ‘simple’ study, suitable for a facebook or flash game, that will give an insight into major usability issues with a game. The focus is on getting the key insights quickly and cheaply, which will allow you to improve your game straight away with actionable results.
I’d be interested in hearing your thoughts on this – please use the comments to add critiques or alternative viewpoints, as I’m sure there will be many! Anyhow, onwards with the 30 minute facebook game usability test.
I’ve been thinking about the social games found on facebook, and I think the most important part is the first 15 minutes of a new player’s experience, e.g. what happens in those fifteen minutes, do they understand the game, and do they want to come back and play again.
This means the most important aspect of user testing is understanding and improving the ‘new user’ experience. For this you’d need some people who’ve never played the game before (and who are close to the target audience of players you want to attract), and simply get them to play the game from the start (without special instructions, just how they would if it was on facebook). Maybe a small incentive (like pizza!) would be enough to recruit people for these short sessions.
a small incentive
Explain to the player that you want them to just play the game as they would at home, and that you are testing the game – not them. Pre planning some notes on this introduction speech will make it easier. It’s probably a legal requirement to let them know if the session is being recorded.
You should have interested developers watch them play through a remote desktop tool (but they shouldn’t interrupt the player!), or record the session to review later. Free tools, such as team viewer, are available to do this. This will be invaluable for getting their buy-in for further user testing, and will prove the benefit of user testing to an often dubious audience.
It’s common to use a ‘think aloud’ methodology, where you ask the player to speak their thoughts aloud as they play. It’s not perfect, but it’ll give some insight into what they’re thinking. You could prompt them by asking non-leading questions such as “what are you doing now” or “what are you thinking” if they’re silent for too long!
what are you thinking?
Then after the fifteen minutes (or completing the tutorial), you can ask them questions to gauge how they understood the game – did they know what they were doing, were they confused by the game, did they know what to do next, would they like to continue playing, etc. Maybe you saw some interesting behaviours while they were playing that you’d like to ask about. Often people don’t remember what they did, and so you’d have to prompt them, or maybe the developers watching have some questions to ask.
Hopefully this quick methodology will show where the user’s are getting confused, or not understanding what to do next, or getting bored, or any other factors that turn player’s away from the game in that crucial first play.
Jakob Nielsen said that most major usability problems can be found by 3-5 users, so you wouldn’t need to run this test many times per iteration of the game. As to when this should be done, its best to get started as soon as there is something playable, as changes will be easier to make the earlier they are found, and then a similar test can be repeated with the next version of the game. Many social games go through an agile design process, with many iterations, and so this process will fit in well.
What do you think? Does this sound helpful? Or do you think that the ‘new user’ experience is not the most important part of a facebook game – maybe converting players to paying players is instead… let me know your thoughts in the comments here, and we can improve this 30 minute facebook game test.
