Posts Tagged ‘Heuristics’
Evaluating User Experience in Games – Book Review
Regina Bernhaupt presents an academic guide to the application of user experience principles to games, as part of a series by Springer Books on HCI, and claims to represent the ‘coming of age’ of video games as a medium. The book is essentially a collection of academic papers, largely from recent conferences, and draws upon the talents of a wide range of authors, including Brighton based Graham McAllister and Gareth White, Katherine Isbister (Editor of Game Usability) and Emily Brown of SCEE. Although largely academic, the book does provide an insight into the cutting edge of this exciting field.
Early chapters in the book try to define what the field of user experience is, and what it means in relation to games. There is a degree of confliction here, with each chapter giving a slightly different interpretation, but they often settle on themes such as immersion, fun, presence, involvement, engagement, flow and playability.
Also wind, fire and heart!
The book gets more exciting when it presents a range of methods for evaluating user experience in games, with a variety of models appropriate for various stages of game development, from prototyping to post-production. This encompasses many custom models for different situations, such as a model for inexperienced gamers, or one for fitness games. The book also presents studies of the usability of game controllers, and the development of heuristics, which is particularly interesting in the last chapter which aims to collate and amalgamate previously created gaming heuristic.
All this content is interesting; however, a liberal spreading of maths means it often comes across as extremely academic (particularly in comparison to Isbister’s book). This can largely be accounted to the background of the various authors, largely coming from academic institutions, compared to the real world perspective of Game Usability.
Where the book excels is the divergence from this academic interpretation, notably in the papers by Vertical Slice and Emily Brown. Vertical Slice cover the current state of user experience evaluation across three Brighton game companies, and give an insight into the methods used through case studies, from the expert evaluation found in the earliest stages of production, through to the user testing close to the end of a project.
Brown gives a comprehensive overview of the range of tools currently in use, and shows optimism for the future, since she recognises only a “lack of knowledge” as a hindrance to the extension of user testing into gaming, rather than opposition to the methods. This conclusion is reinforced by the case studies by McCallister and White, who show game developers are looking to extend their application of user experience testing in the future.
Which will be the same as today, but with more robots
Unlike Remote Research or Game Usability, this book is not a practical how-to guide. Instead it presents the state of user experience in games, and where the cutting edge of research is. Personally I have found it very useful for developing my own research. However the book would be unlikely to be a ‘one stop shop’ for a developer looking to start user experience research at their company.
It will be useful to see how the wide range of interesting ideas found within this book can be integrated into practical solutions for companies to use when producing games. This move from the theoretical to the practical will greatly assist the field of user experience in games, and will truly see the ‘coming of age’ of video game usability.
Replacing the Desktop?
Just under 40 years ago, the desktop metaphor was devised as a way to allow computer users to understand graphical interactions with their computer. Standard tasks, like using a calculator, or deleting files, were presented in a manner familiar to workers from a traditional office place, as an effort to build their experience of computing upon pre-existing knowledge. And, as is evident by this metaphor’s continued existence today, it was a massive success.
Just under 16 years ago Microsoft attempted to reinvent the desktop metaphor, and bring it up to date. The product, Microsoft Bob, aimed to shift computing from an office metaphor to a home metaphor. And it was a massive failure. My first home computer came with Bob installed, and so today I’ll be looking at why it failed, and what we can learn from this failure.
Usability advantages of the desktop metaphor.
So why has the desktop metaphor proved to be a lasting success? Introduced at a time when graphical methods of interacting with a computer were new, it had several key characteristics that led to its success.
...such as fun games
First of all, the desktop was familiar. Rather than having to learn context specific methods of interacting with a computer, it built upon the user’s pre-existing knowledge. For example, when deleting a file, a user could use their existing understanding of a trash can, and drag the file into it (rather than running a deltree command, which doesn’t map with any real-world knowledge). Therefore the desktop metaphor was easy to figure out, and consistent with real life experience, reducing the learning curve upon adoption. This meets Nielsen’s heuristic on a ‘match between the system and the real world’.
Building upon this familiarity was the appropriateness of the desktop metaphor for the tasks at hand. Before home computing, the workplace was the most likely place for users to use a computer, and the computer would be performing office-based tasks, such as word processing or calculating. Hence the adoption of a workplace metaphor seemed appropriate for developing a graphical user interface, as it registered with the target market. This meets Nielsen’s heuristic on ‘recognition rather than recall’.
Furthermore, the desktop metaphor was wide enough to expand to meet the growing roles that computers played. By extending the workplace metaphor through terms such as ‘cut’ and ‘paste’, and the development of graphical tools emulating image manipulation tasks, the desktop metaphor proved that it wasn’t static, and could extend to reach an ever growing range of requirements.
The desktop metaphor also met the heuristic requirement, of having a wide degree of flexibility, by allowing ‘experienced’ users to automate or speed up tasks, such as by selecting groups of objects, or utilizing keyboard shortcuts.
What did Bob try to do?
In the mid 90’s, Microsoft Bob was devised as the successor to the desktop metaphor. Recognising a growth in home computing, Microsoft aimed to shift the graphical interface model for computing from a business/creative focus, to the ‘home’. It was thought that this would open the computing world to a whole range of ‘novice’ users, who would have found the desktop metaphor inaccessible.
Bob presented the user with ‘their room’, covered with clickable objects, such as bookcases, clocks and a notepad. Clicking these things will launch the relevant program (or help you locate files), and you can add your own programs to the shelves. It’s just like your home! (assuming your home is littered with boxes that say ‘Internet Explorer’ and ‘Corel Draw’).
Bob in action
Why did Bob fail?
Microsoft Bob offered an alternative to using the desktop metaphor, aimed at novice users, but its primary failure was that it didn’t offer any significant advantages. For a product that came over twenty years after the ethos of its competitor, this wasn’t a good sign…
Despite being based on the home, Bob still had a learning curve, and so missed it’s key objective of being intuitive. Clicking on a clock to open a calendar, or a pen and paper, still required just as much learning as a calendar or notepad icon in a traditional desktop environment. More complex tasks than just opening programs still require further learning. Also, the enforced ‘home’ layout is just plain inefficient – rifling through a cabinet to find a file offers no advantage to browsing a list of files in a folder.
By attempting to change the way people interacted with computers, Bob alienated itself from existing computer users, and prevented new users from being able to ask for help from power-users. By offering only ‘simple’ ways of interacting with computers, the user was unable to allow users to grow, and learn superior (and more efficient) ways of performing tasks.
It’s also apparent that the ‘cuteness’ of Bob didn’t sit well with users. The two elements of this operating system which outlived the OS itself are among the most hated villains of computing – Virtual assistants like Clippy, and the Comic Sans font. Obviously Microsoft failed to understand the needs of their target market.
The final nail in Bob’s coffin came within a year of its release. Microsoft released Windows 95. It sold… quite well, and offered a fully-powered alternative to Bob based on the traditional desktop metaphor. Bill Gates punished those responsible for the mess that was Bob. He married lead project manager Melinda French. Burn.
What will replace the desktop?
It’s obvious through Bob’s failure that the Desktop cannot be beaten by a simple re-skin or appropriation of another metaphor without offering significant advantages.
As I wrote about in my review of The Humane Interface, Raskin proposes a ‘Zoomworld’ which offers a non-windows environment with no gaps between the operating system and the files. However development by Archy has stalled and they seem to have fallen off the internet…
Or maybe the future will be more like Google, and involve typing queries or commands into a prompt to find answers and perform tasks? Although this does seem like a regression, and breaks several key usability best practises.
So what other systems are out there that offer a viable alternative? Or will it be desktops forever? As ever, I’d be interested in hearing your thoughts in the comments below!
100 Rogues – Playability Heuristics Review
In the words of the Fast Show, this week I’ve mostly been playing 100 Rogues. I’ve never previously been able to get into Rogue-likes, having only played games in this genre briefly, before being scared off by the dungeon crawler’s core mechanics of ‘odds stacked against you’, ‘if you die you lose’, and ‘you will die’. However, 100 Rogues aims to present an accessible Rogue-like, ideal for someone who hasn’t played before. As accessibility is one of their key design goals, a heuristic based playability review seems appropriate.
This review aims to evaluate the playability of the game, including pertinent usability issues, and the effect this has on player experience. This is especially important, given 100 Rogues’ mission of making a difficult genre accessible. I’ve based the review on the heuristic points identified by Heather Desurvire’s paper ‘Using Heuristics to evaluate the playability of games’
What this review doesn’t cover is non-usability or player experience issues. Hence, it’s not a review of the game itself (which I gather is a pretty standard Rogue-like). It’s also not QA, and so doesn’t cover bugs in the game. This is especially important as 100 Rogues has infamously been plagued with a number of bugs since its release last week. The first version would crash if the player equipped a shield. The fix for this introduced several new bugs. And I still haven’t been able to get defeat the first boss without the game killing my player after I’ve defeated the boss. I gather the developers are working on this though.
100 Rogues
Game Play
Game play covers the game’s goals, and player’s involvement in achieving these. 100 Rogues succeeds in many areas here. Crucially, it guides the player through their first steps, and introduces them to the range of actions available to them, by immediately giving them the ability to level up their player, which is reinforced by the low cap for the second level up, allowing the player to practise this mechanic twice within the first 2 minutes of the game.
More complex actions are also introduced to the player, through the game’s challenge mode – a series of short scenarios where only the use of an advanced technique (such as ‘tele-stabbing’) will succeed. These introduce the player to some of the more complex moves available to them, in an intuitive way (rather than just… telling them)
The game handles the inevitable death of your character well, also. As mentioned previously, when playing a Rogue-like, you only have one life, no reprieves, and you will die. They explain this by likening the game to Tetris – the goal with your one life is to see how far you can get, not to reach the ‘end’. However death in any specific situation is never inevitable and the game always seems fair. This isn’t Mario Kart, where the CPU will always cheat at the last moment. Instead, after every death you’ll always believe that you could have done something differently and survived, and hence the game encourages a ‘one more go’ attitude to playing, and challenge comes off as a positive experience.
The only game play element which seems poorly balanced, and takes some of the control away from the player, is the ‘food’ mechanic. If you don’t eat food, your character dies. Makes sense. And sometimes you cannot find anything but rotten food, which will fill you up, but reduce your maximum HP. That’s fine too, if it’s a choice between being weakened, and death. But sometimes the game will spawn no food at all. And then you’re stuck, and nothing you can do will save your character from death. This looks like it needs to be balanced in future games, so the game at least spawns some food (rotten or not) when the player is critically hungry. As it stands, the player doesn’t feel in control of their destiny, and has no ability to prevent their character from dying.
Pictured: 100 Rouges
Story
Story defines how your characters’ actions fit into the world you are presented with and the feeling that the game-space exists as a real world, which you can affect, but which exists without you.
The story in 100 Rogues is simple. Satan is at the bottom of the dungeons, and you must kill him. Why? Because he’s Satan.
This story is introduced by a cut scene at the start of the game, and loading text gives character-related background, however this is where the player’s involvement in the story ends. The in game content, and enemies don’t reflect this final goal (aside from the end boss), and the character development isn’t plot related. Occasional infighting among the enemies gives the potential for further depth within the story, however this is not explored further. Ultimately, like Tetris, the story of ‘why’ you are doing this is not a focal point of the game.
Mechanics
Mechanics covers consistency in how the game reacts, how the AI acts, and how the player controls their character. The AI in the game is a success, with the NPC’s acting consistently, and yet occasionally surprising the player – such as when an archer’s miss-fired arrow will hit another enemy, and they will start battling it out. Hence the game balances allowing the player to understand how NPC’s react, without making them entirely predictable.
The game allows the player to track their own progress, through in game score/stats, and a global high-score table. Although implementation of this isn’t perfect, as I’m not convinced scores are being posted to the internet, the bug has been noted and is being ‘worked on’.
An area where the mechanics of 100 Rogues fails is with its controls. The character is controlled by touching the screen – touch the screen at the top to make the player go up, or touch an enemy to attack them. However, with no confirmation before an action is taken, and no indication of the active areas for each target, I found myself ‘miss-clicking’ numerous times, with often deadly consequences. Hence, when trying to click on an enemy for a ranged attack, I’d instead walk towards them, putting my character in danger. Restricting the range of actions available on a single click, or making ‘attack’ a double click, may help alleviate some of these issues.
Must be my sausage fingers...
Usability
Usability concerns how the game gives feedback for inputting actions, and whether they can achieve their goals. A success for 100 Rogues is how it saves the game state on quit, allowing the game to continue from the last point, as I’ve discussed before
As I discussed above, the game also gives direction to a first time player, by giving them an introduction to levelling up on game start. However, it hinders play the second time you start by… giving the same introduction. Since the game demands multiple play-throughs, I feel that I have grasped this mechanic the 20th time it has been introduced to me.
Conclusion
As has been made clear by a heuristic evaluation of 100 Rogues, the game has a high degree of playability, and provides an accessible entry point to a traditionally difficult genre. As noted, there is room for improvement, yet the game offers significant advances on other games in this genre.
However, I’d be hesitant to recommend the game, as it stands, as a positive player experience. Although, playability shouldn’t include bugs, bugs will undoubtedly have an effect on player experience. Hence, as the game stands, the unexpected crashes and deaths will detract from player’s opinions of the game. What incentive do players have to give the time and effort of playing, when their character could be taken away from them through no fault of their own?
That said, the development team have been dedicated to fixing bugs – having released two patches in the week after the game was released, and are promising up to two-three times more content released periodically, which is an advantage of the iPhone as a platform. Within a few more iterations, I can see this game being the definitive introduction to the Rogue-like genre.
Game Usability: Advancing the Player Experience Book Review
I’ve recently finished reading Game Usability: Advancing the Player Experience, edited by Noah Schaffer and Katherine Isbister, which (as its title may suggest) tries to give a complete overview of the field of usability within computer games. Game usability is a relatively new topic, yet all the key figures of the field are included in the book, including contributions from Microsoft’s User Experience labs, Sauli Laitinen and the Super Mario Club and interviews with figures from many major companies. Game Usability recognises that this field is new, and aims to provide an introduction for complete novices to how usability is developing within computer games, and the shift from ‘hardcore’ games, towards a friendlier user experience.
The topics are widely spread, and try to cover every aspect of usability and UX within computer games, including an introduction to heuristics, how to perform an expert evaluation, and guides to many of the processes of user experience testing. Since the topic is relatively new, there is a wide range of material than can be drawn upon – maybe this book wouldn’t be so useful in ten years time, after a greater degree of precision is applied to each area of usability testing.
Where this book excels is when it covers the actual ‘how to’ of usability processes. If the reader had never performed an expert evaluation before, this book would give them a great introduction and allow them to get started. Similarly for running user tests, articles in the book tell you what to do (and what not to do), common problems encountered, and what results you should be looking for. This is supplemented well by concrete examples, such as Microsoft telling us how they use heat mapping to work out which areas of Halo are causing problems, and what actions they took to fix it.
now try to capture the flag
It also covers the difference between ‘casual’ and ‘power’ gamers, and how games should adapt to the shift towards casual games that can be seen through the success of the Wii. Unlike Alan Cooper (whose book Inmates says that the divide between casual and power users should not exist), this book recognises that ‘power-gamers’ have grown up developing a different skill set to casual gamers, and are more prepared to put up with issues like dying repeatedly, and a higher degree of challenge. By addressing the differences between gamers, and what their expectations are, this book would be a useful aid in the design of personae, and at targeting your game to an intended audience.
Some aspects of the books seemed a tad odd however: An interview with Georgios Yannakakis of Copenhagen University asks only three (very brief) questions (maybe Georgios wasn’t aware it was an interview). Another interview is with someone who shares Schaffer’s last name, and looks to be the author’s dad. Schaffer’s dad has been established in the field of usability, but has little to add with regards to games.
Perhaps the main issue not covered by the book is that all the contributors to the game are established in companies that accept the value of their work. The book briefly covers some counter arguments to common complaints about performing user testing (“It’s too expensive = It’s cheaper than shipping a rubbish product”, or “It takes too long = It’s integrated with the design process so doesn’t take much longer”). However it’s getting into a position where UX is a consideration at all within a company that will be a consideration for many (and is the subject of John Rhodes’ book Selling Usability ). Maybe future revisions of this book will include techniques for making usability a priority within your company.
The focus of the book is maybe sometimes too wide, and only lightly touches each subject before moving on. If you were already familiar with the topics covered in this book, there will be nothing new for you here.
These oversights are only minor though, compared to the large amount of ground the book covers as an introduction to usability. It offers more in terms of practical “how-to” guides than Alan Cooper’s Inmates, and its focus on Games means it can offer fairly comprehensive coverage of the main topics of the field. If you are new to the subject, or a non-usability specialist looking to understand the subject, or can only afford one book, Game Usability would be a great introduction to the theory and practice.
Also, to finish, my quick Usability fail discovery, from Sussex University.
Instead of letting the user break it, how about… not offering the ability to make mistakes (See: Macs)
Conducting an Expert Review
Within our HCI classes, we have started reviewing the UX of an upcoming multi-platform game from a prominent client, and are performing an expert review on it. An expert review, as opposed to a user-based study, involves having usability experts play the game themselves, and uses tools and their expertise to find faults. This is different to a user-based study, where the expert would observe another player playing the game. Because of the time constraints involved, we selected an expert review as the most effective method to review the UX of this game.
To get the best results possible, and be as helpful as possible to the client, we had to choose our methodology carefully. In this blog post, I’ll discuss how we chose to approach this task, why we chose these methods, and what the alternatives are.
The first rule placed on us is that we are to work in groups of 3. As described in an article by Laitinen on performing expert evaluations, the evaluation reaches its optimal group size between 3 and 4. Less experts than this may miss things. More experts than this fail to find a significantly larger number of faults.
plus too many cooks spoil the broth
The other restraint placed upon us is that we would only have a short amount of time with the game. We decided to use this time to play and evaluate the games separately, and then come together to discuss our findings. The alternatives to this would have been having one person play, and the other two take notes, or to have each person play for a bit (as we did), but the experts not playing would take notes then. All of these sessions would involve filming the game screen, and the participant.
Two experts watching one player
Advantages:
- One longer complete play through, so can see player development
- Experts can ask the player questions during their play session
Disadvantages:
- Only one play through, so difficult to see if issues are common or just for this user
- Questions asked during play through may distract/alter playing experience
Three experts playing together, in turns
Advantages:
- Three sessions played through, so can see reoccurring issues
- Experts can get a greater understanding of the game mechanics through playing it
Disadvantages:
- Players wouldn’t get as far as they would with a long session from one player
- Second and third experts play experience will be biased from the experience of the first
Three experts playing separately
Advantages:
- Each player gets an authentic ‘new player’ experience
- Comparing after can show what issues naturally arose for all
Disadvantages:
- Players wouldn’t get as far as in one long play through
- Have to perform expert evaluation after the game play, rather than during.
Since the sessions were all being recorded, we opted to do the last one, and hence have the ‘purest’ play experience recorded for each. There is, of course, no right answer – many other groups chose different approaches, and I’m sure they found equally valid issues. I’d welcome comments below if anyone has reasons for a preference with how to perform an expert evaluation.
Now having a video of a play test, we are individually analyzing them. I’m approaching it using heuristics, such as those made by Nielsen, Nokia, and the work of Federoff as a guide. Having identified the issues, I will then attempt to rate them by severity – the extent to which they will hinder the user’s enjoyment of the game. Then, in a group session with my team members, we will evaluate which issues we all agreed where particularly prominent and severe, and amalgamate our results, ending up with a list of issues with the game.
We will then have to present our data to the client. I posted before about writing a UX report, but the circumstances for this report will differ – Geographical location, and time constraints mean that this report will be an in-person presentation, with some take-aways. I will blog about these soon….!
