An iPad Nano

Jakob Nielsen admits that the study is not as thorough as a typical usability study. However has decided to share it anyway, due to the over-inflated impact that usability studies produced early in a devices’ lifecycle have been seen to have. As an aside, this is an interesting contrary viewpoint to the disadvantages of being first-to-market noted in ‘Inmates’, which argues that being first to market is irrelevant compared to being ‘best’.

The report has some interesting key findings, including that the apps seen on the iPad and the iPhone suffer from the re-emergence of a problem not seen since the early 90’s. Unlike web browsers and desktop software, which has established graphical conventions to highlight buttons and GUI elements, iPhone and iPad software has not implemented standard conventions, such as making a clickable button appear 3D. Hence there is no consistent manner of designating important aspects of the UI, and users just didn’t know what they could click on. Nielsen likens this to the first emergence of graphical interfaces of the early 90s, when anything and everything could be a button.

Its therefore clear that the main recommendation of the study is to standardize common elements, like navigation, among first and third party applications, such as “swipe to turn page”, or “press and hold to delete”. This also links with the studies’ findings that users were unsure what reaction their action would cause, as the apps have yet to find a consistent manner in which to work.

Not just a big iPhone

The study highlights how the iPad is not just a big iPhone, and different usability issues emerged on the larger device – most pertinent is that navigation elements on the bottom of the page, as seen in many iPhone applications, will not work on the iPad. The larger screen means that these elements are too far from the user’s field of vision to be noticed – and hence are not appropriate. What this means for people who make apps is that a custom iPad version is needed, not just relying on the ‘universal’ up scaling of iPhone apps.

The full report is linked below, and worth a look if you’re interested in the usability, the iPad, or designing an application!

Read the full report, here.

Related posts:

1. Five essential iPhone apps for usability professionals
2. Usability Issues in Sharepoint
3. How real-world game usability testing is changing

This review aims to evaluate the playability of the game, including pertinent usability issues, and the effect this has on player experience. This is especially important, given 100 Rogues’ mission of making a difficult genre accessible. I’ve based the review on the heuristic points identified by Heather Desurvire’s paper ‘Using Heuristics to evaluate the playability of games’

What this review doesn’t cover is non-usability or player experience issues. Hence, it’s not a review of the game itself (which I gather is a pretty standard Rogue-like). It’s also not QA, and so doesn’t cover bugs in the game. This is especially important as 100 Rogues has infamously been plagued with a number of bugs since its release last week. The first version would crash if the player equipped a shield. The fix for this introduced several new bugs. And I still haven’t been able to get defeat the first boss without the game killing my player after I’ve defeated the boss. I gather the developers are working on this though.

100 Rogues

Game Play

Game play covers the game’s goals, and player’s involvement in achieving these.  100 Rogues succeeds in many areas here. Crucially, it guides the player through their first steps, and introduces them to the range of actions available to them, by immediately giving them the ability to level up their player, which is reinforced by the low cap for the second level up, allowing the player to practise this mechanic twice within the first 2 minutes of the game.

More complex actions are also introduced to the player, through the game’s challenge mode – a series of short scenarios where only the use of an advanced technique (such as ‘tele-stabbing’) will succeed. These introduce the player to some of the more complex moves available to them, in an intuitive way (rather than just… telling them)

The game handles the inevitable death of your character well, also. As mentioned previously, when playing a Rogue-like, you only have one life, no reprieves, and you will die. They explain this by likening the game to Tetris – the goal with your one life is to see how far you can get, not to reach the ‘end’. However death in any specific situation is never inevitable and the game always seems fair. This isn’t Mario Kart, where the CPU will always cheat at the last moment. Instead, after every death you’ll always believe that you could have done something differently and survived, and hence the game encourages a ‘one more go’ attitude to playing, and challenge comes off as a positive experience.

The only game play element which seems poorly balanced, and takes some of the control away from the player, is the ‘food’ mechanic. If you don’t eat food, your character dies. Makes sense. And sometimes you cannot find anything but rotten food, which will fill you up, but reduce your maximum HP. That’s fine too, if it’s a choice between being weakened, and death.  But sometimes the game will spawn no food at all. And then you’re stuck, and nothing you can do will save your character from death. This looks like it needs to be balanced in future games, so the game at least spawns some food (rotten or not) when the player is critically hungry. As it stands, the player doesn’t feel in control of their destiny, and has no ability to prevent their character from dying.

Pictured: 100 Rouges

Story

Story defines how your characters’ actions fit into the world you are presented with and the feeling that the game-space exists as a real world, which you can affect, but which exists without you.

The story in 100 Rogues is simple. Satan is at the bottom of the dungeons, and you must kill him. Why? Because he’s Satan.

This story is introduced by a cut scene at the start of the game, and loading text gives character-related background, however this is where the player’s involvement in the story ends. The in game content, and enemies don’t reflect this final goal (aside from the end boss), and the character development isn’t plot related.  Occasional infighting among the enemies gives the potential for further depth within the story, however this is not explored further. Ultimately, like Tetris, the story of ‘why’ you are doing this is not a focal point of the game.

Mechanics

Mechanics covers consistency in how the game reacts, how the AI acts, and how the player controls their character. The AI in the game is a success, with the NPC’s acting consistently, and yet occasionally surprising the player – such as when an archer’s miss-fired arrow will hit another enemy, and they will start battling it out.  Hence the game balances allowing the player to understand how NPC’s react, without making them entirely predictable.

The game allows the player to track their own progress, through in game score/stats, and a global high-score table. Although implementation of this isn’t perfect, as I’m not convinced scores are being posted to the internet, the bug has been noted and is being ‘worked on’.

An area where the mechanics of 100 Rogues fails is with its controls. The character is controlled by touching the screen – touch the screen at the top to make the player go up, or touch an enemy to attack them. However, with no confirmation before an action is taken, and no indication of the active areas for each target, I found myself ‘miss-clicking’ numerous times, with often deadly consequences. Hence, when trying to click on an enemy for a ranged attack, I’d instead walk towards them, putting my character in danger.  Restricting the range of actions available on a single click, or making ‘attack’ a double click, may help alleviate some of these issues.

Must be my sausage fingers...

Usability

Usability concerns how the game gives feedback for inputting actions, and whether they can achieve their goals. A success for 100 Rogues is how it saves the game state on quit, allowing the game to continue from the last point, as I’ve discussed before

As I discussed above, the game also gives direction to a first time player, by giving them an introduction to levelling up on game start. However, it hinders play the second time you start by… giving the same introduction. Since the game demands multiple play-throughs, I feel that I have grasped this mechanic the 20^th time it has been introduced to me.

Conclusion

As has been made clear by a heuristic evaluation of 100 Rogues, the game has a high degree of playability, and provides an accessible entry point to a traditionally difficult genre. As noted, there is room for improvement, yet the game offers significant advances on other games in this genre.

However, I’d be hesitant to recommend the game, as it stands, as a positive player experience. Although, playability shouldn’t include bugs, bugs will undoubtedly have an effect on player experience. Hence, as the game stands, the unexpected crashes and deaths will detract from player’s opinions of the game. What incentive do players have to give the time and effort of playing, when their character could be taken away from them through no fault of their own?

That said, the development team have been dedicated to fixing bugs – having released two patches in the week after the game was released, and are promising up to two-three times more content released periodically, which is an advantage of the iPhone as a platform. Within a few more iterations, I can see this game being the definitive introduction to the Rogue-like genre.

Related posts:

1. Conducting an Expert Review
2. Game Usability: Advancing the Player Experience Book Review
3. iPhone Game Design Heuristics

The UXBrighton event was presented in a new format as a series of short talks, from Harry Brignull’s tips on time stamping notes, to Danny Hope’s templates for understanding user roles. Also interesting was a talk on using google analytics, although the length of the talk meant that topic could only be skimmed, dissapointing as I’m an analytics fan. The most interesting idea presented was iCrossing’s presentation on “The iCrossing Connected Brand index: how to measure a brand’s effectiveness online”, given by Ifraz Mughal.

Expert Evaluation

As I’ve mentioned before an expert evaluation is a useful tool for getting an insight into potential usability and user experience issues on a website, or game, with limited resources. Although it can never replace running tests with real users, it can provide a quick approximation, and help highlight the biggest issues.

The ‘method’ for an expert evaluation is simple. Get an expert to look at the site, or game, and tell the client what they think. Job done.

My expert eye tells me you need smarter users...

However an expert evaluation can only ever be subjective, and this is it’s biggest weakness. A client can look at your page full of recommendations, and dismiss it as the opinion of one person. There’s no easy way to see progress with changes, and a comparison with other sites can only ever be abstract.

Quantifying an Expert Evaluation

iCrossing’s solution is to quantify their expert evaluation. As part of their ‘Connected Brand Index’ idea, they rate their clients sites (and competitors), on UX-centric areas such as “usefulness”, “usability” and “desirability”.

A traditional expert evaluation would give a qualitative rating, and give examples to back this up, i.e. “Poor – little emphasis, and diffused call to actions”. Instead iCrossing will give the site a score, on a scale of -2 to 2 (2 being very good). This of course can be backed up with examples in a more in depth report.

after the first few pages, the report can just be pictures of kittens. No-one reads that far.

The advantages:

There are numerous reasons why a client would prefer a scored ‘rating’, rather than comments.

• A ‘score’ makes it easy to benchmark, and compare your own scores against competitors. By dividing the expert evaluation into separate topics, and scoring each, a finely grained comparison can be made, and communicated
• Similarly, a score makes it easy for a client to see progress. If they scored -1 before hiring you, and 1 after, your work can be justified (as long as no-one questions who is doing the scoring!)
• Because this produces a concrete score, clients will be able to handle and communicate the data. Graphs can be made, which wouldn’t be possible for subjective comments. These can be invaluable for justifying and communicating with managers and project sponsors, who do not need to see the details, just get a high-level overview.
• This expert evaluation can be encompassed as one aspect of a larger ‘score’ given to websites, or games. This is the idea behind iCrossing’s connected brands index.

Conclusions:

There is an argument this can be seen as a bit of a scam. Giving arbitrary numbers to your opinions doesn’t make them any less subjective. This method of presenting the data could be misleading if presented incorrectly, and the client should be made aware of the method behind the score system. This could become an issue when running comparative studies before and after your work, since you’d be biased towards giving the site a better score after you’ve worked on it.

The point of this method is to aid communication with the client, and give them data in a format that is useful to them. As I discussed in the review of Selling Usability, management and non-technical people would typically much rather see pretty graphs, and statistics, than a list of comments. This method helps manage client expectations, and gives them what they want.

To make the method more valid, it would be useful to perform a study to ensure the method is sound. Perhaps get a wide range of experts to independently rate a wide range of websites on this scale, and note the correlations between the scores. It’d be first step in countering complaints that this method is still inherently subjective, and help make an art into a science.

Related posts:

1. Conducting an Expert Review
2. How to present a UX Report
3. Understanding players through biometrics

The topics are widely spread, and try to cover every aspect of usability and UX within computer games, including an introduction to heuristics, how to perform an expert evaluation, and guides to many of the processes of user experience testing. Since the topic is relatively new, there is a wide range of material than can be drawn upon – maybe this book wouldn’t be so useful in ten years time, after a greater degree of precision is applied to each area of usability testing.

Where this book excels is when it covers the actual ‘how to’ of usability processes. If the reader had never performed an expert evaluation before, this book would give them a great introduction and allow them to get started. Similarly for running user tests, articles in the book tell you what to do (and what not to do), common problems encountered, and what results you should be looking for. This is supplemented well by concrete examples, such as Microsoft telling us how they use heat mapping to work out which areas of Halo are causing problems, and what actions they took to fix it.

now try to capture the flag

It also covers the difference between ‘casual’ and ‘power’ gamers, and how games should adapt to the shift towards casual games that can be seen through the success of the Wii. Unlike Alan Cooper (whose book Inmates says that the divide between casual and power users should not exist), this book recognises that ‘power-gamers’ have grown up developing a different skill set to casual gamers, and are more prepared to put up with issues like dying repeatedly, and a higher degree of challenge. By addressing the differences between gamers, and what their expectations are, this book would be a useful aid in the design of personae, and at targeting your game to an intended audience.

 Some aspects of the books seemed a tad odd however: An interview with Georgios Yannakakis of Copenhagen University asks only three (very brief) questions (maybe Georgios wasn’t aware it was an interview). Another interview is with someone who shares Schaffer’s last name, and looks to be the author’s dad. Schaffer’s dad has been established in the field of usability, but has little to add with regards to games.

Perhaps the main issue not covered by the book is that all the contributors to the game are established in companies that accept the value of their work. The book briefly covers some counter arguments to common complaints about performing user testing (“It’s too expensive = It’s cheaper than shipping a rubbish product”, or “It takes too long = It’s integrated with the design process so doesn’t take much longer”). However it’s getting into a position where UX is a consideration at all within a company that will be a consideration for many (and is the subject of John Rhodes’ book Selling Usability ). Maybe future revisions of this book will include techniques for making usability a priority within your company.   

The focus of the book is maybe sometimes too wide, and only lightly touches each subject before moving on. If you were already familiar with the topics covered in this book, there will be nothing new for you here.

These oversights are only minor though, compared to the large amount of ground the book covers as an introduction to usability. It offers more in terms of practical “how-to” guides than Alan Cooper’s Inmates, and its focus on Games means it can offer fairly comprehensive coverage of the main topics of the field. If you are new to the subject, or a non-usability specialist looking to understand the subject, or can only afford one book, Game Usability would be a great introduction to the theory and practice.

Also, to finish, my quick Usability fail discovery, from Sussex University.

Instead of letting the user break it, how about… not offering the ability to make mistakes (See: Macs)

Related posts:

1. A Theory of Fun for Game Design by Ralph Koster – Book Review
2. Evaluating User Experience in Games – Book Review
3. Rocket Surgery Made Easy by Steve Krug – Book Review

To get the best results possible, and be as helpful as possible to the client, we had to choose our methodology carefully. In this blog post, I’ll discuss how we chose to approach this task, why we chose these methods, and what the alternatives are.

The first rule placed on us is that we are to work in groups of 3. As described in an article by Laitinen on performing expert evaluations, the evaluation reaches its optimal group size between 3 and 4. Less experts than this may miss things. More experts than this fail to find a significantly larger number of faults.

plus too many cooks spoil the broth

The other restraint placed upon us is that we would only have a short amount of time with the game. We decided to use this time to play and evaluate the games separately, and then come together to discuss our findings. The alternatives to this would have been having one person play, and the other two take notes, or to have each person play for a bit (as we did), but the experts not playing would take notes then. All of these sessions would involve filming the game screen, and the participant.

Two experts watching one player

Advantages:

• One longer complete play through, so can see player development
• Experts can ask the player questions during their play session

Disadvantages:

• Only one play through, so difficult to see if issues are common or just for this user
• Questions asked during play through may distract/alter playing experience

Three experts playing together, in turns

Advantages:

• Three sessions played through, so can see reoccurring issues
• Experts can get a greater understanding of the game mechanics through playing it

Disadvantages:

• Players wouldn’t get as far as they would with a long session from one player
• Second and third experts play experience will be biased from the experience of the first

Three experts playing separately

Advantages:

• Each player gets an authentic ‘new player’ experience
• Comparing after can show what issues naturally arose for all

Disadvantages:

• Players wouldn’t get as far as in one long play through
• Have to perform expert evaluation after the game play, rather than during.

Since the sessions were all being recorded, we opted to do the last one, and hence have the ‘purest’ play experience recorded for each.  There is, of course, no right answer – many other groups chose different approaches, and I’m sure they found equally valid issues. I’d welcome comments below if anyone has reasons for a preference with how to perform an expert evaluation.

Now having a video of a play test, we are individually analyzing them. I’m approaching it using heuristics, such as those made by Nielsen, Nokia, and the work of Federoff as a guide. Having identified the issues, I will then attempt to rate them by severity – the extent to which they will hinder the user’s enjoyment of the game. Then, in a group session with my team members, we will evaluate which issues we all agreed where particularly prominent and severe, and amalgamate our results, ending up with a list of issues with the game.

We will then have to present our data to the client. I posted before about writing a UX report, but the circumstances for this report will differ – Geographical location, and time constraints mean that this report will be an in-person presentation, with some take-aways. I will blog about these soon….!

Related posts:

1. Quantifying the unquantifiable – Expert Evaluations
2. 100 Rogues – Playability Heuristics Review
3. How to present a UX Report