Posts Tagged ‘evaluating existing technologies’
Selling UX in games – everyone else is doing it.
Following recent discussions, I’ve been thinking about how UX can often be misunderstood by developers. I’ve previously blogged about the misconception that usability and user experience testing is ‘just QA’. I’ve also seen people accuse it of ‘homogenising’ a game, and the belief that it’ll remove the game’s individuality and create a race of dull facebook clones.
This inaccurate view is set to change, with the rise of in-house and independent usability and user experience labs, and the breadth of academic research increasing our understanding of user experience. Hence I’m planning a series of articles about my thoughts on selling UX to game companies, helping you to communicate why game companies should be thinking about UX with their games. New posts in this series will come under the category ‘Selling UX in Games’
Everyone else is doing it
So why should your company be interested in games usability and user experience? Well I’m glad you asked…
- Many top selling games have gone through user experience testing
- The companies who have tried it recognise the benefit, and go on to do it again. In fact, one developer said…
- “It adds 10% to the metacritic score”
So let’s look in more depth at some games that have been reaping the benefits of user experience testing. These popular games all found success through user experience testing, and can serve as helpful examples when communicating why your company should be looking into user experience.
oh... well, thats easy to understand
Halo 3
Bungie recognised the value of the player experience when developing their multi-million selling Halo 3, and hence used user experience testing to create a game that was consistently fun, ensuring the player was never confused or lost. As documented in Isbister’s book Game Usability Microsoft’s usability labs were brought on board to ensure that Halo 3 was fun throughout.
Each area was thoroughly tested with real users, using techniques such as video and ‘heatmapping’ to log where the choke points were, where people got lost or stuck, and where they died. This could then be used to easily find problems in the game, and fix these areas. Fixes are typically simple, either by signposting the way to go, altering the enemies or adding points of interest to draw players (like a sparking light to encourage players to look up). This technique could also be applied to multiplayer maps, to balance the game’s flow, and make the game fair.
The success of this technique in Halo 3 has been extended to Halo Reach, where telemetric data was gained from a long public beta testing period. This gave real player information on how to balance and improve the game, and will undoubtedly be a large factor in its success.
To read more about the Halo 3 user testing experience, see the coverage in Wired.
Split Second
Brighton based Black Rock Studios explored user experience testing in their successful racing games ‘Pure’ and ‘Split Second’. With consultation from the usability lab Vertical Slice they tested over 100 players, getting new user’s experiences to fine tune aspects of the game, ensuring the pacing was correct, and that the game was delivering key ‘fun’ moments, which they denoted ‘woah moments’.
They built on this successful experience from Pure in developing Split Second, where they combined in house talent, who worked on testing users, with assistance from a local university, who performed an expert evaluation of the usability and user experience issues.
As documented by Graham McAllister, the developer believed that the process “added 10% to their metacritic score”. More information about how user experience testing helped Pure become a success can be found in the book Evaluating User Experience in Games.
Farmville
It’s not just the expensive AAA games which benefit from user testing. Zynga’s Farmville is everyone’s favourite facebook success story, and has gone from 0 t o 63 million active users in just a few months. This success is largely down to logging the user behaviour, and using this data to improve the player experience.
The online nature of facebook games allow user testing in a way not possible for traditional console games. Metrics can be gathered while people play to log behaviours, such as discovering how long a player spends on each task, which tasks are repeated, and where players get bored and drop off (i.e. only 85% of people make it past the tutorial).
An iterative design process, where the game is evaluated and improved every few days allow Zynga to take advantage of these metrics, and try to improve player behaviour. For example, by running two competing versions of the tutorial, they can judge which one retains more players, and implement that version fully. Repeating this process over every aspect of the game helps make Farmville the addictive success it has become.
Farmville distilled
Conclusion
So, want to help your game be more like these top titles? They all have user testing in common, and have explored them in very different ways, including in house testing, and recording player metrics. It’s likely one of these models will be applicable to your company. These games are all well respected, and most companies want to replicate these success stories.
Not enough to help you convince your company that games usability and user experience is worth pursuing? In the next ‘Selling UX in games’, I’ll be looking at getting team members involved in the UX process.
Usability Issues in Sharepoint
Taking a short break from our normal topics of usability and user experience within games, this week i’ll be focusing on Intranets built with Microsoft Sharepoint.
This post is in response to Jakob Nielsen’s update on Sharepoint, entitled ‘Does Sharepoint Destroy Intranet Design?’. For the last year I’ve been working closely with Sharepoint, and want to wade in to this discussion.
Are intranets all the same?
Nielsen’s update argues for Sharepoint as an effective base for an intranet. Counter to the argument that the standard Sharepoint toolset doesn’t allow much flexibility when designing, he presents four different intranets, all based on Sharepoint, as evidence.
The evidence Nielsen supplies for this conclusion, that Sharepoint still allows intranet design, seems… dubious. For example, he lists the different top navigation categories as evidence that the sites design is different. I would hesitate to define the designing of an intranet to be entirely about information architecture (the categories chosen for top level navigation), and would suggest that the ability to design a flexible intranet goes much deeper than this.
About this much deeper...
Sharepoint isn’t perfect
Having worked closely with Sharepoint for the last year, it’s obvious that the usability problems within Sharepoint are more important than its ability to define navigation categories.
For example, this is how you’d upload an image from your hard drive onto the page you are editing.
- Click ‘edit page’. Wait for new page to load.
- Click in the area, and click on image button.
- New window comes up, click on ‘browse’.
- New window comes up, Only shows 8 images at a time – verify the image you want isn’t already uploaded (through lots of slow page refreshes).
- Decide to upload, click ‘upload’ (note that the directory you have been browsing is not the directory that it’ll upload too – instead it always uploads to the same directory. And has no tools to move the image to a new location, besides opening the directory in windows explorer to do it)
- New window comes up, click browse, find the file, click upload.
- Find the image in the thumbnail viewer (remember, only 8 images at a time) –
- Click ok on every box that’s opened
- Click on ‘check in’ to view page.
A tad unnecessary?
Similarly, small usability ‘mistakes’ seem to exist throughout Sharepoint. For example, Sharepoint keeps old versions of the file or page you’re working on, in case you need to restore it. (note this doesn’t work on ‘web-parts’ – areas of the page which contain JavaScript or custom html). To remove these, which is a common requirement due to file size restrictions, you must:
- Right click on a file, select ‘version history’.
- Right click on an invidual version
- Select ‘delete version’
- Accept confirmation box
- Refresh the page
- Go to 2
I often encounter files with hundreds of versions. This process takes hours.
So the solution to this is obvious and widely seen elsewhere (even in other areas of Sharepoint!). A row of tick boxes down the side of the list of versions, with ‘select all/none’ buttons. Then a ‘delete selected’ button. This would save hours on intranet maintenance, and seems like a massive oversight.
Conclusion
Although Nielsen’s argument that Sharepoint doesn’t prevent intranet customisation is legitimate, I feel Sharepoint has more pertinent usability issues that prevent it from being the perfect CMS to build an intranet upon.
Nielsen over-represents the amount of customisation possible – although you can change the navigation, and the pictures/layout, Microsoft impose definitive restrictions upon what can and cannot be done, often prevent usability issues from being fixed.
Instead, I suggest an open source CMS like Drupal would provide a better base for an intranet. By giving the administrator full control over the CMS’s inner workings, Drupal not only allows a wider range of customisation, and design to take place, but it also benefits from a plethora of user created modules which add almost any pre-generated functionality that can be imagined. As such, it not only allows the limited customisation that Sharepoint can, but gives the user the chance to fix usability problems through customisation, and extend the functionality in ways limited only by imagination.
All Change – Apple’s new social gaming network
Been playing iPhone games recently? Then you’ve probably been bothered by pop-ups asking you to sign in with OpenFeint, Plus+, Crystal, or one of the other many social gaming networks on the iPhone. When I started writing this post, I was going to cover the problems that having many rival social gaming networks causes, and what Apple needs to do to fix it. However, I’m too late. Last week, Apple announced they are going to launch their own social gaming network, called ‘Game Center’. So instead, we’ll be looking at what this new social gaming network needs to do, and what player experience issues it needs to address.
What is a social gaming network?
As seen with Xbox Live, or Playstation Home, social gaming networks essentially all do similar things:
- Store high scores, often with leader boards for comparison with other players, by day/week/all time.
- Gives, and records achievements (or trophies) from within the game: meta-objectives which are publically listed on the player’s profile
- Contains a ‘friends’ list, of other players, with messaging facilities so that multiplayer games with these players can be arranged.
- Match finding, allowing the player to find suitable games that match their criteria, or games with friends.
Admittedly not terribly useful in solitaire...
Problems with social gaming networks on the iPhone.
The problems so far with implementation of these networks on the iPhone are caused by the wide number of competing systems.
Unlike the PS3 and Xbox, which each has one social gaming network on their device, Apple had (until last week) refused to act on implementing their own system, which has lead to the rise of many independent systems. Currently popular are: Crystal, Plus+, Openfeint, Agon and a recently announced competing network by Namco. Even ignoring the smaller, less widely implemented systems, there is still too much diffusion here.
Having a large number of competing systems offers an inconsistent user experience, with similar tasks (i.e. adding a friend) being handled differently on each system, which is ultimately detrimental to the player’s experience. Instead of playing the game, players have to spend too much time setting up accounts and adding the same friends from their other iPhone games.
The lack of a centrally imposed quality control means the implementation of these networks into games is rather haphazard. This can be seen with the free version of the iPhone ‘x-ray’ app, which emulates x-raying the player’s hand. The app has recently added Openfeint support, and so has achievements, leaderboards, a friend list, etc. These features don’t correlate with a single player, non-game. What sort of competition can a faux-x-ray have?
Achievement Unlocked: Found Keys
A closer inspection shows that the achievement points are all given for purchasing the paid version of the game. The social gaming network integration just serves to bribe the player with the chance to pay for points and inflate an artificial score (which can be compared to your friend’s score, see Farmville!).
What will Apple’s new system have to do
Apple’s system will have to improve the disjointed player experience that these systems currently give. To properly emulate the success, and ‘flow’ of the PS3 and Xbox’s networks, Apple should be aiming to entirely replace these competing systems.
The advantages of this would include:
- unifying all players under one system to ensure that friends can find each other, play against each other, without being spread across multiple systems
- The players only need to understand one workflow for each task (i.e. adding a friend), rather than learning the process for each system
- The player will only need to sign up once
- Achievement points earned in one game can be shared across all games, rather than just those on the same network, as currently.
Care would also have to be taken to enforce rules to ensure achievement points remain meaningful, by introducing a form of quality control to prevent poor apps from bribing people to download by handing out points cheaply.
When I ran user tests for an iPhone game last year, the openfeint login/sign up screens confused new users, who were just interested in playing the game. Apple will need to make the network invisible to uninterested parties, to prevent this. Perhaps this can link with their itunes ID, but the implementation of this is not obvious:- families often share an itunes ID across many devices
By introducing their own network, Apple have the opportunity to achieve a consistent, and hence improved, user experience when playing using the iPhone’s social gaming networks, and can only help things get better!
Watching ‘average users’: Word
It’s easy to forget how useful it is to watch less technical people use some common programs, and how helpful observation is as a tool to understand the ‘average’ user. I recently watched someone using MS Word (2003 I think), and it was…enlightening. They made a large number of ‘errors’ when using MS Word, but as we know as usability specialists, its not the the user that creates errors – the software does.
The task was relatively simple – design some worksheets, including textboxes, and pictures, and lay them out in an eye-pleasing manner. I’m sure there are many more appropriate packages to make this in than Word, but it was the user’s software of choice, due to familiarity, and the task shouldn’t be beyond MS Word. I observed, and let them lead the interaction, but advised when they asked for help (naughty I know, but it wasn’t a formal lab setting!)
The lab was busy that day anyway...
How my ‘less-technical user’ used Word:
I noted down (obviously away from the user) some of the more ‘interesting’ characteristics of how they used Word.
- Used the ‘cut’ function as a ‘delete’ (with no understanding of how it links to paste). Taken out of context from “cut and paste”, ‘cut’ would more likely imply removing or ending something, and so this mistake is understandable. Incidentally this method has some pluses. I still don’t know how to remove a table easily (not just the information within it), and cut seems to do this.
- No knowledge of the alignment tools, and so using spaces as a method to align text to the center or right. Obviously ran into problems when editing the text later, as changes would make the text run over the end of the line, ruining the formatting.
- Drew horizontal lines, across the page (i.e. a space to write in your own answer) with –‘s. Seems a pretty effective method, even though I’m sure Word has its own way of doing this. Is there a better way of doing it?
- Displayed difficulty moving images in Word. Is it right that you have to click on an image twice to move it? The first click just gives you resize options, which confused the user.
- Had difficulty with resizing objects. What happens if you make an image so big that it falls off the edge of the paper, and you cannot see the border to make it small again? I guess you could format picture, and manually change the size, but this is an entirely different method of resizing, and isn’t cognitively related to the standard way.
- Constant (constant!) rewriting of words, when word autocapitalised/auto formatted them in an undesired way (which was seemingly every autoformat). User had to delete the word, and re-write each time.
What could word do to improve?
This immediately throws up some questions about how Word was developed. It’s clear that the tools available, such as the alignment, or horizontal lines, are not making their functionality transparent to new users. It wasn’t clear to my user that they existed, or how they should be functioning. Obviously just having the icon on the toolbar isn’t enough, and this should be rethought.
This was also the case with image manipulation. The functions that the user needed do exist in Word (i.e. resizing, moving), but are modal in nature, and so are difficult to find, and don’t offer a consistent user experience to someone who is not familiar with Word’s nuances.
It’s also clear with auto format in particular that the system isn’t adapting to the user’s needs. The constant changes that Word was making to the user’s document, which were then undone each time only created a large degree of frustration in the user. The software should be learning how the user wants auto format to work, and adjust to their preference. In this user’s case, it was causing trouble, and should have turned itself off (or at least given the option)
What they need is some sort of helpful assistant
What should we learn from this?
It occurred to me that these issues were not unique to the user I watched since I encounter similar problems with Word. The difference is I’ve had enough familiarity to learn the workarounds, or solutions to these problems that Word throws at you. For example, it’s an unthinking reaction to press Ctrl+Z after Word incorrectly auto-formats things incorrectly. My user just hadn’t used the program for long enough to train that reaction, and so word’s error became more of a big deal.
Its important when considered usability to realise that users aren’t just like you. If you are in a position to make a difference with usability, it’s very likely you are not an ‘average user’, and as such its difficult to comprehend how ‘average users’ use software.
‘Average users’ are not stupid. They are your mum, and just don’t have the time, or effort to put into learning these workarounds, or making them second nature. The solution, rather than ‘educating’ users, is to make the programs better; make programmers understand who their users are, and how they use the programs. And make them program for the ‘average’ users, rather than the power users. And that is the point of usability.
A Terrible User Experience & how to fix it– Zoomerang.com
When running a website, its important to make sure that the user can achieve their goal with the minimal fuss. This is especially important if you are selling, or trying to sell, a commercial service. I recently had to use Zoomerang.com, a survey site, and had a few notes about the user experience. As you’ll remember, I don’t rant often…
I’m in the process of designing a GPS game, and am currently discovering the functional requirements for the project. As UX practitioners, we know that involving the user is of critical importance at this stage, hence we designed a questionnaire to establish peoples experience, and perception of GPS games, and what they’d like a GPS game to be like. (linked here)
When at university, our internet access goes through a proxy server, which blocks unsuitable content. For some reason, this includes surveymonkey.com, a site I’ve used a few times in the past to construct online surveys. Interestingly, the ‘site blocked’ dialogue said “for survey sites, try zoomerang.com”. However, when I search for some hardcore action, it never gives me alternate suggestions for that. Have I uncovered a conspiracy? Nonetheless, I followed the link.
And so I ended up on zoomerang.com. Being fair, there is one key advantage to zoomerang which immediately put me in a good mood. On surveymonkey, for a free account, you are limited to ten questions. On Zoomerang, you can ask 30 questions before you have to pay. This meant we didn’t have to redo, or concatenate our questions, and made me smile inside
Pictured: an inside smile
Problems with Zoomerang.com
This goodwill was shortlived, when I tried to use the site to implement my questionnaire. Heres why:
- The workflow isn’t clear when making a survey, and so I entirely missed the step where you add your questions. Clicking through the process actually caused me publish a blank questionnaire. Which wouldn’t be a problem, except…
- …You can’t edit an existing survey. Once its published, you cannot add/remove/change questions. Surveymonkey allows this. So I was stuck with my blank survey, and had to start again from scratch.
- Having figured out how to add questions, I got started, and selected “insert question”. It added a header, which then had to be changed to type question. I guessed that was because it was my first item, but no, it always defaults to inserting a header (odd, since you’d only need 1 per page, whereas you’d need multiple questions).
- So I finally got to add a question, and this is when the terribleness of the design struck me. I selected a question where a radio button would select from a number of answers, and typed in my list of 15 or so alternative answers into a rich text field. I hit submit, and … got an error, saying “answers can only be 1000 characters, including HTML”, and even worse…
- …It deleted the data I had entered in that field. All 15 answers. This is a critical failure of any system, since the data a user inputs should be considered sacred.
- There was no counter telling me how many characters I had entered, so I had to retry a few times. Eventually I realised that I could only enter 5 potential one word answers before it’d error that I was over 1000 characters. That had to be a mistake? I investigated further…
- …Looking at the HTML, it turned out that the rich text editor was writing rubbish html. At the start of each answer, it’d add needless style tags, often multiple times. Heres an example of the HTML it generated for my one word answer “complicated”
<p><span style=”font-family: Arial; color: #000000; font-size: small;”><span style=”font-family: Arial; color: #000000; font-size: small;”><span style=”font-family: Arial; color: #000000; font-size: small;”>Complicated</span></span></span></p> - …no wonder it was hitting the character limit after 4 or 5 words. I had to manually enter the html for all the possible answers, just so I could get round this.
- My last fault with zoomerang.com is just a suspicion. I look after my email accounts, and so have never received spam in my current primary address. After signing up for zoomerang last week, I received my first random spam email. Might just be a coincidence, but I didn’t sign up for anything else that week!
I had to call this guy to fix my survey
How to fix zoomerang.com
To improve their user experience, they should look at red-routing the goals the user needs to achieve:
- Make the progression through survey design clearer, highlighting which step questions are added in
- Also make it clear how far through the design process you are, and what steps remain
- Restrict what the user can do, so they cannot post a blank questionnaire. Its obvious if they are about to do this that they’ve made a mistake, tell them!
- Don’t make question types default to “header”. Surely users will only use this type once at most, whereas they’re going to have more than one question on the questionnaire. Make it default to that!
- Fix the WYSIWYG code generator, so that the user doesn’t have to manually code the answers in HTML. A lot of user’s would get stuck at this point!
- Don’t send me spam!
And what can you do, until these fixes are made? Use surveymonkey.com. Or, if you’ve found anything better, leave a comment and let me know!
