Posts Tagged ‘qualitative’
Watching ‘average users’: Word
It’s easy to forget how useful it is to watch less technical people use some common programs, and how helpful observation is as a tool to understand the ‘average’ user. I recently watched someone using MS Word (2003 I think), and it was…enlightening. They made a large number of ‘errors’ when using MS Word, but as we know as usability specialists, its not the the user that creates errors – the software does.
The task was relatively simple – design some worksheets, including textboxes, and pictures, and lay them out in an eye-pleasing manner. I’m sure there are many more appropriate packages to make this in than Word, but it was the user’s software of choice, due to familiarity, and the task shouldn’t be beyond MS Word. I observed, and let them lead the interaction, but advised when they asked for help (naughty I know, but it wasn’t a formal lab setting!)
The lab was busy that day anyway...
How my ‘less-technical user’ used Word:
I noted down (obviously away from the user) some of the more ‘interesting’ characteristics of how they used Word.
- Used the ‘cut’ function as a ‘delete’ (with no understanding of how it links to paste). Taken out of context from “cut and paste”, ‘cut’ would more likely imply removing or ending something, and so this mistake is understandable. Incidentally this method has some pluses. I still don’t know how to remove a table easily (not just the information within it), and cut seems to do this.
- No knowledge of the alignment tools, and so using spaces as a method to align text to the center or right. Obviously ran into problems when editing the text later, as changes would make the text run over the end of the line, ruining the formatting.
- Drew horizontal lines, across the page (i.e. a space to write in your own answer) with –‘s. Seems a pretty effective method, even though I’m sure Word has its own way of doing this. Is there a better way of doing it?
- Displayed difficulty moving images in Word. Is it right that you have to click on an image twice to move it? The first click just gives you resize options, which confused the user.
- Had difficulty with resizing objects. What happens if you make an image so big that it falls off the edge of the paper, and you cannot see the border to make it small again? I guess you could format picture, and manually change the size, but this is an entirely different method of resizing, and isn’t cognitively related to the standard way.
- Constant (constant!) rewriting of words, when word autocapitalised/auto formatted them in an undesired way (which was seemingly every autoformat). User had to delete the word, and re-write each time.
What could word do to improve?
This immediately throws up some questions about how Word was developed. It’s clear that the tools available, such as the alignment, or horizontal lines, are not making their functionality transparent to new users. It wasn’t clear to my user that they existed, or how they should be functioning. Obviously just having the icon on the toolbar isn’t enough, and this should be rethought.
This was also the case with image manipulation. The functions that the user needed do exist in Word (i.e. resizing, moving), but are modal in nature, and so are difficult to find, and don’t offer a consistent user experience to someone who is not familiar with Word’s nuances.
It’s also clear with auto format in particular that the system isn’t adapting to the user’s needs. The constant changes that Word was making to the user’s document, which were then undone each time only created a large degree of frustration in the user. The software should be learning how the user wants auto format to work, and adjust to their preference. In this user’s case, it was causing trouble, and should have turned itself off (or at least given the option)
What they need is some sort of helpful assistant
What should we learn from this?
It occurred to me that these issues were not unique to the user I watched since I encounter similar problems with Word. The difference is I’ve had enough familiarity to learn the workarounds, or solutions to these problems that Word throws at you. For example, it’s an unthinking reaction to press Ctrl+Z after Word incorrectly auto-formats things incorrectly. My user just hadn’t used the program for long enough to train that reaction, and so word’s error became more of a big deal.
Its important when considered usability to realise that users aren’t just like you. If you are in a position to make a difference with usability, it’s very likely you are not an ‘average user’, and as such its difficult to comprehend how ‘average users’ use software.
‘Average users’ are not stupid. They are your mum, and just don’t have the time, or effort to put into learning these workarounds, or making them second nature. The solution, rather than ‘educating’ users, is to make the programs better; make programmers understand who their users are, and how they use the programs. And make them program for the ‘average’ users, rather than the power users. And that is the point of usability.
Rocket Surgery Made Easy by Steve Krug – Book Review
Steve Krug’s new book, Rocket Surgery Made Easy, aims to be the only book someone new to user experience will need to start performing usability testing on their company’s website. And, with a few qualifying remarks, it is.
Krug’s previous book, Don’t Make Me Think, aimed to introduce the idea of web usability to a general audience, and had a number of key characteristics. It was short, designed to be read during a plane journey, and it was accessible, so that anyone could read it and understand usability. It also had a chapter on performing usability testing, which was removed in the second edition. Rocket Surgery Made Easy is that chapter.
In the decade since the emergence of this first book, Krug has been honing his usability testing skills, and through running numerous workshops, has perfected how to teach usability testing. This book is the summary of that experience, and in a wonderfully brief and readable style, is the only book someone new to user experience will need to start performing usability testing. Looking at that statement in more detail will give a deeper insight into the book.
Pictured: Steve Krug, international man of mystery
“The only book”
Krug’s book is a complete introduction to testing your company’s website, and guides the reader through every step of the process, from recruitment, and pre-test setup (including the equipment required), through to running the session and the debrief. The book also offers all the test-scripts and task cards you’ll need. Throughout the process Krug offers his best practise techniques, useful maxims to remember, and pitfalls to avoid. Essentially, you could start performing usability tests with this book alone as a reference.
“someone new to user experience”
This book aims to be an introduction, and so requires no prior knowledge (although reading his other book wouldn’t be a bad start). It also assumes that you are not a usability professional, just an interested party at a company, and that there isn’t a large degree of company buy-in, and believes that many rounds of semi-amateur testing is superior to one round of professional testing, which is valid. Because it assumes you don’t have a large amount of support from your company, in particular monetarily, it focuses on how to perform usability testing with a large degree of constraints – how to do it cheaply, how to do it quickly (just one morning a month) & how to get people interested through pastry!
“will need to start”
Because of its small size, Krug’s book is only designed as an introduction to usability testing, and won’t contain any new information if you’ve done this before. At best you could pick up a few best practises, but you’re not the intended audience. Just for you, the book has suggestions for further reading “for overachievers only”, with some interesting books to continue studying this field, but the book is largely aimed at newcomers to the field.
“performing usability testing”
Also due to its brevity, the book only covers one on one think aloud tests with websites, and only hints that other forms of usability testing possible. The book ends with a short chapter on remote research, but only hints at the possibilities that we saw at the UX Brighton event. Other types of usability research, such as quantitative methods, expert analysis, or focus groups, are left out entirely of the books remit. Although the book offers a complete solution, it’s a complete solution for one particular type of usability testing, and only covers websites (although the skills could easily be applied elsewhere).
Its not rocket scien.... oh.
Conclusion
Krug’s new book is short, and simple. However neither of these points are negative. It’s short because it has to be, to introduce a wide audience to usability testing, and to emphasise how easy it is to begin. It’s simple, because usability testing can be simple, and done for just a morning a month. If you’re someone who has been doing usability testing for years, or a professional who does a large amount of usability testing, perhaps it’s better to look elsewhere. However if you’re looking for an introduction to usability testing, having never donae it before, or just need a short walkthrough guide to getting usability testing started at your workplace, Krug’s book should definitely be your first choice.
Why you should always video players when testing games
Video game developers around the country are missing a cheap, easy, and useful aspect of running player experience tests by not videoing the players being tested. I believe recording players gives a better understanding of how people experience games, and I’m going to explain why this leads to the development of better games.
Recently I’ve been involved with the testing of the user experience for a number of high profile upcoming games. The results of these sessions have always been very interesting, and undeniably shown that running tests with real players gives a true understanding of the issues in the games, finding issues that often cannot be seen by the developers. However, all the places I’ve worked at have all been making the same, easily fixable mistake of not videoing the player’s reactions while playing.
a mistake never made by this man
There are many possible excuses given why people running player tests on games fail to record the player. In a hectic working environment, it can be difficult to find time to review the video with tight deadlines hanging overhead. Similarly, the initial set up costs, of a camera or software that will record the player seems like an area where cutbacks can be made with a tight budget. Often tester’s will try to justify this – surely they can get the same information, and cheaper, by using surveys and interviews after the event?
Here is just of few of the things missed when you fail to video a player’s experience
- A timeline of the player experience, not just their end of play feelings
- The ability to pinpoint the ‘key’ fun moments
- The ability to pinpoint the dullest moments.
- The ability to verify the opinions given to you by a post-game survey
- A key ‘visual’ way of showing the stakeholders real player’s experiences (useful for presentations)
Would this experience be the same without video?
Having this data leads to the production of better games – the incredibly dull level, that is only redeemed by an exciting boss battle need not be an incredibly dull level, and videoing the players can mean the difference between picking up on these moments and missing them. If you miss them, you’ll never fix these problem moments. It is well worth the time to review what aspects of the game the player especially enjoyed, or hated, without the ‘bias’ introduced by the time delay before getting their opinions. If you are asking them to ‘think aloud’ (and you should be!), a video will also keep notes of the opinions given throughout the session. The problems identified can then be fixed.
User (or player) experience is of utmost importance for game designers – unlike software, which businesses can force upon their employees, everyone who plays games does so because they enjoy doing so. Videoing their reactions can lead to a more enjoyable game, and makes your game more consistently fun, and how can that be a bad thing?
Five essential iPhone apps for usability professionals
As you may have noticed, I like my iPhone. But the iPhone is not just for playing games, or twitter, but can also be a useful tool for every stage of usability testing, from the discovery of customer needs to the iterative testing of potential designs and implementation. I’ve tried to compile five essential apps for usability professionals, available on the iPhone now. If anyone has any suggestions/additions, or disagrees with the choices I’ve made, please add your comments!
Working with usability requires a wide range of abilities, and hence these tools reflect the wide range of roles a usability professional may be performing. When talking about each app, I’ll try to cover where in the usability process it may come in useful, and how it can help you to increase efficiency and get better results.
but don't forget lo-fi alternatives
1. Pidoco°
Starting off, this isn’t strictly an iPhone app, but Pidoco’s software has been developed to be compatible with mobile browsers and the iPhone. I saw them demo the software at a recent UX Brighton event, and they impressed upon me that it was built with usability in mind. Pidoco offers a quick way to collaborate on the design of wire frames, and quickly show these to the user or client, without the need to be in the same geographic location as them. The easy to use nature of the system means it’d be ideal when iteratively testing an interface with users, and is a mid-fi alternative to paper prototyping. Currently offering a month’s free trial, there’s no excuse not to give it a go!
Pidoco°’s website: https://pidoco.com/en/
2. Mocha VNC
Need to mock up a mid-fi prototype of an iPhone App? You obviously don’t have time to make the app in Xcode, buy the iPhone developers license, link the customer’s phone to the computer and copy the app over. Instead, how about making a prototype on your Mac, or PC and using a VNC client to make the iPhone look at the prototype. The user will still be able to interact with the prototype by clicking, or navigating around, and the ‘controller’ sat at the server can display the correct pages to the user. Easy mid-fi prototyping, on an iPhone, without the long set up costs and investments required to actually have the app running on the iPhone.
Mocha VNC’s website: http://www.mochasoft.dk/iphone_vnc.htm
3. Camera
Watching someone perform a task is an important part qualitative usability testing. What about those moments where you weren’t looking, or noticed something that you wanted to ask about later, but forgot about? Looks like you need to be recording your user tests. Camera comes with all iPhones, but is only available for video recording on the 3GS. Since you are always likely to have your phone with you, it’s a handy mobile solution that allows you to record your usability sessions without requiring any additional equipment, so that they can be analysed later. Maybe buy a stand for it though!
4. exZact Data Collection
Quantitative research is also an important part of establishing user requirements, and working towards a design that meets their goals. The Data Collection iPhone app aims to allow you to create, and answer, custom surveys on the iPhone, collecting information like geographic location, and responses in a variety of forms (1-10 ratings, drop down lists, yes/no responses, and comment boxes). They then offer real time analysis, so you can see the results as they come in, and claim to be massively scalable. Offering a month’s free trial, it seems an effective solution if you require your data collection method to be mobile, and accessible anywhere.
exZact’s website: http://www.iformbuilder.com
5. Insight – Basecamp on iPhone
You’ve seen from the diversity of the apps above that a usability specialist has to be experienced in a lot of areas, and its often hard to keep on top of these. A tool that I have found invaluable in project management is Basecamp, which allows you to collaborate on projects, assign tasks and see what everyone is up to. Insight is a mobile version of Basecamp, and will allow you to keep up to date with how everyone working with you on the usability project is progressing from one easy interface, and keep on top of projects. As a busy person, it’s likely to be an invaluable tool to any usability specialist or contractor!
Insight’s website: http://www.encampapp.com/
Conclusion
Its clear from looking for the most useful usability apps for the iPhone that this is a largely untapped market – although there are tools available to help with the individual steps in the process, there is no one ‘go-to’ app for usability and UX professionals. What is needed is a more holistic solution, one that can work with a usability professional in every step of the process, from the initial requirement gathering stage, through building up personas of the customers, to the iterative design process and beyond. Until then we’ll have to make use of the wide range of apps covered today to achieve these tasks.
Mobile Games should start quickly (lets get down to business!)
Part of a series on iPhone Game Design Issues. For an introduction see here, or use the categories on the right.
PopCap, they of Peggle and Plants Vs Zombies, commissioned a survey on where most people play mobile games. Results showed that men play more at work (28%) than women (17%). They were also asked when do they play these games. “while waiting for an appointment” came first for both genders, although I imagine this is because “on the toilet” wasn’t an option in the survey (I’m not judging… just saying!).
waiting for an appointment
What these results show is that people want mobile games to be ‘pick up and play’, and use them to fill time, as opposed to committing a large amount of time to playing them. It is therefore important that game designers facilitate this method of game playing, and make it easy to pick up and play games. The main elements of this are:
- fast start up time (from app load to actually playing)
- low number of extraneous menus to navigate before playing
- app resumes from where it left off
- app shuts down quickly, but doesn’t lose progress
A previous blog post touched on the issue of apps resuming your progress after an exit, and so this topic will focus on the user experience starting (or resuming) a game.
The player, when they start the game, wants to play as soon as possible. Initially they have to sit through a load screen, which beyond the abilities to reduce this by programming optimization is a necessary evil. However, as developers interested in ux, steps should be taken to reduce the number of steps a user has to go through after this to reach the game.
There is a trade off that has to be made here, based on our assumptions of what the user wants to do. With many types of games we can assume that the user will already be on their right profile, will want to resume a current game, or start a new game. More complex games may have a wider degree of options that they need to present the user (particularly on a first boot). So how do we decide what options the majority of users will require? More effective than experts’ ‘educated guesses’ would be qualitative testing – either through a limited release prior to the app store (I believe apple allows you to distribute your app to 100 people, enough for a good sample group), or through ‘hooks’ in the code of early releases, which will log, and send back, details on user activity. If you then found that 87% of users, on loading a game, went straight to ‘resume game’, you could make the game do this automatically, and reduce their wait.
The design of an effective ‘entry method’ into the game is incredibly well suited to large user tests, based on qualitative tests of user behaviour, and can have an enormous effect on a player’s good will and ‘feeling’ about a game. Put simply, if player’s know that they can load your game quickly, compared to one with a 30 second wait, and 5 menu screens to navigate, they are more likely to pick your game when waiting in a queue or for a meeting.
"anyone for peggle?"
Who does it well?
Textropolis
Textropolis, the word guessing game by Ian Marsh. Click on the app, and after a short load screen you’re playing the game – no menu options (obviously you can get back to the menu, but the game assumes, correctly, that most players will not need this). The nature of the game (no time crucial element), lend the game to a quick start, so this is also a fine example of a game design being suited to its platform. The game isn’t lacking in features too – the ability to sign in as separate user profiles exists, the game assumes that you will be less likely to want to do this than just play. Textropolis’ quick start up time would make this game a suitable choice for the sort of quick gaming that Popcap’s user survey says mobile gamers are into.
NB: I was going to feature Geared, the puzzle game by Bryan Mitchell here, as it loads quickly, and I cant remember having seen the title screen (it throws you straight into the game). However when writing this, I realised it doesn’t save your progress on a puzzle when you exit/re-enter, and hence it loses user experience points!
Who does it badly?
hero of sparta
Hero of Sparta, the (rather epic) hack and slash game by Gameloft (Think those PS2 Star Wars/LOTR games).
So, we’ve got five minutes before the meeting starts, lets play. App started. A load screen, then a one minute cut scene, ok, I’ve seen it before, so I’ll skip this. Another load screen. A title screen. Touch the screen to continue, ok, I’ll do this. New game, or Continue. Obviously I want continue, so I select this. Select a chapter. Well, I’ll select the one I’ve been playing on. Great. Another load screen. And I’m in.
That’s a lot of stuff to get through between deciding I want to play, and actually getting to play. Probably one of the reasons why I haven’t devoted much time to playing this game.
How could it be improved?
After the first load, I’ll be unlikely to want to watch the opening cutscene again (despite the flashy graphics). The title screen adds needless clicking to my experience. Although I haven’t verified this with testing, I believe the player is likely to select ‘continue’ rather than ‘new game’ after they’ve started playing. And th
