Kick Ass in action...

Today Erik tells us about his experiences with finding testers, getting user feedback, and why you should look at what people do, and not what they say!

First of all, I want you to bare in mind that I am in no way experienced in the field of creating computer games. I’m 18 years old, I have been programming for around 5 years, and most of that time I have been making websites. I have been successful in it, I’ve already had several jobs with some really large Swedish websites (where I’m from). The original kick ass was something that I started at 10 pm and was finished with a couple of days later. A couple of months after that I made a post on reddit where I linked to it. A week later it went viral and just exploded. Fast forward another few months and I begin with the iPhone app.

My idea of testing has always been to just write some code, try it out, if it works, great, if not, fix bugs. My twin brother, who is more the design monkey (if I’m the code monkey), has been the one talking about good UI vs bad UI. And we have worked together a lot, and now I also have a need for making good UIs. But the notion of a good UI is not something that I have scientifically proved to be one through user testing. It’s more what I can gather myself, so all my knowledge about UX is totally subjective. Some things are simple as daylight to see. If your mom does not know how to use your app you lost the market for +40 year olds.

Take that iTunes!

… And that is the way I make stuff. The idea of using “qualified” testers is something that hit me shortly after I released the iOS app. Before that I had only tested it on me, my girlfriend, my brother and my father (and that was only once!). I guess that shows how inexperienced I am in game development. If you read about professionals who make games you read about testers, beta testing periods etc. And the fact that it can take 40 people more than two years to make ONE game is totally alien to me, seeing as my experience is that it took one night and one person. That also shows in the iOS app. The concept has HUGE potential. For example, because the code is open source, many people have taken it and adapted it for their ad campaigns. One campaign, which took the script, changed the image of the ship, and launched a microsite with it had 300 000 unique visitors to it. The campaign won several awards and was nominated for a Cannes Lion (luckily, they had me in the credits too, which would be totally absurd if they hadn’t). Paramount used the idea too, which also had huge success (that ad is also going up for awards, I’ve heard). So anyway…

What I mean is that if an experienced game developer/company decided to do something out of this concept, it would be huge. But that would require many difference instances of knowledge. UX, menu designers, music composers, level designers, programmers, marketing, and that is not me. The iOS app gets a couple of sales a day, yet the bookmarklet has had over 1 million unique visitors. Why? Because the bookmarklet is a silly thing that you play with for 15 minutes, and the game (iOS app) has to be a game.

And on the topic of feedback: I mostly don’t trust user feedback. Back when I worked for the large swedish site, every time we launched a new feature, they hated it. 90% of the comments were about how it sucked and along the lines of “it sucks. i liked it more before”. The target audience for these sites were 13-17 year olds, and they are apparently really hard to please. During that period I learned a lot. Key things were: If it ain’t broke, don’t fix it. If you think it’s broke, but the users don’t, change it gradually. Sneak in changes. Fix it over time. Don’t make a complete overhaul and release over one night. Your users will become disoriented and like you less. That is what I’m planning to do for the bookmarklet. Change the little things first, and let those changes grow, so the users know what is happening, and can follow the development.

When I ask users for feedback, I’m always very skeptical to their responses. One user might say something that only he doesn’t like or wants. It’s your job as the creator of something to know what is bad and what needs done. Asking users for feedback should only be done when you want to validate your opinion, but that might not even be the right time to do it. A wild estimate is that only 5% of your users are sharing their opinion. The other 95% you will never know what they want.

Feedback can be great when you want know how the users use your application/game. But don’t trust their words, look at their actions instead. Why don’t they click there, why did they choose that option, why don’t they know what do. Those are the questions you should be asking yourself when you’ve seen their actions. You have to analyze the feedback.

Erik has a new blog, including a interesting post-mortem of his game, at erikrothoff.com

Related posts:

1. Unintentionally designing with users.
2. Ian Marsh, creator of Tiny Tower, on user research, play-testing and F2P
3. Test with real users – not your team

UCD in action

User Centred Design, and participatory design, typically allows users to lead the design process, with their input and feedback being sought throughout an agile development. Unlike the reflective practitioner, where the impetus to improve comes from self-reflection, UCD externalises this and hence the impetus to improve comes from users surrounding the ‘expert’, rather than the ‘expert’ themselves.

So do you still get the benefits from reflective practice? Since understanding users, and applying this knowledge is a complex and ill-defined problem, there is are a large degree of potential approaches with no one approach being considered ‘best’. Hence there is the space, and need, in UCD for improvement. Contact with external people, such as the users, also gives a clear route for feedback from the results of UCD, and the ability to judge success in an objective manner. Hence, user insight seems like a suitable vehicle and impetus for reflection when improving practice.

If the impetus is coming from outside, rather than self reflection, does this avoid the self-reflection typically needed for expertise? Does this mean that no-one can ever become an expert in UCD? An interesting parallel that can help unravel this is considering how one becomes an expert in chess. Expert players in Chess have been found to be no better at conscious thought than a beginner, yet are much better at recognising established patterns or plays, and acting on their previous knowledge to make decisions quickly.

Hence expertise in these fields, and seemingly UCD too, is the ability to be a better reflective practitioner, and quickly dismissing areas that have been evaluated before. Although it isn’t a pure implementation, it does seem that the idea of the reflective practitioner is applicable to the field of user centred design!

Related posts:

1. User Centered Design vs. Genius Method – Which Approach Is Best for you?
2. Understanding Cognition, User Experience Winners & Losers, and a Design Failure.
3. UX needs an Agile environment

Ethnography

Ethnography is the idea of conducting research in the ‘real world’, through observation of the end-user in their natural setting. It has a historical basis in the field of anthropology, when explorers would go live with groups of people from unexplored cultures, to come to a greater understanding of their lifestyle. As a research tool for software design, it usually encompasses visiting a workplace for designers to come to a greater understanding of the context in which the program needs to work.

So where can I plug in my modem?

The advantages of ethnography will become visible through the discussion of the third space, below, however it’s important to note that ethnography isn’t a perfect design tool, and so the inherent disadvantages with using ethnography as an aspect of a design methodology have to be considered too.

Primarily there’s the practical issue of cost and distance. Often in a project, the designers are physically remote from the end-users, and so there is the expense of sending the designers out to the user’s location (although this can be mitigated by remote research techniques and software). There is also the additional cost of the time spent observing users – it’s time when the designer’s aren’t designing. There is a strong argument however that this time spent properly understanding the design problem prevents unnecessary re-work later, which would be at much greater expense!

Secondly there are lesser issues when bringing ‘outside’ designers into a workplace, including issues of security and data-handling, as well as the disruption that outside observers may bring to a typical work day. It’s also important to keep in mind that the act of observation will alter the thing being observed, and so the end-user’s may not represent their typical workflow accurately, intentionally or not.  I know I’d work harder if someone was looking over my shoulder and making notes!

Ethnography and the 3^rd space.

Ethnography can be seen as the direct application of Muller’s idea of a ‘third space’ in design. Muller describes the third space as a design space that takes place neither in the worker’s domain, (i.e. the air traffic controllers in our example), or the designer’s own domain (the dark cupboard in which the programmer lives).

The idea of moving the designers into the worker’s space is that it’ll create a ‘third space’ – a region bridging these two experience sets. According to Muller this allows all the participants, including the end-users and the developers, to combine their knowledge and approach the design problem with a fresh perspective, rather than relying on ‘traditional’ solutions from their own space.

Such as grass roots marketing campaigns

As an aspect of participatory design, this is therefore important for giving all stakeholders – including the end-users – an equal opportunity to influence the design of a product or service, and not be considered subsidiary to the designer’s ideas or traditions. Bringing designers into the end-user’s space gives the opportunity for the designer’s long-held assumptions to be challenged by first-hand experience, and for the problem to be negotiated, and designed co-operatively, encompassing a range of voices.

Therefore we can see that ethnography is a way in which this ‘third space’ can be accessed and achieved in the design process. This means that the designers and end-users can benefit from mutual learning, and improve the final product. Similarly, giving designers a true understanding of the context in which their solution needs to operate  prevents ‘obvious’ errors from being made – for example creating a system that relies on audio alerts in a noisy workplace.  Ethnography, and incorporating the idea of a ‘third space’ into the design process makes better software. And isn’t that what it’s all about?

Related posts:

1. A Theory of Fun for Game Design by Ralph Koster – Book Review
2. The Reflective Practitioner in User Centred Design
3. Telling Tales – Stories for promoting user experience.

Sometimes this handing over of control can happen accidentally, when user’s behaviour or meme’s become formally incorporated into, or define the very core of how a service works. Today I’ll be considering a few examples of this.

Unintentional user-led design can be seen in the development of Twitter. Originally there were no common ways of directing a tweet towards an individual (@), or indicating an ongoing conversation between multiple people (#). It wasn’t until a year after twitter’s launch was introduced that Chris Messina suggested using # for group discussion, which he later pitched to the Twitter team. They rejected it, stating it was ‘too nerdy’. However it was users who adopted the practise, such as during the reaction to San Diego’s forest fires, and now the practise is commonplace.

Such as #outsideyourwindow

The adoption of hashtags (and the @ sign) by users caused the Twitter team to incorporate it into their service, with ‘trending topic’ hashtags now being displayed on twitter’s front page. Although the idea was initially rejected by Twitter, we can see that users did, unintentionally, guide design decisions in the evolution of Twitter.

A second example of unintentional user behaviour being incorporated into the design of a product was found during the production of Burnout Paradise. Play testers were brought in to play the game and allow developers to understand a ‘real player’s perspective’. However, they noticed that some play testers developed some interesting behaviour. Within the game there were spaces between parked cars or scenery, and play testers were accelerating towards them, pulling a handbrake turn, and performing ‘extreme parking’. Think Arnie in True Lies, or Jim Carrey in that terrible Ace Ventura sequel style parking.

Something like this...

Having noticed that multiple players were acting in this way, making their own ‘meta game’ during play, developers then incorporated this as official challenges and objectives within the game’s campaign. Hence, user’s once again become unintentional design partners.

A similar incorporation of user behaviour and design can be seen in forums and ‘link’ websites such as reddit. When linking to …sensitive… material, users commonly denote this by tagging a link with NSFW (“Not Safe for Work”), or somethingawful’s NMS (“Not Mind Safe”) for the more surreal content. This practise, created by users, has been formally incorporated by many websites, as can be seen in reddit where, if an item is marked ‘NSFW’, the thumbnail preview image is turned off.

In all these examples we’ve seen that users can not only provide useful insight into how a system should works, but, particularly with collaborative services with the ability to iterate behaviour quickly over time, can create and define the very way in which a service works. Unintentionally, they have become design partners, and have come closer to embracing the idea of participatory design.

Related posts:

1. Erik Rothoff Andersson (Kick Ass) on testing with users & why you shouldn’t listen to what users say!
2. Does WarioWare DIY make users into designers?
3. Designing Interfaces by Jenifer Tidwell – Book Review

The uncanny valley was originally discovered in the field of robotics, but also frequently applies to video games. It describes a phenomenon with replications of humans, whether they are life-like androids or avatars on a computer game. Initially, as the reproduction of a human and its movement becomes more lifelike we react more positively towards the object, so we’d like Lara Croft more than Leisure Suit Larry. However a point is reached, when the reproduction becomes too life-like, and the emotional response drops rapidly, meaning we feel repulsed from the object. Consider Keanu Reeves’ acting. Almost human, but utterly repulsive!

The term ‘uncanny valley’ therefore comes from plotting a graph showing our emotional response against how lifelike the reproduction is, with a sharp ‘valley’ appearing in the emotional response between a very lifelike reproduction, and the real thing.

Got that?

A similar phenomena can be seen in the production of wireframes, and hence is of critical importance to UX designers. After spending hours producing beautiful wireframes in Omnigraffle, I presented them to a client, to show how their ‘event registration’ pages would function. They came back and said “yeah it looks good, but we need to change that label text… and we need to make the dropdown arrow bolder… and can we make the heading font bigger”. This was their first view of the wireframes, to approve whether they functioned correctly, and it’s obvious what I’d done wrong.

The time and effort I’d put into making the wireframes look good, and look like a real website weren’t just wasted, they were actually hindering the process – since the mock up looked like a real webpage, the client was focusing on the small presentational details, and not the functionality itself. They expected it to look and function like the final product. If instead, I’d done a rough sketch on paper to demonstrate how the registration process should work, the client would have focused on the functionality instead. A design that looks to be in the early stages will encourage more far-reaching comments and criticism, rather than ‘fine-tuning’.

This is most important when you’re trying to focus the user experience when performing tests, especially with people not overly familiar with your site or game. Performing tests to ascertain the correct information architecture, or user’s experiences with a website’s functionality would be useless if all your comments ended up being about the site’s colour scheme. To make it clear that the designs are rough, and the presentation is not the focus, it is important not to create overly realistic designs.

Similarly care should be taken to pick an appropriate prototyping method. Paper isn’t used just because it’s quick and easy, but it also helps manage the client’s expectations. If you spent the time making the webpage on a computer, they’d be expecting it to work like a real product. On paper, this isn’t the case. Like the uncanny valley, getting too close to the real thing will be detrimental to the client’s perception of your work.

And you wouldn't want your work to look like this...

So, what steps do we need to take to ensure that the client, or user will focus on the right areas of your wireframes and designs?

• When performing initial designs, use lo-fi methods, like paper, post-stick notes, and whiteboards, where possible
• If using design software, like Pidoco or Omnigraffle, use a ‘sketches’ template, which renders your design in a pseudo-drawn method.
• Avoid drawing/designing unnecessary parts of the design – focus only on the essential
• Use filler text, and don’t work on the copy until later.
• Make it clear to the user/client that these are rough, disposable prototypes.

So, whether you’re working with a client to design their site, or conducting user testing, take care not to over-present the design, in order to manage expectations, and prevent unnecessary complaints!

Related posts:

1. How to present a UX Report

How to make a Persona

            We’re going to assume you’ve already take some steps towards understanding the users. You could have designed a survey, or performed one on one interviews, and now you have a pretty good understanding of who your users are, and what they think about your product.

            First we need to divide the people into demographics. Did all the women over 50 find your game idea too violent? Did most of the children think it sounded like it would get dull?  Try to focus on the key areas where people picked up on things, and ignore the outliers for now – you want to turn all your results into 3-5 personae, of which only one or two will be the ‘key’ users you are designing for.

            Now you want to write about these groups you’ve divided the people into. Make them real people; give them names, occupations and a back story. Make their name an in-joke, that’ll make you laugh, but no-one else will ever get (Hello Bibi Andersson). Include some text about how they’d use the product, and what they’d expect from it. Also include some concerns that they may have. You want these people to be treated as real people, and so they need to be complete! Include a picture, so that people can see the people they are designing for.

pictures make everything more fun

Why you should be making personae

            There are many advantages to making usability and user experience personae. As a usability expert, it can help you by:

• Communicating your findings in a way that will be read (unlike that big 50 page report)
• Can be stuck on the wall, keeping your role in the team prominent
• Allowing you to make a definitive impact on the design of the product
• Incorporating usability and user experience principles from the start of the design process, promoting the use of your field as a holistic practice, rather than a ‘last stage’ of product development.

not that sort of persona

Why they need you to make personae

            You obviously don’t just make personae for your own benefit. The design team benefits in many ways from your deliverable as well. Not only does it give them an easy way to understand your findings, and focuses on the aspects that are important in their roles, but it gives them a concrete person, or group of people, to design for.

A persona gives an important baseline for future developments by the design team. By understanding who they are designing for, it reduces coder’s natural urge to spend too long designing for the ‘edge case’ users, and focuses their attention on the core functionality required (perhaps an example of how to maximise with the 80/20 rule ). This reduces feature creep, as the team can consider whether the personae identified would actually use the new suggested features, and as such helps deliver games and projects on time, and to budget. I’m guessing that 3D realms did not use personae.

Related posts:

1. No user testing? Oops! – The Digiscent iSmell
2. The right environment for user testing
3. Effective UI by the EffectiveUI team – Book Review

Straight back onto UX issues -
For my HCI Coursework, I’m designing a new product, with a focus on evaluating the user requirements -
To do this, I’ve made a survey (keeping in mind the issues I raised on making a UX Survey), to help me understand user’s needs.

It’d make my decade if you’d fill out the survey, it only takes 2-3 minutes and doesnt require any personal details

Click here to fill it out.

Feel free to comment here aswell, if you find any issues with the survey itself.

I have lots of ideas for UX blog posts for 2010, so expect alot more to come soon…

Steve

Related posts:

1. Who are you?
2. The Problems with Surveys for User Experience Tests

A less adept user

A software engineer is unlikely to be best placed to consider usability issues – they are too close to the code, and have a deep understanding of how it works. As such it can be difficult to distance themselves from their knowledge, and try the system as a new user, or one who is less adept with technology. It is therefore the job of a usability expert, using tools such as ‘voice aloud’ playtesting, and user interview to assess the usability of a system, and target aspects that need particular revision.

It is important to start usability testing early on in the development process of a product, and continue it throughout the lifecycle of a project. Catching issues early on will make them easier to fix than having to redesign major sections of the project at a late stage, and ignoring them altogether will may make the entire project redundant, such as with the Pocketmail

The Pocketmail is an example of a product that went all the way to market without obviously thinking about what the users want, and how they want to use it. It gave the ability to send and receive email from a handheld device, pretty impressive for the year 2000, even if it wouldn’t fit in your pocket. The only downside being that to send and receive mail, you’d have to hold it up to a land line telephone. Oh. Surely then, you’d call the person you need to contact? Or text them? SMS was widely used by then.

But back to the HCI course. The topic seems really interesting, and something I’m genuinely interested in. As a smug iPhone user (and former employee of a Mac computer store), I’ve seen the results of year of usability refinement, and it’s a field that is both really interesting, and that has a huge amount of room to expand into. It affects every user’s experience of a product, and can be the deciding factor in whether a product succeeds, or fails miserably. Within the course we are getting the opportunity to work with the university, and a real client on delivering usability solutions, and I am really excited about this opportunity. I’m currently reading Game Usability by Katherine Isbister & Noel Schaffer – its seems a useful introduction to the field, although its authority does seem a bit diminished by the author misspelling his name, and that he seems to interview his dad.. hmm! I’ve also signed up for the UPA. Can’t wait for my new ‘designing the user experience’ poster to come through the post!  

We’ve been asked to bring in examples of good and bad design for next week. The only bad example that springs to mind is craiglist – does it really need to give all that screen real estate to a list of cities/countries that a user won’t use after their first visit? Craig Newman is famously resistant to change though, so its too easy of an example (no Web 2.0 here!)

Related posts:

1. Ethnography as an application of third space theory
2. Desperation & Disappointment: Social interaction in We Dare