Erik Rothoff Andersson (Kick Ass) on testing with users & why you shouldn’t listen to what users say!

Erik Andersson is the creator of Kick Ass, the new iPhone game based on his hit web-app. Kick-Ass is an adaption of asteroids that allows you to attack your favourite (or least favourite) websites, destroying them with a selection of ships, while earning achievements.
The iOS game can be downloaded here, and the original bookmark-let is available to play for free

Kick Ass

Kick Ass in action...

Today Erik tells us about his experiences with finding testers, getting user feedback, and why you should look at what people do, and not what they say!

First of all, I want you to bare in mind that I am in no way experienced in the field of creating computer games. I’m 18 years old, I have been programming for around 5 years, and most of that time I have been making websites. I have been successful in it, I’ve already had several jobs with some really large Swedish websites (where I’m from). The original kick ass was something that I started at 10 pm and was finished with a couple of days later. A couple of months after that I made a post on reddit where I linked to it. A week later it went viral and just exploded. Fast forward another few months and I begin with the iPhone app.

My idea of testing has always been to just write some code, try it out, if it works, great, if not, fix bugs. My twin brother, who is more the design monkey (if I’m the code monkey), has been the one talking about good UI vs bad UI. And we have worked together a lot, and now I also have a need for making good UIs. But the notion of a good UI is not something that I have scientifically proved to be one through user testing. It’s more what I can gather myself, so all my knowledge about UX is totally subjective. Some things are simple as daylight to see. If your mom does not know how to use your app you lost the market for +40 year olds.

Attacking iTunes

Take that iTunes!

… And that is the way I make stuff. The idea of using “qualified” testers is something that hit me shortly after I released the iOS app. Before that I had only tested it on me, my girlfriend, my brother and my father (and that was only once!). I guess that shows how inexperienced I am in game development. If you read about professionals who make games you read about testers, beta testing periods etc. And the fact that it can take 40 people more than two years to make ONE game is totally alien to me, seeing as my experience is that it took one night and one person. That also shows in the iOS app. The concept has HUGE potential. For example, because the code is open source, many people have taken it and adapted it for their ad campaigns. One campaign, which took the script, changed the image of the ship, and launched a microsite with it had 300 000 unique visitors to it. The campaign won several awards and was nominated for a Cannes Lion (luckily, they had me in the credits too, which would be totally absurd if they hadn’t). Paramount used the idea too, which also had huge success (that ad is also going up for awards, I’ve heard). So anyway…

What I mean is that if an experienced game developer/company decided to do something out of this concept, it would be huge. But that would require many difference instances of knowledge. UX, menu designers, music composers, level designers, programmers, marketing, and that is not me. The iOS app gets a couple of sales a day, yet the bookmarklet has had over 1 million unique visitors. Why? Because the bookmarklet is a silly thing that you play with for 15 minutes, and the game (iOS app) has to be a game.

And on the topic of feedback: I mostly don’t trust user feedback. Back when I worked for the large swedish site, every time we launched a new feature, they hated it. 90% of the comments were about how it sucked and along the lines of “it sucks. i liked it more before”. The target audience for these sites were 13-17 year olds, and they are apparently really hard to please. During that period I learned a lot. Key things were: If it ain’t broke, don’t fix it. If you think it’s broke, but the users don’t, change it gradually. Sneak in changes. Fix it over time. Don’t make a complete overhaul and release over one night. Your users will become disoriented and like you less. That is what I’m planning to do for the bookmarklet. Change the little things first, and let those changes grow, so the users know what is happening, and can follow the development.

When I ask users for feedback, I’m always very skeptical to their responses. One user might say something that only he doesn’t like or wants. It’s your job as the creator of something to know what is bad and what needs done. Asking users for feedback should only be done when you want to validate your opinion, but that might not even be the right time to do it. A wild estimate is that only 5% of your users are sharing their opinion. The other 95% you will never know what they want.

Feedback can be great when you want know how the users use your application/game. But don’t trust their words, look at their actions instead. Why don’t they click there, why did they choose that option, why don’t they know what do. Those are the questions you should be asking yourself when you’ve seen their actions. You have to analyze the feedback.

Erik has a new blog, including a interesting post-mortem of his game, at erikrothoff.com

0 Comments On “Erik Rothoff Andersson (Kick Ass) on testing with users & why you shouldn’t listen to what users say!”

Leave a Reply

Your email address will not be published. Required fields are marked *