{"id":2205,"date":"2017-05-26T17:50:29","date_gmt":"2017-05-26T16:50:29","guid":{"rendered":"http:\/\/www.stevebromley.com\/blog\/?p=2205"},"modified":"2019-11-06T17:44:57","modified_gmt":"2019-11-06T16:44:57","slug":"measuring-and-minimising-sickness-in-vr-games","status":"publish","type":"post","link":"https:\/\/www.stevebromley.com\/blog\/2017\/05\/26\/measuring-and-minimising-sickness-in-vr-games\/","title":{"rendered":"Measuring and Minimising Sickness in Virtual Reality"},"content":{"rendered":"

This week I gave a talk at The Research Thing<\/a> on how Simulator Sickness can be identified and measured in virtual reality games, and some best practises for avoiding it. It’s based on my experience from PlayStation, and a lot of good research done by others.<\/p>\n

I’ve shared the slides below, and my notes. The video of the talk is available here.<\/a><\/p>\n

<\/p>\n

\"\"
\nSo here we go…<\/p>\n

\n

\"\"
\nI was the lead researcher for two of the launch games for PlayStation VR, PlayStation VR Worlds<\/a> and Until Dawn: Rush of Blood<\/a><\/p>\n

\n

\"\"
\nI also worked with other researchers from PlayStation’s team on their other launch titles, including Rigs<\/a> and the installation process for the VR headset itself, in addition to working on many other PlayStation titles.<\/a><\/p>\n

\n

\"\"
\nA few months ago I left PlayStation to lead the research team at Parliament. They are a great team working on some really exciting stuff, but nothing related to VR – so for today my talk is based on my time at PlayStation and as a member of the Games User Research community.<\/p>\n

\n

\"\"
\nWhich brings me to today. I’d like to talk about Video Games. One of the many reasons video games are an exciting medium is because they allow you to have experiences that you couldn’t have in real life.<\/p>\n

\n

\"\"
\nFor example, they let you travel to exciting new worlds. See amazing sights and vistas. Meet new civilisations of alien life forms, completely different to anything you could possibly imagine.<\/p>\n

\n

\"\"
\nAnd when you meet them, you can shoot them in the face…<\/p>\n

\n

\"\"
\nOr you can travel to a city above the clouds – representing a lost American state following Christian Utopian ideals, and explore ideas of community and individualism beyond our every day experiences.<\/p>\n

\n

\"\"
\nAnd, when you meet them, you can also shoot them in the face…<\/p>\n

\n

\"\"
\nOr you could travel back in time and explore Roman or Greek Mythology. The Colliseum, The Fields of Elysium, Mount Olympus…<\/p>\n

\n

\"\"
\n…and when you are there, you can punch Zeus in the face.<\/p>\n

\n

\"\"
\nMy point being that games lets you have experiences that you would’nt be able to experience in real life. Virtual Reality takes that one step further. Rather than being separated from the experience by the screen, it allows you to actually be *inside* the action, and creates presence – the feeling of being in that environment.
\nWhen Virtual Reality started becoming a technical possibility, this lead to the potential of whole new types of experience, perhaps even some that don’t involve shooting someone. <\/p>\n

For a while things were good. People experimented with new ideas and scenarios, and explored the breadth of things that could be done in VR. But then reports started coming back that some experiences were wrong, and were making people feel sick. Some so sick that they wouldn’t want to play VR again. This is … simulator sickness!<\/p>\n

\n

\"\"
\nFor the purposes of today, Simulator Sickness is very similar to Motion Sickness, with symptoms such as nausea, stomach awareness and dizziness.
\nIt is triggered by the perception of motion despite being sat still. There are some great talks about the reasons why this occurs, including one by Ben Lewis Evans<\/a> that I’ve referenced many times, but one theory is that it’s caused by the disconnect between your eyes and your brain.<\/p>\n

Your eyes report that you are moving, because they can see things moving past them. Your brain has other ways of telling if you’re moving, and it says that you’re not. There’s a disconnect, and when there’s a disconnect, bad things happen.<\/p>\n

\n

\"\"
\nSo, simulator sickness is a thing. If you are building a VR game or experience, the question is what should we do about it?<\/p>\n

\n

\"\"
\nLuckily we have a tool to help. In the nineties, the army discovered this issue with sickness when working on flight simulators, and developed the Simulator Sickness Questionnaire<\/a>. This is a list of 16 symptoms associated with simulator sickness, and each can be assigned a rating from None->Severe based on how the participant is feeling. This gives us something we can use to measure sickness.<\/p>\n

\n

\"\"
\nThe first thing we want to do is work out if there is a problem. A potential way of doing this is by running a small study, playing a representative part of the game, and using the sickness questionnaire before and after. It’s important to take a pre-score as people may come in feeling sick already, and you don’t want that to impact your results.<\/p>\n

Each of the symptoms can be converted to a score of between 0 and 3 based on the level of sickness reported, and these can be aggregated and averaged to come up with a ‘sickness score’, somewhere between 0 and 48.<\/p>\n

The challenge is deciding what score is ‘ok’. Because you’ve primed people to think about sickness, many will report some level of sickness. Also some people will feel sickness regardless of the contents of the VR experience, so a score of 0 is probably impossible. Perhaps as a rule of thumb a score between 0-10 is fine, a score between 11-30 can be improved, and a score of 30+ is a irredeemably bad experience and you should give up with the whole thing.<\/p>\n

\n

\"\"
\nSo, if your sickness score came back acceptable, that’s great – you can relax and not read on any further.<\/p>\n

\n

\"\"
\nHowever we’re still here, so presumably there is a problem with your game or experience. The next thing to do is to ask yourself – do I have a lot of money and VR is very very important to my organisation. If yes, you should A\/B test. If no, we’ll look at heuristic review.<\/p>\n

\n

\"\"
\nLet’s imagine we are the big company, and look at what we could do.<\/p>\n

\n

\"\"
\nA\/B testing is a really powerful tool in this case. VR experiences are large and complicated and have many aspects, and we need to work out what aspect of the experience is causing sickness. A\/B testing could allow us to do that.
\nOnce we’ve identified what the problem is, we need to tell if we are fixing it or not. Again, A\/B testing can help with that.
\nIt’s science.<\/p>\n

\n

\"\"
\nPerhaps everyone is familiar with the idea of A\/B testing. In short, it’s changing one single aspect of an experience, and measuring it to see if it’s had an effect. In this context, we can alter aspects of the VR experience (the controls, the movement, the textures) and the sicknesss questionnaire gives us the ability to measure the effect. This will generate insights about what is causing sickness in the experience, and what we can do to fix it. <\/p>\n

Sounds simple right?<\/p>\n

\n

\"\"<\/p>\n

It’s not. I have to thank Cyril<\/a> for explaining this to me, many times, in simple words.<\/p>\n

First of all, sickness is hard to measure.<\/p>\n