{"id":763,"date":"2010-03-22T11:06:07","date_gmt":"2010-03-22T10:06:07","guid":{"rendered":"http:\/\/www.stevebromley.com\/blog\/?p=763"},"modified":"2019-11-06T17:45:03","modified_gmt":"2019-11-06T16:45:03","slug":"quantifying-the-unquantifiable-expert-evaluations","status":"publish","type":"post","link":"https:\/\/www.stevebromley.com\/blog\/2010\/03\/22\/quantifying-the-unquantifiable-expert-evaluations\/","title":{"rendered":"Quantifying the unquantifiable \u2013 Expert Evaluations"},"content":{"rendered":"

At a recent UXBrighton<\/a> talk, iCrossing presented an interesting idea about applying metrics to expert evaluation. This is a potentially controversial topic, yet has numerous benefits if it can successfully make qualitative data quantitative (and turn impressions and thoughts into numbers). I\u2019ve outlined the method, and my thoughts on the issues around this.<\/p>\n

The UXBrighton event was presented in a new format as a series of short talks, from Harry Brignull\u2019s tips on time stamping notes<\/a>, to Danny Hope\u2019s templates for understanding user roles<\/a>. Also interesting was a talk on using google analytics, although the length of the talk meant that topic could only be skimmed, dissapointing as I’m an analytics fan. The most interesting idea presented was iCrossing’s<\/a> presentation on \u201cThe iCrossing Connected Brand index: how to measure a brand\u2019s effectiveness online\u201d, given by Ifraz Mughal.<\/p>\n

Expert Evaluation<\/h4>\n

As I\u2019ve mentioned before<\/a> an expert evaluation is a useful tool for getting an insight into potential usability and user experience issues on a website, or game, with limited resources. Although it can never replace running tests with real users, it can provide a quick approximation, and help highlight the biggest issues.<\/p>\n

The \u2018method\u2019 for an expert evaluation is simple. Get an expert to look at the site, or game, and tell the client what they think. Job done.<\/p>\n

\n
\"Scientist

My expert eye tells me you need smarter users<\/p><\/div>\n<\/div>\n

However an expert evaluation can only ever be subjective, and this is it\u2019s biggest weakness. A client can look at your page full of recommendations, and dismiss it as the opinion of one person. There\u2019s no easy way to see progress with changes, and a comparison with other sites can only ever be abstract.<\/p>\n

Quantifying an Expert Evaluation<\/h4>\n

iCrossing\u2019s solution is to quantify their expert evaluation. As part of their \u2018Connected Brand Index\u2019<\/a> idea, they rate their clients sites (and competitors), on UX-centric areas such as \u201cusefulness\u201d, \u201cusability\u201d and \u201cdesirability\u201d.<\/p>\n

A traditional expert evaluation would give a qualitative rating, and give examples to back this up, i.e. \u201cPoor \u2013 little emphasis, and diffused call to actions\u201d. Instead iCrossing will give the site a score, on a scale of -2 to 2 (2 being very good). This of course can be backed up with examples in a more in depth report.<\/p>\n

\n
\"kittens<\/a>

after the first few pages, the report can just be pictures of kittens. No-one reads that far.<\/p><\/div>\n<\/div>\n

The advantages:<\/h4>\n

There are numerous reasons why a client would prefer a scored \u2018rating\u2019, rather than comments.<\/p>\n