6
$\begingroup$

I work in a company that gathers work environment surveys. When a survey is done we create reports that are handed out to the managers in the company to show where they need to focus their efforts and such.

In these reports we show a comparison of the variance of the calculated group compared to the variance of a reference population. We "normalize" this score on a 0-100 scale. (The in-code calculation gives another scale from -inf to +inf, but for simplicity we say that 0 is 50, and just chop off everything over 100 and under 0.) Our biggest problem is that our customers gets very conscious of the actual number. Even though we try to tell them that a high/low number isn't necessarily bad, it just shows how your groups variance compares to the reference populations variance.

We are thinking of moving away from showing the number in the end-user report, and going for a visual representation of "high", "normal", "low" variance instead. But I can't figure out any good visual representation of it. It needs to be kind of neutral looking, but still show something... (.. i know ..)

Does anyone have any suggestions on how this could be achieved?

(Disclaimer: I am not a statistician, I'm a developer ;). I had one class of Statistics in my higher education, and that is over 5 years ago. So both the terms I use and my explanations could be totally meaningless.)

|cite|improve this question
$\endgroup$
  • $\begingroup$ I don't understand your question (too much English for me, perhaps). Do you want to display the variance on a picture ? $\endgroup$ – Stéphane Laurent Oct 8 '13 at 8:30
  • $\begingroup$ I guess since you write "good visual representation of it". But do you want to add this representation to a graphic showing your data ? And if, what kind of graphic ? $\endgroup$ – Stéphane Laurent Oct 8 '13 at 8:44
  • $\begingroup$ See the comment on the answer below ;) $\endgroup$ – Christian Wattengård Oct 8 '13 at 9:24
  • $\begingroup$ Would be more clear if we could see this graph. $\endgroup$ – Stéphane Laurent Oct 8 '13 at 9:42
7
$\begingroup$

If the main concern is "that our customers gets very conscious of the actual number. Even though we try to tell them that a high/low number isn't necessarily bad", then I think you should formally address them by plotting the confidence intervals. Variance is a bad choice because its unit is the square of whatever you're measuring with and they are much bigger and can be potentially very misleading. Standard deviation is a better approach but that does not answer your customers' concern because just by SD itself one cannot tell if the point estimates are really different from the reference mean.

Some kind of plot modified based on a forest plot would be a better candidate. It's compact and easy to integrate with text fields (where you can show the summary statistics.) And what's more, it answers your client question head on. If they are worried that 3.5 is so much lower than 4.6, then show them statistically they are not different. (Or maybe your clients are right.)

And somewhat contrary to what you propose to do (eliminating numeric output altogether), I'd perhaps try to enrich the graph so that it shows more data. Devices like panel histogram or violin plot (see below) allows you to show the distribution of the actual data, which perhaps will give a strong visual cue that the data do spread and it's not about just one point.

enter image description here

Also, I'd recommend evaluating your score distribution for skewness or other deviation from normal distribution, and see if augmenting with some non-parametric plot like box plot would be a good idea.

Side comment: I feel that your trimming criterion is very rigid, but I wouldn't question your familiarity with the scale. Anyhow, if such a trimming scheme is being used, I feel you're also obligated to report how many of the people are trimmed. It's because the variation you're using to convince them that things are not that different can be potentially altered by how you define the trimming threshold. It'd be awkward if they find out later.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Nice idea. A similar option is to replace the green droplets with histograms or interpolated densities $\endgroup$ – Itamar Oct 8 '13 at 13:39
6
$\begingroup$

The question can be reduced to "How do I show one value of interest against a reference distribution?". The former, showing a value of interest is the simple part; any dramatic marking at that point on the graph will do. So it will be useful to show different ways of displaying the reference distribution. We need not know what exactly that reference distribution is to give pertinent advice.

One of the most usual ways to show a distribution is to plot its probability density function or its cumulative density function (most often referred to as PDF and CDF respectively). The below plot shows a reference distribution that is normally distributed with a mean of 40 and a standard deviation of 15. A value of interest, at 80, is superimposed as an unmistakable large red dot. The grey line on the left plot shows the estimate of the CDF from the reference distribution, and the PDF in the right plot.

enter image description here

This type of graph is amenable to not as well defined reference distributions as well. For example you could plot the smoothed kernel density estimate of the PDF (or CDF) based on the prior reference values and superimpose the current value of interest just the same. From these plots one can either estimate the probability of getting a value above or below the current value of interest. The CDF it is read directly off the chart, PDF one has to make that estimate based on the area to the left or right of the value of interest. Another alternative (which Penguin shows) is to reflect the PDF and show its area as a violin plot. This provides some more visual gurth for the area in the tail of the distribution. Here the value of interest is marked by a black horizontal line, and the area above the value is colored red.

Violin Plot

Another popular alternative to showing distributions are box plots (or error bar charts). The error bars in the left chart covers the middle 80% of the reference distribution, and the box plot on the right plots the interquartile range within the grey bar and outside of the whiskers are typically considered to be a robust estimate of outliers.

enter image description here

These are potentially suspect to the rote worrying you noticed though - everything is fine if within the bars and the sky is falling if it is outside. Depending on how well the reference distribution is estimated, you could plot letter values beyond the interquartile range, or plot a continuous density strip to show the reference distribution. Below is an example of a continuous gradient, where the darker grey symbolizes a higher PDF for the reference distribution. (See 40 years of boxplots by Wickham & Stryjewski.)

enter image description here

|cite|improve this answer
$\endgroup$
3
$\begingroup$

As I understand from his comments, Christian wants to add an iconish representation of the variance to an existing plot. We don't know what kind of plot yet. For a dotplot, the moment of inertia representation of the variance possibly is a solution. Taking the standard deviation of the sample as the horizontal radius is a good option, and one can choose three colors for a "low-medium-high" scale.

enter image description here

|cite|improve this answer
$\endgroup$
1
$\begingroup$

The square root of variance is on the same scale as your data. For a normal distribution, this is known as the standard deviation.

It is a common practise to normalize values to multiples of the standard deviation, such that $+3\sigma$ is considered an unusually high value, whereas $-3\sigma$ is considered unusually low.

This is known as "standardization", or the $z$-score.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ The $\pm 3\sigma$ idea does probably not help since the aim is to grade variances (not means or individual values) $\endgroup$ – Michael M Oct 8 '13 at 7:13
  • 1
    $\begingroup$ Thanks. But some of the point is to present this data to "evryman". Managers at our customers can be everything from university professors to mechanics, and everything inbetween. So it needs to be easily understood. $\endgroup$ – Christian Wattengård Oct 8 '13 at 7:29
  • $\begingroup$ Well, you can map $+3\sigma$ to 100, and $-3\sigma$ to 0. And clip as you already did. The point is, to take variance into account when scaling the values. $\endgroup$ – Anony-Mousse Oct 8 '13 at 7:43
  • $\begingroup$ Yes, but the problems isn't the values. The problem is showing the variance without the reader getting too hung up on the actual number. Which is why we were looking for a visual icon-ish representation of high/low/normal variance. $\endgroup$ – Christian Wattengård Oct 8 '13 at 8:33
  • $\begingroup$ If your number already includes variance, then you don't really need to show it; because it has been standardized. Otherwise, I'd suggest adding a "variance" column that contains "high", "med", or "low", as desired... In particular, if you want it to be a neutral display. $\endgroup$ – Anony-Mousse Oct 8 '13 at 9:15
1
$\begingroup$

I would suggest using bar plots with pairs of bars for the target group next to the reference group, and on top of each bar plot confidence intervals (I's) centered around the top of the bar with length of $2\sigma$, where $\sigma$ is the standard deviation. See example from Wikipedia's Error Bar article:

enter image description here

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.