What can I say with mean, variance and standard deviation?

Question

I wrote this code in R:

getinfoNumeric <- function(attr) {
  cat(min(attr),  "   ")
  cat(max(attr),  "   ")
  cat(mean(attr), "   ")
  cat(var(attr),  "   ")
  cat(sd(attr),   "   ")
}

When I apply it to an attribute, it gives me the following result:

• 50
• 100
• 71.89536
• 37.50461
• 6.124101

I don't understand the meaning of the last two values. Can you help me? I learnt that:

• variance measures how far a set of numbers are spread out from their average value

• the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean of the set, while a high standard deviation indicates that the values are spread out over a wider range

But, looking at this data, what does it mean? My data is about cocoa percentage in chocolate bars. So the minimum percentage is 50%, the maximum is 100% and the mean value is 71.89%. But what about variance and standard deviation? Does the variance mean that the percentage of chocolate is concentrated between 71.89 - 37.5 and 71.89 + 37.5? And what about standard deviation? Does it mean that the percentage tends to be close to the mean?

Histogram:

Answer 1

Your histogram looks roughly normal, which makes for an easy interpretation of standard deviation.

In a normal distribution, 68% of observations are $\pm$ one standard deviation of the mean, 95% of observations are within $\pm$ two standard deviations of the mean, and 99.7% of observations are within $\pm$ three standard deviations of the mean. You can test that with a few lines of code.

# should be about 68
length(attr[attr<50+6.124 & attr>50-6.124]/length(attr)*100 
#
# should be about 95
length(attr[attr<50+6.124*2 & attr>50-6.124*2]/length(attr)*100 
#
# should be about 99.7
length(attr[attr<50+6.124*3 & attr>50-6.124*3]/length(attr)*100 

This characterization fails for non-normal distributions. However, we can bound how many observations lie within $k$ standard deviations of the mean. This is called Chebyshev’s inequality:

https://en.m.wikipedia.org/wiki/Chebyshev%27s_inequality

In words, no more than $100 \times \frac{1}{k^2}$ percent of the observations will be beyond $k$ standard deviations of the mean.

Getting back to the original question, smaller standard deviations (and smaller variances) tend to indicate more clustering around the mean than larger standard deviations (and larger variances).

There are other ways to measure spread, but variance remains popular because it is easy to calculate, it is easy to test (hypothesis testing), and because of its unique role in a very important theorem called the central limit theorem. Then one might want to take the square root of variance to get the standard deviation, as variance is expressed in the square of the original units (e.g. $€^2$ when the original unit is $€$).