Can the standard deviation of non-negative data exceed the mean?

Question

I have some triangulated 3D meshes. The statistics for the triangle areas are:

• Min 0.000
• Max 2341.141
• Mean 56.317
• Std dev 98.720

So, does it mean anything particularly useful about the standard deviation or suggest there are bugs in calculating it, when the figures work out like the above? The areas are certainly far from being normally distributed.

And as someone mentioned in one of their responses below, the thing that really surprised me that it only took one SD from the mean for the numbers to go negative and thus out of the legal domain.

Thanks

Answer 1

There is nothing that states that the standard deviation has to be less than or more than the mean. Given a set of data you can keep the mean the same but change the standard deviation to an arbitrary degree by adding/subtracting a positive number appropriately.

Using @whuber's example dataset from his comment to the question: {2, 2, 2, 202}. As stated by @whuber: the mean is 52 and the standard deviation is 100.

Now, perturb each element of the data as follows: {22, 22, 22, 142}. The mean is still 52 but the standard deviation is 60.

Answer 2

Of course, these are independent parameters. You can set simple explorations in R (or another tool you may prefer).

R> set.seed(42)     # fix RNG
R> x <- rnorm(1000) # one thousand N(0,1)
R> mean(x)          # and mean is near zero
[1] -0.0258244
R> sd(x)            # sd is near one
[1] 1.00252
R> sd(x * 100)      # scale to std.dev of 100
[1] 100.252
R> 

Similarly, you standardize the data you are looking at by subtracting the mean and dividing by the standard deviation.

Edit And following @whuber's idea, here is one an infinity of data sets which come close to your four measurements:

R> data <- c(0, 2341.141, rep(52, 545))
R> data.frame(min=min(data), max=max(data), sd=sd(data), mean=mean(data))
  min     max      sd    mean
1   0 2341.14 97.9059 56.0898
R> 

Answer 3

I am not sure why @Andy is surprised at this result, but I know he is not alone. Nor am I sure what the normality of the data has to do with the fact that the sd is higher than the mean. It is quite simple to generate a data set that is normally distributed where this is the case; indeed, the standard normal has mean of 0, sd of 1. It would be hard to get a normally distribute data set of all positive values with sd > mean; indeed, it ought not be possible (but it depends on the sample size and what test of normality you use... with a very small sample, odd things happen)

However, once you remove the stipulation of normality, as @Andy did, there's no reason why sd should be larger or smaller then the mean, even for all positive values. A single outlier will do this. e.g.

x <- runif(100, 1, 200) x <- c(x, 2000)

gives mean of 113 and sd of 198 (depending on seed, of course).

But a bigger question is why this surprises people.

I don't teach statistics, but I wonder what about the way statistics is taught makes this notion common.

Answer 4

Just adding a generic point that, from a calculus perspective, $$ \int x f(x) \text{d}x $$ and $$ \int x^2 f(x) \text{d}x $$ are related by Jensen's inequality, assuming both integrals exist, $$ \int x^2 f(x) \text{d}x \ge \left\{ \int x f(x) \text{d}x \right\}^2\,. $$ Given this general inequality, nothing prevents the variance to get arbitrarily large. Witness the Student's t distribution with $\nu$ degrees of freedom, $$ X \sim \mathfrak{T}(\nu,\mu,\sigma) $$ and take $Y=|X|$ whose second moment is the same as the second moment of $X$, $$ \mathbb{E}[|X|^2] = \frac{\nu}{\nu-2}\sigma^2 + \mu^2, $$ when $\nu>2$. So it goes to infinity when $\nu$ goes down to $2$, while the mean of $Y$ remains finite as long as $\nu>1$.

Answer 5

Perhaps the OP is surprised that the mean - 1 S.D. is a negative number (especially where the minimum is 0).

Here are two examples that may clarify.

Suppose you have a class of 20 first graders, where 18 are 6 years old, 1 is 5, and 1 is 7. Now add in the 49-year-old teacher. The average age is 8.0, while the standard deviation is 9.402.

You might be thinking: one standard deviation ranges for this class ranges from -1.402 to 17.402 years. You might be surprised that the S.D. includes a negative age, which seems unreasonable.

You don't have to worry about the negative age (or the 3D plots extending less than the minimum of 0.0). Intuitively, you still have about two-thirds of the data within 1 S.D. of the mean. (You actually have 95% of the data within 2 S.D. of the mean.)

When the data takes on a non-normal distribution, you will see surprising results like this.

Second example. In his book, Fooled by Randomness, Nassim Taleb sets up the thought experiment of a blindfolded archer shooting at a wall of inifinte length. The archer can shoot between +90 degrees and -90 degrees.

Every once in a while, the archer will shoot the arrow parallel to the wall, and it will never hit. Consider how far the arrow misses the target as the distribution of numbers. The standard deviation for this scenario would be inifinte.

Answer 6

A gamma random variable $X$ with density $$ f_X(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x} I_{(0,\infty)}(x) \, , $$ with $\alpha,\beta>0$, is almost surely positive. Choose any mean $m>0$ and any standard deviation $s>0$. As long as they are positive, it does not matter if $m>s$ or $m<s$. Putting $\alpha=m^2/s^2$ and $\beta=m/s^2$, the mean and standard deviation of $X$ are $\mathbb{E}[X]=\alpha/\beta=m$ and $\sqrt{\mathbb{Var}[X]}=\sqrt{\alpha/\beta^2}=s$. With a big enough sample from the distribution of $X$, by the SLLN, the sample mean and sample standard deviation will be close to $m$ and $s$. You can play with R to get a feeling about this. Here are examples with $m>s$ and $m<s$.

> m <- 10
> s <- 1
> x <- rgamma(10000, shape = m^2/s^2, rate = m/s^2)
> mean(x)
[1] 10.01113
> sd(x)
[1] 1.002632

> m <- 1
> s <- 10
> x <- rgamma(10000, shape = m^2/s^2, rate = m/s^2)
> mean(x)
[1] 1.050675
> sd(x)
[1] 10.1139

Answer 7

As pointed out in the other answers, the mean $\bar{x}$ and standard deviation $\sigma_x$ are essentially unrelated in that it is not necessary for the standard deviation to be smaller than the mean. However, if the data are nonnegative, taking on values in $[0,c]$, say, then, for large data sets (where the distinction between dividing by $n$ or by $n-1$ does not matter very much), the following inequality holds: $$\sigma_x \leq \sqrt{\bar{x}(c-\bar{x})} \leq \frac{c}{2}$$ and so if $\bar{x} > c/2$, we can be sure that $\sigma_x$ will be smaller. Indeed, since $\sigma_x = c/2$ only for an extremal distribution (half the data have value $0$ and the other half value $c$), $\sigma_x < \bar{x}$ can hold in some cases when $\bar{x} < c/2$ as well. If the data are measurements of some physical quantity that is nonnegative (e.g. area) and have an empirical distribution that is a good fit to a normal distribution, then $\sigma_x$ will be considerably smaller than $\min\{\bar{x}, c - \bar{x}\}$ since the fitted normal distribution should assign negligibly small probability to the events $\{X < 0\}$ and $\{X > c\}$.

Answer 8

What you seem to have in mind implicitly is a prediction interval that would bound the occurrence of new observations. The catch is: you must postulate a statistical distribution compliant with the fact that your observations (triangle areas) must remain non-negative. Normal won't help, but log-normal might be just fine. In practical terms, take the log of observed areas, calculate the mean and standard deviation, form a prediction interval using the normal distribution, and finally evaluate the exponential for the lower and upper limits -- the transformed prediction interval won't be symmetric around the mean, and is guaranteed to not go below zero. This is what I think the OP actually had in mind.

Answer 9

Felipe Nievinski points to a real issue here. It makes no sense to talk in normal distribution terms when the distribution is clearly not a normal distribution. All-positive values with a relatively small mean and relatively large standard deviation cannot have a normal distribution. So, the task is to figure out what sort of distribution fits the situation. The original post suggests that a normal distribution (or some such) was clearly in mind. Otherwise negative numbers would not come up. Log normal, Rayleigh, Weibull come to mind ... I don't know but wonder what might be best in a case like this?