# Can the standard deviation of non-negative data exceed the mean?

I have some triangulated 3D meshes. The statistics for the triangle areas are:

• Min 0.000
• Max 2341.141
• Mean 56.317
• Std dev 98.720

So, does it mean anything particularly useful about the standard deviation or suggest there are bugs in calculating it, when the figures work out like the above? The areas are certainly far from being normally distributed.

And as someone mentioned in one of their responses below, the thing that really surprised me that it only took one SD from the mean for the numbers to go negative and thus out of the legal domain.

Thanks

-
In the dataset $\{2,2,2,202\}$ the sample standard deviation is $100$ while the mean is $52$--pretty close to what you observe. –  whuber Nov 18 '11 at 16:35
For a familiar (to some) example, the mean result of someone playing blackjack for one hour might be negative \$25 but with a standard deviation of say$100 (numbers for illustration). This large coefficient of variation makes it easier for someone to be tricked into thinking they are better than they really are. –  Michael McGowan Nov 18 '11 at 19:27
The follow-up question is quite informative, too: it places bounds on the SD of a set of (nonnegative data), given the mean. –  whuber Nov 22 '11 at 20:58

There is nothing that states that the standard deviation has to be less than or more than the mean. Given a set of data you can keep the mean the same but change the standard deviation to an arbitrary degree by adding/subtracting a positive number appropriately.

Using @whuber's example dataset from his comment to the question: {2, 2, 2, 202}. As stated by @whuber: the mean is 52 and the standard deviation is 100.

Now, perturb each element of the data as follows: {22, 22, 22, 142}. The mean is still 52 but the standard deviation is 60.

-
If you add to each element, you change the location parameter, i.e. the mean. You change dispersion (i.e. the standard deviation) by multiplying with a scale factor (provided your mean is zero). –  Dirk Eddelbuettel Nov 18 '11 at 16:42
@DirkEddelbuettel You are correct. I fixed the answer and provided an example for clarity. –  varty Nov 18 '11 at 16:50
I don't follow the example. The new dataset clearly is not derived from the original by "adding or subtracting a positive number" from each of the original values. –  whuber Nov 18 '11 at 17:15
I cannot edit it because I don't know what you're trying to say. If you can arbitrarily add separate values to each of the numbers in a dataset, you are merely changing one set of $n$ values into a completely different set of $n$ values. I don't see how that's relevant to the question or even to your opening paragraph. I think anyone would grant that such changes can alter the mean and SD, but that doesn't tell us why the SD of a set of nonnegative data can be any positive multiple of its mean. –  whuber Nov 18 '11 at 17:33
You are right: the quoted assertion is mine and it does not appear in your reply. (It happens to be correct and relevant, though. :-) One point I'm trying to get across is that the mere ability to change the SD while keeping the mean the same doesn't answer the question. How much can the SD be changed (while keeping all data non-negative)? The other point I have tried to make is that your example does not illustrate a general, predictable process of making such alterations to data. This makes it appear arbitrary, which is not of much help. –  whuber Nov 18 '11 at 17:49

Of course, these are independent parameters. You can set simple explorations in R (or another tool you may prefer).

R> set.seed(42)     # fix RNG
R> x <- rnorm(1000) # one thousand N(0,1)
R> mean(x)          # and mean is near zero
[1] -0.0258244
R> sd(x)            # sd is near one
[1] 1.00252
R> sd(x * 100)      # scale to std.dev of 100
[1] 100.252
R>


Similarly, you standardize the data you are looking at by subtracting the mean and dividing by the standard deviation.

Edit And following @whuber's idea, here is one an infinity of data sets which come close to your four measurements:

R> data <- c(0, 2341.141, rep(52, 545))
R> data.frame(min=min(data), max=max(data), sd=sd(data), mean=mean(data))
min     max      sd    mean
1   0 2341.14 97.9059 56.0898
R>

-
I am not sure I understand your point. They are not exactly independent as one could change the mean by perturbing one data point and thereby change the standard deviation as well. Did I misinterpret something? –  varty Nov 18 '11 at 16:32
Noting that triangle areas cannot be negative (as confirmed by the minimum value quoted in the question), one would hope for an example consisting solely of non-negative numbers. –  whuber Nov 18 '11 at 16:39
(+1) Re the edit: Try using 536 replications of 52.15 :-). –  whuber Nov 18 '11 at 17:12
Nice one re 536 reps. Should have done a binary search :) –  Dirk Eddelbuettel Nov 18 '11 at 18:43
@Dirk "these are independent parameters", consider the case when $X$ is a bernouilli. variance and mean are not independent: $var(X)=p(1-p)$. Consider a random variable $100>X>0$, maximum possible variance is $(50)^2$ now if you force the mean to be equal to one (i.e. lower than $50$) the maximum variance cannot be greater than $99/100*(1)^2+(1/100)*99^2$. There are more examples of bounded variables in nature than gaussians? –  robin girard Nov 18 '11 at 20:46

I am not sure why @Andy is surprised at this result, but I know he is not alone. Nor am I sure what the normality of the data has to do with the fact that the sd is higher than the mean. It is quite simple to generate a data set that is normally distributed where this is the case; indeed, the standard normal has mean of 0, sd of 1. It would be hard to get a normally distribute data set of all positive values with sd > mean; indeed, it ought not be possible (but it depends on the sample size and what test of normality you use... with a very small sample, odd things happen)

However, once you remove the stipulation of normality, as @Andy did, there's no reason why sd should be larger or smaller then the mean, even for all positive values. A single outlier will do this. e.g.

x <- runif(100, 1, 200) x <- c(x, 2000)

gives mean of 113 and sd of 198 (depending on seed, of course).

But a bigger question is why this surprises people.

I don't teach statistics, but I wonder what about the way statistics is taught makes this notion common.

-
I have never studied statistics, just a couple of units of engineering math and that was thirty years ago. Other people at work, who I thought understood the domain better, have been talking about representing bad data by "number of std devs away from the mean". So, it's more about "how std dev is commonly mentioned" than "taught" :-) –  Andy Dent Nov 18 '11 at 23:11
@Andy having a large number of std away from the mean simply means that the variable is not significantly different from zero. Then it depend on the context (was is the meaning of the random variable) but in some case you might want to remove those ? –  robin girard Nov 19 '11 at 9:13
@Peter see my comment to Dirk, this might explain the "surprise" in some context. Actually I have teached statistic for some time and I have never seen the surprise you are talking about. Anyway, I prefer studient that are surprised by everything I am pretty sure this is a good epistemologic position (better than fainting the absolutly no surprise position :)). –  robin girard Nov 19 '11 at 9:20
@AndyDent "bad" data, to me, means data that is incorrectly recorded. Data that is far from the mean are outliers. For example, suppose you are measuring people's heights. If you measure me and record my height as 7'5' instead of 5'7, that's bad data. If you measure Yao Ming and record his height as 7'5", that's an outlier but not bad data. Regardless of the fact that it is very far from the mean (something like 6 sds) –  Peter Flom Nov 19 '11 at 11:57
@Peter Florn, In our case, we have outliers which we want to get rid of because they represent triangles that will cause algorithmic problems processing the mesh. They may even be "bad data" in your sense if they were created by faulty scanning devices or conversion from other formats :-) Other shapes may have outliers which are legitimately a long way from the mean but don't represent a problem. One of the more interesting things about this data is we have "bad data" at both ends but the small ones are not far from the mean. –  Andy Dent Nov 20 '11 at 13:04
show 1 more comment

Just adding a generic point that, from a calculus perspective, $$\int x f(x) \text{d}x$$ and $$\int x^2 f(x) \text{d}x$$ are related by Jensen's inequality, assuming both integrals exist, $$\int x^2 f(x) \text{d}x \ge \left\{ \int x f(x) \text{d}x \right\}^2\,.$$ Given this general inequality, nothing prevents the variance to get arbitrarily large. Witness the Student's t distribution with $\nu$ degrees of freedom, $$X \sim \mathfrak{T}(\nu,\mu,\sigma)$$ and take $Y=|X|$ whose second moment is the same as the second moment of $X$, $$\mathbb{E}[|X|^2] = \frac{\nu}{\nu-2}\sigma^2 + \mu^2,$$ when $\nu>2$. So it goes to infinity when $\nu$ goes down to $2$, while the mean of $Y$ remains finite as long as $\nu>1$.

-
Please note the explicit restriction to nonnegative values in the question. –  whuber Nov 22 '11 at 20:47
The Student example gets easily translated into the absolute-value-of-a-Student's-t-distribution example... –  Xi'an Nov 22 '11 at 21:45
But that changes the mean, of course :-). The question concerns the relationship between the SD and the mean (see its title). I am not saying you're wrong; I'm just (implicitly) suggesting that your reply could, with little work, more directly address the question. –  whuber Nov 22 '11 at 22:39
@whuber: ok, I edited the above to consider the absolute value (I also derived the mean of the absolute value but <a href="ceremade.dauphine.fr/~xian/meanabs.pdf">it is rather ungainly</a>...) –  Xi'an Nov 24 '11 at 8:56

Perhaps the OP is surprised that the mean - 1 S.D. is a negative number (especially where the minimum is 0).

Here are two examples that may clarify.

Suppose you have a class of 20 first graders, where 18 are 6 years old, 1 is 5, and 1 is 7. Now add in the 49-year-old teacher. The average age is 8.0, while the standard deviation is 9.402.

You might be thinking: one standard deviation ranges for this class ranges from -1.402 to 17.402 years. You might be surprised that the S.D. includes a negative age, which seems unreasonable.

You don't have to worry about the negative age (or the 3D plots extending less than the minimum of 0.0). Intuitively, you still have about two-thirds of the data within 1 S.D. of the mean. (You actually have 95% of the data within 2 S.D. of the mean.)

When the data takes on a non-normal distribution, you will see surprising results like this.

Second example. In his book, Fooled by Randomness, Nassim Taleb sets up the thought experiment of a blindfolded archer shooting at a wall of inifinte length. The archer can shoot between +90 degrees and -90 degrees.

Every once in a while, the archer will shoot the arrow parallel to the wall, and it will never hit. Consider how far the arrow misses the target as the distribution of numbers. The standard deviation for this scenario would be inifinte.

-
The rule about 2/3 of the data within 1 SD of the mean is for normal data. But the classroom data is clearly non-normal (even if it passes some test for normality because of small sample size). Taleb's example is terrible. It's an example of poor operationalization of a variable. Taken as is, both the mean and the SD would be infinite. But that's nonsense. "How far the arrow misses" - to me, that's a distance. The arrow, no matter how it is fired, will land somewhere. Measure the distance from there to the target. No more infinity. –  Peter Flom Nov 18 '11 at 21:27
Yup, the OP was sufficiently surprised the first time I saw mean - 1 SD went negative that I wrote a whole new set of unit tests using data from Excel to confirm at least my algorithm was calculating the same values. Because Excel just has to be an authoritative source, right? –  Andy Dent Nov 18 '11 at 23:13
@Peter The 2/3 rule (part of a 68-95-99.7% rule) is good for a huge variety of datasets, many of them non-normal and even for moderately skewed ones. (The rule is quite good for symmetric datsets.) The non-finiteness of the SD and mean are not "nonsense." Taleb's example is one of the few non-contrived situations where the Cauchy distribution clearly governs the data-generation process. The infiniteness of the SD does not derive from the possibility of missing the wall but from the distribution of actual hits. –  whuber Nov 25 '11 at 15:09
@whuber I was aware of your first point, which is a good one. I disagree about your second point re Taleb. It seems to me like another contrived example. –  Peter Flom Nov 25 '11 at 22:31

What you seem to have in mind implicitly is a prediction interval that would bound the occurrence of new observations. The catch is: you must postulate a statistical distribution compliant with the fact that your observations (triangle areas) must remain non-negative. Normal won't help, but log-normal might be just fine. In practical terms, take the log of observed areas, calculate the mean and standard deviation, form a prediction interval using the normal distribution, and finally evaluate the exponential for the lower and upper limits -- the transformed prediction interval won't be symmetric around the mean, and is guaranteed to not go below zero. This is what I think the OP actually had in mind.

-

A gamma random variable $X$ with density $$f_X(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x} I_{(0,\infty)}(x) \, ,$$ with $\alpha,\beta>0$, is almost surely positive. Choose any mean $m>0$ and any standard deviation $s>0$. As long as they are positive, it does not matter if $m>s$ or $m<s$. Putting $\alpha=m^2/s^2$ and $\beta=m/s^2$, the mean and standard deviation of $X$ are $\mathbb{E}[X]=\alpha/\beta=m$ and $\sqrt{\mathbb{Var}[X]}=\sqrt{\alpha/\beta^2}=s$. With a big enough sample from the distribution of $X$, by the SLLN, the sample mean and sample standard deviation will be close to $m$ and $s$. You can play with R to get a feeling of this. Here are examples with $m>s$ and $m<s$.

> m <- 10
> s <- 1
> x <- rgamma(10000, shape = m^2/s^2, rate = m/s^2)
> mean(x)
[1] 10.01113
> sd(x)
[1] 1.002632

> m <- 1
> s <- 10
> x <- rgamma(10000, shape = m^2/s^2, rate = m/s^2)
> mean(x)
[1] 1.050675
> sd(x)
[1] 10.1139

-
As pointed out in the other answers, the mean $\bar{x}$ and standard deviation $\sigma_x$ are essentially unrelated in that it is not necessary for the standard deviation to be smaller than the mean. However, if the data are nonnegative, taking on values in $[0,c]$, say, then, for large data sets (where the distinction between dividing by $n$ or by $n-1$ does not matter very much), the following inequality holds: $$\sigma_x \leq \sqrt{\bar{x}(c-\bar{x})} \leq \frac{c}{2}$$ and so if $\bar{x} > c/2$, we can be sure that $\sigma_x$ will be smaller. Indeed, since $\sigma_x = c/2$ only for an extremal distribution (half the data have value $0$ and the other half value $c$), $\sigma_x < \bar{x}$ can hold in some cases when $\bar{x} < c/2$ as well. If the data are measurements of some physical quantity that is nonnegative (e.g. area) and have an empirical distribution that is a good fit to a normal distribution, then $\sigma_x$ will be considerably smaller than $\min\{\bar{x}, c - \bar{x}\}$ since the fitted normal distribution should assign negligibly small probability to the events $\{X < 0\}$ and $\{X > c\}$.