17
$\begingroup$

In the discussion following a recent question about whether the standard deviation can exceed the mean, one question was raised briefly but never fully answered. So I am asking it here.

Consider a set of $n$ nonnegative numbers $x_i$ where $0 \leq x_i \leq c$ for $1 \leq i \leq n$. It is not required that the $x_i$ be distinct, that is, the set could be a multiset. The mean and variance of the set are defined as $$\bar{x} = \frac{1}{n}\sum_{i=1}^n x_i, ~~ \sigma_x^2 = \frac{1}{n}\sum_{i=1}^n (x_i - \bar{x})^2 = \left(\frac{1}{n}\sum_{i=1}^n x_i^2\right) - \bar{x}^2$$ and the standard deviation is $\sigma_x$. Note that the set of numbers is not a sample from a population and we are not estimating a population mean or a population variance. The question then is:

What is the maximum value of $\dfrac{\sigma_x}{\bar{x}}$, the coefficient of variation, over all choices of the $x_i$'s in the interval $[0,c]$?

The maximum value that I can find for $\frac{\sigma_x}{\bar{x}}$ is $\sqrt{n-1}$ which is achieved when $n-1$ of the $x_i$ have value $0$ and the remaining (outlier) $x_i$ has value $c$, giving $$\bar{x} = \frac{c}{n},~~ \frac{1}{n}\sum x_i^2 = \frac{c^2}{n} \Rightarrow \sigma_x = \sqrt{\frac{c^2}{n} - \frac{c^2}{n^2}} = \frac{c}{n}\sqrt{n-1}.$$ But this does not depend on $c$ at all, and I am wondering if larger values, possibly dependent on both $n$ and $c$, can be achieved.

Any ideas? I am sure that this question has been studied in the statistical literature before, and so references, if not the actual results, would be much appreciated.

$\endgroup$
  • $\begingroup$ I think you're right about that being the largest possible value, and I am also surprised that $c$ doesn't matter. Cool. $\endgroup$ – Peter Flom - Reinstate Monica Nov 19 '11 at 12:07
  • 7
    $\begingroup$ $c$ should not affect the result as $\frac{\sigma_x}{\bar{x}}$ does not change if all the values are multiplied by any positive constant $k$. $\endgroup$ – Henry Nov 19 '11 at 18:13
15
$\begingroup$

Geometry provides insight and classical inequalities afford easy access to rigor.

Geometric solution

We know, from the geometry of least squares, that $\mathbf{\bar{x}} = (\bar{x}, \bar{x}, \ldots, \bar{x})$ is the orthogonal projection of the vector of data $\mathbf{x}=(x_1, x_2, \ldots, x_n)$ onto the linear subspace generated by the constant vector $(1,1,\ldots,1)$ and that $\sigma_x$ is directly proportional to the (Euclidean) distance between $\mathbf{x}$ and $\mathbf{\bar{x}}.$ The non-negativity constraints are linear and distance is a convex function, whence the extremes of distance must be attained at the edges of the cone determined by the constraints. This cone is the positive orthant in $\mathbb{R}^n$ and its edges are the coordinate axes, whence it immediately follows that all but one of the $x_i$ must be zero at the maximum distances. For such a set of data, a direct (simple) calculation shows $\sigma_x/\bar{x}=\sqrt{n}.$

Solution exploiting classical inequalities

$\sigma_x/\bar{x}$ is optimized simultaneously with any monotonic transformation thereof. In light of this, let's maximize

$$\frac{x_1^2+x_2^2+\ldots+x_n^2}{(x_1+x_2+\ldots+x_n)^2} = \frac{1}{n}\left(\frac{n-1}{n}\left(\frac{\sigma_x}{\bar{x}}\right)^2+1\right) = f\left(\frac{\sigma_x}{\bar{x}}\right).$$

(The formula for $f$ may look mysterious until you realize it just records the steps one would take in algebraically manipulating $\sigma_x/\bar{x}$ to get it into a simple looking form, which is the left hand side.)

An easy way begins with Holder's Inequality,

$$x_1^2+x_2^2+\ldots+x_n^2 \le \left(x_1+x_2+\ldots+x_n\right)\max(\{x_i\}).$$

(This needs no special proof in this simple context: merely replace one factor of each term $x_i^2 = x_i \times x_i$ by the maximum component $\max(\{x_i\})$: obviously the sum of squares will not decrease. Factoring out the common term $\max(\{x_i\})$ yields the right hand side of the inequality.)

Because the $x_i$ are not all $0$ (that would leave $\sigma_x/\bar{x}$ undefined), division by the square of their sum is valid and gives the equivalent inequality

$$\frac{x_1^2+x_2^2+\ldots+x_n^2}{(x_1+x_2+\ldots+x_n)^2} \le \frac{\max(\{x_i\})}{x_1+x_2+\ldots+x_n}.$$

Because the denominator cannot be less than the numerator (which itself is just one of the terms in the denominator), the right hand side is dominated by the value $1$, which is achieved only when all but one of the $x_i$ equal $0$. Whence

$$\frac{\sigma_x}{\bar{x}} \le f^{-1}\left(1\right) = \sqrt{\left(1 \times (n - 1)\right)\frac{n}{n-1}}=\sqrt{n}.$$

Alternative approach

Because the $x_i$ are nonnegative and cannot sum to $0$, the values $p(i) = x_i/(x_1+x_2+\ldots+x_n)$ determine a probability distribution $F$ on $\{1,2,\ldots,n\}$. Writing $s$ for the sum of the $x_i$, we recognize

$$\eqalign{ \frac{x_1^2+x_2^2+\ldots+x_n^2}{(x_1+x_2+\ldots+x_n)^2} &= \frac{x_1^2+x_2^2+\ldots+x_n^2}{s^2} \\ &= \left(\frac{x_1}{s}\right)\left(\frac{x_1}{s}\right)+\left(\frac{x_2}{s}\right)\left(\frac{x_2}{s}\right) + \ldots + \left(\frac{x_n}{s}\right)\left(\frac{x_n}{s}\right)\\ &= p_1 p_1 + p_2 p_2 + \ldots + p_n p_n\\ &= \mathbb{E}_F[p]. }$$

The axiomatic fact that no probability can exceed $1$ implies this expectation cannot exceed $1$, either, but it's easy to make it equal to $1$ by setting all but one of the $p_i$ equal to $0$ and therefore exactly one of the $x_i$ is nonzero. Compute the coefficient of variation as in the last line of the geometric solution above.

$\endgroup$
  • $\begingroup$ Thanks for a detailed answer from which I have learned a lot! I assume that the difference between the $\sqrt{n}$ in your answer and the $\sqrt{n-1}$ that I obtained (and Henry confirmed) is due to the fact that you are using $$\sigma_x = \sqrt{\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar{x})^2}$$ as the definition of $\sigma_x$ while I used $$\sigma_x = \sqrt{\frac{1}{n}\sum_{i=1}^n(x_i-\bar{x})^2}?$$ $\endgroup$ – Dilip Sarwate Nov 20 '11 at 17:46
  • 1
    $\begingroup$ Yes Dilip, that's right. Sorry about the discrepancy with the question; I should have checked first and I should have defined $\sigma_x$ (which I intended to do but forgot). $\endgroup$ – whuber Nov 20 '11 at 17:50
10
$\begingroup$

Some references, as small candles on the cakes of others:

Katsnelson and Kotz (1957) proved that so long as all $x_i \ge 0$, then the coefficient of variation cannot exceed $\sqrt{n − 1}$. This result was mentioned earlier by Longley (1952). Cramér (1946, p.357) proved a less sharp result, and Kirby (1974) proved a less general result.

Cramér, H. 1946. Mathematical methods of statistics. Princeton, NJ: Princeton University Press.

Katsnelson, J., and S. Kotz. 1957. On the upper limits of some measures of variability. Archiv für Meteorologie, Geophysik und Bioklimatologie, Series B 8: 103–107.

Kirby, W. 1974. Algebraic boundedness of sample statistics. Water Resources Research 10: 220–222.

Longley, R. W. 1952. Measures of the variability of precipitation. Monthly Weather Review 80: 111–117.

I came across these papers in working on

Cox, N.J. 2010. The limits of sample skewness and kurtosis. Stata Journal 10: 482-495.

which discusses broadly similar bounds on moment-based skewness and kurtosis.

$\endgroup$
8
$\begingroup$

With two numbers $x_i \ge x_j$, some $\delta \gt 0$ and any $\mu$:

$$(x_i+\delta - \mu)^2 + (x_j - \delta - \mu)^2 - (x_i - \mu)^2 - (x_j - \mu)^2 = 2\delta(x_i - x_j +\delta) \gt 0.$$

Applying this to $n$ non-negative datapoints, this means that unless all but one of the $n$ numbers are zero and so cannot be reduced further, it is possible to increase the variance and standard deviation by widening the gap between any pair of the data points while retaining the same mean, thus increasing the coefficient of variation. So the maximum coefficient of variation for the data set is as you suggest: $\sqrt{n-1}$.

$c$ should not affect the result as $\frac{\sigma_x}{\bar{x}}$ does not change if all the values are multiplied by any positive constant $k$ (as I said in my comment).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.