# Covariance vs variance of order statistics

I wonder if the following statement is true? Let $x_i$ be i.i.d. drawn from distribution $F$ with non-negative support and $f(0)>0$, i.e. the density at zero is positive. Let $X_{i:n}$ be the $i$-th order statistics of a sample of size $n$, where $X_{1:n}=\min\{x_1,...,x_n\}$. Is the following always true?

$Var(X_{1:n})> Cov(X_{1:n+1},X_{2:n+1})$

I've tried simulations with many different parent distributions and it seems to hold true, but I cannot analytically verify this.

• Let $F$ be Bernoulli$(1/2)$ and $n=1$. If my quick calculation is correct, I believe your statement then asserts $1/4 \gt 1/3$. Are you making some implicit assumptions about the size of $n$ or the nature of $F$? – whuber May 26 '16 at 20:51
• @whuber I did forget to say that F is assumed to have non-negative support, but I'm not sure if you are right with the Bernoulli case. When F is Bernoulli, $Var(X_{1:1})=Var(x)=1/4$. On the other hand, $$Cov(X_{1:n+1},X_{2:n+1}) = E[X_{1:n+1} X_{2:n+1}] - E[X_{1:n+1}] E[X_{2:n+1}] = 1/8$$. – user341296 May 26 '16 at 21:17
• Are you sure that covariance is not 1/16, with correlation = 1/3? – Mark L. Stone May 26 '16 at 23:14
• You're right--I did miscalculate the covariance. To make amends, I have posted a more careful reply. – whuber May 27 '16 at 15:08

It's not true.

Intuitively, we can make the variance of the minimum of $n$ iid samples small by using a distribution $F$ that has a high concentration of probability near its minimum. To make the covariance of the two lowest values high, we need to create some chance that both the lowest and second lowest will frequently be very much greater than the minimum.

This is a delicate balancing act, but it's possible. As an example I offer a mixture distribution (whose parameters were found through trial and error): $5\%$ of it is a standard Lognormal (giving a concentrated probability in the range from $0$ to $10$) and the remaining $95\%$ is a Gamma$(30)$ distribution, scaled by a factor of $2$, so that most of its values are between $40$ and $100$:

On the left of the figure is the bivariate density of $(X_{[1;n+1]}, X_{[2;n+1]})$. It was drawn by simulating $100,000$ pairs of values and plotting them as transparent gray dots. On the right is the histogram of $X$ (based on the same simulation).

Since I don't care to work out the integrals that give these values, I will offer results of a simulation. In each iteration $n+1=100$ iid values are drawn from this distribution, then $X_{[1;n+1]}$ and $X_{[2;n+1]}$ are found by sorting them. One of the $n+1$ values is then randomly removed and $X_{[1;n]}$ is found as their minimum. This is repeated $20,000$ times and the variance of $X_{[1;n]}$ as well as the covariance of $(X_{[1;n+1]}, X_{[2;n+1]})$ are estimated. The entire process is iterated one hundred times. The average of the variances is around $8.2$ and the average of the covariances around $9.2$, reversing the expected inequality. In all hundred instances the variance is less than the covariance, providing convincing evidence that these results are significant.

The conjecture in the question supposes that on average the dots will plot below the line. That clearly is not true in this case: the covariances (on the vertical axis) exceed the variances (on the horizontal axis).

Here is the R code that was used (modified to make the simulation smaller; it still produces significant results).

f <- function(n=100, n.sim=2e3) {
x1 <- exp(rnorm(n*n.sim))
x2 <- rgamma(n*n.sim, 30, scale=2)
p <- runif(n*n.sim) < 5/n
x <- apply(matrix(ifelse(p, x1, x2), nrow=n), 2, sort)
x.n <- apply(x, 2, function(y) min(sample(y, n-1)))
return(c(var(x.n), cov(x[1, ], x[2, ])))
}
set.seed(17)
x <- replicate(1e1, f())

plot(t(x), xlab="Var(X[1:n])", ylab="Cov(X[1:n+1],X[2:n+1])")
abline(c(0,1), col="Red", lwd=2)

• Thanks again for the help last week. I wonder if I can bother you a little more on this topic? I am curious if my conjecture is true when the PDF, $f$, is weakly decreasing? – user341296 May 31 '16 at 14:52
• There is no such PDF, because then the integral out to $-\infty$ would diverge. You might try unimodal instead if you want to block the idea I used here of capturing the minimum within an isolated mode of the distribution. – whuber May 31 '16 at 15:01
• I meant $F$ with non-negative support and $f$ that is weakly decreasing. I doubt unimodal would ensure the conjecture to be true. Thanks much :) – user341296 May 31 '16 at 17:02
• Hmmm... I think I convinced myself earlier that it must be true for unimodal distributions. A counterexample might be interesting. Incidentally, a weakly decreasing $f$ over non-negative support is a special case of a unimodal distribution. – whuber May 31 '16 at 18:22
• Good point. I don't have any intuition on why unimodal would work, except that it blocks your example. – user341296 May 31 '16 at 21:46