# How does Huber compute the $\operatorname{var}(s_n)/E[s_n]^2$ and $\operatorname{var}(d_n)/E[d_n]^2$?

(N.B. I am cross posting this question from math stackexchange since after x days I have still not received any responses.)

How does Huber in book 'Robust statistical procedures' in chapter 1 compute the variance of certain statistical functions? He defines the mean square deviation to be $$s_n = \sqrt{\frac{1}{n} \sum \left(x_i - \bar{x} \right)^2}$$ and the mean absolute deviation as $$d_n = \frac{1}{n} \sum \| x_i - \bar{x} \|.$$ Then he goes on to obtain the asymptotic relative efficiency expression ARE($$\epsilon$$) as shown in the photo. (So, we should have at least for arguments in the ARE(), namely, ARE($$\epsilon;n,x_i,x$$),right? even if $$n \to \infty.$$ What does ARE() look like before we send $$n \to \infty$$? How on earth does one obtain the expression in the 2nd step?)

What is $$\operatorname{var}(s_n), E[s_n]$$? How to obtain these expressions?

Added after comment by @whuber about actual assumptions of $$x_i$$'s. Huber makes the following assumption. "We have a large, randomly mixed batch of "good' observations which are normal $$N(\mu, \sigma^2)$$ and "bad" ones which are normal $$N(\mu, 9\sigma^2)$$, i.e., all observations have the same mean but the errors of some are increased by a factor of 3. Now each single $$x_i$$ is a good one with probability $$(1-\epsilon)$$, a bad one with probability $$\epsilon$$, where $$\epsilon$$ is a small number.

Finally, why does he state that just two bad observations in 1000 suffice to offset the 12% advantage of the mean square error? Is is not 5 bad observations? See:

In the photo, values for $$\epsilon = 0.005$$ is 1.198 which is the (12%) I guess?

• Statistical functions have neither variances nor expectations. Functions of random variables like the $x_i$ are themselves random variables and therefore do have variances, etc. but they depend on the distributions of the $x_i.$ What does Huber assume about the distributions here?
– whuber
Commented Sep 24, 2023 at 16:37
• @peter Re your last line -- you need to look at the $\varepsilon=0.002$ row, which has ARE just slightly above $1$, offsetting all of the approximately $12\%$ advantage that $0.876$ from the first row has over an ARE of $1$. That is $0.2\%$ contamination of this kind "undoes" the advantage you have at $0$ contamination, taking the ARE back to $1$, or no efficiency advantage. Commented Sep 24, 2023 at 16:58
• @whuber many thanks for this comment. Yes, now it makes more sense, but still how do we compute it. Huber makes the following assumption. "We have a large, randomly mixed batch of "good' observations which are normal $N(\mu, \sigma^2)$ and "bad" ones which are normal $N(\mu, 9\sigma^2)$, i.e., all observations have the same mean but the errors of some are increased by a factor of 3. Now each single $x_i$ is a good one with probability $(1-\epsilon)$, a bad one with probability $\epsilon$, where $\epsilon$ is a small number. Commented Sep 25, 2023 at 2:46
• In effect, he rounds 0.876 to 0.88. That 0.88 means $s_n$ has a 12% advantage over $d_n$ when there's no contamination. This part uses information ONLY in the first row. He then looks down the remainder of the table in your post until the ARE reaches a value close to 1, to see approximately how much contamination (of the kind he looked at there) it would take to eliminate that 12% advantage, discovering that even extremely tiny amounts suffice. Commented Sep 25, 2023 at 3:05
• @Glen_b : impressive. many thanks now I understood exactly what is done!! Many thanks!!! Commented Sep 25, 2023 at 3:10

At present I'll begin by outlining how to do it for the numerator which will at least give you some sense of what's involved, and then I'll see about doing the denominator.

Let $$Y$$ be a random variable distributed as an $$\varepsilon$$-contaminated Gaussian (mixture of Gaussians with common center). As in the question, we'll take the contaminating part having $$9$$ times the variance of the main distribution. That is, $$f_Y(y)=(1-\varepsilon)f_X(x)+\varepsilon f_W(w)$$ with $$X\sim N(\mu,\sigma^2)$$ and $$W\sim N(\mu,9\sigma^2)$$.

Of course the estimators we're comparing will be based on $$n$$ such (i.i.d.) random variables, $$Y_i, i=1, 2, ..., n$$, but for the moment we just need to worry about the distribution of one of them.

Note that $$\frac{\text{Var}(s_n)}{E(s_n)^2}=\frac{E(s_n^2)}{E(s_n)^2}-1$$. Similarly for $$d_n$$. Presumably this will be where the "-1" terms in the numerator and denominator come from.

Now the usual sample variance is unbiased for population variance, irrespective of distribution (assuming the distribution has finite variance).

So $$E(s_n^2)=\frac{n-1}{n}E(s_{n-1}^2)=\frac{n-1}{n}\text{Var}(Y)$$

The expectation, expectation of the square and the variance of a mixture is straightforward, especially when they have a common mean.

I think we get $$\text{Var}(Y)=(1-\varepsilon)(\sigma^2+\mu^2)+\varepsilon (9\sigma^2+\mu^2)-\mu^2 = \sigma^2(1+8\varepsilon)$$.

So $$E(s_n^2) = \frac{n-1}{n}\sigma^2(1+8\varepsilon)$$.

For the expectation, by writing the integral, we get a weighted average of the two component distribution expectations, each of which comes down to an expectation of a scaled chi-distribution.

That is (if I didn't make an error)
$$E(s_n)= \sigma\sqrt{\frac{n-1}{n}}\sqrt{2}\sqrt{n-1}\frac{\ \Gamma( \tfrac{n + 1}{2} )\ }{\Gamma( \tfrac{n}{2})}[(1-\varepsilon)+3\varepsilon]$$.

So $$E(s_n)^2=(1+2\varepsilon)^2 \sigma^2 \frac{2(n-1)^2}{n}\left(\frac{\ \Gamma( \tfrac{n + 1}{2} )\ }{\Gamma( \tfrac{n}{2})}\right)^2$$

There's a $$(1+2\varepsilon)^2$$ in the denominator of the image in the question, which is perhaps encouraging, and of course since the result is asymptotic presumably the combined $$n$$-terms that don't just cancel with another one will go to $$1$$.

I'll try to come back and have a go at the ratio for the $$d_n$$ terms later and see whether it's all correct.

• Happy New Year, Glen! Nice contribution.
– whuber
Commented Jan 1 at 15:21