# Limiting joint distribution of estimators; Functional Statistics; Influence curves;

Let $X_1,...,X_n$ iid r.v. with distribution F, with mean $\mu$ and median $\theta$.Assume that $Var(X_i)=\sigma^2$ and $F'(\theta)>0$. If $\hat{\mu}_n$ is the sample mean, and $\hat{\theta}_n$ the sample median, then using the influence curve/functions, show that

$\left( \begin{array}{ccc} \sqrt{n} (\hat{\mu}_n-\mu) \\ \sqrt{n} (\hat{\theta}_n-\theta) \end{array} \right)\xrightarrow[d]{}N(\mathbf{0},C)$, with $C$ a variance-covariance matrix.

Well, we know that $\sqrt{n} (\hat{\theta}_n-\theta)\xrightarrow[d]{}N(0,(2F'(\theta))^{-2})$ (Here I used some properties of influence curves of parameters that give 'linear approximation' similar to delta method) and also that

$\sqrt{n} (\hat{\mu}_n-\mu)\xrightarrow[d]{}N(0,\sigma^2)$

So, I was thinking that I could immediately conclude what the exercise asks, remaining only the entries of C matrix to be explicitly determined. Can I do this?

Thus, $\mathbf{C}=\left( \begin{array}{ccc} \sigma^2 & \lim Cov(\hat{\mu}_n,\hat{\theta}_n) \\ \lim Cov(\hat{\theta}_n,\hat{\mu}_n) & (2F'(\theta))^{-2} \end{array} \right)$

Next, using properties/definition of influence curves (IC), and one other assumption,

$\sqrt{n} (\hat{\mu}_n-\mu)=\frac{1}{\sqrt{n}}\sum IC_{\mu}(X_i,F)+R_1=\frac{1}{\sqrt{n}}\sum (X_i-\mu)+R_1$

$\sqrt{n} (\hat{\theta}_n-\theta)=\frac{1}{\sqrt{n}}\sum IC_{\theta}(X_i,F)+R_2=\frac{1}{\sqrt{n}}\sum \frac{sign(X_i-\theta)}{2F'(\theta)}+R_2$

With $R_1,R_2\rightarrow_p0$.

Using the RHS expressions I've tried computing the Covariance in the matrix C, and obtained $\frac{\mu-2E(X_iI(X_i<\theta))}{2F'(\theta)}$

Is this deduction correct?

Any help would be appreciated.

• In any case you could use this math.ucla.edu/~tom/papers/unpublished.html – Alecos Papadopoulos Jul 30 '14 at 2:54
• @AlecosPapadopoulos this paper indeed helped me a lot. I'll be editing some things later on. I'm not sure of 1 thing in the paper, but it seems that when he adapts the main theorem to the mean and median, it seems that the expression he obtains is valid only when mean=median, otherwise he would get $E((X-\mu)sign(X-\theta))/(2f(\theta))$. At least that is what I get... – An old man in the sea. Jul 30 '14 at 10:19

These results hold for absolutely continuous random variables (see this question for the sample median behavior for discrete random variables). $F$ will be the cumulative distribution function, $f$ the probability density function, $\mu$ the mean. A hat will indicate sample quantities, and a bar sample means. Denote the quantile associated with probability $p$ by $\xi_p$, so that the median is $\xi_{1/2}\equiv \theta$.

The Bahadur representation of a sample quantile is

$$\hat \xi_p = \xi_p +\frac{1-\hat F_n(\xi_p)-(1-p)}{f(\xi_p)} +R_{n,p},\;\; R_{n,p} =o(1/\sqrt n)$$

$$\Rightarrow \sqrt n(\hat \xi_p - \xi_p) = \sqrt n\frac{1-\hat F_n(\xi_p)-(1-p)}{f(\xi_p)} +\sqrt nR_{n,p} \tag{1}$$

Define the indicator function $I_i\equiv I(X_i>\xi_p)$ (this is the complementary indicator function of the one the OP uses). We have that

$$E(I_i)= P(X_i>\xi_p) = 1-p,\;\; \operatorname{Var}(I_i)=p(1-p)\tag {2}$$

and also

$$\bar S_I \equiv \frac 1n\sum_{i=1}^nI_i = 1-\hat F_n(\xi_p), \;\; E(\bar S_I) = 1-p, \operatorname{Var}(\bar S_I) = p(1-p)/n \tag {3}$$

Using $(2)$ and $(3)$ we can write $(1)$ as

$$\sqrt n(\hat \xi_p - \xi_p) = \sqrt n\frac{1}{f(\xi_p)}\left(\bar S_I-E(\bar S_I)\right) +\sqrt nR_{n,p} \tag{4}$$

The quantity in the big parenthesis is a sample mean centered, the term in front is a constant, the last term asymptotically vanishes, so we are led to the CLT for sample medians. But this is not what we are after.

Consider now the bivariate random vector $(X_i, I_i)'$. The covariance between the two components is

$$\operatorname{Cov}(X_i,I_i) = E(X_iI_i) - \mu(1-p) \tag{5}$$

By an application of a multivariate CLT, $(5)$ will also be the covariance of the asymptotic distribution of the bivariate vector $\sqrt n\big(\bar X -\mu,\, \bar S_I-E(\bar S_I)\big)'$.

We are interested in the (asymptotic) quantity

$$\operatorname{Cov}\Big(\sqrt n(\bar X -\mu),\sqrt n(\hat \xi_p - \xi_p)\Big) = \operatorname{Cov}\Big(\sqrt n(\bar X -\mu),\sqrt n\frac{1}{f(\xi_p)}\left(\bar S_I-E(\bar S_I)\right)\Big)$$

$$=\frac{1}{f(\xi_p)}\operatorname{Cov}\Big(\sqrt n(\bar X -\mu),\sqrt n\left(\bar S_I-E(\bar S_I)\right)\Big)$$

$$= \frac{1}{f(\xi_p)}\left(E(X_iI_i) - \mu(1-p)\right) \tag {6}$$

For the sample median, $(6)$ becomes

$$\operatorname{Cov}\Big(\sqrt n(\bar X -\mu),\sqrt n(\hat \theta - \theta)\Big) = \frac{1}{2f(\theta)}\left(2\int_{\theta}^{\infty}xf(x)dx - \mu\right) \tag{7}$$

Eq. $(7)$ is the same as what the OP found (remember, we have defined the indicator function complementarily). Now, Ferguson's paper gives this quantity as $E(|X-\theta|)/2f(\theta)$. We have to verify that this is the same as $(7)$.

$$E(|X-\theta|) = \int_{-\infty}^{\infty}|x-\theta|f(x)dx = \int_{-\infty}^{\theta}(\theta-x)f(x)dx+\int_{\theta}^{\infty}(x-\theta)f(x)dx$$

$$=\theta\int_{-\infty}^{\theta}f(x)dx - \int_{-\infty}^{\theta}xf(x)dx + \int_{\theta}^{\infty}xf(x)dx - \theta\int_{\theta}^{\infty}f(x)dx$$

Bring the terms not involving $x$ together and add and subtract $\int_{\theta}^{\infty}xf(x)dx$ to obtain

$$E(|X-\theta|) =\theta F(\theta) -\theta[1-F(\theta)] + \\ +2\int_{\theta}^{\infty}xf(x)dx - \int_{-\infty}^{\theta}xf(x)dx - \int_{\theta}^{\infty}xf(x)dx$$

$$= \theta/2 - \theta/2 +2\int_{\theta}^{\infty}xf(x)dx -\int_{-\infty}^{\infty}xf(x)dx$$

$$\Rightarrow E(|X-\theta|) = 2\int_{\theta}^{\infty}xf(x)dx -\mu$$

So Ferguson's expression for the asymptotic covariance between sample mean and sample median is correct, without needing to impose $\mu = \theta$.

So, I took this exercise from Keith Knight Mathematical Statistics, page 229.

Some things of the OPost are wrong. What we need is the multivariate CLT.

This theorem ensures us that $\sqrt{n} \left( \begin{array}{ccc} \hat{\mu}_n-\mu \\ \hat{\theta}_n-\theta \end{array} \right)\xrightarrow[d]{}N(\mathbf{0},C)$. We'll see later on the elements of $\mathbf{C}$.

First notice that $\sqrt{n} \left( \begin{array}{ccc} \hat{\mu}_n-\mu \\ \hat{\theta}_n-\theta \end{array}\right)=\sqrt{n} \left( \begin{array}{ccc} \frac{1}{n}\sum IC_{\mu}(X_i,F)+R_1 \\ \frac{1}{n}\sum IC_{\theta}(X_i,F)+R_2 \end{array}\right)$

will have the same limiting distribution as

$\sqrt{n} \left( \begin{array}{ccc} \frac{1}{n}\sum IC_{\mu}(X_i,F) \\ \frac{1}{n}\sum IC_{\theta}(X_i,F)\end{array}\right)$ since $R_1, R_2\rightarrow_p 0$.

Because $X_i$ 's are independent, $IC_{\mu}(X_i,F)=X_i-\mu$ are independent among themselves; and so are $IC_{\theta}(X_i,F)=\frac{sign(X_i-\theta)}{2f(\theta)}$.

Note that $E\left(\begin{array}{} IC_{\mu}(X_i,F)\\ IC_{\theta}(X_i,F)\end{array}\right)= \left(\begin{array}{} 0\\ (2f(\theta))^{-1}(P(X_i>\theta)-P(X_i<\theta))\end{array}\right)= \left(\begin{array}{} 0\\ 0 \end{array}\right)$

Now we can state the elements of the $\mathbf{C}$ variance-covariance matrix in a much simplified manner. We have $\mathbf{C}= \left( \begin{array}{ccc} Var(IC_{\mu}(X_i,F)) & Cov(IC_{\mu}(X_i,F),IC_{\theta}(X_i,F)) \\ Cov(IC_{\mu}(X_i,F),IC_{\theta}(X_i,F)) & Var(IC_{\theta}(X_i,F)) \end{array} \right) =\left( \begin{array}{ccc} \sigma^2 & E((X_i-\mu)\frac{sign(X_i-\theta)}{2f(\theta)}) \\ E((X_i-\mu)\frac{sign(X_i-\theta)}{2f(\theta)}) & (2f(\theta))^{-2} \end{array} \right)$

To confirm that this deduction is correct, the second part of the exercise (4.11) gives a density function, and the final expression of $\mathbf{C}$. When calculating $E((X_i-\mu)sign(X_i-\theta)$, we'll need to use some coordinate changes to reach the result of $E((X_i-\mu)sign(X_i-\theta))=\frac{\Gamma(2/p)}{\Gamma(1/p)}$, which will imply that $E((X_i-\mu)\frac{sign(X_i-\theta)}{2f(\theta)})=\frac{\Gamma(2/p)}{2f(\theta)\Gamma(1/p)}=\frac{\Gamma(2/p)}{2\frac{p}{2\Gamma(1/p)}\Gamma(1/p)}=\Gamma(2/p)/p$

• @AlecosPapadopoulos what do you think of this answer? ;) – An old man in the sea. Jul 30 '14 at 20:48
• You will have to correct a few square-roots, but apart from that, there's nothing wrong with it, since : $$E\left[(X_i-\mu)\frac{sign\{X_i-\theta\}}{2f(\theta)}\right] = \frac 1{2f(\theta)}E\left[(X_i-\mu)sign\{X_i-\theta\}\right] = \frac 1{2f(\theta)}E[(X_i\cdot sign\{X_i-\theta\}] =$$ $$=\frac 1{2f(\theta)}\cdot \Big(E(X_i; X_i> \theta) - E(X_i; X_i< \theta)\Big)$$ $$=\frac 1{2f(\theta)}\cdot \Big(\int_{\theta}^{\infty}xf(x)dx - \int_{-\infty}^{\theta}xf(x)dx\Big)$$ $$=\frac 1{2f(\theta)}\cdot \Big(2\int_{\theta}^{\infty}xf(x)dx - \mu\Big)$$ – Alecos Papadopoulos Jul 31 '14 at 0:45
• @AlecosPapadopoulos, Many Thanks for your answer and comment. – An old man in the sea. Jul 31 '14 at 8:09