Limiting joint distribution of estimators; Functional Statistics; Influence curves; Let $X_1,...,X_n$ iid r.v. with distribution F, with mean $\mu$ and median $\theta$.Assume that $Var(X_i)=\sigma^2$ and $F'(\theta)>0$. If $\hat{\mu}_n$ is the sample mean, and $\hat{\theta}_n$ the sample median, then using the influence curve/functions, show that 
$\left( \begin{array}{ccc}
\sqrt{n} (\hat{\mu}_n-\mu) \\
\sqrt{n} (\hat{\theta}_n-\theta) \end{array} \right)\xrightarrow[d]{}N(\mathbf{0},C)$, with $C$ a variance-covariance matrix.
Well, we know that
$\sqrt{n} (\hat{\theta}_n-\theta)\xrightarrow[d]{}N(0,(2F'(\theta))^{-2})$ (Here I used some properties of influence curves of parameters that give 'linear approximation' similar to delta method) and also that
$\sqrt{n} (\hat{\mu}_n-\mu)\xrightarrow[d]{}N(0,\sigma^2) $ 
So, I was thinking that I could immediately conclude what the exercise asks, remaining only the entries of C matrix to be explicitly determined. Can I do this?
Thus,
$\mathbf{C}=\left( \begin{array}{ccc}
\sigma^2 & \lim Cov(\hat{\mu}_n,\hat{\theta}_n)  \\
\lim Cov(\hat{\theta}_n,\hat{\mu}_n) & (2F'(\theta))^{-2} \end{array} \right)$
Next, using properties/definition of influence curves (IC), and one other assumption,
$\sqrt{n} (\hat{\mu}_n-\mu)=\frac{1}{\sqrt{n}}\sum IC_{\mu}(X_i,F)+R_1=\frac{1}{\sqrt{n}}\sum (X_i-\mu)+R_1$
$\sqrt{n} (\hat{\theta}_n-\theta)=\frac{1}{\sqrt{n}}\sum IC_{\theta}(X_i,F)+R_2=\frac{1}{\sqrt{n}}\sum \frac{sign(X_i-\theta)}{2F'(\theta)}+R_2$
With $R_1,R_2\rightarrow_p0$.
Using the RHS expressions I've tried computing the Covariance in the matrix C, and obtained $\frac{\mu-2E(X_iI(X_i<\theta))}{2F'(\theta)}$
Is this deduction correct?
Any help would be appreciated.
 A: These results hold for absolutely continuous random variables (see this question for the sample median behavior for discrete random variables). $F$ will be the cumulative distribution function, $f$ the probability density function, $\mu$ the mean. A hat will indicate sample quantities, and a bar sample means. Denote the quantile associated with probability $p$ by $\xi_p$, so that the median is $\xi_{1/2}\equiv \theta$.  
The Bahadur representation of a sample quantile is
$$\hat \xi_p = \xi_p +\frac{1-\hat F_n(\xi_p)-(1-p)}{f(\xi_p)} +R_{n,p},\;\; R_{n,p} =o(1/\sqrt n)$$
$$\Rightarrow \sqrt n(\hat \xi_p - \xi_p) = \sqrt n\frac{1-\hat F_n(\xi_p)-(1-p)}{f(\xi_p)} +\sqrt nR_{n,p} \tag{1}$$
Define the indicator function $I_i\equiv I(X_i>\xi_p)$ (this is the complementary indicator function of the one the OP uses). We have that
$$E(I_i)= P(X_i>\xi_p) = 1-p,\;\; \operatorname{Var}(I_i)=p(1-p)\tag {2}$$
and also
$$\bar S_I \equiv  \frac 1n\sum_{i=1}^nI_i = 1-\hat F_n(\xi_p), \;\; E(\bar S_I) = 1-p, \operatorname{Var}(\bar S_I) = p(1-p)/n \tag {3}$$
Using $(2)$ and $(3)$ we can write $(1)$ as
$$\sqrt n(\hat \xi_p - \xi_p) = \sqrt n\frac{1}{f(\xi_p)}\left(\bar S_I-E(\bar S_I)\right) +\sqrt nR_{n,p} \tag{4}$$
The quantity in the big parenthesis is a sample mean centered, the term in front is a constant, the last term asymptotically vanishes, so we are led to the CLT for sample medians.  But this is not what we are after. 
Consider now the bivariate random vector $(X_i, I_i)'$. The covariance between the two components is
$$\operatorname{Cov}(X_i,I_i) = E(X_iI_i) - \mu(1-p) \tag{5}$$
By an application of a multivariate CLT, $(5)$ will also be the covariance of the asymptotic distribution of the bivariate vector $\sqrt n\big(\bar X -\mu,\, \bar S_I-E(\bar S_I)\big)'$.
We are interested in the (asymptotic) quantity
$$\operatorname{Cov}\Big(\sqrt n(\bar X -\mu),\sqrt n(\hat \xi_p - \xi_p)\Big) = \operatorname{Cov}\Big(\sqrt n(\bar X -\mu),\sqrt n\frac{1}{f(\xi_p)}\left(\bar S_I-E(\bar S_I)\right)\Big) $$
$$=\frac{1}{f(\xi_p)}\operatorname{Cov}\Big(\sqrt n(\bar X -\mu),\sqrt n\left(\bar S_I-E(\bar S_I)\right)\Big)$$
$$= \frac{1}{f(\xi_p)}\left(E(X_iI_i) - \mu(1-p)\right) \tag {6}$$
For the sample median, $(6)$ becomes
$$\operatorname{Cov}\Big(\sqrt n(\bar X -\mu),\sqrt n(\hat \theta - \theta)\Big) = \frac{1}{2f(\theta)}\left(2\int_{\theta}^{\infty}xf(x)dx - \mu\right) \tag{7}$$
Eq. $(7)$ is the same as what the OP found (remember, we have defined the indicator function complementarily). Now, Ferguson's paper gives this quantity as $E(|X-\theta|)/2f(\theta)$. We have to verify that this is the same as $(7)$.  
$$E(|X-\theta|) = \int_{-\infty}^{\infty}|x-\theta|f(x)dx = \int_{-\infty}^{\theta}(\theta-x)f(x)dx+\int_{\theta}^{\infty}(x-\theta)f(x)dx$$
$$=\theta\int_{-\infty}^{\theta}f(x)dx - \int_{-\infty}^{\theta}xf(x)dx + \int_{\theta}^{\infty}xf(x)dx - \theta\int_{\theta}^{\infty}f(x)dx$$
Bring the terms not involving $x$ together and add and subtract $\int_{\theta}^{\infty}xf(x)dx$ to obtain
$$E(|X-\theta|) =\theta F(\theta) -\theta[1-F(\theta)] + \\
+2\int_{\theta}^{\infty}xf(x)dx - \int_{-\infty}^{\theta}xf(x)dx - \int_{\theta}^{\infty}xf(x)dx$$
$$= \theta/2 - \theta/2 +2\int_{\theta}^{\infty}xf(x)dx -\int_{-\infty}^{\infty}xf(x)dx$$
$$\Rightarrow E(|X-\theta|) = 2\int_{\theta}^{\infty}xf(x)dx -\mu$$
So Ferguson's expression for the asymptotic covariance between sample mean and sample median is correct, without needing to impose $\mu = \theta$.
A: So, I took this exercise from Keith Knight Mathematical Statistics, page 229. 
Some things of the OPost are wrong.
What we need is the multivariate CLT.
This theorem ensures us that $\sqrt{n} \left( \begin{array}{ccc}
 \hat{\mu}_n-\mu \\
 \hat{\theta}_n-\theta \end{array} \right)\xrightarrow[d]{}N(\mathbf{0},C)$. 
We'll see later on the elements of $\mathbf{C}$. 
First notice that 
 $\sqrt{n} \left( \begin{array}{ccc}
 \hat{\mu}_n-\mu \\
 \hat{\theta}_n-\theta \end{array}\right)=\sqrt{n} \left( \begin{array}{ccc}
 \frac{1}{n}\sum IC_{\mu}(X_i,F)+R_1 \\
 \frac{1}{n}\sum IC_{\theta}(X_i,F)+R_2 \end{array}\right)$
will have the same limiting distribution as
$\sqrt{n} \left( \begin{array}{ccc}
 \frac{1}{n}\sum IC_{\mu}(X_i,F) \\
 \frac{1}{n}\sum IC_{\theta}(X_i,F)\end{array}\right)$ since $R_1, R_2\rightarrow_p 0$.
Because $X_i$ 's are independent,
$IC_{\mu}(X_i,F)=X_i-\mu$ are independent among themselves; 
and so are $IC_{\theta}(X_i,F)=\frac{sign(X_i-\theta)}{2f(\theta)}$.
Note that $E\left(\begin{array}{}
IC_{\mu}(X_i,F)\\
IC_{\theta}(X_i,F)\end{array}\right)=
\left(\begin{array}{}
0\\
(2f(\theta))^{-1}(P(X_i>\theta)-P(X_i<\theta))\end{array}\right)=
\left(\begin{array}{}
0\\
0 \end{array}\right)$
Now we can state the elements of the $\mathbf{C}$ variance-covariance matrix in a much simplified manner.
We have $\mathbf{C}=
\left( \begin{array}{ccc}
Var(IC_{\mu}(X_i,F)) & Cov(IC_{\mu}(X_i,F),IC_{\theta}(X_i,F))  \\
Cov(IC_{\mu}(X_i,F),IC_{\theta}(X_i,F)) & Var(IC_{\theta}(X_i,F)) \end{array} \right)
=\left( \begin{array}{ccc}
\sigma^2 & E((X_i-\mu)\frac{sign(X_i-\theta)}{2f(\theta)})  \\
 E((X_i-\mu)\frac{sign(X_i-\theta)}{2f(\theta)}) & (2f(\theta))^{-2} \end{array} \right)$
To confirm that this deduction is correct, the second part of the exercise (4.11) gives a density function, and the final expression of $\mathbf{C}$.

When calculating $E((X_i-\mu)sign(X_i-\theta)$, we'll need to use some coordinate changes to reach the result of $E((X_i-\mu)sign(X_i-\theta))=\frac{\Gamma(2/p)}{\Gamma(1/p)}$, which will imply that $E((X_i-\mu)\frac{sign(X_i-\theta)}{2f(\theta)})=\frac{\Gamma(2/p)}{2f(\theta)\Gamma(1/p)}=\frac{\Gamma(2/p)}{2\frac{p}{2\Gamma(1/p)}\Gamma(1/p)}=\Gamma(2/p)/p$
