Suppose I observe $n$ iid realizations of two random variables $X$ and $Y$, denoted respectively $x_i$ and $y_i$. Observations can be groupped into two subsamples, with $n_1$ and $n_2$ observations. I want to decompose the sample covariance $\widehat{\sigma_{XY}}$ by the contribution of each group of observations plus possibly a residual term. Here is what I have:

\begin{eqnarray} \widehat{\sigma_{XY}} & = & \frac{1}{n} \sum^n x_i y_i -n \overline{x} \overline{y} \\ & = & \frac{n_1}{n} \frac{1}{n_1} \sum^{n_1} x_i y_i + \frac{n_2}{n} \frac{1}{n_2} \sum^{n_2} x_i y_i \\ &&- n\left(\frac{n_1}{n} \frac{1}{n_1} \sum^{n_1} x_i + \frac{n_2}{n} \frac{1}{n_2} \sum^{n_2} x_i\right) \left(\frac{n_1}{n} \frac{1}{n_1} \sum^{n_1} y_i + \frac{n_2}{n} \frac{1}{n_2} \sum^{n_2} y_i\right) \\ & =& \frac{n_1}{n} \frac{1}{n_1} \left(\sum^{n_1} x_i y_i -n_1 \overline{x}_1 \overline{y}_1\right) + \frac{n_2}{n} \frac{1}{n_2} \left(\sum^{n_2} x_i y_i -n_2 \overline{x}_2 \overline{y}_2\right) - \frac{n_1n_2}{n}\left(\overline{x}_1\overline{y}_2 + \overline{x}_2 \overline{y}_1\right)\\ & = & \frac{n_1}{n} \widehat{\sigma_{XY, 1}} + \frac{n_2}{n} \widehat{\sigma_{XY, 2}} - \frac{n_1n_2}{n}\left(\overline{x}_1\overline{y}_2 + \overline{x}_2 \overline{y}_1\right) \end{eqnarray}

That is, the overall sample covariance is a weighted average of the sample covariances within each group minus a residual term.

However, in a numerical example this decomposition does not hold as I do not obtain the overall sample covariance on the LHS. Especially the residual term seems too large.

I would appreciate if somebody can double-check whether there is a mistake in the above.

Many thanks


1 Answer 1


Covariance of $X$ and $Y$ is $\mathrm{Cov}(X,Y) = \mathrm{E}(XY) - \mathrm{E}(X)\mathrm{E}(Y)$, so the estimate $\widehat{\sigma_{XY}} = \sigma$

$\sigma = \frac{1}{n} \left(\sum^n x_i y_i - n \overline{x} \overline{y}\right) = -\overline{x} \overline{y} + \frac{1}{n} \sum^n x_i y_i$

NB: factor $\frac{1}{n}$ does not multiply $\overline{x} \overline{y}$.

It follows that \begin{eqnarray} \sigma & = & \frac{n_1}{n} \sigma_1 + \frac{n_2}{n} \sigma_2 - \frac{n_1 n_2}{n^2}\left(\overline{x}_1\overline{y}_2 + \overline{x}_2 \overline{y}_1 - \overline{x}_1 \overline{y}_1 - \overline{x}_2 \overline{y}_2 \right) \end{eqnarray}

  • $\begingroup$ Many thanks, Alexander. There is actually a typo in the fourth line: it must read $\frac{n_1}{n} \frac{1}{n_1} \left(\sum^{n_1} x_i y_i -n_1 \overline{x}_1 \overline{y}_1\right) + \frac{n_2}{n} \frac{1}{n_2} \left(\sum^{n_2} x_i y_i -n_2 \overline{x}_2 \overline{y}_2\right)$ instead of $\frac{n_1}{n} \frac{1}{n_1} \left(\sum^{n_1} x_i y_i -n \overline{x}_1 \overline{y}_1\right) + \frac{n_2}{n} \frac{1}{n_2} \left(\sum^{n_2} x_i y_i -n \overline{x}_2 \overline{y}_2\right) $. Also, notice that I am factoring $\frac{n_1}{n}$ and $\frac{n_2}{n}$ which is why you do not see quadratic terms. $\endgroup$
    – LuckyLuke
    Aug 25, 2017 at 7:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.