Population covariance, are these two formulas equivalent?

Question

For the population covariance, you can write it as:

$$\sigma_{x,y} = \frac{\sum_i(x_i-\bar{x})(y_i-\bar{y})}{N},$$

where $N$ is the population size. I think you can also write it in terms of expected values as:

$$\sigma_{x,y} = \mathbb{E}((X-\mu_X)(Y-\mu_Y)).$$

Are these two formulations actually equivalent? If you had the total $N$, and plugged in equation (1), does that converge to the true expectation value?

I am just confused why the former equation is used to denote the population value if this is so, is it just a intuitive way to formulate the population covariance?

Answer 1

The problem is unclear as presently written, because you are comparing a set of population values $(x_i,y_i)$ with a pair of random variables $(X,Y)$ where the latter have not been clearly defined. However, suppose we let $I \sim \text{U} \{ 1,...,N \}$ denote a random value from the population and we then define the values:

$$X \equiv x_I \quad \quad \quad Y \equiv y_I.$$

This definition means that $(X,Y)$ is now a random pair of points from the finite population. With a bit of work, we can now show the equivalence between these two formulae. As a preliminary result we can use the law of iterated expectation to get:

$$\begin{align} \mu_X \equiv \mathbb{E}(X) &= \mathbb{E}(\mathbb{E}(X|I)) \\[12pt] &= \mathbb{E}(X_I) \\[6pt] &= \frac{1}{N} \sum_{i=1}^N x_i \\[6pt] &= \bar{x}_N, \\[18pt] \mu_Y \equiv \mathbb{E}(Y) &= \mathbb{E}(\mathbb{E}(Y|I)) \\[12pt] &= \mathbb{E}(Y_I) \\[6pt] &= \frac{1}{N} \sum_{i=1}^N y_i \\[6pt] &= \bar{y}_N. \\[12pt] \end{align}$$

Another application of the law of iterated expectation then gives:

$$\begin{align} \mathbb{E}((X-\mu_X)(Y-\mu_Y)) &= \mathbb{E}(\mathbb{E}((X-\mu_X)(Y-\mu_Y)|I)) \\[16pt] &= \mathbb{E}((X_I-\mu_X)(Y_I-\mu_Y)) \\[8pt] &= \frac{1}{N} \sum_{i=1}^N (x_i-\mu_X)(y_i-\mu_Y) \\[6pt] &= \frac{1}{N} \sum_{i=1}^N (x_i-\bar{x}_N)(y_i-\bar{y}_N). \\[6pt] \end{align}$$

As you can see, these two formulae are equivalent if you define the pair $(X,Y)$ to be a random pair in the population. You needn't worry about "convergence" here, since the two formulae are equivalent for all $1 \leqslant N < \infty$.

This equality is part of the "design-based" approach to sampling theory, where we implicitly condition on the empirical distribution of the population, and take our random variable version of a data value to be a random value from this distribution. Note that things are different in the "model-based" approach, where we would usually define a random observation as coming from a higher-level "superpopulation" (infinite population) in which the finite population is embedded. In the latter case the moment quantities for a random observation usually refer to moments of the superpopulation distribution, rather than the finite population empirical distribution.

Answer 2

The best way to understand the difference between two formulae is to notice that the first one is in terms of observations, and the second one is in terms of outcomes. They will get you the same answer.

In the first equation the index $i$ refers to an observation in your population.

The second equation can be spelled out as follows: $$\sigma_{xy}=\sum_{jk} Pr(x_j,y_k)(x_j-\mu_x)(y_k-\mu_y)$$ Where $Pr(x,y)$ is the joint probability mass function (PMF) of variables $x,y$, and $j,k$ is the index of all possible outcomes of $x,y$. Outcomes, not observations or population members.

Example

Your population is $x,y=(0,1)(0,1)(1,1)(1,0)$

$\bar x=1/2$ and $\bar y=3/4$

PMF is $f(0,1) = 1/2\\ f(0,0) = 0\\ f(1,0) = 1/4\\ f(1,1) = 1/4$

Marginals: $$f_x(0)=1/2\\f_x(1)=1/2$$ $$f_y(0)=1/4\\f_y(1)=3/4$$

$\mu_x=1/2,\mu_y=3/4$

Plug the equations and get the same results for variances.

Answer 3

As whuber said in his comment, the two equations are equivalent, as long as the considered population is finite ($N$ being its size).

If a population is not finite, we cannot compute its parameters without some estimation error, but if the population is finite, we can compute any statistic we may want from the totality of data, without any uncertainty (if you do have all data), because expected values are just population averages.