Here are three equations for some basic parameters.

Average of deviations:

$$\lim_{N\to \infty} \bar{d} = \lim_{N\to \infty} \left[\frac{1}{N}\sum(x_i-\mu) \right]= \lim_{N\to \infty} \left(\frac{1}{N}\sum x_i\right) - \mu = 0$$

Variance: $$\sigma^2 \equiv \lim_{N\to \infty} \left[\frac{1}{N}\sum(x_i-\mu)^2 \right]= \lim_{N\to \infty} \left(\frac{1}{N}\sum x_i^2\right) - \mu^2$$

Variance of a continuous distribution: $$\sigma^2 = \int_{-\infty}^\infty(x-\mu)^2p(x)dx = \int_{-\infty}^\infty x^2p(x)dx - \mu^2$$

In all of the above cases, the mean $\mu$ has been taken out of the brackets $()$ in what seems to be a violation of algebra laws. What is the justification for that?

Intuitively I think that because $\mu$ is a constant, it is unrelated to all of the other things that are going on in there, but I can't fully understand it yet.


You maybe need to consider some intermediate steps to understand what is going on, as these are more or less just the consequences algebraic equalities: Let us use ${\bar{x}_N}$ for the sample mean of $x_1,x_2,\ldots,x_N$. As @MartijnWeterings mentioned, under the right conditions ${\bar{x}_N} \to \mu$.

In the first case we have

$$ \begin{align}\frac{1}{N} \sum_i (x_i - \mu) &= \frac{1}{N}\left[ \left(\sum_i x_i \right) - \left(\sum_i \mu \right)\right] \\ &= \underbrace{\frac{1}{N} \left(\sum_i x_i \right)}_{={\bar{x}_N}} - \underbrace{\frac{1}{N}\underbrace{\left(\sum_i \mu \right)}_{=N\mu} }_{=\mu} \\ &= {\bar{x}_N} - \mu \to 0 \end{align}$$

In the first equation I use that the sum commutes, in the second equation I use the distributive law.

You can expand the other two cases with similar intermediate steps, so it is not just a matter of "taking $\mu$ out of the brackets".

In the second case consider following steps:

$$\begin{align} \frac{1}{N} \sum (x_i - \mu)^2 &= \frac{1}{N} \sum (x_i^2 - 2x_i\mu + \mu^2) \\ &= \left(\frac1N\sum x_i^2 \right) - \left(\frac1N\sum 2x_i \mu\right) + \underbrace{\left(\frac1N\sum \mu^2 \right)}_{=\mu^2} \\ &= \left(\frac1N\sum x_i^2 \right) - 2\mu\underbrace{\left(\frac1N\sum x_i \right)}_{=\bar{x}_N} + \underbrace{\left(\frac1N\sum \mu^2 \right)}_{=\mu^2} \\ &\to \lim_{N\to\infty} \left(\frac1N\sum x_i^2 \right) - \mu^2 \end{align}$$

The third case is just the continuous analogue of the second one and you can use the exact same steps.

  • $\begingroup$ There is a slight complication that has slipped. You have more exactly $\frac{1}{N} \left( \sum_i x_i \right) = \bar{x}$, ie. the sample mean. It is only equal to the popuation mean $\mu$ in the limit $N \to \infty$ (and if the right conditions for the law of large numbers are met). $\endgroup$ Apr 3 '19 at 13:35
  • $\begingroup$ @MartijnWeterings Thanks for clarifying I have mistaken $\mu$ for $\bar x$ indeed. $\endgroup$
    – flawr
    Apr 3 '19 at 13:50
  • $\begingroup$ It is also a bit confusing due to the third case, which doesn't seem to be about the sample but uses the population distribution density $p(x)$. And "average of deviations" is a bit ambiguous (although the formula, with terms like $x_i$, looks like a sample to me). Is it the average of residuals, the average of error-terms, etc. ? $\endgroup$ Apr 3 '19 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.