2
$\begingroup$

How does the addition of one unit affect the population variance of a finite population if everything else remains unchanged? What are the conditions such that the new unit leaves the variance unchanged (increases/decreases it)?

I was able to find the following paper regarding sample variances for changing finite populations: http://www.amstat.org/sections/srms/Proceedings/papers/1987_087.pdf. But I am asking specifically about population variances. Any help is appreciated.

$\endgroup$
  • $\begingroup$ Are you looking for another answer? What you have seems to cover it. $\endgroup$ – Dale M Sep 28 '14 at 23:41
  • $\begingroup$ My dissertation chair said my math didn't make any sense. I wanted to see what a stats expert would say. $\endgroup$ – Jon Page Sep 28 '14 at 23:46
  • $\begingroup$ The only difference between this and the corresponding sample calculation (which may be found in a number of places, including a few questions here) is the absence of Bessel's correction for the variance, so you have a simple double check on your algebra from that. You can also confirm your calculation by simply calculating it from scratch on small populations that differ by one element and showing that it does produce the correct answer, and help check the condition on when it increases or decreases. Such checks help increase the confidence that the algebraic calculation is fine. $\endgroup$ – Glen_b Sep 28 '14 at 23:56
  • $\begingroup$ Thanks Glen_b! I'll do that and post an answer following that advice. $\endgroup$ – Jon Page Sep 29 '14 at 0:07
2
$\begingroup$

I was unable to find the sample calculations that correspond to the specific problem here (as suggested by Glen_b), but I was able to confirm the following answer with numerical calculations in R at the bottom of this answer.

Let $N$ be the initial number of units in the population and $N + 1$ be the number of units in the population after the change. Denote the initial set of observations $X = \{x_1, \ldots, x_N\}$ (i.e., one observation corresponding to each population unit). Denote the set of observations after the change $Y = X \cup \{x_{N+1}\}$.

The mean of $X$ is

$\mu_X = \frac{\sum_{i=1}^N{x_i}}{N}$.

The mean of Y is

$\mu_Y = \frac{\sum_{i=1}^{N+1}{x_i}}{N+1} = \mu_X \frac{N}{N+1} + \frac{x_{N+1}}{N+1}$

Define $x_{N+1}$ as the original mean, $\mu_X$, plus some $\varepsilon$. Then, the mean of $Y$ is

$\mu_Y = \mu_X \frac{N}{N+1} + \frac{\mu_X + \varepsilon}{N+1} = \mu_X + \frac{\varepsilon}{N+1}$

The variance of $Y$ is

$\sigma^2_Y = \frac{\sum_{i=1}^{N+1} \left(x_i - \mu_Y \right)^2}{N+1} = \frac{\sum_{i=1}^{N+1} \left(x_i - \mu_X - \frac{\varepsilon}{N + 1} \right)^2}{N+1}$

$= \frac{\sum_{i=1}^{N} x_i^2 + \mu_X^2 + \frac{\varepsilon^2}{\left(N+1\right)^2} - 2x_i\mu_X - 2x_i\frac{\varepsilon}{N+1} + 2\mu_X\frac{\varepsilon}{N+1}}{N + 1}$

$\frac{\left(\mu_X + \varepsilon - \mu_X - \frac{\varepsilon}{N + 1}\right)}{N + 1} $

$ = \frac{N}{N+1}\sigma^2_X + \frac{N\varepsilon^2}{\left(N+1\right)^3} - \frac{2N\mu_X\varepsilon}{\left(N+1\right)^2} + \frac{2N\mu_X\varepsilon}{\left(N+1\right)^2} + \frac{N^2\varepsilon^2}{\left(N+1\right)^3}$

$ = \frac{N}{N+1} \sigma^2_X + \frac{N}{\left(N+1\right)^2}\varepsilon^2$

When $x_{N+1}$ is equal to $\mu_X$, the variance of $Y$ is

$\frac{N}{N+1}\sigma^2_X < \sigma^2_X $

Thus, when $\varepsilon$ is sufficiently small $\sigma^2_Y$ is less than $\sigma^2_X$. To determine how large $\varepsilon$ should be so that the variance of $Y$ is greater than the variance of $X$, I set the two variances equal.

$ \frac{N}{N+1} \sigma^2_X + \frac{N}{\left(N+1\right)^2}\varepsilon^2 = \sigma^2_X$

$ \frac{N}{\left(N+1\right)^2}\varepsilon^2 = \frac{1}{N+1} \sigma^2_X$

$ \varepsilon^2 = \frac{N+1}{N} \sigma^2_X $

$ \varepsilon = \pm \sigma_X \sqrt{\frac{N+1}{N}}$

Thus, adding a unit who's observation is within $\sqrt{\frac{N+1}{N}}$ standard deviations of the old mean will lead to a lower variance.


The following R script verifies the above conclusion:

N <- 10
X <- runif(N)
width <- sqrt((N+1)/N)
# on the boundary
var(c(X, mean(X) + width * sqrt(var(X)))) - var(X) == 0
# outside the boundary
var(c(X, mean(X) + width * sqrt(var(X)) + 1)) - var(X) > 0
# inside the boundary
var(c(X, mean(X))) - var(X) < 0
$\endgroup$
  • $\begingroup$ +1 A simpler point of departure is a standard algorithm for computing variances, which (in effect) you first had to rederive. $\endgroup$ – whuber Sep 29 '14 at 14:59
  • $\begingroup$ Thanks @whuber! I wish I had found that link before deriving the above results. $\endgroup$ – Jon Page Sep 29 '14 at 15:04
1
$\begingroup$

I have the feeling that you may be confusing a finite population with a sample from it. The fact that a population is finite, does not make it "equivalent" to a sample (which is always finite of course).

When we examine populations that are comprised of "identically and independently distributed" random variables, we have the habit of talking about the population mean or variance: strictly speaking this is wrong language -what we mean is the mean/variance of the common marginal distribution that each member of the population follows.

If this is your case, then, the expression $\frac{\sum_{i=1}^N{x_i}}{N}$ represents the sample mean of a specific sample from this finite population, namely, of a specific set of realizations of the random variables comprising this finite population. It is not the mean (expected value) of the population, i.e. it is not the common expected value of the i.i.d. variables comprising the population.

And all your calculations are consistent with examining the sample mean and the sample variance, not their population counterparts.

Viewed in this light, your calculations are correct and intuitive: if the additional observation is exactly equal to the sample mean of the previous observations included in $X$, then dispersion lessens and the sample variance of $Y$ will be smaller.

I guess it is evident that adding an i.i.d. random variable to an i.i.d. population does not change "the" population mean or variance (i.e. the moments of the common marginal distribution).

$\endgroup$
  • $\begingroup$ I have in mind the finite population. I am looking for changes to the population variance (not the sample variance) from adding a unit to the population. The population itself is changed and I am calculating population statistics. $\endgroup$ – Jon Page Sep 29 '14 at 1:01
  • $\begingroup$ How do you define "population mean" and "population variance"? Do you use the formulas in your question, or expected values? $\endgroup$ – Alecos Papadopoulos Sep 29 '14 at 1:02
  • $\begingroup$ I use the formulas in my question. $N$ is the size of the initial population. The observations themselves are considered fixed and not draws from a distribution. $\endgroup$ – Jon Page Sep 29 '14 at 1:05
  • $\begingroup$ Then it is a matter of semantics. What you call "population" is a) either a sample of realizations drawn from this population, a sample that happens to have the same size with the population (in which case my answer applies), or b) a series of deterministic numbers that are not drawn from a statistical distribution, in which case you are essentially out of statistical territory, and what you are calculating are descriptive metrics of series of numbers, not statistics, for which we just use the same names, but their interpretation is totally different. $\endgroup$ – Alecos Papadopoulos Sep 29 '14 at 1:17
  • $\begingroup$ I suppose I am calculating (as opposed to estimating) population parameters. Is there a different forum to which I should direct this question? $\endgroup$ – Jon Page Sep 29 '14 at 1:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.