It seems to be that the minimum number of observations needed to calculate variance is 2. I can see the logic behind this because by logic, there can not be variance for a single point. But on the other hands, I don't think 2 points are really enough to calculate variance in general. This is because you need to observe more data to understand how much variance there truly is.

Ex: if it rains 2 days but does not rain 1 day, the variance might tell me that there is a a lot of variability in rain, but more data would really be needed to be certain. This is actually why you need something like min 25-30 data points to calculate the variance.

So if this is true, how can we have variance in repeated measure longitudinal regression models when there is usually just 3-4 obs max per person? If you combine everything into one dataset, I think you might be able to calculate variance for whole population .... but how can you calculate individual variance for each person individually when you only have 3-4 obs? Isn't it like the rain example?

  • 2
    $\begingroup$ You are confusing the ability to calculate an estimate with the accuracy of the estimator. You can calculate an estimate of variance with just two observations; the fact that it isn't a very accurate estimate at that sample size is irrelevant to that fact. Depending upon your purposes, and the availability of data, small sample sizes might be OK. $\endgroup$
    – jbowman
    Sep 28 at 18:13
  • $\begingroup$ Look at this page and its linked and related pages to see what's going on. You do need a large sample to get a precise estimate of the variance. How large to get a particular precision depends on the nature of the underlying distribution. But 2 is enough for a (very imprecise) estimate (assuming that the distribution in fact has a variance). $\endgroup$
    – EdM
    Sep 28 at 18:22
  • $\begingroup$ Could you explain what you mean by "calculating" variance? If it's a property of the sample, then evidently the variance of a sample of $1$ is always zero. If it's an estimator, then some variance estimators need only one random sample. For instance, when $X$ has a Poisson$(\lambda)$ distribution, $X$ estimates the variance (and it's pretty accurate when $\lambda$ is large). $\endgroup$
    – whuber
    Sep 28 at 18:47
  • $\begingroup$ thx everyone! I still dont think i understand unfort ... how it is possible to reliably calculate variance when you only have very small number of observations per each subject? $\endgroup$
    – stats_noob
    Sep 28 at 19:44
  • $\begingroup$ @jwolof The standard error of the variance estimate gets bigger the fewer data points you have - with very little data, you can't get a reliable estimate of the variance. You can always apply the variance formula with even with little data, but the point estimate becomes an increasingly unreliable estimate of the true variance the less data you have. $\endgroup$ Sep 29 at 16:00

1 Answer 1


One can argue that you only need $n=1$ to estimate variance, except that the variance is always estimated to be $0$ when you calculate $E[X^2] - E[X]^2 = X_1^2 - X_1^2 = 0$ and that this is a biased estimator.

"Mixed models" comprise a large, large class of models. In the simplest example: using random intercepts, it's relatively straightforward to understand why the estimates are reliable in moderate sample sizes (even with "3-4 obs max per person"). The mixed model uses a complicated estimation approach called profile likelihood which has the desirable property of shrinkage. For instance, the Neyman-Scott problem illustrates why maximum likelihood breaks down when the number of parameters is large. Stated briefly, Neyman-Scott concerns the problem of estimating the variance using maximum likelihood in a process of IID dyads having $$ [X_{i,1}, X_{i,2}] \sim \mathcal{N}(\mu_i, \sigma^2 \mathcal{I})$$

More background here but the asymptotic MLE for $\sigma^2 $ $\rightarrow$ 2$\sigma^2$ as $n \rightarrow \infty$. The mixed model approach would model the $\mu_i$ as a normal distribution and use the quantiles as plugin estimates for the $\mu_i$ before maximizing the joint normal likelihood as an estimator for $\sigma^2$. Ridge estimators also enforce shrinkage. The specific estimates of $\mu_i$ are biased a little but the estimates for $\sigma^2$ have far more favorable small and large sample properties. It is precisely because of this distributional modeling that the number of observations per cluster can be extremely small and yet the estimators of fixed effects and variance structures have highly favorable properties.

In spite of this, there are many missteps in the literature and elsewhere where the wrong mixed model is applied and the variance components are subject to sparse estimation. Here, just as in any ordinary OLS, unreliable estimates lead to unreliable inference, and much of it can be missed without doing thorough diagnostics.

  • $\begingroup$ You offer one biased estimator of variance, but that doesn't logically demonstrate anything except that this is possibly not a good estimator. $\endgroup$
    – whuber
    Sep 28 at 18:46
  • $\begingroup$ An interesting twist of N=1 is that an unbiased estimate of the standard error of the sample variance is undefined - with N=1 we can generate a point estimate of zero variance, but that point estimate is totally uninformative for the true variance, which could credibly take any positive value at all. You can calculate Var=0 for N=1, but it provides no information whatsoever about the actual variance in the underlying population. $\endgroup$ Sep 28 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.