For a static normal distribution with fixed parameters $\mu$ and $\sigma^2$, the sample mean and sample variance can be used to estimate $\mu$ and $\sigma^2$, maybe additionally using a trimmed or winsored estimator.

What is the case when both $\mu(t)$ and $\sigma^2(t)$ are a (continuous) function of the time $t$? Is there any way to update the estimates based on previous values?

An example might be market bids and asks, where several people state their buying price for an item (say, a stock, or apples). Over time, the price may move higher or lower, but it can be frequently sampled to observe even small changes.

  • $\begingroup$ Can you say something about what these functions might look like? The case where $\sigma^2(t) = \sigma^2$ for instance is addressed by standard regression techniques. $\endgroup$ – dsaxton Mar 13 '16 at 15:21
  • $\begingroup$ @dsaxton: They can be sampled often enough that you can observe small movements. (But without them being continuous, it might be impossible). $\endgroup$ – serv-inc Mar 13 '16 at 16:07
  • $\begingroup$ One possibility might be to estimate the functions with splines, perhaps using some sort of weighting of the residuals to account for heteroskedasticity of the error term. You could first fit the function $\hat{\mu}(t)$ and then fit another model to the squared errors $(\hat{\mu}(t) - y(t))^2$, although some care would have to be taken to avoid overfitting (i.e., underestimation of $\sigma^2(t)$). Are you able to provide a sample of your data and / or a scatter plot? $\endgroup$ – dsaxton Mar 13 '16 at 17:18
  • $\begingroup$ @dsaxton: Would you still like some sample data even though there are already answers, or not? $\endgroup$ – serv-inc Mar 14 '16 at 16:04
  • 1
    $\begingroup$ No, it seems other users are figuring it out. Thanks. $\endgroup$ – dsaxton Mar 14 '16 at 16:29

Yes, mean and variance can be updated incrementally.

The details are in the seminal book

Donald E. Knuth (1998).
The Art of Computer Programming, volume 2: Seminumerical Algorithms

  • $\begingroup$ Searching 'knuth art of computer programming update mean and variance' points to Wikipedia: en.wikipedia.org/wiki/Algorithms_for_calculating_variance. That references volume two of "The Art of Computer Programming". $\endgroup$ – serv-inc Mar 14 '16 at 10:50
  • $\begingroup$ A blog gives a reference of "Donald Knuth’s Art of Computer Programming, Vol 2, page 232, 3rd edition" $\endgroup$ – serv-inc Mar 14 '16 at 11:11

@Anony-Mousse's post gave some terms to google, which led to this dsp.SX question. There are two approaches, one of which is the Knuth from the other answer. The other

weighs more recent samples stronger than samples from the distant past.

The update algorithm (slightly edited) is given as

% update the estimate of the mean and the mean square:
mean = (1-a)*mean + a*x
meansq = (1-b)*meansq + b*(x^2)

% calculate the estimate of the variance:
var = meansq - mean^2;

% and, if you want standard deviation:
std = sqrt(var);

The choice of a and b determines how strongly to weigh the newer values. The dsp.sx question has answers which partly contain how to determine it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.