2
$\begingroup$

For a static normal distribution with fixed parameters $\mu$ and $\sigma^2$, the sample mean and sample variance can be used to estimate $\mu$ and $\sigma^2$, maybe additionally using a trimmed or winsored estimator.

What is the case when both $\mu(t)$ and $\sigma^2(t)$ are a (continuous) function of the time $t$? Is there any way to update the estimates based on previous values?

An example might be market bids and asks, where several people state their buying price for an item (say, a stock, or apples). Over time, the price may move higher or lower, but it can be frequently sampled to observe even small changes.

Improve this question
$\endgroup$
5
  • $\begingroup$ Can you say something about what these functions might look like? The case where $\sigma^2(t) = \sigma^2$ for instance is addressed by standard regression techniques. $\endgroup$
    – dsaxton
    Commented Mar 13, 2016 at 15:21
  • $\begingroup$ @dsaxton: They can be sampled often enough that you can observe small movements. (But without them being continuous, it might be impossible). $\endgroup$
    – serv-inc
    Commented Mar 13, 2016 at 16:07
  • $\begingroup$ One possibility might be to estimate the functions with splines, perhaps using some sort of weighting of the residuals to account for heteroskedasticity of the error term. You could first fit the function $\hat{\mu}(t)$ and then fit another model to the squared errors $(\hat{\mu}(t) - y(t))^2$, although some care would have to be taken to avoid overfitting (i.e., underestimation of $\sigma^2(t)$). Are you able to provide a sample of your data and / or a scatter plot? $\endgroup$
    – dsaxton
    Commented Mar 13, 2016 at 17:18
  • $\begingroup$ @dsaxton: Would you still like some sample data even though there are already answers, or not? $\endgroup$
    – serv-inc
    Commented Mar 14, 2016 at 16:04
  • 1
    $\begingroup$ No, it seems other users are figuring it out. Thanks. $\endgroup$
    – dsaxton
    Commented Mar 14, 2016 at 16:29

2 Answers 2

Reset to default
1
$\begingroup$

Yes, mean and variance can be updated incrementally.

The details are in the seminal book

Donald E. Knuth (1998).
The Art of Computer Programming, volume 2: Seminumerical Algorithms

Improve this answer
$\endgroup$
2
  • $\begingroup$ Searching 'knuth art of computer programming update mean and variance' points to Wikipedia: en.wikipedia.org/wiki/Algorithms_for_calculating_variance. That references volume two of "The Art of Computer Programming". $\endgroup$
    – serv-inc
    Commented Mar 14, 2016 at 10:50
  • $\begingroup$ A blog gives a reference of "Donald Knuth’s Art of Computer Programming, Vol 2, page 232, 3rd edition" $\endgroup$
    – serv-inc
    Commented Mar 14, 2016 at 11:11
1
$\begingroup$

@Anony-Mousse's post gave some terms to google, which led to this dsp.SX question. There are two approaches, one of which is the Knuth from the other answer. The other

weighs more recent samples stronger than samples from the distant past.

The update algorithm (slightly edited) is given as

% update the estimate of the mean and the mean square:
mean = (1-a)*mean + a*x
meansq = (1-b)*meansq + b*(x^2)

% calculate the estimate of the variance:
var = meansq - mean^2;

% and, if you want standard deviation:
std = sqrt(var);

The choice of a and b determines how strongly to weigh the newer values. The dsp.sx question has answers which partly contain how to determine it.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.