2
$\begingroup$

Let's say, for instance, you have normally distributed data that is coming in one at a time. At some unknown instance, the distribution parameters (mean and variance) change. Assuming that there is not infinite memory, what would be the best way to determine how many samples are need to estimate the new distribution?

In this case, I was trying to play with a toy example where the first 100 events are drawn from N(100,20) and then at some unknown time, events are drawn from N(120,5). Intuitively, I think that it would take much longer to realize the variance shift in the distribution (i.e. it would take longer to detect a change moving from a very wide distribution to a narrow distribution, vs it would quickly be detected if you had a very narrow distribution previously and then shifted to a large distribution), but I'm not sure what the proper way to show this would be.

Any advice is greatly appreciated. Thanks!

Edit: Also in this case, the mean and variance of both the pre and post break point are unknown. Thanks

$\endgroup$
2
  • $\begingroup$ For the mean, at least, this sounds similar to a problem from control theory where a PID control must estimate the derivative of a noisy input signal. In this case, the derivative of the mean can tell us if the mean is actually changing, but a time filter must be employed to avoid false positives. $\endgroup$ – Ron Jensen Apr 22 '20 at 1:36
  • $\begingroup$ This area of statistics is known as changepoint detection $\endgroup$ – Dedekind Cuts Jul 15 '20 at 14:33
0
$\begingroup$

From a Bayesian point of view: $$ P(\theta|X_{new}) = \frac{P(X_{new}|\theta)P(\theta)}{P(X)} $$ Or in words: the probability the parameters, $\theta$, are what we think they are, given the data we've just observed, $X_{new}$, is given by the probability of getting the data we've just observed given the parameters times our confidence in the parameters.

If there have been hundreds, or thousands, of samples supporting $ X \sim N(100,20) $, and the process is believed stable, then $P(\theta=[100, 20])$ will be close to 1. When new data, $X_{new}\sim N(120,5)$, comes in, the original parameters, $\theta = [100,20] $ still look like a reasonable fit and $P(\theta =[100, 20] | X_{new})$ remains high. It's not until a large number of samples from $X_{new}\sim N(120,5)$ are observed that our confidence in $\theta=[100, 20]$ decreases and we can say we detect a change. On the other hand, if our confidence in $\theta=[100, 20]$ is low to begin with, $P(\theta=[100, 20])$ will be small and $P(\theta=[100, 20] | X_{new})$ gets smaller much quicker and we can say we detect a change much earlier.

This raises the question: are we willing to sacrifice being wrong about a change in distributions in order to claim to detect such a change sooner?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.