2
$\begingroup$

I am reading a book on time series analysis and I am having problems understanding the section about outlier detection.

The authors say that when you want to know whether at a certain time $T$ there was an outlier, you should use a certain test statistic and a test with size less than $\alpha$. But when you don't know where an outlier could be and you have a time series of size $n$ then you should use the same test statistic for each point but you should use tests of size $\alpha/n$. They say that this is an application of the conservative Bonferroni correction.

I just don't understand this. Doesn't this mean that there will be lots of outliers that you detect in short time series but don't detect in large ones? After all, spam filters don't have stronger spam criteria for people with more incoming email, right?

$\endgroup$
  • $\begingroup$ Welcome to the site. The question has little to do with time series, so I added a few other tags. To do accurate multiple comparison analysis, you would need to divide by some sort of effective sample size, which will be lower for (positively) correlated time series. Bonferroni will likely be too conservative, so in a way what you say may be true. There likely to be time-series-specific methods of outlier detection that work around this difficulty. $\endgroup$ – StasK Sep 6 '11 at 18:24
4
$\begingroup$

Try generating some data from a normal distribution, first generate a small sample and look at the spread of the points, now add a few more points, then more, then more. You will notice that as the sample size gets bigger you will see more extreme values (potential outliers) just by chance alone. If you don't do some adjustment for multiple comparisons then you will see much more significance in large sample sizes just due to the large sample size when the underlying process is stable and all the data points are legitimate (inliers?).

$\endgroup$
  • $\begingroup$ When I keep the ordinary size of, say, $\alpha=0.05$, no matter how many elements there are in the series, then I will get more and more false positives when the time series grows. Then I could perhaps check whether there are too many positives and could build a hypothesis test based on the number of positives but that would only tell me, that there might be outliers somewhere, not at which time. ... $\endgroup$ – frank Sep 7 '11 at 9:35
  • $\begingroup$ (continuing from above): But to find outliers, I think it would be just wrong to divide by $n$ because that way I would fail to detect more and more outliers, just by growing my time series. It's the same with spam filters: no email provider is strengthening the spam criteria for people with more email only to keep the number of false positives from increasing. When you have more mail, you have more false positives. ... $\endgroup$ – frank Sep 7 '11 at 9:49
  • $\begingroup$ continuing: maybe I should add that I need outlier detection for network intrusion detection. $\endgroup$ – frank Sep 7 '11 at 9:50
  • $\begingroup$ It sounds as if you are not after a correction for the family-wise Type I error rate, but a method for controlling the false positive (or false discovery) rate. Have you taken a look at the Wikipedia page? en.wikipedia.org/wiki/False_discovery_rate I think this may be useful. $\endgroup$ – Wolfgang Sep 7 '11 at 12:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.