3
$\begingroup$

I am reading a book on time series analysis and I am having problems understanding the section about outlier detection.

The authors say that when you want to know whether at a certain time $T$ there was an outlier, you should use a certain test statistic and a test with size less than $\alpha$. But when you don't know where an outlier could be and you have a time series of size $n$ then you should use the same test statistic for each point but you should use tests of size $\alpha/n$. They say that this is an application of the conservative Bonferroni correction.

I just don't understand this. Doesn't this mean that there will be lots of outliers that you detect in short time series but don't detect in large ones? After all, spam filters don't have stronger spam criteria for people with more incoming email, right?

Improve this question
$\endgroup$
1
  • $\begingroup$ Welcome to the site. The question has little to do with time series, so I added a few other tags. To do accurate multiple comparison analysis, you would need to divide by some sort of effective sample size, which will be lower for (positively) correlated time series. Bonferroni will likely be too conservative, so in a way what you say may be true. There likely to be time-series-specific methods of outlier detection that work around this difficulty. $\endgroup$
    – StasK
    Commented Sep 6, 2011 at 18:24

1 Answer 1

Reset to default
4
$\begingroup$

Try generating some data from a normal distribution, first generate a small sample and look at the spread of the points, now add a few more points, then more, then more. You will notice that as the sample size gets bigger you will see more extreme values (potential outliers) just by chance alone. If you don't do some adjustment for multiple comparisons then you will see much more significance in large sample sizes just due to the large sample size when the underlying process is stable and all the data points are legitimate (inliers?).

Improve this answer
$\endgroup$
4
  • $\begingroup$ When I keep the ordinary size of, say, $\alpha=0.05$, no matter how many elements there are in the series, then I will get more and more false positives when the time series grows. Then I could perhaps check whether there are too many positives and could build a hypothesis test based on the number of positives but that would only tell me, that there might be outliers somewhere, not at which time. ... $\endgroup$
    – frank
    Commented Sep 7, 2011 at 9:35
  • $\begingroup$ (continuing from above): But to find outliers, I think it would be just wrong to divide by $n$ because that way I would fail to detect more and more outliers, just by growing my time series. It's the same with spam filters: no email provider is strengthening the spam criteria for people with more email only to keep the number of false positives from increasing. When you have more mail, you have more false positives. ... $\endgroup$
    – frank
    Commented Sep 7, 2011 at 9:49
  • $\begingroup$ continuing: maybe I should add that I need outlier detection for network intrusion detection. $\endgroup$
    – frank
    Commented Sep 7, 2011 at 9:50
  • $\begingroup$ It sounds as if you are not after a correction for the family-wise Type I error rate, but a method for controlling the false positive (or false discovery) rate. Have you taken a look at the Wikipedia page? en.wikipedia.org/wiki/False_discovery_rate I think this may be useful. $\endgroup$
    – Wolfgang
    Commented Sep 7, 2011 at 12:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.