Timeline for Outlier detection for generic time series
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 1, 2011 at 13:38 | comment | added | IrishStat | Another citation for data cleansing might be of interest autobox.com/pdfs/modellingunitcasestudy.pdf | |
| Jul 1, 2011 at 10:25 | comment | added | IrishStat | @asdfion: The number of false positives (or false negatives!) depends on the autocorrelative structure AND the appropriateness of the filter being used.Now to precisely answer your question if the series is positively autocorrelated say ar(1) of .5 then your procedure will over estimate the variance of the noise and thus will under estimate (mask) the number of outliers. Good luck on rolling your own!. I have been at it for nearly 50 years. if the series is negatively correlated ar(1)of -.5 my gut reaction is a first differnce filter will also yield an increase in false negatives | |
| Jul 1, 2011 at 3:39 | comment | added | Affine | @Irish - Will certainly take a look, especially at the citations. I'm actually interested in the actual process (and the theory/math behind it) as I will shortly be pursuing a MS. In addition, give my firm's inclination, we'd probably end up rolling our own, if I can convince them of an improved method. In addition, I kind of wonder what exactly the implications of our current model. For example, if the series is an AR(1), I think the current strategy would run more false positives? $X_t - X_{t-1} \sim N(0, something>\sigma^2)$. Would there be any case where the variance end up lower? | |
| Jun 30, 2011 at 21:24 | history | edited | IrishStat | CC BY-SA 3.0 |
added 4 characters in body
|
| Jun 30, 2011 at 18:28 | history | edited | IrishStat | CC BY-SA 3.0 |
added 1022 characters in body
|
| Jun 30, 2011 at 13:01 | comment | added | IrishStat | @Dm: For Transparency considerations I should advise you that I am one of the developers of AUTOBOX which is cited in both of the above references. For other commercial sites significantly less powerful in my opinion as they don't detect time trends or seasonal pulsesyou might look at support.sas.com/documentation/cdl/en/etsug/60372/HTML/default/… . | |
| Jun 30, 2011 at 11:54 | comment | added | IrishStat | @Dm: Since you asked 1)An overview stpete.usf.edu/gkearns/Articles_Fraud/Fraud%20Magazine1.pdf ; 2)Details on how to program Intervention Detection autobox.com/pdfs/forestdisturbance.pdf | |
| Jun 30, 2011 at 11:22 | comment | added | Dmitrij Celov | (+1) I was pretty sure that you will answer your favorite topic :) @IrishStat, may be you have also your preferred reviews (just for those who has no time to dig in search engines). | |
| Jun 30, 2011 at 11:00 | history | answered | IrishStat | CC BY-SA 3.0 |