1
$\begingroup$

I want to extract outliers from time series data which contains faulty values. I want to filter out outliers from stock prices. I have around 14,000 such timeseries and I can't analyse them manually. I need a robust technique that would do this task.

What statistical methods can be used to detect outliers apart from IQR and Percentiles?

$\endgroup$

1 Answer 1

1
$\begingroup$

I wouldn't use IQR nor Percentiles to detect outliers in a times series since these statistics are computed on the overall sample (at least how I understand it with your question) but you have some effects (dynamic/cyclic/trends) in time series. In fact, if your value grows over time (or is cyclical) you may detect outliers regarding previous values.

First thing you do when you hear time series is to decompose your values, as explained here or here.

Once you remove the trend, seasonal and cyclical effects, you can use an ARMA (or simple moving average) to detect what can be modeled as time series (shocks, return to mean, etc) and what is noise.

You can detect if a value is an outlier with IQR and Percentile of the noise. You can also use true value vs prediction confidence interval (a good example here with the moving average).
Another way to assess an outlier is influence: how much an estimator is sensitive to some value. This paper is theoretical but I hope it will give you some insights.

FB's Prophet (R and Python) offers a good toolbox to address Time Series (and outliers for instance).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.