How to fit a model for a time series that contains outliers

Question

I have fitted ARIMA(5,1,2) model using auto.arima() function in R and by looking order we can say this is not a best model to forecast. If outliers exist in the data series, what is the method to fit a model to such data?

Answer 1

Michael Chernick points you in the right direction. I would also look at Ruey Tsay's work as that added to this body of knowledge. See more here.

You can't compete against today's automated computer algorithms. They look at many ways to approach the time series that you haven't considered and often not documented in any paper or book. When one asks how to do an ANOVA, a precise answer can be expected when comparing against different algorithms. When one asks the question how do I do pattern recognition, many answers are possible as heuristics are involved. Your question involves the use of heuristics.

The best way to fit an ARIMA model, if outliers exist in the data is to evaluate possible states of nature and to select that approach that is deemed optimal for a particular data set. One possible state of nature is that the ARIMA process is the primary source of explained variation. In this case one would "tentatively identify" the ARIMA process via the acf/pacf function and then examine the residuals for possible outliers. Outliers can be Pulses, i.e., one-time events OR seasonal pulses which are evidenced by systematic outliers at some frequency (say, 12 for monthly data). A third type of outlier is where one has a contiguous set of pulses, each having the same sign and magnitude, this is called a step or level shift. After examining the residuals from the tentative ARIMA process one can then tentatively add the empirically identified deterministic structure to create a tentative combined model. Nor if the primary source of variation is one of the 4 kinds or "outliers" then one would be better served by identifying them ab initio (first) and then using the residuals from this "regression model" to identify the stochastic (ARIMA) structure. Now these two alternative strategies get a little more complicated when one has a "problem" where the ARIMA parameters change over time or the error variance changes over time due to a number of possible causes, possibly the need for weighted least squares or a power transform like logs / reciprocals, etc. Another complication / opportunity is how and when to form the contribution of user-suggested predictor series to form a seamlessly integrated model incorporating memory, causals and empirically identified dummy series. This problem is further exacerbated when one has trending series best modeled with indicator series of the form $0,0,0,0,1,2,3,4,...$, or $1,2,3,4,5,...n$ and combinations of level shift series like $0,0,0,0,0,0,1,1,1,1,1$. You might want to try and write such procedures in R, but life is short. I would be glad to actually solve your problem and demonstrate in this case how the procedure works, please post the data or send it to [email protected]

---

Additional comment after receiving / analyzing the data / daily data for a foreign exchange rate / 18=765 values starting 1/1/2007

The data had an acf of:

Upon identifying an arma model of the form $(1,1,0)(0,0,0)$ and a number of outliers the acf of the residuals indicates randomness since the acf values are very small. AUTOBOX identified a number of outliers:

The final model:

included the need for a variance stabilization augmentation a la TSAY where variance changes in the residuals were identified and incorporated. The problem that you had with your automatic run was that the procedure you were using, like an accountant, believes the data rather than challenging the data via Intervention Detection (a.k.a., Outlier Detection). I have posted a complete analysis here.

Answer 2

There is no ready to use robust counterpart to arima function in R (yet); should one appear, it will be listed here. Maybe an alternative is to down-weight those observations that are outlying with respect to a simple univariate outlier detection rule, but I don't see ready to use packages to run weighted ARMA regression either. Another possible alternative would then be to Winsorize the outlying points:

#parameters
para     <- list(ar=c(0.6,-0.48), ma=c(-0.22,0.24))
#original series
y1 <- y0 <- arima.sim(n=100, para, sd=sqrt(0.1796)) 
#outliers
out      <- sample(1:100, 20)               
#contaminated series
y1[out]  <- rnorm(20, 10, 1)                
plot( y1, type="l")
lines(y0, col="red")

#winsorized series
y2      <- rep(NA, length(y1))
a1      <- (y1-median(y1)) / mad(y1)
a2      <- which(abs(a1)>3)
y2[-a2] <- y1[-a2]
for(i in 2:length(y2)){
   if(is.na(y2[i])){ y2[i] <- y2[i-1] }
}       

Answer 3

There is a sizable literature on robust time series models. Martin and Yohai are among the major contributors. Their work goes back to the 1980s. I did some work on detecting outliers in time series myself, but Martin was really one of the many contributors to both the detection of outliers and parameter estimation in the presence of outliers or heavy-tailed residuals in time series.

• Here is a link to a survey article on the topic with a list of over 100 references. It even includes my 1982 JASA paper.
• Here is a 2000 PhD thesis (pdf) that covers the theory, methods and applications of robust time series analysis and includes a nice bibliography.
• Here is a link on software that includes some robust time series tools.

Answer 4

is the purpose of your model to forecast or analyze the history? if this is not for forecasting, and you know that these are the outliers, then simply add the dummy variable, which is 1 in those dates and 0 in other dates. this way the dummy coefficients will take care of the outliers, and you'll be able to interpret the other coefficients in the model.

if this is for forecasting, then you have to ask yourself two questions: will these outliers happen again? if they would, do I have to account for them?

For instance, let's say your data series have outliers when Lehman brothers went down. it's an event which you have no way of predicting, obviously, yet you can't simply ignore it because something like this is bound to happen in future. if you throw in the dummy for outliers, then you effectively remove the uncertainty of this event from the error variance. your forecast will underestimate the tail risk - not a good thing, perhaps, for risk management. however, if you are to produce the baseline forecast of sales, the dummy will work, because you're not interested in the tail, you're interested in most likely scenarios - so you don't have to account for the unpredictable event for this purpose.

Hence, the purpose of your model impacts the way you deal with outliers.