# How to fit a model for a time series that contains outliers

I have fitted ARIMA(5,1,2) model using auto.arima() function in R and by looking order we can say this is not a best model to forecast. If outliers exist in the data series, what is the method to fit a model to such data?

• Do you have any data/theory as to what points are outliers? You can't simply assume that "far away" points are outliers, but if you know that something special happened on a particular date and that event would affect your data, you can add an indicator variable to your model for that date. See IrishStat's comment, below. Commented Jul 18, 2012 at 14:12
• If something special happened on 1 or 2 weeks in that period and it affect to the model, then the model may be incorrect. since there's no other reason like seasonal variation, I assumed that outliers is the cause that affect to the model. Commented Jul 18, 2012 at 18:01

Michael Chernick points you in the right direction. I would also look at Ruey Tsay's work as that added to this body of knowledge. See more here.

You can't compete against today's automated computer algorithms. They look at many ways to approach the time series that you haven't considered and often not documented in any paper or book. When one asks how to do an ANOVA, a precise answer can be expected when comparing against different algorithms. When one asks the question how do I do pattern recognition, many answers are possible as heuristics are involved. Your question involves the use of heuristics.

The best way to fit an ARIMA model, if outliers exist in the data is to evaluate possible states of nature and to select that approach that is deemed optimal for a particular data set. One possible state of nature is that the ARIMA process is the primary source of explained variation. In this case one would "tentatively identify" the ARIMA process via the acf/pacf function and then examine the residuals for possible outliers. Outliers can be Pulses, i.e., one-time events OR seasonal pulses which are evidenced by systematic outliers at some frequency (say, 12 for monthly data). A third type of outlier is where one has a contiguous set of pulses, each having the same sign and magnitude, this is called a step or level shift. After examining the residuals from the tentative ARIMA process one can then tentatively add the empirically identified deterministic structure to create a tentative combined model. Nor if the primary source of variation is one of the 4 kinds or "outliers" then one would be better served by identifying them ab initio (first) and then using the residuals from this "regression model" to identify the stochastic (ARIMA) structure. Now these two alternative strategies get a little more complicated when one has a "problem" where the ARIMA parameters change over time or the error variance changes over time due to a number of possible causes, possibly the need for weighted least squares or a power transform like logs / reciprocals, etc. Another complication / opportunity is how and when to form the contribution of user-suggested predictor series to form a seamlessly integrated model incorporating memory, causals and empirically identified dummy series. This problem is further exacerbated when one has trending series best modeled with indicator series of the form $0,0,0,0,1,2,3,4,...$, or $1,2,3,4,5,...n$ and combinations of level shift series like $0,0,0,0,0,0,1,1,1,1,1$. You might want to try and write such procedures in R, but life is short. I would be glad to actually solve your problem and demonstrate in this case how the procedure works, please post the data or send it to [email protected]

Additional comment after receiving / analyzing the data / daily data for a foreign exchange rate / 18=765 values starting 1/1/2007

The data had an acf of:

Upon identifying an arma model of the form $(1,1,0)(0,0,0)$ and a number of outliers the acf of the residuals indicates randomness since the acf values are very small. AUTOBOX identified a number of outliers:

The final model:

included the need for a variance stabilization augmentation a la TSAY where variance changes in the residuals were identified and incorporated. The problem that you had with your automatic run was that the procedure you were using, like an accountant, believes the data rather than challenging the data via Intervention Detection (a.k.a., Outlier Detection). I have posted a complete analysis here.

• @IrishStat: In the future, can you use the formatting options? (the correct key to insert links is ctrl-l, not ctrl-c). Commented Jul 19, 2012 at 10:53
• @user603 I have been simply using the add a graph option. I don't know what you mean by ctrl-l or cntrl-c. Perhaps you could help me , please contact me at dave /@/ autobox.com and perhaps you can walk/talk me through this. Commented Jul 19, 2012 at 14:22
• @IrishStat: have a look here . Commented Jul 19, 2012 at 14:29
• @user603 OK but I still don't know what you mean by ctrl-l and cntrl-c . I was using the image transfer where I am able to insert an image. If I type cntrl-g I just get a blank pop-up screen. Commented Jul 19, 2012 at 14:51
• @IrishStat: it's okay -- the link's method is even better than ctrl-l :). P.S.: my aim was to help you make your answers/comments more appealing, cosmetically, by hiding these long links that cut through the flow of the text. The link explains you how to achieve that. Commented Jul 19, 2012 at 14:53

There is no ready to use robust counterpart to arima function in R (yet); should one appear, it will be listed here. Maybe an alternative is to down-weight those observations that are outlying with respect to a simple univariate outlier detection rule, but I don't see ready to use packages to run weighted ARMA regression either. Another possible alternative would then be to Winsorize the outlying points:

#parameters
para     <- list(ar=c(0.6,-0.48), ma=c(-0.22,0.24))
#original series
y1 <- y0 <- arima.sim(n=100, para, sd=sqrt(0.1796))
#outliers
out      <- sample(1:100, 20)
#contaminated series
y1[out]  <- rnorm(20, 10, 1)
plot( y1, type="l")
lines(y0, col="red")


#winsorized series
y2      <- rep(NA, length(y1))
a1      <- (y1-median(y1)) / mad(y1)
a2      <- which(abs(a1)>3)
y2[-a2] <- y1[-a2]
for(i in 2:length(y2)){
if(is.na(y2[i])){ y2[i] <- y2[i-1] }
}


• A rule based procedure works for this series bu having looked at time seriesdata for nearly42 years may not work in general.Consider a simple series that has an auto-projective process e.g.1,9,1,9,1,9,5,9,1,9,1,9 where the 7th value is exceptional OR in general a series that hasa strong autoprojective structure of some unknown form OR a series that has 1,2,1,1,1,2,1,1,1,2,1,20,1,2,1,2,1,1,1,2,1,2,2,19,2,1,1,2,1,2,1,1,2,2,2,21 a monthly series that has no pulses BUT does have an unspecified SEASONAL DUMMY. Detecting 12,24,and 36 as BAD would be throwing out the baby with the bathwater. Commented Jul 18, 2012 at 13:56
• All that you say is true i.e. you were providing a band-aid that would help with the wound if 1) the series was stationary which assumes among other things no level shifts , no trends et al , no ARIMA structure ; and 2) the series has to be non-seasonal or has been previously de-seasonalized ; . It is my experience that such assumptions hardly if ever hold and users ignore your caveats as they try to wrestle with time series that has "imperfections". My comments here are not meant to be argumentative but rather illuminating ( to some ! ) Commented Jul 18, 2012 at 14:49
• @IrishStat the OP has updated his question and stated that there is no seasonality. Commented Jul 19, 2012 at 8:47
• Are you Christophe ?. Commented Jul 19, 2012 at 10:48

There is a sizable literature on robust time series models. Martin and Yohai are among the major contributors. Their work goes back to the 1980s. I did some work on detecting outliers in time series myself, but Martin was really one of the many contributors to both the detection of outliers and parameter estimation in the presence of outliers or heavy-tailed residuals in time series.

• Here is a link to a survey article on the topic with a list of over 100 references. It even includes my 1982 JASA paper.
• Here is a 2000 PhD thesis (pdf) that covers the theory, methods and applications of robust time series analysis and includes a nice bibliography.
• Here is a link on software that includes some robust time series tools.
• What is the software used for Robust forecasting? Is it suitable for univariate series? Commented Jul 22, 2012 at 7:54
• @Anthony That is a very good question. I have not done any robust time series modeling. dug Martin founded the company called Insightful (they actually had several name chnages) that marketed SPlus. I am pretty sure that he included robust methids in the SPlus software. There is probably now a version in R. I will check into it. Here is a Wikipedia article including the history of SPlus. en.wikipedia.org/wiki/S-PLUS Commented Jul 22, 2012 at 9:48
• @Anthony Here is a source provided by Rob Hyndman on what is available in R on time series analysis. It includes what is in CRAN and covers a wide variety of methods including robust filtering. cran.r-project.org/web/views/TimeSeries.html Commented Jul 22, 2012 at 10:32

is the purpose of your model to forecast or analyze the history? if this is not for forecasting, and you know that these are the outliers, then simply add the dummy variable, which is 1 in those dates and 0 in other dates. this way the dummy coefficients will take care of the outliers, and you'll be able to interpret the other coefficients in the model.

if this is for forecasting, then you have to ask yourself two questions: will these outliers happen again? if they would, do I have to account for them?

For instance, let's say your data series have outliers when Lehman brothers went down. it's an event which you have no way of predicting, obviously, yet you can't simply ignore it because something like this is bound to happen in future. if you throw in the dummy for outliers, then you effectively remove the uncertainty of this event from the error variance. your forecast will underestimate the tail risk - not a good thing, perhaps, for risk management. however, if you are to produce the baseline forecast of sales, the dummy will work, because you're not interested in the tail, you're interested in most likely scenarios - so you don't have to account for the unpredictable event for this purpose.

Hence, the purpose of your model impacts the way you deal with outliers.