2
$\begingroup$

I have modeled a stationary time series with another related stationary time series. I'm having a problem with cyclicality in the residuals and I don't know how to fix it.

Here is my model:

$\text{TS}_1(t) = \beta_0 + \beta_1\cdot \text{TS}_2(t) + \epsilon$

Here is a chart of the residuals:

Residuals

There is clearly a strong trend. I tried to take out the trend by adding a 15 day lagged variable. The new residuals are looking a lot better, but there still looks like there is some kind of trend or abnormality (they don't look random to me).

Here is the model with the Lag:

$\text{TS}_1(t) = \beta_0 + \beta_1\cdot \text{TS}_2(t) + \beta_2\cdot(\text{TS}_2(t-15)) + \epsilon$

Residual_Lagged_model

I haven't ever done anything like this before. I know adding a lagged variable in AR models can remove seasonality, but I don't know if that applies to the errors on a time series regressed on a different time series.

Is adding a lagged variable to the model the appropriate way of removing trends in the residuals? What tests can I run (other than just looking at the chart) to decide of the trend is still an issue? I ran the Durbin-Watson test (both models failed), but I don't know if the test applies when modeling one time series from another.

$\endgroup$
  • 1
    $\begingroup$ If you post your data I will be able to advise you as to how to deal with your problem which is probably an omitted ARIMA term required to deal with auto-correlation in the residuals. $\endgroup$ – IrishStat Jan 9 '17 at 20:34
  • $\begingroup$ @IrishStat Thank you for your willingness to help! Unfortunately for me the data I am working with has privacy restrictions that prevent me from being able to share the actual data points. I will look into the omitted ARIMA terms and try to see if I can find a solution. Thank you again for you help. $\endgroup$ – Jarom Jan 9 '17 at 21:11
  • $\begingroup$ It could also be an artifact of not having the correct lag structure in X or it could be an artifact of a change in model parameters or error variance over time or even the need to transform Y in some manner as suggested by the Box-Cox test OR even a reflection of untreated outliers/level step shifts . You are observing a symptom and analysis can suggest a possible remedy/cause. $\endgroup$ – IrishStat Jan 9 '17 at 21:35
  • $\begingroup$ @IrishStat Thanks for the direction. I'll have to do some research and some testing of the data. I'll post if I find a solution. $\endgroup$ – Jarom Jan 9 '17 at 23:24
  • 1
    $\begingroup$ Sorry for the ambiguity. TS1 and TS2 are two different related time series. $\endgroup$ – Jarom Jan 10 '17 at 18:37
2
$\begingroup$

Try adding a moving average MA(q) term, it looks like a spike upwards in your errors is followed by a sharp drop in your errors. An MA(1) would add the term $\theta$ε$_{t-1}$ which would factor in the error from the day before. This might smooth out large movements in your trend.

AR(1),MA(q): TS$_{t}$ = β$_{0}$ + β$_{1}$*TS$_{t-1}$ + β$_{2}$*TS$_{t-15}$ + $\theta_{1}$ε$_{t-1}$+ ...+ $\theta_{q}$ε$_{t-q}$ + ε

$\endgroup$
  • $\begingroup$ I added a seven day moving average (which I think is logical given the data). It smoothed out my residuals a lot and increased the R-Squared. I'm going to have to look up the technical aspect of adding a previous error term to the regression in SAS. Is the DW test appropriate to quantify if my errors are random enough? $\endgroup$ – Jarom Jan 10 '17 at 15:38
  • 1
    $\begingroup$ Yep, Durbin Watson tests the autocorrelation of your residuals, since you are not fitting a regression with Arima errors, rather an ARMAX model, you should check for that. By the way, differencing could be an idea since it seems like you have some trend there $\endgroup$ – Tommaso Guerrini Jan 10 '17 at 16:17
  • $\begingroup$ @TommasoGuerrini Great advice! I didn't think that I needed to do differencing because my data looked stationary based on the Dickey-Fuller test. But maybe the DF does not catch seasonal non-stationarity? I put a week and month seasonal difference and it has taken care of the auto correlation issue. My quesion now is, does it make sense to use this model? I apalogize that I don't know the proper notation: TS1Diff(Month) = β0 + β 1 ⋅TS 2 Diff(Month) + β 2 ⋅TS 2 Diff(week) $\endgroup$ – Jarom Jan 10 '17 at 17:37
  • $\begingroup$ @Jarom you should use ACF and PACF graphs to explore your data set to determine whether it is stationary as well as statistical tests. Here is a great tutorial on ARIMA modelling and contains R code if your an R user. otexts.org/fpp/8/1 $\endgroup$ – Morgan Ball Jan 10 '17 at 17:47
  • $\begingroup$ @TommasoGuerrini Like I said, I did the seasonal differencing. Now I have stationary data. I created a valid model on that data. Now my question is, is there a technique to take the predicted values from the differenced data and apply it to the original data? $\endgroup$ – Jarom Jan 11 '17 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.