3
$\begingroup$

I am building a regression model of time series data in R, where my primary interest is the coefficients of the independent variables. The data exhibit strong seasonality with a trend.

Original data

The model looks good, with four of the six regressors significant: Model

Here are the OLS residuals: Residuals

I used auto.arima to select the sARIMA structure, and it returns the model (0,1,1)(1,1,0)[12].

fit.ar <- auto.arima(at.ts, xreg = xreg1, stepwise=FALSE, approximation=FALSE)
summary(fit.ar)

Series: at.ts 
ARIMA(0,1,1)(1,1,0)[12]                    

Coefficients:
          ma1    sar1      v1       v2      v3       v4         v5
      -0.7058  0.3974  0.0342  -0.0160  0.0349  -0.0042  -113.4196
s.e.   0.1298  0.2043  0.0239   0.0567  0.0555   0.0333   117.1205

sigma^2 estimated as 3.86e+10:  log likelihood=-458.13
AIC=932.26   AICc=936.05   BIC=947.06

Training set error measures:
                   ME     RMSE      MAE       MPE     MAPE      MASE
Training set 7906.896 147920.3 103060.4 0.1590107 3.048322 0.1150526

My question is this: based on the parameter estimates and s.e. of the regressors, I believe that none of them are significant - is this correct, and if so, what does it imply if my goal is to interpret the relative importance of these predictors as opposed to forecasting?

Any other advice relative to the process of building this model is welcome and appreciated.

Here are the ACF and PACF for the residuals:

ACF-PACF

> durbinWatsonTest(mod.ols, max.lag=12)
 lag Autocorrelation D-W Statistic p-value
   1     0.120522674     1.6705144   0.106
   2     0.212723044     1.4816530   0.024
   3     0.159828108     1.5814771   0.114
   4     0.031083831     1.8352377   0.744
   5     0.081081308     1.6787808   0.418
   6    -0.024202465     1.8587561   0.954
   7    -0.008399949     1.7720761   0.944
   8     0.040751905     1.6022835   0.512
   9     0.129788310     1.4214391   0.178
  10    -0.015442379     1.6611922   0.822
  11     0.004506292     1.6133994   0.770
  12     0.376037337     0.7191359   0.000
 Alternative hypothesis: rho[lag] != 0
Improve this question
$\endgroup$
11
  • 1
    $\begingroup$ It looks like some of the t statistics are favorable (ma1, sar1, v1). Also, could you add to the post some information on how autocorrelated the residuals are, such as the Durbin-Watson statistic? They don't look that autocorrelated. If you do believe there is still autocorrelation, add another autocorrelation term to the arima and check how the AIC changes. $\endgroup$
    – John
    Commented Mar 4, 2014 at 16:23
  • $\begingroup$ do you have acf/pacf of residuals? why do you think they're autocorrelated? $\endgroup$
    – Aksakal
    Commented Mar 4, 2014 at 16:45
  • $\begingroup$ Added ACF/PACF for the residuals of the OLS model, as well as DW statistics. It should be noted that v6 in the OLS model is a dummy seasonal variable (0/1 marking the month of interest), which I excluded from the xreg in auto.arima in favor of letting it decide the seasonality structure. I'm sensing that the opinion so far is that sARIMA is not necessary, which confuses me because looking at the original data it sure looks like an obvious candidate for ARIMA errors. Perhaps I'm making this harder than it needs to be? $\endgroup$
    – Geoffrey
    Commented Mar 4, 2014 at 17:18
  • 1
    $\begingroup$ @Geoffrey, whatever the autocorrelations, they're benign. I thought you ran the ARIMA model. maybe I'm confused. If you ran OLS, and got these nice residuals, then i'd stick to OLS, as long as your coefficients look good. $\endgroup$
    – Aksakal
    Commented Mar 4, 2014 at 17:46
  • 1
    $\begingroup$ @Geoffrey, you see the peak at 12 in your ACF? that's the seasonality, the only obvious issue. I would try first OLS with the seasonal dummy. start with monthly dummies, then try to reduce the number of dummies, you may end up with just one month dummy. e.g. in financial series, you often see jumps around the tax season in April, so a single dummy takes care of it in many cases $\endgroup$
    – Aksakal
    Commented Mar 4, 2014 at 18:43

1 Answer 1

Reset to default
2
$\begingroup$

Just to be certain, the residual plot does not demonstrate autocorrelation. If you did want to visually describe the extent of autocorrelation in these data, I think a variogram would be a much better descriptive tool. Nonetheless, autocorrelation is present in these data based on your understanding of the time series and your belief that the predictors in your model do not adequately handle the extent of residual variance due to autocorrelation that they could. An example of when that might not be the case is in clinical trials with adaptive dosing where the dosage is based on severity of disease, so you make the time series of subsequent maladies conditionally independent by adjusting for the dosage.

Nonetheless, at any point if there is unmeasured correlation in the data, the interpretation of the coefficients remains exactly the same. The least squares regression slope is still a measure of expected difference in outcomes comparing a unit difference in exposures/regressors. What you lose by ignoring correlation is efficiency/validity of inference. Your standard errors can be inflated or shrunk, so it's the confidence intervals that are messed up. Of course, if you accurately identify the correlation structure, you can iteratively estimate correlation and parameters, and obtain the BLUE which gives valid inference. This is all detailed in Seber & Lee.

What's not in Seber & Lee is that if you use robust standard errors, these will give you correct inference even if the correlation structure is misspecified. The data could be AR-1, you can specify independence, and you still get valid inference... you just lose a little efficiency relative to estimating robust standard errors with the correct correlation structure.

Improve this answer
$\endgroup$
7
  • $\begingroup$ My concern is precisely that I can not trust my confidence interval for the parameter estimates. The estimates themselves do not worry me, but my goal is to understand their relative significance (or lack thereof). I wish to obtain s.e.'s which leverage the autocorrelation I suspect to be present, thus the SARIMA structured model. The problem (which may simply be my lack of understanding) is that this structure tells me that none of the predictor variables are significant. This may be true, but it is surprising and I want to validate the process and conclusions I've reached. $\endgroup$
    – Geoffrey
    Commented Mar 4, 2014 at 17:45
  • $\begingroup$ Based on the lack of autocorrelation in the residual plot, would you conclude the OLS regression is an acceptable model for this data, where acceptable means robust s.e. for the parameter estimates? $\endgroup$
    – Geoffrey
    Commented Mar 4, 2014 at 17:49
  • $\begingroup$ I did not say that there is no autocorrelation. But I would never produce a residual plot to describe autocorrelation since I can't trust my eyeballs. (A variogram is a much better tool). You asked about the interpretation of the parameters. Since the parameter estimates unbiased, the interpretation remains the same: difference in outcome for unit difference in exposure. A linear regression with robust standard errors is a perfectly valid way to address this and obtain correct confidence intervals. $\endgroup$
    – AdamO
    Commented Mar 4, 2014 at 19:37
  • $\begingroup$ But are the s.e.'s robust in this situation? Or are you saying that a variogram is necessary to find out? $\endgroup$
    – Geoffrey
    Commented Mar 4, 2014 at 19:43
  • 1
    $\begingroup$ I read the paper on NW SE estimators. Apparently, "Newey-West" estimators are yet another derivation of sandwich based SEs. It is NOT a correction, though. But it seems like you've mostly come to the same conclusion that I recommended. $\endgroup$
    – AdamO
    Commented Mar 4, 2014 at 22:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.