I'm working with a time series of macroeconomic data (independent variables) and bank loss rates (dependent variable) to show how bank losses vary based on the state of the economy.
To estimate the relationship between the dependent and independent varaibles, I am considering two regression techniques: standard OLS and Regression with ARMA errors as outlined in this extremely helpful article https://robjhyndman.com/hyndsight/arimax/.
When comparing the coefficients and residuals calculated by the two methodologies, I am struggling to interpret which is preferable. I'll walk you through my thought process with the hope that others can better intuit which is the preferable model.
When I estimate the model using OLS, I get a decently fit model (63% Adj R-squared) with all coefficients showing some degree of significance:
> summary(varLM) Call: lm(formula = DepVar ~ modelFactors) Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -0.260879 0.061043 -4.274 4.73e-05 *** NAT.5_Year_Treasury_Yield 0.012234 0.005059 2.418 0.01759 * NAT.House_Price_Index_Growth -0.002837 0.001129 -2.514 0.01371 * REG.Real_Disposable_Income_Growth -0.011284 0.003674 -3.072 0.00281 ** REG.Unemployment_Rate 0.060506 0.006767 8.942 4.18e-14 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 0.08086 on 91 degrees of freedom Multiple R-squared: 0.6491, Adjusted R-squared: 0.6337
The model residuals pass stationarity tests, so I feel fairly confident that my model isn't misspecified. Below are my residuals plotted over time as well as the ADF test on the OLS residuals.
> adf.test(varLM$residuals) Augmented Dickey-Fuller Test data: varLM$residuals Dickey-Fuller = -3.9243, Lag order = 4, p-value = 0.01605 alternative hypothesis: stationary
Now where my models got dinged is that the residuals show autocorrelation. Below is the Durbin-Watson test for this model:
Durbin-Watson test data: varLM DW = 1.0976, p-value = 1.709e-07 alternative hypothesis: true autocorrelation is greater than 0
A model validator could not get over that 1) my models had autocorrelation and 2) that I didn't use an arima structure. In fact they told me that the above model was basically garbage. On the autocorrelation point, I thought I could get around it using Newey-West Standard errors as I've read that serial correlation only affects the model's standard errors...
Regression with ARMA Errors
Based on this feedback, I investigated using a regression with ARMA errors to resolve the serial correlation issue in my models. I used the auto.arima function in R to fit an ARIMA model on my residuals:
> auto.arima(DepVar, xreg = modelFactors) Series: DepVar ARIMA(1,0,0) with non-zero mean Coefficients: ar1 intercept NAT.5_Year_Treasury_Yield NAT.House_Price_Index_Growth 0.4674 -0.2429 0.0075 -0.0029 0.0985 0.0894 0.0080 0.0017 REG.Real_Disposable_Income_Growth REG.Unemployment_Rate -0.0078 0.0586 s.e. 0.0051 0.0101 sigma^2 estimated as 0.005294: log likelihood=118.34 AIC=-222.67 AICc=-221.4 BIC=-204.72
Comparing the coefficients on this model with the coefficients on the OLS model, I see little change. The obvious difference is now we are modeling the errors using an AR(1) process. The difference can be seen in the residuals which now look like true white noise.
Am I wrong to say that the OLS model is fine, despite the serial correlation of the residuals? My preference is to stick to a simpler model structure (OLS) because I'd prefer to avoid a more complex model if possible. The other issue with the ARMA model is that I"m building multiple - what I've shown here is a subset. And when use the auto.arima function, I'm getting a different mix of AR() and MA() terms for each model.
I guess in general, is there a good rule of them of when OLS is appropriate for times series and when an ARIMA model is preferrable?