I'm working with a time series of macroeconomic data (independent variables) and bank loss rates (dependent variable) to show how bank losses vary based on the state of the economy.
To estimate the relationship between the dependent and independent varaibles, I am considering two regression techniques: standard OLS and Regression with ARMA errors as outlined in this extremely helpful article https://robjhyndman.com/hyndsight/arimax/.
When comparing the coefficients and residuals calculated by the two methodologies, I am struggling to interpret which is preferable. I'll walk you through my thought process with the hope that others can better intuit which is the preferable model.
OLS Model
When I estimate the model using OLS, I get a decently fit model (63% Adj R-squared) with all coefficients showing some degree of significance:
> summary(varLM)
Call:
lm(formula = DepVar ~ modelFactors)
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.260879 0.061043 -4.274 4.73e-05 ***
NAT.5_Year_Treasury_Yield 0.012234 0.005059 2.418 0.01759 *
NAT.House_Price_Index_Growth -0.002837 0.001129 -2.514 0.01371 *
REG.Real_Disposable_Income_Growth -0.011284 0.003674 -3.072 0.00281 **
REG.Unemployment_Rate 0.060506 0.006767 8.942 4.18e-14 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.08086 on 91 degrees of freedom
Multiple R-squared: 0.6491, Adjusted R-squared: 0.6337
The model residuals pass stationarity tests, so I feel fairly confident that my model isn't misspecified. Below are my residuals plotted over time as well as the ADF test on the OLS residuals.
> adf.test(varLM$residuals)
Augmented Dickey-Fuller Test
data: varLM$residuals
Dickey-Fuller = -3.9243, Lag order = 4, p-value = 0.01605
alternative hypothesis: stationary
Now where my models got dinged is that the residuals show autocorrelation. Below is the Durbin-Watson test for this model:
Durbin-Watson test
data: varLM
DW = 1.0976, p-value = 1.709e-07
alternative hypothesis: true autocorrelation is greater than 0
A model validator could not get over that 1) my models had autocorrelation and 2) that I didn't use an arima structure. In fact they told me that the above model was basically garbage. On the autocorrelation point, I thought I could get around it using Newey-West Standard errors as I've read that serial correlation only affects the model's standard errors...
Regression with ARMA Errors
Based on this feedback, I investigated using a regression with ARMA errors to resolve the serial correlation issue in my models. I used the auto.arima function in R to fit an ARIMA model on my residuals:
> auto.arima(DepVar, xreg = modelFactors)
Series: DepVar
ARIMA(1,0,0) with non-zero mean
Coefficients:
ar1 intercept NAT.5_Year_Treasury_Yield NAT.House_Price_Index_Growth
0.4674 -0.2429 0.0075 -0.0029
0.0985 0.0894 0.0080 0.0017
REG.Real_Disposable_Income_Growth REG.Unemployment_Rate
-0.0078 0.0586
s.e. 0.0051 0.0101
sigma^2 estimated as 0.005294: log likelihood=118.34
AIC=-222.67 AICc=-221.4 BIC=-204.72
Comparing the coefficients on this model with the coefficients on the OLS model, I see little change. The obvious difference is now we are modeling the errors using an AR(1) process. The difference can be seen in the residuals which now look like true white noise.
Conclusion
Am I wrong to say that the OLS model is fine, despite the serial correlation of the residuals? My preference is to stick to a simpler model structure (OLS) because I'd prefer to avoid a more complex model if possible. The other issue with the ARMA model is that I"m building multiple - what I've shown here is a subset. And when use the auto.arima function, I'm getting a different mix of AR() and MA() terms for each model.
I guess in general, is there a good rule of them of when OLS is appropriate for times series and when an ARIMA model is preferrable?
AIC(mod)
)? That would provide you with a number that is designed to compare models, and would help make it clearer whether the ARMA error model was really a substantial improvement or whether the apparent improvement was within the range you'd expect to see from adding another, albeit fundamentally useless, parameter. In your case, though, given that thear1
parameter is quite large relative to its std. error., I'd expect that the ARMA error model really is better. $\endgroup$ – jbowman Jan 22 '18 at 23:13