3
$\begingroup$

I'm working with a time series of macroeconomic data (independent variables) and bank loss rates (dependent variable) to show how bank losses vary based on the state of the economy.

To estimate the relationship between the dependent and independent varaibles, I am considering two regression techniques: standard OLS and Regression with ARMA errors as outlined in this extremely helpful article https://robjhyndman.com/hyndsight/arimax/.

When comparing the coefficients and residuals calculated by the two methodologies, I am struggling to interpret which is preferable. I'll walk you through my thought process with the hope that others can better intuit which is the preferable model.

OLS Model

When I estimate the model using OLS, I get a decently fit model (63% Adj R-squared) with all coefficients showing some degree of significance:

> summary(varLM)

Call:
lm(formula = DepVar ~ modelFactors)

Coefficients:
                                   Estimate Std. Error t value  Pr(>|t|)    
(Intercept)                       -0.260879   0.061043  -4.274  4.73e-05 ***
NAT.5_Year_Treasury_Yield          0.012234   0.005059   2.418   0.01759 *  
NAT.House_Price_Index_Growth      -0.002837   0.001129  -2.514   0.01371 *  
REG.Real_Disposable_Income_Growth -0.011284   0.003674  -3.072   0.00281 ** 
REG.Unemployment_Rate              0.060506   0.006767   8.942  4.18e-14 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.08086 on 91 degrees of freedom
Multiple R-squared:  0.6491,    Adjusted R-squared:  0.6337

The model residuals pass stationarity tests, so I feel fairly confident that my model isn't misspecified. Below are my residuals plotted over time as well as the ADF test on the OLS residuals.

OLS Residuals

> adf.test(varLM$residuals)

Augmented Dickey-Fuller Test

data:  varLM$residuals
Dickey-Fuller = -3.9243, Lag order = 4, p-value = 0.01605
alternative hypothesis: stationary

Now where my models got dinged is that the residuals show autocorrelation. Below is the Durbin-Watson test for this model:

Durbin-Watson test

data:  varLM
DW = 1.0976, p-value = 1.709e-07
alternative hypothesis: true autocorrelation is greater than 0

A model validator could not get over that 1) my models had autocorrelation and 2) that I didn't use an arima structure. In fact they told me that the above model was basically garbage. On the autocorrelation point, I thought I could get around it using Newey-West Standard errors as I've read that serial correlation only affects the model's standard errors...

Regression with ARMA Errors

Based on this feedback, I investigated using a regression with ARMA errors to resolve the serial correlation issue in my models. I used the auto.arima function in R to fit an ARIMA model on my residuals:

> auto.arima(DepVar, xreg = modelFactors)
Series: DepVar 
ARIMA(1,0,0) with non-zero mean 

Coefficients:
     ar1  intercept  NAT.5_Year_Treasury_Yield  NAT.House_Price_Index_Growth
  0.4674    -0.2429                     0.0075                       -0.0029
  0.0985     0.0894                     0.0080                        0.0017
        REG.Real_Disposable_Income_Growth  REG.Unemployment_Rate
                             -0.0078                 0.0586
 s.e.                         0.0051                 0.0101

sigma^2 estimated as 0.005294:  log likelihood=118.34
AIC=-222.67   AICc=-221.4   BIC=-204.72

Comparing the coefficients on this model with the coefficients on the OLS model, I see little change. The obvious difference is now we are modeling the errors using an AR(1) process. The difference can be seen in the residuals which now look like true white noise. ARMA Error Model Residuals

Conclusion

Am I wrong to say that the OLS model is fine, despite the serial correlation of the residuals? My preference is to stick to a simpler model structure (OLS) because I'd prefer to avoid a more complex model if possible. The other issue with the ARMA model is that I"m building multiple - what I've shown here is a subset. And when use the auto.arima function, I'm getting a different mix of AR() and MA() terms for each model.

I guess in general, is there a good rule of them of when OLS is appropriate for times series and when an ARIMA model is preferrable?

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ Can you get the AIC for the OLS model (AIC(mod))? That would provide you with a number that is designed to compare models, and would help make it clearer whether the ARMA error model was really a substantial improvement or whether the apparent improvement was within the range you'd expect to see from adding another, albeit fundamentally useless, parameter. In your case, though, given that the ar1 parameter is quite large relative to its std. error., I'd expect that the ARMA error model really is better. $\endgroup$ – jbowman Jan 22 '18 at 23:13
  • $\begingroup$ > AIC(Linear Model) = -203.577, so the ARMA error model does show improvement over the linear model. $\endgroup$ – RLH Jan 23 '18 at 15:09
  • $\begingroup$ ... and what's more, a big improvement. Improvements on the order of 1 or 2 are pretty marginal, but that's an improvement of about 18 points. I would treat a jump that big as definitive - not necessarily with respect to the ARMA process being AR(1), but with respect to there being an ARMA process at all. You could just fit all your models forcing an AR(1) or an ARMA(1,1) model on the residuals if you are concerned about different models for different data sets, that would probably cost you very little. $\endgroup$ – jbowman Jan 23 '18 at 15:34
2
$\begingroup$

Am I wrong to say that the OLS model is fine?

I'd say no, you're not wrong, but it does depend on what you mean by "fine." An economist introduced me to the book "A Guide to Econometrics" by Peter Kennedy, which is a really nice book for telling you what happens when classical assumptions are violated but you use the classical technique anyways. Usually the sky does not fall, but you do lose something.

In the case of regression with autocorrelated errors, Section 7.4 states:

"With positive first-order autocorrelated errors this implies that several succeeding error terms are likely also to be positive, and once the error term becomes negative it is likely to remain negative for a while...In repeated samples these estimates will average out, since we are as likely to start with a negative error as with a positive one, leaving the OLS estimator unbiased, but the high variation in these estimates will cause the variance of [the OLS estimator] to be greater than it would have been had the errors been distributed randomly."

Additionally, it goes on to mention that the variance of beta_OLS is also underestimated, which of course will mess up your test statistics. But you still have unbiasedness!

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your thorough answer. My current solution to deal with the autocorrelated errors has been to calculate the test statistics using Newey-West robust standard errors. My argument has essentially been that my coefficients are unbiased, rather it's the standard errors that are being affected by autocorrelation. Thus robust standard errors treat the problem. Do you think that is sufficient? $\endgroup$ – RLH Jan 23 '18 at 15:50
  • $\begingroup$ Newey-West basically estimates the covariance matrix of the errors using a simplified model of same. You are just hiding the estimation in the Newey-West part of the overall estimation procedure instead of making it explicit in the ARMA modeling of the errors. Your results will be different, as Newey-West doesn't use an ARMA model of the errors, but it's not as though no estimation is occurring at all. It's up to you as to which approach to take. $\endgroup$ – jbowman Jan 23 '18 at 16:39
  • $\begingroup$ @RLH, it's pretty standard to only adjust the standard errors. You are "leaving something on the table" however - OLS is no longer the lowest variance estimator. I met (another) economist who wondered why standard statistical packages ever printed traditional standard errors, rather than a variant that was robust to heteroskedasticity. "You get it almost for free," he said. While I think the strategy might be more common in econ, GEE modeling is one place in statistics that the strategy is employed. $\endgroup$ – Ben Ogorek Jan 24 '18 at 0:47
  • $\begingroup$ "But you still have unbiasedness" Is this correct? I was certain that when the errors and predictors are correlated, this results in biased (but still produces asymptotically consistent) estimates. E.g. en.wikipedia.org/wiki/Omitted-variable_bias $\endgroup$ – Confounded Apr 3 at 14:55
  • $\begingroup$ @Confounded, if the incorporation of the lagged response variable removes the serial correlation, then the model error at time t is uncorrelated with the "predictor," i.e., the response at t-1. In that case you'd be ok from a bias standpoint. But you're right to be skeptical, because if after adding the lagged response there's still some residual correlation left over, then you've got endogeneity bias. That lagged regression model had better be pretty good at whitening the residuals. $\endgroup$ – Ben Ogorek Apr 4 at 14:01
0
$\begingroup$

Simple regression using OLS, or any modeling technique, is primarily used to predict the relationship among population parameters, using sample data. When you want to use OLS on the sample data, and extrapolate your relationship to the population data, there are certain assumptions which are required. For. eg: the errors are independent, variance of errors is constant, etc. Unless these assumptions are satisfied, your results from sample data cannot be used for making inference on the relationship between population parameters.

The fact that the errors in OLS are autocorrelated indicate that your model, which you built using the sample data, cannot be used to comment on the relationship between population parameters.

If you have used all the entire data in your population for your model, your R-square may be high - indicating some sort of linear relationship. But, even then model cannot be used for forecasting accurately for the future - meaning that one cannot be sure of the validity of coefficients of the regression output on unseen future data.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

You are perfectly fine to use the OLS Method. I would just recommend utilizing robust p-values to show that your variables are still significant and running Bruesch-Pagan, Durbin-Watson and ADF tests against your results to further show the model is solid. No one model is ever better than another, it all depends upon the context of the utilization of the model. The approach I have shared is current and common practice across the financial industry.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.