Suppose I have fitted an standard linear regression mode $Y=\beta_0+\beta_1X_1+\beta_2X_2+\beta_3X_3+\epsilon$. Based on the ACF plot or PACF plot of the residuals for this regression model, I found that the $\epsilon$'s are serially auto-correlated that can be modeled as an AR(1) i.e. $\epsilon=\phi\epsilon_{t-1}+\nu_t$, where $\nu_t$ are white noise. I decided to fit the above model with Generalized least squares approach with the following code

model.gls=gls(Y~X_1+X_2+X_3,data=dat, correlation=corARMA(p=1), method="ML")

A part of output in R is as follows:

Correlation Structure: AR(1)
 Formula: ~1 
 Parameter estimate(s):

                Value  Std.Error    t-value p-value
(Intercept) -3.172158 0.01504180 -210.88950  0.0000
x1          -3.890222 0.14141947  -27.50839  0.0000
x2           0.089368 0.11667430    0.76596  0.4456
x3          -0.155348 0.10190694   -1.52441  0.1308

Clearly $X_2$ and $X_3$ are not significant based on the t-test.

Q1: Is it a correct approach to drop $X_2$ and $X_3$ and refit a new Generalized least squares model with the same correlation structure (i.e. AR(1))? In this case the code should be like:

model.gls=gls(Y~X_1,data=dat, correlation=corARMA(p=1), method="ML")

I am asking this because the AR(1) has been selected while all $X_1$, $X_2$ and $X_3$ have been included in the linear regression model. So dropping two terms may change the correlation structure (i.e AR(1)) to another ARMA model.

If the approach in Q1 is not correct, how can we proceed and handle the non-significant terms, $X_2$ and $X_3$ in the fitted Generalized least squares?

  • 1
    $\begingroup$ There is no reason to drop "insignificant" terms, and it will create a bias to do so. $\endgroup$ – Frank Harrell Dec 19 '12 at 22:06
  • $\begingroup$ Can you provide any reference to support your comment? $\endgroup$ – Stat Dec 19 '12 at 22:13
  • 1
    $\begingroup$ @FrankHarrell - isnt that the whole idea of dropping insignificant terms? You trade increased bias for decreased variance. $\endgroup$ – probabilityislogic Dec 19 '12 at 23:10
  • 1
    $\begingroup$ Do you have an a apriori reason for believing that you have AR(1) error? Is Y a time series, where you might expect a carry-over from one observation to the next? If so, continue to model it. Other point: the p-values of individual coefficients can mislead -- not always the best way to do variable selection. Either one of X2 or X3 might be significant "were other dear charmer away." $\endgroup$ – Placidia Dec 20 '12 at 1:11
  • $\begingroup$ I checked the ACF and PACF of the residuals in the linear model and AR(1) seems to be a good model for that. Yes, $Y$ is a time series. What do you mean by "continue to model it"? I think my questions are clear ... $\endgroup$ – Stat Dec 20 '12 at 1:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.