I have created regression models using robust regression - in particular, LTS and MM-estimators (using the R package robustbase). I am now looking to creation prediction intervals.

The standard formula for prediction intervals for linear regression is: $$ \hat{y_0} \pm t_{\alpha/2, n-p} \sqrt{\hat{\sigma^2}(1+x_0'(X'X)^{-1}x_0)} $$ (see Montgomery and Peck, Introduction to Linear Regression Analysis, 1992)

For robust regression, obviously, the term $x_0'(X'X)^{-1}x_0$ cannot be used. The Hat Matrix is different due to the weights. It would seem to me that we can instead use the Hat Matrix modified with the weights:

$$ x_0'(X'WX)^{-1}x_0 $$

(see page 44 of the PhD thesis by Christopher Assaid at Virigina Tech)(http://scholar.lib.vt.edu/theses/available/etd-3649212139711101/unrestricted/Ch6.PDF)

We can approximate $\hat{\sigma^2}$ from the data as $$ \hat{\sigma^2} = \frac{1}{df}\sum e_i^2 $$ where $df$ are the number of degrees of freedom.

I have three questions on this formulation for prediction intervals:

Is my formula correct? Is the simple adjustment by factoring in the Weight Matrix enough to adapt the OLS formula for prediction intervals to robust regression.

If it is correct, does it apply to all types of robust regression, or just a subset?

Is it correct to estimate the variance as above? If so, it would seem to me that robust regression will always have a larger variance, and thus larger prediction intervals, than OLS. The reason is is that OLS, by definition, is set up to minimize the residual sum of squares. Robust regression, on the other hand, by definition of down-weighting potential outliers, even though it may give an overall better fit, will see a larger net residual sum of squares because of the contribution of the squared residuals from the outlier points. Consequently, since the length of the interval is $\sqrt{\hat{\sigma}^2(1+\delta)}$ (where granted $\delta$ is not necessarily small, but the interval is always $\sigma$ plus something), if $\hat{\sigma}_{RR} > \hat{\sigma}_{OLS}$, in general, the length of the interval for RR will be larger. It seems counter-intuitive to me that if data is fit with both OLS and robust regression and prediction intervals are made, those from OLS will be by definition narrower and may even be contained within the robust ones. It seems to thus minimize the power of robust regression.

Any answers to these questions or other suggestion/advice on creating prediction intervals for LTS and MM regression would be appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.