# Robust regression - a better understanding

I looked at robust regression for the first time today and I am a bit confused, comparing it to something like ordinary least squares and I am not sure if I am on the right track.

I read a few articles and they say that with robust regression you don't need to worry too much about outliers and heteroscedasticy and the normality restrictions on the residuals is not as important as with OLS. When I do robust regression in R however, I also don't get any significant indicators regarding coefficients (I am using the rlm() function). So when robust regression like M or MM-estimation is applied in a regression model, is the significance of coefficients still important or is the idea of robust regression just to fit the best possible plane through the data points and find the coefficients?

• use the robustbase package instead. Its more more recent and contains the information (t statistics) you seek. Apr 23, 2014 at 19:34

To be somewhat nitpicky, I would not quite say that outliers, heteroscedasticity, and non-normality don't matter with robust regression methods. Rather, I would say that robust methods are less likely to be impaired or harmed by those conditions. However, they could still have a negative effect.

The issue of whether the significance of the coefficients or the accuracy of their estimation is what's important is really unrelated to robust regression. Which of those is more important to you depends on the questions you are trying to answer, not what tools you use to try to answer them. Instead, consider a case where you want to test the hypothesis that a given variable is unrelated to the response variable. You wouldn't want the answer you get to that question (either yes or no) to be driven by an outlier. So you would use robust methods to help ensure that your answer is representative of the bulk of your data. Likewise, consider a case where you want to know the slope of the relationship between a predictor variable and the response variable as accurately as possible. You wouldn't want the estimated slope value that you get to have been driven by an outlier. So you would use robust regression to protect against that possibility. In short, robust methods diminish the extent to which your results might be influenced by violations of the classical statistical assumptions.

I recognize your frustration that you did not get any significant results when you used these methods. There are a couple of possibilities here. It may be that what appeared to be the case prior to using robust regression (perhaps the results from a prior OLS regression analysis) were driven by violations of the OLS assumptions and the null hypothesis is actually true. The other possibility is that, when OLS assumptions do hold, standard methods will have more power than robust methods.

With OLS regression, when all the assumptions hold, then the t-statistics will follow a t distribution (when the null is true) and the p-value is straight forward to compute.

With the robust regressions it is no longer straight forward and we don't really know what the distribution of the "t" statistic is, so it is better to not provide a p-value then to provide one that is most likely wrong (but may be trusted if seen). You can still use the rule of thumb that "t" statistics near 0 are probably not significant and ones far from 0 are probably significant, but we don't really know where the divide is between "close" and "far". One option for producing p-values is to bootstrap the process, see the boot package for tools that will help with this.

• you don t want to use a generic bootstrap package to bootstrap robust estimates. There are enormously more efficient ways to get bootstrapped CI's (computer wize) for robust estimated. Have a look at the frb package and this joss paper Apr 24, 2014 at 6:55
• @user603 In many cases using a generic package is not only sufficient but the better choice. It's not worth it to spend some hours of a scientist just to shorten a computer calculation by a few minutes. Only when you start doing very similar things a lot of time it is worth to look at efficiency.
– Erik
Apr 24, 2014 at 7:02
• @Erik: you might want to read the JOSS paper before commenting. We're talking of 2 orders of magnitude saving in computing times with no compromise on the validity of the bootstraped results. Apr 24, 2014 at 9:15
• @user603 A saving in two orders of magnitude only matters if the computing time is significant to start with. If I have to do it once for a paper I don't care if the calculation takes 5 minutes or 5 seconds. My time is much more valuable than that of the computer.
– Erik
Apr 24, 2014 at 9:26
• @user603 What do I care about exponential time complexity in the number of samples when I never have more than say 10? Listen, there are times and places where execution speed matters and I once wrote myself a fast C function to bootstrap AUCs of classifiers since I had to do it for a lot of them. But those are special cases. I don't think using special tools for everything is good general advice. And I think we should stop this conversation now, since it does not really contribute to the answer anymore.
– Erik
Apr 24, 2014 at 9:42