Adjust the "Threshold" in a robust regression

Question

I am trying to perform a robust regressions using the lmrob function in R. I am getting this error Message:

Warning message:
In lmrob.S(x, y, control = control) :
  S-estimated scale == 0:  Probably exact fit; check your data

Then I see that 244 of my observations are weighted with 0, the other 355 are weighted with 1 (see plot as illustration). My criterion variable is a continuous variable ranged 1-5, but with a lot of mass at 1. In fact, any other value than the 1 was considered as an outlier in the lmrob model.

Regardless of whether it is appropriate to finally rely on a linear regression model given these circumstances, is there an intuitive way to adjust the "threshold" to consider an observation as outlier when using this function? I have never worked with such a model or this function before, but it seems like a potential solution to me to increase the threshold in a way that prevents the model from considering half of the sample as outliers. However, the explanations in the help sections are too technical for me.

Or is it more appropriate to exclude outliers before fitting the model? The reason I fitted the robust model was to compare it to the results of a normal OLS regression and a bootstrap regression.

Answer 1

Robust regression methods with breakdown point close to 50% have the "exact fit property", meaning that if more then 50% of observations can be fitted with variance zero, this is the fit that will be chosen. All the other observations will then be identified as outliers, as the identification is relative to the estimated error variance.

The reason why this happens here is that you have so many $y$-values equal to 1 that apparently a regression constant 1 can fit more than half of the observations perfectly. This is what the warning message indicates; discrete data like these are a problem for the S/MM robust regression estimator.

Assuming that a linear regression still makes sense (which is questionable), it would be better to tune the initial S-estimator to have a lower breakdown point than 50%, i.e., to enforce that more then 50% of observations need to be taken into account. Unfortunately the documentation is not very clear on how to do this. If I understand things correctly, the parameter tuning.chi of lmrob.control (control argument in lmrob) needs to be set larger than the default of 1.54764 to achieve this. I can't tell you how precisely to choose it, but if I were you I'd try out 3 (it should be smaller than the tuning.psi-value of 4.685) and see what happens, maybe then look at 2.5 or 3.5 (I think one should try to achieve 25% or 20% breakdown, i.e., enforce the initial S-estimator to take into account at least 75 or 80% of observations; there is a mathematical way how to do this which is probably at least indirectly explained in the "Robust Statistics Theory and Methods" book by Maronna, Martin, and Yohai; the larger tuning.chi, the more observations are taken into account, but unfortunately it's not a linear relation). It'd be experimental on your behalf as I don't have the time to test this right now.

Do not remove outliers before fitting. It may however be that you don't have an outlier problem (the robust outlier identification is apparently led astray by the "exact fit") and OLS-estimation may work just fine for these data.