2
$\begingroup$

I am trying to perform a robust regressions using the lmrob function in R. I am getting this error Message:

Warning message:
In lmrob.S(x, y, control = control) :
  S-estimated scale == 0:  Probably exact fit; check your data

Then I see that 244 of my observations are weighted with 0, the other 355 are weighted with 1 (see plot as illustration). My criterion variable is a continuous variable ranged 1-5, but with a lot of mass at 1. In fact, any other value than the 1 was considered as an outlier in the lmrob model.

Regardless of whether it is appropriate to finally rely on a linear regression model given these circumstances, is there an intuitive way to adjust the "threshold" to consider an observation as outlier when using this function? I have never worked with such a model or this function before, but it seems like a potential solution to me to increase the threshold in a way that prevents the model from considering half of the sample as outliers. However, the explanations in the help sections are too technical for me.

Or is it more appropriate to exclude outliers before fitting the model? The reason I fitted the robust model was to compare it to the results of a normal OLS regression and a bootstrap regression.

enter image description here

$\endgroup$
2
  • $\begingroup$ How many variables do you have? $\endgroup$ Apr 19, 2023 at 23:22
  • $\begingroup$ What have you specified for this object you created control? $\endgroup$
    – AdamO
    Apr 19, 2023 at 23:34

1 Answer 1

3
$\begingroup$

Robust regression methods with breakdown point close to 50% have the "exact fit property", meaning that if more then 50% of observations can be fitted with variance zero, this is the fit that will be chosen. All the other observations will then be identified as outliers, as the identification is relative to the estimated error variance.

The reason why this happens here is that you have so many $y$-values equal to 1 that apparently a regression constant 1 can fit more than half of the observations perfectly. This is what the warning message indicates; discrete data like these are a problem for the S/MM robust regression estimator.

Assuming that a linear regression still makes sense (which is questionable), it would be better to tune the initial S-estimator to have a lower breakdown point than 50%, i.e., to enforce that more then 50% of observations need to be taken into account. Unfortunately the documentation is not very clear on how to do this. If I understand things correctly, the parameter tuning.chi of lmrob.control (control argument in lmrob) needs to be set larger than the default of 1.54764 to achieve this. I can't tell you how precisely to choose it, but if I were you I'd try out 3 (it should be smaller than the tuning.psi-value of 4.685) and see what happens, maybe then look at 2.5 or 3.5 (I think one should try to achieve 25% or 20% breakdown, i.e., enforce the initial S-estimator to take into account at least 75 or 80% of observations; there is a mathematical way how to do this which is probably at least indirectly explained in the "Robust Statistics Theory and Methods" book by Maronna, Martin, and Yohai; the larger tuning.chi, the more observations are taken into account, but unfortunately it's not a linear relation). It'd be experimental on your behalf as I don't have the time to test this right now.

Do not remove outliers before fitting. It may however be that you don't have an outlier problem (the robust outlier identification is apparently led astray by the "exact fit") and OLS-estimation may work just fine for these data.

$\endgroup$
1
  • $\begingroup$ Thank you for the explanation. It makes sense, but I did not manage to solve the problem by experimenting with the tuning parameters (my fault). As I was more concerned about heteroscedasticity than about the outliers per so, I chose a different approach that was said to deal with my issues (using robust estimators from the "sandwich" package). $\endgroup$ Apr 20, 2023 at 11:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.