Will the coefficient of determination $R^2$ increase if we remove the outliers in linear regression? Or will the correlation coefficient $p$ would get closer to $0$?
$\begingroup$
$\endgroup$
2
-
3$\begingroup$ Removing the outliers will result in a better fit of the regression line, and this should increase the value of $R^2$. For a detailed discussion, please see aip.scitation.org/doi/pdf/10.1063/1.4907473?class=pdf $\endgroup$– Nash J.Commented Dec 28, 2019 at 20:09
-
2$\begingroup$ Did you try yourself on some examples? What did you see? $\endgroup$– kjetil b halvorsen ♦Commented Dec 29, 2019 at 15:50
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
It depends on what kind of outlier / the location of the outlier with respect to the regression line. Imagine a bivariate data cloud without the outlier. X and Y covary, so the cloud is oblong. Draw the regression line. If the outlier lies along that regression line, but far from the cloud so that the observation has high leverage, then the outlier strengthens the X-Y relationship--indeed, one outlier along the regression line could produce a "significant" regression relationship even if X and Y are otherwise orthogonal (spherical). But an outlier lying far off the regression line will pull the regression line away from the other observations, worsening fit and biasing parameter estimates. Your Studentized-deleted residuals will tell you if the observation is likely having such an effect.
If you have multiple outliers, it is more difficult to tell the effect of removing all of them at the same time.
-
$\begingroup$ I'm going to quibble with the middle third of your main paragraph, of an otherwise good answer. Such an X-outlier alongside a purely orthogonal relationship would lie along a fit line that is horizontal at the mean of Y. It would extend that fit line without improving fit. Such an outlier would not substantially change b or p. I simulated these conditions and found as much (details on request: [email protected]). $\endgroup$– rolando2Commented Apr 10, 2020 at 19:45