# Outliers change the sign of linear relation

I have a scatter plot with Hgb A1C as the explanatory variable ('x' axis) and FPG ('y' axis)as the result variable. There is a positive linear relationship but it is weak. There is 1 outlier in the y direction. There is 1 outlier in the x direction. When each of these points is removed, r changes in opposite directions. I am trying to explain why this is happening, in simple language.

Would it make sense to say: When the outlier in the y direction is removed, r increases because an outlier that normally falls a distance away from the regression line would decrease the size of the correlation coefficient.

When the outlier in the x direction is removed, r decreases because an outlier that normally falls near the regression line would increase the size of the correlation coefficient.

Does this make sense?

-
Yes, I think you're quite right. And overplotting regression lines on four scatterplots (all points, only Y outlier removed, only X outlier removed, and both outliers removed) should illustrate. –  AKE Oct 6 '12 at 4:16

This is usually called the leverage problem; it is easiest to understand in the language of optimization in which regression is done.

Basically, we want to have such a line that sum of squared residuals (distances on y between points and regression line) is minimal; because of that a single point with extreme coordinates may have a dominant contribution to this sum.
Here is an example of such a situation. Green line seems more correct, but its cost is huge because of the outlier's residual (black) -- because of that the red line is generated.

In your case, the line is simply made between those two outliers effectively ignoring or treating the rest of points as one.

-