2
$\begingroup$

I'm posting this in this forum because I think the question is related to stats. If it's not, I'm fine with it being moved over to Stackoverflow.

I have data in R and am trying to fit a linear model to it. Here's what the data looks like (sorry that it's not reproducible, it's just too much data to type out). The colored dots are based on density of points. The black line is what a linear model is returning as the best fit line (via lm(y~x) which spits out a slope of 0.67 and an intercept of 0.002) and the dashed line is what I would think the best fit line should be (slope of 5, intercept of -3).

Why is R's lm method giving me a line which looks like it doesn't fit at all. Is it true that that line is better fitting than the dashed line I propose?

enter image description here

Improve this question
$\endgroup$
2
  • $\begingroup$ Looks right to me, considering that the y vs x relationship looks very weak. Your dashed line is predicting that for x values below .6 and above .8, y will be below 0 and above 1 respectively, which is always false. (Indeed, that will have an enormous residual for high values below .6 and low values above .8, like those red values around x=.85). $\endgroup$
    – David Robinson
    Nov 25, 2013 at 1:46
  • 1
    $\begingroup$ If you want to make the data reproducible, you could post dput(data), which would probably be large but doable. $\endgroup$
    – David Robinson
    Nov 25, 2013 at 1:47

3 Answers 3

Reset to default
5
$\begingroup$

Your dashed line doesn't look like it's least squares (minimizing the sum of squares of the vertical distances) to me, but more like a line that attempts to minimize the orthogonal distances. To get an idea where the LS line should go, divide the range of x's into vertical strips and find the average of y in each strip.

enter image description here

If a straight line is appropriate those averages should lie relatively close to a straight line... the regression line.

Below I have taken your plot and marked two such strips (delimited by two red lines on the strip on the left and two purple lines for the strip on the right):

enter image description here

I've also marked a rough (by eye) guess at where the mean y in each strip is, and indicated it with a "+" of the corresponding color.

As you see they both lie close to the regression line, and nowhere near your line.

So the regression line R gave you looks just about exactly right to me.

Now, if your data are bounded by 0 and 1 ... why on earth would you fit a straight line? How can that be right?

Improve this answer
$\endgroup$
1
$\begingroup$

As you say, the points have different density in the data set. That means the data points have different weights in the lm model.

Maybe you can compare the R squares between the R's lm model and the lm model you propose.

By visually, not taking different density of different colour into account, it seems that R's model is more reasonable. There is not clear upward trend about the scatter plot, informing that the slope of the line should be small.

Improve this answer
$\endgroup$
0
$\begingroup$

This isn't an issue of the data (either different density or providing reproducible data), but rather an understanding of what OLS does.

Minimizing the cost function in OLS will never produce a result like the dotted line. You'll need to extend the y-axis down to -1 to better understand this visually and remember what the cost function in OLS is.

In some textbooks you'll see graphics like this where both the x and y-axis represent IV/predictors and then you'll see the cost function producing results like this, so this can cause confusion.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.