6
$\begingroup$

I was going back through linear regression and I found two equations for finding the best fit slope given X and y. This is one: $$m=\frac{\sum_{i=0}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=0}^{n}(x_i - \bar{x}^2)}$$ and this is the other: $$m=\frac{\bar{x}.\bar{y}-\bar{xy}}{\bar{x}^2 - \bar{x^2}}$$ I programmed both of them with Numpy and tested the second implementation with the same set of numbers and got the same slope value, but the first implementation yielded different results). What's the difference between those three and which should I use?

Here is my implementation of the first equation:

m = (np.sum(np.dot(X - np.mean(X), y - np.mean(y)))) / (np.sum(np.square(X - np.mean(X))))

and the second:

m = (np.dot(np.mean(X), np.mean(y)) - np.mean(np.dot(X, y))) / (np.square(np.mean(X)) - np.mean(np.square(X)))
$\endgroup$
3
  • $\begingroup$ What are the sums in your second line of code? There are no sums in your question. $\endgroup$
    – Dave
    Commented Jul 20, 2021 at 15:55
  • $\begingroup$ en.wikipedia.org/wiki/Simple_linear_regression $\endgroup$ Commented Jul 20, 2021 at 16:06
  • $\begingroup$ @Dave yeah sorry I mixed them up, the sums are in the first equation. $\endgroup$
    – Kareem Amr
    Commented Jul 20, 2021 at 17:32

1 Answer 1

8
$\begingroup$

It looks like there are a couple of problems.

First, the python code for the 1st equation does not implement it correctly. The first line of code (for the first equation in the OP) should be

m = (np.sum(np.dot(X - np.mean(X), y - np.mean(y)))) / np.sum(X - np.square(np.mean(X)))

However the first equation is wrong. It's not:

$$m=\frac{\sum_{i=0}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=0}^{n}(x_i - \bar{x}^2)}$$

It's

$$ m=\frac{\sum_{i=0}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=0}^{n}(x_i - \bar{x})^2} $$

So in (Num)Python that would be:

m = (np.sum(np.dot(X - np.mean(X), y - np.mean(y)))) / (np.sum(np.square(X - np.mean(X))))
$\endgroup$
3
  • $\begingroup$ Thank you for your help Robert, I implemented the first equation correctly I guess I just wrote it in Latex wrong, my original question was what is the difference, if any, between the two equations as to getting the best fit slope? $\endgroup$
    – Kareem Amr
    Commented Jul 22, 2021 at 1:25
  • $\begingroup$ You're welcome. They are equavalent (just expand the brackets in the first one and rearrange). There is a comutational benefit in using the 2nd one, since it does not require any summations and therefore will be more efficient. $\endgroup$ Commented Jul 22, 2021 at 10:46
  • $\begingroup$ Does this answer your question ? If so please consider marking it as the accepted answer. If not, please let us know why. Also, if you haven't already, please consider upvoting it $\endgroup$ Commented Aug 21, 2021 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.