0
$\begingroup$

I'm new to much of this and find that a good way to wrap my head around some of these concepts is to calculate them by hand. After working through linear regression, I thought multiple regression would be straightforward because I read that multiple regression is the linear combination of the independent variables. But I can't seem to learn how to calculate the y-intercept ($b_0$) for multiple regression. Maybe I'm using the wrong terminology because I'm finding this difficult to google.

For simple linear regression, the y-intercept is:

$b_0 = \bar y -b_1\bar x$

The equation for multiple regression is:

$ \hat y = b_0 + b_1x_1 + b_2x_2 +b_3x_3... $

I don't understand where the $b_0$ comes from in the multiple regression equation because wouldn't we have a different $b_0$ for every independent variable? How do they turn into a single y-intercept?

Thanks in advance!

$\endgroup$
2
  • $\begingroup$ I would have thought $b_0 = \bar y -b_1\bar x_1 -b_2\bar x_2 -b_3\bar x_3 - \cdots$ would be plausible in the multiple regression. The hypersurface should pass though the mean of the regression data $\endgroup$
    – Henry
    Jan 9 at 1:38
  • $\begingroup$ You might feel better thinking of the intercept being the sum of those “one intercept for each feature”, since they would just be numbers. $\endgroup$
    – Dave
    Jan 9 at 1:59

1 Answer 1

0
$\begingroup$

The intercept is just a free scalar parameter that is optimized to minimize the squared error loss (i.e. find the best fit line).

Note that optimizing $b_0$ in the fit: $$\hat{y} = b_0 + b_1 x_1 + b_2 x_2 + ...$$ is actually just as flexible as optimizing $b_{0,1}, b_{0,2}, ...$ in the fit: $$\hat{y} = b_{0,1} + b_1 x_1 + b_{0,2} + b_2 x_2 + ...$$

because we could always simply define: $b_0 = b_{0,1} + b_{0,2} + ...$.

So there is no additional benefit that would come from having a different intercept for each independent variable, the resulting squared error would not be any lower.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.