1
$\begingroup$

I'm new to much of this and find that a good way to wrap my head around some of these concepts is to calculate them by hand. After working through linear regression, I thought multiple regression would be straightforward because I read that multiple regression is the linear combination of the independent variables. But I can't seem to learn how to calculate the y-intercept ($b_0$) for multiple regression. Maybe I'm using the wrong terminology because I'm finding this difficult to google.

For simple linear regression, the y-intercept is:

$b_0 = \bar y -b_1\bar x$

The equation for multiple regression is:

$ \hat y = b_0 + b_1x_1 + b_2x_2 +b_3x_3... $

I don't understand where the $b_0$ comes from in the multiple regression equation because wouldn't we have a different $b_0$ for every independent variable? How do they turn into a single y-intercept?

Thanks in advance!

$\endgroup$
2
  • $\begingroup$ I would have thought $b_0 = \bar y -b_1\bar x_1 -b_2\bar x_2 -b_3\bar x_3 - \cdots$ would be plausible in the multiple regression. The hypersurface should pass though the mean of the regression data $\endgroup$
    – Henry
    Commented Jan 9, 2023 at 1:38
  • $\begingroup$ You might feel better thinking of the intercept being the sum of those “one intercept for each feature”, since they would just be numbers. $\endgroup$
    – Dave
    Commented Jan 9, 2023 at 1:59

1 Answer 1

1
$\begingroup$

The intercept is just a free scalar parameter that is optimized to minimize the squared error loss (i.e. find the best fit line).

Note that optimizing $b_0$ in the fit: $$\hat{y} = b_0 + b_1 x_1 + b_2 x_2 + ...$$ is actually just as flexible as optimizing $b_{0,1}, b_{0,2}, ...$ in the fit: $$\hat{y} = b_{0,1} + b_1 x_1 + b_{0,2} + b_2 x_2 + ...$$

because we could always simply define: $b_0 = b_{0,1} + b_{0,2} + ...$.

So there is no additional benefit that would come from having a different intercept for each independent variable, the resulting squared error would not be any lower.

$\endgroup$
1
  • $\begingroup$ Saved my day. I was following a book and it kinda jumped without this equation. Thanks! $\endgroup$ Commented May 10 at 1:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.