2
$\begingroup$

Let's say I have a equation like as below (linear regression)

Y = intercept + x1+x2+x3...xn

intercept = 30

positive coeff sum = 40

negative coeff sum = -10

So, the final outcome becomes like as below

30 + 40 - 10 = 60

My question is why does intercept have to be +30?

It could have been -30 as well?

Whether logistic or linear regression, how is the sign of intercept determined?

Does intercept only take positive values always?

$\endgroup$
3
  • 3
    $\begingroup$ ?? It doesn't have to be +30 or -30, this is what was estimated from your data, theoretically it could be any value. It doesn't have to be positive. $\endgroup$ Feb 16 at 10:59
  • 1
    $\begingroup$ Why the neural-networks tag? That makes me think you have some other question for which this question is a proxy. Do you know about the XY problem? $\endgroup$
    – Dave
    Feb 16 at 11:17
  • $\begingroup$ It's fine if you removed the neural-networks tag because it was a mistake to include it in the first place, but your original inclusion of that tag leads me to wonder if you have some other question for which this is a proxy. $\endgroup$
    – Dave
    Feb 16 at 17:44

1 Answer 1

1
$\begingroup$

It is determined by the estimation method you select. In OLS, the regression parameters $\hat\beta_{OLS}$ are determined by the following.

$$ \hat\beta_{OLS}=(X^TX)^{-1}X^Ty $$

Whatever the result of that calculation turns out to be is the answer.

Logistic regression does not have a closed-form equation for its parameter vector, but it can be calculated as the parameter vector that minimizes the crossentropy loss function (same as how OLS minimizes the square loss function, which turns out to be the same as the $\hat\beta_{OLS}$ above). Whatever the intercept value is in the parameter vector is the intercept in the logistic regression; any real number is possible.

$\endgroup$
13
  • $\begingroup$ is there any logic to this? I ask this because, my input variables explain only 30% of the outcome. And model has predicted the class probability to be 70%. So, intercept is 40.. $\endgroup$
    – The Great
    Feb 16 at 13:04
  • $\begingroup$ @TheGreat Logistic regression has its coefficients calculated a different way, and there is not a closed-form solution like there is in OLS. I gave that as an example of how the intercept is whatever the intercept in the parameter vector is that minimizes the loss function (square loss in OLS). // What do you mean that the input variables explain $30\%$ of the outcome? How do you calculate that? // What do you mean that the predicted class probability is $70\%?$ Is that for one observation? What about the other observations? // Do you mean an intercept of $40$ because $70-30=40?$ $\endgroup$
    – Dave
    Feb 16 at 13:11
  • $\begingroup$ I have put a bounty on the below uestion. If you have time and are interested, can help me with the related question - $\endgroup$
    – The Great
    Feb 16 at 13:29
  • $\begingroup$ datascience.stackexchange.com/questions/107928/… $\endgroup$
    – The Great
    Feb 16 at 13:30
  • $\begingroup$ How are you getting the $30$ and $70?$ $\endgroup$
    – Dave
    Feb 16 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.