0
$\begingroup$
  1. what is the purpose of adding the intercept in regression. why we are adding the bias.

  2. How we can predict if we have only dependent variable not any independent variable.

$\endgroup$
3
  • 1
    $\begingroup$ For the first question, check stats.stackexchange.com/questions/7948/… Could you clarify what you mean by the second case? No independent variables is intercept-only regression model (=mean of dependent variable). $\endgroup$ – Tim Dec 4 '18 at 7:53
  • $\begingroup$ Thanks Tim. I am very much satisfied with your answers. (Could you clarify what you mean by the second case) . My query was that why we are adding bias value to the our model. if i am not wrong intercept is the bias value. $\endgroup$ – karunesh Dec 8 '18 at 6:38
  • $\begingroup$ @Tim means your second question. Anyhow, your second Q seems unrelated to the first one and should be asked separately (after being clarified). $\endgroup$ – kjetil b halvorsen Jun 26 '20 at 18:08
1
$\begingroup$

For linear models, the intercept is the value of the linear predictor when all covariates are zero. In linear regression, this is equivalent to the y-intercept of the line of best fit. In logistic regression, it is the log odds of the baseline group. Suppose we did not add an intercept term for the regression. We would then be positing that when all covariates are 0, the linear predictor is 0. In summation, we add the bias to improve interpretability and add flexibility to the model.

Having no independent variables means no prediction can be made. Suppose you have a model for cancer incidence based on age. Since you do not know my age, you can not put my data into your model, and thus you can not make predictions.

$\endgroup$
0
0
$\begingroup$

The intercept represents the base assumption. Let's say you build an algorithm to classify cats and dogs where +1 is cat and -1 is dog. A model with an intercept of -1 will always assume the case is a dog unless enough variables prove it is a cat. This is especially likely to happen when you have severe class imbalance.

If your goal is performance, keep the intercept. If your goal is to determine feature importance, do not fit an intercept. By setting the base assumption to zero you force the model to select features for both cat and dog classes - no cheating!

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.