2
$\begingroup$

I'm relatively new to regression analysis in Python. I'm running a logistic regression on a dataset in a dataframe using the Statsmodels package.

I've seen several examples, including the one linked below, in which a constant column (e.g. 'intercept') is added to the dataset and populated with 1.0 for every row. And then the intercept variable is included as a parameter in the regression analysis.

My question is: what is the purpose of this, and is it necessary? (How do I know if it's necessary?)

(Reference: Logistic Regression: Scikit Learn vs Statsmodels)

Thank you!

$\endgroup$
9
$\begingroup$

It is almost always necessary. I say almost always because it changes the interpretation of the other coefficients. Leaving out the column of 1s may be fine when you are regressing the outcome on categorical predictors, but often we include continuous predictors.

Let's compare a logistic regression with and without the intercept when we have a continuous predictor. Assume the data have been mean centered. Without the column of 1s, the model looks like

$$ \operatorname{logit}\left( \dfrac{p(x)}{1-p(x)} \right) = \beta x $$

When $x=0$ (i.e. when the covariate is equal to the sample mean), then the log odds of the outcome is 0, which corresponds to $p(x) = 0.5$. So what this says is that when $x$ is at the sample mean, then the probability of a success is 50% (which seems a bit restrictive).

If we do have the intercept, the model is then

$$ \operatorname{logit}\left( \dfrac{p(x)}{1-p(x)} \right) = \beta_0 + \beta x $$

Now, when $x=0$ the log odds is equal to $\beta_0$ which we can freely estimate from the data.

In short, unless you have good reason to do so, include the column of 1s.

$\endgroup$
  • $\begingroup$ Upvoted for the clarity and excellence of the answer. $\endgroup$ – James Phillips Dec 11 '19 at 0:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.