Analyzing logistic regression coefficients

Question

Here is a list of logistic regression coefficients (first one is an intercept)

-1059.61966694592
-1.23890500515482
-8.57185269220438
-7.50413155570413
 0
 1.03152408392552
 1.19874787949191
-4.88083274930613
-5.77172565873336
-1.00610998453393

I find it weird how the intercept is so low and I have a coefficient that is actually equal to 0. I am not fully sure how I would interpret this. Does the 0 indicate that the specific variable has no affect at all on the model? But the intercept that is made by inputting a column of one's is suddenly really important? Or is my data just crap and the model is unable to properly fit to it.

Answer 1

You are getting some very good information in the comments, in my opinion. I wonder if some basic facts about logistic regression would help make these things more comprehensible, so with that in mind, let me state a couple of things. In logistic regression, coefficients are on the logistic scale (hence the name...). If you were to plug in your covariate values for an observation, multiply them by the coefficients, and sum them, you would get a logit.
$$ \text{logit}=\beta_0+\beta_1x_1+\beta_2x_2+...+\beta_kx_k $$ A logit is a number that makes no intuitive sense to anyone, so it is very difficult to know what to do with a number looks funny (e.g., very high or very low). The best way to understand these things is to convert them from their original scale (logits) to one that you can understand, specifically probabilities. To do that, you take your logit and exponentiate it. That means you take the number e ($e\approx 2.718281828$) and raise it to the power of the logit. Imagine your logit were 2:
$$ e^2=7.389056 $$ This will give you the odds. You can convert the odds to a probability by dividing the odds by one plus the odds: $$ \frac{7.389056}{1+7.389056}=0.880797 $$ People typically find the probability much easier to deal with.

For your model, imagine you had an observation in which the value of all of your variables is exactly 0, then all of your coefficients would drop out and you would be left with only your intercept value. If we exponentiate your value, we get 0 as the odds (if it were -700, the odds would be $9.8\times 10^{-305}$, but I can't get my computer to give me a value for -1060, it is too small given the numerical limits of my software). Converting those odds to a probability, ( $0/(1+0)$ ), gives us 0 again. Thus, what your output is telling you is that your event (whatever it is) simply does not occur when all of your variables are equal to 0. Of course, it depends on what we are talking about, but I find nothing too remarkable about this. A standard logistic regression equation (say, without a squared term, for instance) necessarily assumes that the relationship between a covariate and the probability of success is either monotonically increasing or monotonically decreasing. That means that it always gets larger and larger (or smaller and smaller), and so, if you go far enough in one direction, you will get to numbers so small my computer can't tell them apart from 0. That is just the nature of the beast. As it happens, for your model, going really far is going to where your covariate vales equal 0.

As for the coefficient of 0, it does mean that that variable has no effect, as you suggest. Now, it is quite reasonable that a variable will not have an effect, nonetheless, you will basically never get a coefficient of exactly 0. I don't know why it occurred in this case; the comments offer some possible suggestions. I can offer another, which is that there may be no variation in that variable. For example, if you had a variable that coded for sex, but only women in your sample. I don't know if that's the real answer (R, for example, returns NA in that case, but software differ)--it's just another suggestion.

Answer 2

Interpreting the intercept

You can think of logistic regression as giving you a posterior probability of being a '1'. The intercept represents a prior on categories derived from the dataset: specifically, it is the empirical estimate of log(p(Y=1)/p(Y=0), by itself when the model has only an intercept, for the cases in the 'reference' classes when there are categorical covariates, and for cases when the covariates are at 0 more generally (but less interpretably). So your strongly negative number is probably telling you that '1's are rare among the cases in your sample characterised by having all covariates at 0. Again, there may be no observations there, so it's not worth worrying about the intercept value. This discussion is fairly clear.

Because of this handy separation of concerns among the parameters, you can correct for category imbalance by training on a better balanced sample and only adjusting the intercept. See King and Zeng for a thorough discussion.