3
$\begingroup$

In a linear regression, when you standardize your numeric variables, the resulting intercept has the same value as the mean of your sample. Is there any way in a logistic regression, with numeric continuous variables, to have the intercept to express the odd-ratios of the baseline probability in the data (average probability of response)?

$\endgroup$
  • $\begingroup$ Do you have multiple values per person (ie longitudinal data)? $\endgroup$ – gung - Reinstate Monica Oct 10 '15 at 13:50
2
$\begingroup$

The intercept might be interpreted as the estimated baseline log odds when all independent variables are set to 0, or the reference category in case of categorical variables. The probability when all independent variables are set to 0 is log(intercept)/(1+log(intercept)).

With a standardized continuous variable, the intercept is the estimated log odds for the event when the standardized variable is 0.

The problem is that mean probability in your sample is not the same as probability when the standardized variable is 0. If the probability of having an event (or whatever the dependent variable is) is 0.1 when the standardized variable x is 0, and the estimated coefficient for x is 1, this means that for an individual whose value for x is 1, the odds ratio will be exp(1)=2.71. We can calculate the expected probability for an event in such an individual:

base odds = 0.1/(1-0.1) = 0.11. odds for this individual: 0.11 * 2.71 = 0.3 probability for this individual = 0.3/(1+0.3) = 0.23

Now, for an individual who is one standard deviation below the mean on the x variable, the odds ratio will be exp(-1) = 0.37:

odds for this individual: 0.11 * 0.37 = 0.03 probability for this individual = 0.04

So the +1 sd of x means a probability of 0.23 and -1 sd means a probability of 0.03. If we instead calculate the probability for +2 sd we get a probability of 0.45, and for -2 sd we get a probability of 0.01.

It's easy to see that the average probability in the sample will be higher than the probability for individuals whose value on x is 0, because the probabilities are skewed because of how odds and odds ratios work.

As for your question, I don't think it's possible to make the intercept represent the mean probability, because in logistic regression, (log) odds and odds ratios are estimated, not probabilities, and the mean probability is not really meaningful to consider in a logistic regression.

$\endgroup$
  • $\begingroup$ The OP did not mention standardization by the SD and I don't recommend it. The SD is an arbitrary measure in this context. Note also the error in your formula for $Prob(Y=1)$. $\endgroup$ – Frank Harrell Oct 10 '15 at 19:53
  • $\begingroup$ I was under the impression that standardization usually means that the mean is set to 0 and sd is set to 1? Anyway, it doesn't matter in this context as you say. $\endgroup$ – JonB Oct 10 '15 at 20:10
  • $\begingroup$ That is a common approach, and not recommended. $\endgroup$ – Frank Harrell Oct 10 '15 at 20:35
2
$\begingroup$

There is no simple interpretation in binary logistic models other than the intercept and slopes satisfy the property that the the average predicted probability equals the observed prevalence of $Y=1$ in the dataset used to fit the model. But I don't find it very useful to think about this in either linear models or logistic models, because the idea of reference values is arbitrary. For example one person may think of the median or mode as the reference and another the mean. When categorical variables are included things are more complex.

I like to think of the intercept as an arbitrary constant that makes the model work no matter what the numeric origin is for the predictors. In R when you request predictions everything is handled automatically.

$\endgroup$
  • $\begingroup$ I think that "satisfy the property that the the average predicted probability equals the observed prevalence of Y=1" holds only for the sample that was used to estimate the coefficents ? $\endgroup$ – user83346 Oct 10 '15 at 15:15
  • $\begingroup$ I added that to the answer above - thanks for clarifying. $\endgroup$ – Frank Harrell Oct 10 '15 at 19:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.