3
$\begingroup$

I'm new to logistic regression. Can you help me understand how to read this?

Here's what I understand - For every +1 of continous_variable, the probability of the outcome goes up by 4.88%. If they have 0 for continous variable, then the probability of the outcome is -314.49%.

Both of these are significant and that means that we can reject the null hypothesis that there is no relationship between outcome and continous variable.

glm(formula = outcome ~ ., family = binomial(link = "logit"), 
    data = dataframe)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-4.2917  -0.2904  -0.2904  -0.2904   2.5247  

Coefficients:
                     Estimate Std. Error z value Pr(>|z|)    
(Intercept)         -3.144947   0.095262  -33.01   <2e-16 ***
continous_variable  0.048831   0.004774   10.23   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 1256.6  on 2817  degrees of freedom
Residual deviance: 1065.2  on 2816  degrees of freedom
AIC: 1069.2

Is that correct? Is there stuff that I'm missing?

Thank you!

Context -

continuous_variable is the number of a particular action that was taken by one person. outcome is whether that person took a final action coded in 1 or 0. The dataset was analyzed in R and was a dataframe with two fields: continuous_variable and outcome.

Also, for sample sizes, there are about 3,000 records and about 2,700 where outcome = 1.

$\endgroup$
  • 3
    $\begingroup$ Hint: can -314.49% be a probability in the first place? $\endgroup$ – Kodiologist Jan 24 '18 at 21:42
  • $\begingroup$ What's a better way to understand these results? $\endgroup$ – Sebastian Jan 24 '18 at 21:43
4
$\begingroup$

The units in logistic regression are logits, not probabilities. Probabilities have to be somewhere 0 to 1 (e.g. a negative probability is meaningless), whereas logits run from -infinity to +infinity. A probability of .5 corresponds to a logit of 0, so negative logits mean less than 50% chance and positive logits mean greater than 50% chance.

You can interpret just the direction of the effect (e.g. is the outcome more or less likely as continuous_var increases) by looking at the output in logits, as you presented it. Because the logit coefficient for continuous_var is positive and significant, you can say that the probability of outcome increases as continuous_var increases.

If you want to get more specific about the size of the effect (not just the direction), then I recommend converting from logits to units that are more interpretable, such as odds or probabilities. You can convert your logit coefficients to probabilities pretty easily:

odds = e^logit
odds = e^0.048831 = 1.05

prob = e^logit/(1 + e^logit) = odds/(1 + odds)
prob = e^0.048831/(1 + e^0.048831) = 0.512

Be very careful in interpreting these, though --- because you used a generalized linear regression, the relationships between coefficients and predictors and the outcome aren't just additive. You can't just say "the probability of outcome goes up by 51% for every unit increase in continuous_var" (and you can see how that would quickly become nonsense for high levels of continuous_var). The change in outcome logits per unit continuous_var is linear, but the change in outcome probability isn't. I recommend plotting to interpret your results. Alternatively, you can examine the resulting probability from different values of continuous_var by plugging them into the logit regression equation and converting the final value to probabilities.

For example, for someone with a continuous_var score of 10, their probability of outcome is about 6.6%:

logit = b0 + b1*continuous_var = -3.144947 + 0.048831*10 = -2.656637
prob  = e^logit/(1 + e^logit) = e^-2.656637/(1 + e^-2.656637) = 0.06558112

Better yet, use the predict function in R with type = 'response' to get probabilities for any set of continuous_var values you like. You can also use predict to quickly plot your results.

$\endgroup$
  • 1
    $\begingroup$ @Sebastian You're very welcome! If you found my answer helpful, you can upvote it. You can also select it (or another) as the "accepted" answer, but you might want to wait a little longer to see if anyone else offers something helpful before making a selection. $\endgroup$ – Rose Hartman Jan 24 '18 at 23:07
3
$\begingroup$

Your understanding of the results is not correct.

You can just write down the formula for the logistic regression model

$$\log\frac{\hat{p}}{1-\hat{p}}=\beta_0+\beta_1x$$

Here $\beta_0$ is your intercept $x$ is the continuous_variable, $\beta_1$ is the coefficient of the continuous variable. $\hat{p}$ is the probability of outcome "1" happens.

According to your results, the formula becomes

$$\log\frac{\hat{p}}{1-\hat{p}}=-3.144947+0.048831x$$

So when $x=0$,

$$\frac{\hat{p}}{1-\hat{p}}=e^{-3.144947}\Rightarrow\hat{p}=0.04129084$$ Which means when $x=0$, the probablity of "outcome" happens is 0.041.

Next, you can compare when $x_2=x_0+1$ with $x_1=x_0$ which mean one unit increase of the continuous variable.

When $x_1=x_0$ $$\log\frac{\hat{p_1}}{1-\hat{p_1}}=-3.144947+0.048831*x_0 \tag{1}$$

Then when $x_2=x_0+1$ (i.e one unite increase) $$\log\frac{\hat{p_2}}{1-\hat{p_2}}=-3.144947+0.048831*(x_0+1) \tag{2}$$

Next we calculate $(2)-(1)$ we get

$$\log\frac{\hat{p_2}}{1-\hat{p_2}}-\log\frac{\hat{p_1}}{1-\hat{p_1}}=0.048831$$

Note we call $\frac{p}{1-p}$ as Odds, We denote $\frac{\hat{p_2}}{1-\hat{p_2}}=O_2$ i.e Odds when $x_2=x_0+1$, and similarly, $\frac{\hat{p_1}}{1-\hat{p_1}}=O_1$

Therefore,

$$\log O_2-\log O_1=0.048831$$

$$\log\frac{O_2}{O_1}=0.048831$$

$$OR(\text{odds ratio})=\frac{O_2}{O_1}=e^{0.048831}=1.05003$$

Which mean with one unit of increase of your continous_variable, the Odds of "outcome" happens increase by about 5%(1.05003)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.