Analyzing the results from a logistic regression

Question

I'm new to logistic regression. Can you help me understand how to read this?

Here's what I understand - For every +1 of continous_variable, the probability of the outcome goes up by 4.88%. If they have 0 for continous variable, then the probability of the outcome is -314.49%.

Both of these are significant and that means that we can reject the null hypothesis that there is no relationship between outcome and continous variable.

glm(formula = outcome ~ ., family = binomial(link = "logit"), 
    data = dataframe)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-4.2917  -0.2904  -0.2904  -0.2904   2.5247  

Coefficients:
                     Estimate Std. Error z value Pr(>|z|)    
(Intercept)         -3.144947   0.095262  -33.01   <2e-16 ***
continous_variable  0.048831   0.004774   10.23   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 1256.6  on 2817  degrees of freedom
Residual deviance: 1065.2  on 2816  degrees of freedom
AIC: 1069.2

Is that correct? Is there stuff that I'm missing?

Thank you!

Context -

continuous_variable is the number of a particular action that was taken by one person. outcome is whether that person took a final action coded in 1 or 0. The dataset was analyzed in R and was a dataframe with two fields: continuous_variable and outcome.

Also, for sample sizes, there are about 3,000 records and about 2,700 where outcome = 1.

Answer 1

The units in logistic regression are logits, not probabilities. Probabilities have to be somewhere 0 to 1 (e.g. a negative probability is meaningless), whereas logits run from -infinity to +infinity. A probability of .5 corresponds to a logit of 0, so negative logits mean less than 50% chance and positive logits mean greater than 50% chance.

You can interpret just the direction of the effect (e.g. is the outcome more or less likely as continuous_var increases) by looking at the output in logits, as you presented it. Because the logit coefficient for continuous_var is positive and significant, you can say that the probability of outcome increases as continuous_var increases.

If you want to get more specific about the size of the effect (not just the direction), then I recommend converting from logits to units that are more interpretable, such as odds or probabilities. You can convert your logit coefficients to probabilities pretty easily:

odds = e^logit
odds = e^0.048831 = 1.05

prob = e^logit/(1 + e^logit) = odds/(1 + odds)
prob = e^0.048831/(1 + e^0.048831) = 0.512

Be very careful in interpreting these, though --- because you used a generalized linear regression, the relationships between coefficients and predictors and the outcome aren't just additive. You can't just say "the probability of outcome goes up by 51% for every unit increase in continuous_var" (and you can see how that would quickly become nonsense for high levels of continuous_var). The change in outcome logits per unit continuous_var is linear, but the change in outcome probability isn't. I recommend plotting to interpret your results. Alternatively, you can examine the resulting probability from different values of continuous_var by plugging them into the logit regression equation and converting the final value to probabilities.

For example, for someone with a continuous_var score of 10, their probability of outcome is about 6.6%:

logit = b0 + b1*continuous_var = -3.144947 + 0.048831*10 = -2.656637
prob  = e^logit/(1 + e^logit) = e^-2.656637/(1 + e^-2.656637) = 0.06558112

Better yet, use the predict function in R with type = 'response' to get probabilities for any set of continuous_var values you like. You can also use predict to quickly plot your results.

Answer 2

Your understanding of the results is not correct.

You can just write down the formula for the logistic regression model

$$\log\frac{\hat{p}}{1-\hat{p}}=\beta_0+\beta_1x$$

Here $\beta_0$ is your intercept $x$ is the continuous_variable, $\beta_1$ is the coefficient of the continuous variable. $\hat{p}$ is the probability of outcome "1" happens.

According to your results, the formula becomes

$$\log\frac{\hat{p}}{1-\hat{p}}=-3.144947+0.048831x$$

So when $x=0$,

$$\frac{\hat{p}}{1-\hat{p}}=e^{-3.144947}\Rightarrow\hat{p}=0.04129084$$ Which means when $x=0$, the probablity of "outcome" happens is 0.041.

Next, you can compare when $x_2=x_0+1$ with $x_1=x_0$ which mean one unit increase of the continuous variable.

When $x_1=x_0$ $$\log\frac{\hat{p_1}}{1-\hat{p_1}}=-3.144947+0.048831*x_0 \tag{1}$$

Then when $x_2=x_0+1$ (i.e one unite increase) $$\log\frac{\hat{p_2}}{1-\hat{p_2}}=-3.144947+0.048831*(x_0+1) \tag{2}$$

Next we calculate $(2)-(1)$ we get

$$\log\frac{\hat{p_2}}{1-\hat{p_2}}-\log\frac{\hat{p_1}}{1-\hat{p_1}}=0.048831$$

Note we call $\frac{p}{1-p}$ as Odds, We denote $\frac{\hat{p_2}}{1-\hat{p_2}}=O_2$ i.e Odds when $x_2=x_0+1$, and similarly, $\frac{\hat{p_1}}{1-\hat{p_1}}=O_1$

Therefore,

$$\log O_2-\log O_1=0.048831$$

$$\log\frac{O_2}{O_1}=0.048831$$

$$OR(\text{odds ratio})=\frac{O_2}{O_1}=e^{0.048831}=1.05003$$

Which mean with one unit of increase of your continous_variable, the Odds of "outcome" happens increase by about 5%(1.05003)