2
$\begingroup$

Is it normal to get values for response that are $> 1$ even though in logistic regression the response has meaning only in the range $[0,1]$?

Does one then have to truncate all $> 1$ values to mean just $1.0$?


The reason for asking is that

I got something that looked like it might have response $> 1$ using a skin variable (1 means skin that's more prone to get sunburn and 0 means normal skin), a trt variable (1 signifies that the patient took beta carotene and 0 that the patient didn't) and then a cancer variable that signifies, whether the patient got skin cancer during the study. So $cancer \text{ ~ } trt+skin$ produces $$cancer=0.1331410+0.5570949⋅trt+0.627890⋅skin$$ (these coefficients are after taking invlogit() of the coefficients returned by glm())
which is $>1$ if both variables $==1$.

Although, when seeing whether such rows actually exist (where $skin=trt=1$), no such rows exist.

$\endgroup$
  • $\begingroup$ Are you referring to the predicted values of the logistic regression? It's not entirely clear. $\endgroup$ – Ian_Fin Oct 19 '16 at 15:31
  • $\begingroup$ @Ian_Fin Yes, also known as response variable or Y. $\endgroup$ – mavavilj Oct 19 '16 at 15:32
  • $\begingroup$ This should be impossible, that is the entire idea of logit, predicted probabilities exactly equal to 1 can happen, and typically indicates a degenerate model or perfect separation in the data. $\endgroup$ – Repmat Oct 19 '16 at 15:37
  • 2
    $\begingroup$ "The above coefficient values are after taking invlogit()." - this does not make sense to me. What does it mean? It's better if you update your Q with this information, btw, instead of posting it in the comments. $\endgroup$ – amoeba says Reinstate Monica Oct 19 '16 at 16:02
  • $\begingroup$ One must wonder why you are hand-cranking these computations rather than using R's prepackaged methods for making predictions with a glm model object. $\endgroup$ – Reinstate Monica Oct 19 '16 at 17:01
12
$\begingroup$

You are not getting probabilities because you are confusing the order of operation between adding the coefficients and taking the inverse logit.

Explanation: You have a logistic model given by $$logit(Pr(cancer=1|trt,skin,\beta)) = \beta_0 + \beta_1 * trt + \beta_2 * skin$$

You are trying to calculate a probability (I denote this quantity by $Pr^*$, since it's not a probability) by doing the following:

$$Pr^*(cancer=1|trt,skin,\beta) = logit^{-1}(\beta_0) + logit^{-1}(\beta_1 * trt) + logit^{-1}(\beta_2 * skin)$$

When it should be this:

$$Pr(cancer=1|trt,skin,\beta) = logit^{-1}(\beta_0 + \beta_1 * trt + \beta_2 * skin)$$

Note that, in general $$logit^{-1}(A+B) \neq logit^{-1}(A) + logit^{-1}(B)$$

We would say that a non-linear transformation on a sum of two variables is not equal to the sum of the transformed variables, except in special, trivial cases.

This would be easier to see if you used a log-linear model instead of a logistic model: $$exp(A+B)\neq exp(A) + exp(B)$$ Since we know that $$exp(A+B)= exp(A)*exp(B)$$

In other words, to get a probability, you first find the log-odds at trt=skin=1 as

$logit(Pr(cancer=1|trt,skin,\beta)) = -1.873468 + 0.229380*trt + 0.5231755*skin$ $logit(Pr(cancer=1|trt=1,skin=1,\beta)) = -1.873468 + 0.229380*1 + 0.5231755*1$ $logit(Pr(cancer=1|trt=1,skin=1,\beta)) = -1.120912$

And then transform: $$Pr(cancer=1|trt=1,skin=1,\beta) = logit^{-1}(-1.120912) = 0.2458422$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.