1
$\begingroup$

I have the following issue in R.

I perform logistic regression in R:

logitMod <- glm(dependent_var ~ var1, var2, varN, data=traindata, family=binomial())

Then, I run predict on the first record of the testdata set, to get the log odds:

>predict(logitMod, testdata[1])
-44.81362 

Then I calculate the probability from the log odds:

> 1/(1+ exp(-predict(logitMod, testdata[1])))
3.449006e-20 

Then, I check with the predict function what the built-in probability-conversion would yield, and the result is very different:

>predict(logitMod, testdata[1], type="response")
2.220446e-16 

So my question is, what am I overlooking?

$\endgroup$
  • $\begingroup$ Both are basically zero. Surely not "very different" in my book. (I haven't made an effort to check whether you got things right though.) $\endgroup$ – Lewian Jun 26 at 12:56
  • $\begingroup$ I'm with Lewian on this. Sure, it looks like you're off by many orders of magnitude, but your numbers are so small that a tiny rounding difference could make for considerable differences. Maybe there's an issue, but try your predictions with testdata[2], 3, etc to see if you get matching results. Your probability calculation looks right. Maybe try $\dfrac{exp(x)}{1+exp(x)}$ and see if the numerical precision improves for testdata[1]. $\endgroup$ – Dave Jun 26 at 13:04
  • $\begingroup$ I see no error in your code and it indeed looks like predict(logitMod, testdata[1], type="response") produces erroneous predictions. But maybe there are good reasons for this behaviour of predict.glm. $\endgroup$ – Jarle Tufto Jun 26 at 13:07
  • $\begingroup$ @JarleTufto Why is that prediction erroneous? $\endgroup$ – David Jun 26 at 13:11
  • $\begingroup$ @David Well, there is certainly an unexpected numerical difference. While the difference may not have any practical consequences, it is worth asking why predict.glm was implemented to have this behaviour. Btw, 2.22e-16 is the same as .Machine$double.eps. $\endgroup$ – Jarle Tufto Jun 26 at 13:32
0
$\begingroup$

Those two results are exactly the same: zero! Try to see if the problem persists when using a more "normal" number, i.e: predict(logitMod, testdata[1]) in a range like $(-2,2)$ or $(-3, 3)$

$\endgroup$
  • $\begingroup$ For testdata[2], the result of predict(logitMod, testdata[2], type="response") 6.653410e-01 = 0.665341. With the same manual formula as above, the results indeed seem to be matching. Thanks for this tip! In the end, even though the difference between those approximately zero numbers are 4 orders of magnitude, this issue is probably related to decimal conversion indeed. $\endgroup$ – itarill Jun 26 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.