1
$\begingroup$

I have the following issue in R.

I perform logistic regression in R:

logitMod <- glm(dependent_var ~ var1, var2, varN, data=traindata, family=binomial())

Then, I run predict on the first record of the testdata set, to get the log odds:

>predict(logitMod, testdata[1])
-44.81362

Then I calculate the probability from the log odds:

> 1/(1+ exp(-predict(logitMod, testdata[1])))
3.449006e-20

Then, I check with the predict function what the built-in probability-conversion would yield, and the result is very different:

>predict(logitMod, testdata[1], type="response")
2.220446e-16

So my question is, what am I overlooking?

Improve this question
$\endgroup$
7
  • $\begingroup$ Both are basically zero. Surely not "very different" in my book. (I haven't made an effort to check whether you got things right though.) $\endgroup$
    – Christian Hennig
    Jun 26, 2019 at 12:56
  • $\begingroup$ I'm with Lewian on this. Sure, it looks like you're off by many orders of magnitude, but your numbers are so small that a tiny rounding difference could make for considerable differences. Maybe there's an issue, but try your predictions with testdata[2], 3, etc to see if you get matching results. Your probability calculation looks right. Maybe try $\dfrac{exp(x)}{1+exp(x)}$ and see if the numerical precision improves for testdata[1]. $\endgroup$
    – Dave
    Jun 26, 2019 at 13:04
  • $\begingroup$ I see no error in your code and it indeed looks like predict(logitMod, testdata[1], type="response") produces erroneous predictions. But maybe there are good reasons for this behaviour of predict.glm. $\endgroup$
    – Jarle Tufto
    Jun 26, 2019 at 13:07
  • $\begingroup$ @JarleTufto Why is that prediction erroneous? $\endgroup$
    – David
    Jun 26, 2019 at 13:11
  • $\begingroup$ @David Well, there is certainly an unexpected numerical difference. While the difference may not have any practical consequences, it is worth asking why predict.glm was implemented to have this behaviour. Btw, 2.22e-16 is the same as .Machine$double.eps. $\endgroup$
    – Jarle Tufto
    Jun 26, 2019 at 13:32

1 Answer 1

Reset to default
0
$\begingroup$

Those two results are exactly the same: zero! Try to see if the problem persists when using a more "normal" number, i.e: predict(logitMod, testdata[1]) in a range like $(-2,2)$ or $(-3, 3)$

Improve this answer
$\endgroup$
1
  • $\begingroup$ For testdata[2], the result of predict(logitMod, testdata[2], type="response") 6.653410e-01 = 0.665341. With the same manual formula as above, the results indeed seem to be matching. Thanks for this tip! In the end, even though the difference between those approximately zero numbers are 4 orders of magnitude, this issue is probably related to decimal conversion indeed. $\endgroup$
    – itarill
    Jun 26, 2019 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.