0
$\begingroup$

I'm using a binomial GLM to model what kind of student would pass or fail a certain class. When I use predict with type "response" on my model, I see a vector of probabilities. Per my understanding, probabilities near 1 = passing while probabilities near 0 = failing. My book initially said to use a cutoff of 0.5 to determine pass or fail. Essentially, if the predicted probability is greater than 0.5, then we say pass, and if less than 0.5, then fail.

Then, the book said that there may be a more accurate way to gauge the cutoff. I thought I should generate a vector of random numbers ~ uniform[0,1] the same length as my data. Then I would compare each predicted probability and see if it's greater than the respective randomly generated uniform number. If what I'm doing correct or even necessary? I'm not exactly sure why we shouldn't use 0.5 in the first place.

$\endgroup$
  • $\begingroup$ If you're modeling a low probability outcome and you use a cutoff of .5, then all your predictions will be zero, so there can be no such general rule. First question is why do you need a cutoff? $\endgroup$ – Heteroskedastic Jim Nov 18 '18 at 2:56
  • $\begingroup$ I'm using response as the prediction type which gives me probabilities, but the target variable is binomial ("Fail" or "Pass"). $\endgroup$ – mistersunnyd Nov 18 '18 at 20:27
  • $\begingroup$ The target variable is fail/pass but why do you need to go from probabilities to fail/pass? The question remains. What is your goal when you attempt this? $\endgroup$ – Heteroskedastic Jim Nov 18 '18 at 20:31
  • $\begingroup$ Hmm, is there a method for R to output just the fail/pass result without any probabilities even if I'm using "response" as the prediction type? $\endgroup$ – mistersunnyd Nov 18 '18 at 20:33
  • $\begingroup$ There is no canned method that is defensible. That is why I keep asking for your motivations for doing this. Any defensible method for turning probabilities into 0/1 has to be context-dependent. $\endgroup$ – Heteroskedastic Jim Nov 18 '18 at 21:04
1
$\begingroup$

You need to generate an ROC curve. A 0.5 cutoff is often the default but not necessarily the best cutoff.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.