2
$\begingroup$

I performed classification analysis as using the glm(). The dependent variable has a value of 0 and 1, and the probability is measured using the predict function as follows.

glm.prob<-predict(A, newdata=test, type="response")
glm.prob

 4         11         15         17         22         23         30         31         35         36         42         50 
0.83793310 0.51753857 0.54858443 0.76921368 0.82107932 0.07838337 0.83934274 0.84484728 0.61028261 0.74274305 0.84628820 0.88751409 

If it exceeds 0.5, it is 1, and if it is less than 0.5, it is considered as 0. But I have a question. Why is the standard 0.5? 0.7 or 0.3 etc .. Can not other values?

$\endgroup$
  • 2
    $\begingroup$ Logistic regression is not a classification technique, it's a prediction technique. Specifically, the probability of being a "1" in terms of your dependent variable. Making a classification from this by using a cutpoint such as 0.5 ultimately decides your categories and it can be any point you wish. $\endgroup$ – prince_of_pears Jun 23 '17 at 2:09
  • 1
    $\begingroup$ Could you elaborate on the difference between your use of classification vs. prediction? It seems like people generally use classification whenever there is a categorical/qualitative/dichotomous/nominal dependent/response/output variable $\endgroup$ – Mark White Jun 23 '17 at 2:42
  • 1
    $\begingroup$ @Mark Can you elaborate on where your sense of "seems like" arises? Perhaps you're mostly reading literature which ignores what logistic regression is actually about (certainly some of the machine learning literature actively misrepresents it). prince_of_pears statement is correct; logistic regression is explicitly a model for a probability, not classification. It can be used to obtain a classification, but that's something imposed on top of logistic regression by adding a classification rule to it, it's not actually part of logistic regression itself. $\endgroup$ – Glen_b Jun 24 '17 at 5:15
  • 1
    $\begingroup$ @Glen_b I see. But naive Bayes and random forest can output probabilities too, right? Are those just for probability, too, with classification of on top of it? How do we know what is "imposed on top of" versus actually part of the algorithm? $\endgroup$ – Mark White Jun 24 '17 at 5:43
  • $\begingroup$ Note that I referred to a model, rather than an algorithm (an algorithm is simply the steps you use to achieve something, not the thing you're trying to estimate by using the algorithm). As I said logistic regression is a model for a probability -- specifically the probability parameter in a model for a count proportion, though it's not the only one; probit regression is another, for example. To answer the same question in relation to other models, you must consider what it is they are modelling (what quantity do they estimate?). $\endgroup$ – Glen_b Jun 24 '17 at 7:34
2
$\begingroup$

The cutoff 0.5 is not a standard, and if it is communicated as such, you should have some suspicion about any other information you recieve from the same source.

It is the job of the regression only to estimate the predicted conditional probabilities

$$ P(y = 1 \mid X) $$

Assigning hard class assignments is another layer of decision making above and beyond estimating the probabilities. It should not be done unless there is a pressing need, and if there is a need, it should be done in accordance of that need. One way to do this is to threshold the predicted probabilities, but the threshold chosen should be in service of some objective.

There should not be any need for a standard threshold, or a rule of thumb. If you find yourself in need of one, it's better to think more carefully about whether you really need hard classification, or about what objective you are attempting to accomplish with the hard classification.

| cite | improve this answer | |
$\endgroup$
-3
$\begingroup$

Typically, unless you make changes, it will create the cutoff at whatever the average response rate of the training data is. If your training data has 13.5% y=1, then it will classify anything where predicted probability >0.135 as a 1.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ This is wrong, logistic regression does not create any cutoff at all. $\endgroup$ – Matthew Drury Jun 23 '17 at 2:34
  • $\begingroup$ @MatthewDrury I was merely describing your 2nd paragraph. The way I described it is exactly how it does that out-of-the-box. I don't know why you disagree. $\endgroup$ – Josh Jun 26 '17 at 23:05
  • $\begingroup$ It doesn't do that out of the box. Logistic regression doesn't do anything with a threshold or hard classification out of the box. $\endgroup$ – Matthew Drury Jun 26 '17 at 23:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.