4
$\begingroup$

Logistic regression creates a model that predicts one of two possible responses for a given set of predictors. What if I want to choose between three responses?

I tried this by using a logistic regression, except instead of defining my two outcomes as 0 and 1; I defined a third outcome at 0.5. This seemed to be going great, until I realized that this model would never produce an estimate of probability in all three categories. For example, if outcome A = 0; B = 0.5; and C = 1, and if the regression result for a certain set of predictor variables is 0.75, I could interpret that as saying the probability of B is 0.5 an the probability of C is 0.5. However, the observed probability of all three outcomes is > 0 for any set of predictor variables, so this model is not so useful.

What sorts of regression-type models will estimate probabilities for three or more outcomes, the way that logistic regression estimates probabilities for two outcomes?

$\endgroup$
2
  • $\begingroup$ why does it have to be a a regression model? From what you're writing, this sounds like a standard multi-class classification problem $\endgroup$
    – deemel
    Dec 12, 2017 at 21:22
  • 1
    $\begingroup$ @Rickyfox, typical ML classifiers (ANNs, SVMs, random forests, etc.) can be considered regression models in an abstract sense. $\endgroup$ Dec 14, 2017 at 16:16

1 Answer 1

3
$\begingroup$

You need to determine what the nature of the response categories are. They could be ordered (e.g., normal, mild, severe), or they could be unordered (e.g., animal, vegetable, mineral).

If your outcome categories are ordered, you would use (some form of) ordinal logistic regression. Note that there are actually multiple versions of OLR, but the standard one is more formally called the proportional odds model. The key assumption is that the odds are proportional; this means that adjacent categories can be closer or further apart, but the relationship between the covariates and 'moving up' from one level to the next is the same for all levels. There are other versions for situations where the proportional odds assumption is not met (several are listed in Agresti's tutorial pdf in section 2 starting on p. 35).

If the outcome categories are not ordered, you would use multinomial logistic regression. This model makes the assumption of the independence of irrelevant alternatives. This means that the odds that event $A$ occurs instead of event $B$ would be the same whether the event $C$ is among the possible outcomes or not. However, this is known to not hold of human decision making (e.g., marketers can nudge consumers towards one product vs. another by including a specially designed third product). Alternative models that do not require this assumption are discussed starting on p. 28 of the R mlogit package's vignette (pdf).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.