# Why can't we use linear regression for discrete Y?

This text is from An Introduction to Statistical Learning with Applications in R(by • Gareth James • Daniela Witten • Trevor Hastie • Robert Tibshirani) List item Can anyone help me by making linear models with these different variable so that I can practically understand what is wrong with these

Suppose that we are trying to predict the medical condition of a patient in the emergency room on the basis of her symptoms. In this simpliﬁed example, there are three possible diagnoses: stroke, drug overdose, and epileptic seizure. We could consider encoding these values as a quantitative response variable, Y , as follows:

y=1 if stroke;

y= 2 if drug overdose;

y= 3 if epileptic seizure.

Using this coding, least squares could be used to ﬁt a linear regression model to predict Y on the basis of a set of predictors X1,...,Xp. Unfortunately, this coding implies an ordering on the outcomes, putting drug overdose in between stroke and epileptic seizure, and insisting that the diﬀerence between stroke and drug overdose is the same as the diﬀerence between drug overdose and epileptic seizure. In practice there is no particular reason that this needs to be the case.

For instance, one could choose an equally reasonable coding,

y=1 if epileptic seizure;

y=2 if stroke;

y=3 if drug overdose.

which would imply a totally diﬀerent relationship among the three conditions. Each of these coding would produce fundamentally diﬀerent linear models that would ultimately lead to diﬀerent sets of predictions on test observations.

1)how to produce linear model using any of the relationship

2)how different relationship among the three condition produce different linear model

• What is unclear for you in this example?
– Tim
Apr 1 '16 at 11:41
• Hint: what would it mean if your model returned prediction of 2.5 or 4..?
– Tim
Apr 1 '16 at 11:44
• @Tim I didn't understand how these coding produce different linear model and how to produce linear model using these Apr 1 '16 at 12:16
• @NickCox Now i update . Thanks to your suggestion. Apr 1 '16 at 12:20

There are two viable models for this situation and neither of them is close to the linear model (ordinary least squares).

1. Multinomial logistic model, which doesn't assume an ordering of $Y$ but which requires twice as many $\beta$ coefficients with a 3-level $Y$.
2. Ordinal regression (e.g., proportional odds ordinal logistic model) which makes a strong ordering assumption about $Y$ (but no spacing assumptions) and only needs as many $\beta$s as the linear model, plus 2 intercepts for 3-level $Y$.

To me it is not useful to conceptualize this using a model for continuous $Y$.

I changed the original title of the question, which included the word classification. Classification has nothing to do with the issues at hand.

• They mentioned in the text that it is possible to build linear model using binary variable and i am curious to find out how .Just wanted to know from you is it possible to build linear model using binary variable 0 and 1 ? Please clear the confusion as you are an expert in this field Apr 1 '16 at 12:51
• Why would anyone be interested in a linear model for binary $Y$ when it would yield "probabilities" <0 or >1? To answer your question it is possible. Lots of things are possible that are terrible ideas. Apr 1 '16 at 12:56
• Well I am bit curious to learn by going to the bottom of the subject. Helpful if you provide any link or way to create linear model using binary variable. Apr 1 '16 at 12:58
• What you want in terms of a "linear model using binary variable" is explained farther down the page in the text that you cite (page 130): the section on logistic regression. You use a transformation of the probabilities, the logit, as the outcome variable rather than raw probabilities. That avoids the inconsistencies that Frank Harrell points out in his comment. Some ways to handle multiple discrete outcomes are covered later in the same Chapter 4 of ISLR. Try the exercises at the end of the chapter.
– EdM
Apr 1 '16 at 13:53
• @FrankHarrell "Classification has nothing to do with the issues at hand." I would love to hear your explanation about why this isn't a classification problem.
– Dave
Sep 12 '19 at 10:27