I am reading An Introduction to Statistical Learning. In this book, section 4.2, page 130 (text given below) mentions that linear regression would not be useful to predict ordinal variables because the difference between one pair of values of the dependent variable might not be equal to difference between the other pair of values of that dependent variable.

If the response variable’s values did take on a natural ordering, such as mild, moderate, and severe, and we felt the gap between mild and moderate was similar to the gap between moderate and severe, then a 1, 2, 3 coding would be reasonable. Unfortunately, in general there is no natural way to convert a qualitative response variable with more than two levels into a quantitative response that is ready for linear regression.

  • 3
    $\begingroup$ The explanation of why not is right there in your question. Can you please indicate the way in which that is not adequate? $\endgroup$
    – Glen_b
    Jun 6 '16 at 14:52
  • $\begingroup$ I treat nominal dependent variables always as classification problems, and if they are ordinal then special resolution schemes follow. Dunno if this works better than ordinal regression. $\endgroup$
    – Firebug
    Jun 7 '16 at 13:55
  • $\begingroup$ For a linear regression it is assumed that the value of the dependent variable $y$ given the independent variables $x$ has a normal distribution, i.e $y|_x$ has a normal density. It is known that in that case $y$ is a real number. $\endgroup$
    – user83346
    Jun 7 '16 at 15:13

Linear regression assumes linear relationships between the independent and dependent variables, e.g. $y=\beta_0+\beta_1 x$. The meaning of $\beta_1$ is that per unit change in $x$ your $y$ changes by $\beta_1$. If your $x$ is ordinal, then there is no unit of $x$ per se, hence, the slope $\beta_1$ is meaningless.

The unit of measure of $x$ could be meters or yards, in this case $x=1,2,3$ are one meter or yard away from each other. If you have some kind of an ordinal distance measure such as far away, not too far and very close, and you denote them with 0, 1 and 2 then the distance between 0 and 1 is not necessarily equal to the distance between 1 and 2. It makes no sense to use linear regression then

  • $\begingroup$ Thanks a lot for your answer. However, it seems that you have explained the need of interval variables as independent variable. I have asked this in context of dependent variable. $\endgroup$
    – kusur
    Jun 7 '16 at 20:20
  • $\begingroup$ It's the same argument $\endgroup$
    – Aksakal
    Jun 7 '16 at 20:25

One problem with using linear regression to predict ordinal variables is that you can't interpret the regression coefficients.

Imagine you wanted to predict someone's yearly income with a set of variables (gender, race, education level, etc.). But instead of having exact yearly incomes, you only have a variable indicating whether their income, such as:

y = 1 if yearly income < \$10,000
y = 2 if \$10,000 $\leq$ yearly income $\leq$ \$50,000
y = 3 if \$50,000 $<$ yearly income $\leq$ \$60,000
y = 4 if yearly income > $60,000

y is clearly an ordinal variable because it's ordered but the gap in income between each level is not the same. You can run a regression and you will get an answer. For example, you might get that $\beta$ = 1 for gender, meaning that men earn more than women if men is coded as gender = 1. But how can you quantify the amount of the difference in income? You can't, because a change from 3 to 4 can mean a big difference in income (a person where y = 4 could be making $1,000,000 a year), while a change from 2 to 3 has a maximum amount of income difference.

Remember that ordinal variables are not capturing the exact values, but rather are buckets for a continuous variable.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.