2
$\begingroup$

While thinking about methods on how to predict a dependent variable that is ordinal and constrained (0, 1, 2, 3, 5 are the only possible values), I found that an ordered logistic regression is an common approach used in the community and as a strategy, that make sense to me.

However, I was trying to understand why another method that came into my mind is a bad idea.

The method in question would be to use a simple linear regression and then map the predicted outcome to the closest integer in the range [0, 5].

Would that be bad? Why?

Explanations with references to theory would be greatly appreciated to fill the all the gaps I have in this matter.

$\endgroup$
  • 1
    $\begingroup$ If the input data to be modeled is outside the range [-0.5, 5.5] then there would be clipping, and of course data that would extrapolate outside that range would be clipped. With those limitations in mind, in my opinion this could be a useful technique. $\endgroup$ – James Phillips Feb 9 at 22:45
3
$\begingroup$

If the dependent variable is ordinal, then the coding into integer values is arbitrary, so long as the ordering is maintained.

Instead of 0, 1, 2, 3, and 5, you could just as well code them as 0, 1, 2, 3, and 9999, for example. It should not be difficult to see that this will in general change the predicted categories under your proposed method; predictions in this case would tend to be moved in the direction of that highest category.

But if your predictions depend on an arbitrary encoding, then your predictions are themselves arbitrary.

Ordinal regression avoids this problem.

$\endgroup$
  • $\begingroup$ With your example, it is clear that the proposed approach would not work well, but if a unitary distance is maintained between the possible values of the DV, why the suggested method would not be a good way to tackle the problem? $\endgroup$ – datapug Feb 10 at 16:17
  • $\begingroup$ There is no unitary distance to be maintained in the first place. $\endgroup$ – The Laconic Feb 10 at 18:24
1
$\begingroup$

If you use classical linear regression, your model assumes only a linear relationship between the dependent and independent variables (i.e. $y=w^Tx+\epsilon$), and fits the regressor coefficients based on this assumption. If you visualise the situation in 1D, you'll see that you end up fitting a line to a piecewise staircase function. I think, even intuitively, that doesn't seem so safe.

More generally, ordinal regression models assumes a linear relationship between a latent dependent variable and the independent variables. The latent DV is not present in the data, (e.g. the actual opinion of a person, say $4.312$ instead of $4$). The actual outputs are the converted versions of these latent outputs. Here, ordinal regression assumes a set of thresholds and solves the entire system to find both the thresholds and the regressors. The ordinary regression doesn't have thresholds, so basically, it's like $y_{latent}=y$, which is a more constrained model.

$\endgroup$
  • $\begingroup$ I see how the approach proposed might not be safe, but I was looking for a proof of why it would be bad to tackle the problem in that way. $\endgroup$ – datapug Feb 10 at 16:12
  • 1
    $\begingroup$ @datapug, It is enforcing that your dependent variable has linear relationship with IV. But, including thresholds makes the model nonlinear. You are trying to model a possibly non-linear relationship with a linear model. $\endgroup$ – gunes Feb 10 at 16:55
1
$\begingroup$

Severe problems with using linear models to analyze ordinal Y have been extremely well documented in the literature. Spend a little time searching the literature. See for example this paper. Think of an ordinal variable as non-numeric, e.g., letters A,B,C,...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.