We have an independent ordinal variable which may or may not explain a dependent binary target. We would like to know if this effect is significant, for example linearly. Doing a logistic regression and testing the parameter for a significant difference from 0 makes sense. Someone proposed to compute the observed probabilities per ordinal group and do a linear regression or correlation test over this. I feel like this is wrong but I can't properly articulate why. I feel like you throw away a lot of information, mostly about sample size. What are other reasons why you shouldn't do this?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ It's not entirely clear what is being suggested, but surely unless you have a large number of levels in your ordinal variable (say >10) then surely you're going to have too little data to regress the probability per level onto the ordinal variable? $\endgroup$– Ian_FinCommented Oct 14, 2016 at 8:04
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
To begin with, as you state, the suggested alternative throws away information about the sizes. So, for example, say the ordinal variable takes values in $\{0, 1, 2, 3, 4, 5\}$. Furthermore, say there were
1,000,000 samples with 2 as the independent value, where the dependent value tended to be 0.
1,000,000 samples with 3 as the independent value, where the dependent value tended to be 1.
A single value each for ${0, 1}$ as the independent variable, where the dependent value was 1.
A single value each for ${4, 5}$ as the independent variable, where the dependent value was 0.
A linear predictor (at least an unweighted one) over the aggregation for each value of the independent variable, would not recognize that that the linear function should be ascending, which doesn't make sense when considering the problem.
Another thing to note is that there's a fundamental difference between logistic regression and linear regression over the aggregates. Logistic regression, finds a classification with a linear boundary between the two classes. Linear regression over the aggregates finds the best fit mapping an increase in the independent variable to a linear increase in the probability of obtaining a specific class in the dependent variable. These two things are not the same.
In fact, if the point is to find the significance of a linear boundary (which is what logistic regression does), it can be found more directly. Say the ordinal variable takes a small number $m$ of values.
For each of the possible $m - 1$ possible threshold values for $t$, divide the dependent variable into two sets depending on whether the corresponding independent variable is below or above $t$, and run a Mann-Whitney test.
For the best threshold, say the significance of the Mann-Whitney test was $\alpha$. Using the Bonferroni correction justification, you can claim the significance is at least $\frac{\alpha}{m - 1}$.
-
$\begingroup$ The OP's question is what's wrong with calculating the probabilities per ordinal group and regressing these probabilities onto the ordinal variable. Your answer appears to suggest an alternative, but never actually addresses the OP's specific question. $\endgroup$– Ian_FinCommented Oct 14, 2016 at 8:55
-
$\begingroup$ @Ian_Fin Thanks for your correct comment - you're right. I've updated the answer to focus on the difference between what logistic regression, and linear regression on aggregates, checks. $\endgroup$ Commented Oct 14, 2016 at 9:28