Logistic Regression - outcome influences a predictor - help conceptualizing

Question

I could really use some help conceptualizing why I think this is a problem. In a logistic regression analysis I am reviewing the one of the predictor variables is conceptualized using the outcome variable. Due to confidentiality, I can't give you the exact variables, but I will try to give a parallel example.

Outcome - Whether or not someone took medical leave? Predictors - age gender severity of the incident

Severity of the incident is conceptualized as the percentage of time someone with that particular accident/injury took medical leave. So losing a limb would have a severity rating of 99 because 99% of the time someone would take medical leave, but breaking an arm would have a score of 20 because only 20% of people who break their arm take medical leave.

Isn't there a problem with assessing severity in that manner? This seems inappropriate, but I am having trouble conceptualizing exactly why from a statistical perspective.

I'm more of a consumer of statistics. My statistics days are far gone so I don't need an overly technical answer. I appreciate your help!

Answer 1

If, in your example, you are trying to build a system to predict future instances of taking medical leave, then there might be a danger but perhaps not the one you fear.

There does seem to be a circular logic in measuring severity of an injury by the probability of taking leave with a particular type of injury, and if the severity indices were determined from your data set then you would presumably have a completely over-determined model. If the severity indices were established from some other data set and just applied here, however, there might not be a problem. In principle in your example, you might then be able to examine how age and gender are related to taking leave once the severity (as separately defined) is taken into account.

A danger in logistic regression is that your variables explain so much of the difference in outcomes that the model is completely fit and might not generalize well. That is called perfect separation, and it can happen in what might seem naively to be under-determined models if there are multiple predictors. This would be expected to be more of a danger when one of your predictors is so directly tied to outcome, even if the predictor wasn't defined by the data set you are trying to analyze.

Then again, you presumably would want your model to predict that someone takes medical leave after losing a limb. So you do have to consider the purpose of building your model when you evaluate the use of such a predictor.