I am trying to predict diabetes using the BRFSS dataset by using a supervised learning classification model. But I see that the target variable which is having diabetes or not is skewed. That is 90% of the records are non-diabetic and only 10% of the records are diabetic. How do I handle the skewness in the target variable?
When your data is skewed you may:
- use specific error metrics like precision, recall, F-score
- trade of between precision and recall accordingly:
- want to predict diabetes with confidence => adjust for higher precision, lower recall
- want to avoid missing too many diabetes cases => adjust for lower precision, higher recall
- (for example, in logistic regression, by adjusting the separating threshold)
- use F-score to find a good balance between precision and recall, that maximizes both as much as possible