When should I balance my data for GLM/GAM?

Question

I'm running GLMs and GAMs and I can't find a clear answer about if I should be balancing my data or not.

I'm trying to get useful descriptive models, not predictive models. So I haven't split my data into training and test. Two reasons, I didn't think that was necessary (since it isn't a predictive model) and I don't have a huge amount of data to begin with (200 samples for males, 100 samples for females). I'm investigating what factors (10) impact mortality after heart failure.

For some factors I have a massive imbalance. For example, I've separated the data on sex and 97% of females are non-smokers.

Would be it best to just omit this variable or do I attempt some balancing techniques? Do I leave it alone since it may just be representative of the real-world? From what I've read, balancing data means the model may no longer reflect reality. But in the case of the female smokers, it seems I just do not have enough data to see whether smoking impacts mortality after heart failure in females.

I wanted to apply a confusion matrix but I'm not predicting, so it doesn't seem applicable.

Answer 1

+1 to usεr11852's answer.

Don't worry about "unbalanced" data, as long as you use appropriate models. Are unbalanced datasets problematic, and (how) does oversampling (purport to) help? GLM/GAM are appropriate.

I didn't think that was necessary (since it isn't a predictive model)

Over-/undersampling does not make sense, whether you are using it in a predictive or inferential model.

I don't have a huge amount of data to begin with (200 samples for males, 100 samples for females). I'm investigating what factors (10) impact mortality after heart failure.

Assessing 10 factors with a sample size of only 300 is dubious, especially since you apparently also include interactions, like sex by smoking status. Consider using a more parsimonious model.

But in the case of the female smokers, it seems I just do not have enough data to see whether smoking impacts mortality after heart failure in females.

That is quite possible. Low information content, like having only three female smokers, will mean that your parameter estimates will be uncertain. That is unfortunately just a fact of your data. Oversampling these individuals will pretend to the model that you have more data than you really do, and parameter estimates will be more "certain" after oversampling - but of course you do not really have more data, you are just counting some observations multiple times. The only way to address this issue is to collect more data.

Answer 2

I would strongly suggest to avoid rebalancing your data at first instance.

Especially when using a GLM/GAM to describe the tendencies of the sample at hand it makes little sense to try to up- or down-sample the original sample as this would immediately change the base-rates. GLM/GAM predictions are generally "well-calibrated" (i.e. there are accurate as probabilities estimated by the model) and resampling with destroy the intercept ($\beta_0$) that anchor our estimates.

We might want to use some re-sampling techniques (e.g. bootstrapping) to better inform our audience for aspects of sampling variance and finite-sample precision but that's a different point. Similarly if we care about predictive performance more, using regularisation (e.g. a ridge regression model) might be preferable, but again not a necessity for a "descriptive model"). Please note that, yes, it is perfectly reasonable to be worried about certain covariates having small absolute counts but that uncertainty will be captured by the resampling and/or the standard error associated with that coefficient.