I am implementing a Multinomial Logistic Regression, but I am encountering the possible issue of having very small groups when I create a frequency table of the dependent variable Y and one of the independent variables X (see table). Would this affect my results in a problematic way? Or should I for example just exclude this variable X in my regression (and with that ignore that these 2 variables seem to "correlate" to some extent)?

-------- y0 --- y1

x0: --- 180 ---   2

x1: ---  50 ---   2

x2: --- 300 ---   4

x3: ---  50 --- 350

x4: ---  50 ---  20
  • 1
    $\begingroup$ Small sizes are always problematic for inference, as in you will have very low power to detect any significant change, however in terms of doability, as long as it works you can do it. More importantly, why do x0-x2 have such small y1 sizes? $\endgroup$ – user2974951 Jan 11 at 12:51
  • $\begingroup$ The small y1-sizes are due to the measuring of rare events I would say. Gathering more data is in practice not a option at this point. $\endgroup$ – Zeven Jan 11 at 13:56
  • $\begingroup$ Try to run the model then and see if there are issues. $\endgroup$ – user2974951 Jan 11 at 14:13
  • $\begingroup$ One thing you might consider doing if you can't seem to get enough rare events by each of your predictors is to collapse categories of categorical predictors where it makes sense. For example, if you have race as a predictor and you have five race categories like White, Black, Indian, Asian, and Pacific Islander, and too few Pacific Islanders have the rare event, you could consider re-coding race as White vs. Non-white (collapsing Black, Indian, Asian, and Pacific Islander in 1 category) if this made sense in your study and you didn't need to break your estimates out by race. $\endgroup$ – StatsStudent Jan 11 at 19:35

There is nothing wrong with taking different analytic and reporting approaches for different predictors. For predictors other than X you might report regression coefficients, while for X you might simply say that Y1 events were rare when X was 0-2, uncommon when X = 4, and very common when X = 3. But honestly the X-Y relationship seems so obvious, has such an interocular effect, that one might wonder why an investigation of it through regression would be necessary. Sometimes analysts feel the need to include whatever variables can add to their model's fit, even if some such variables are more or less proxies for the outcome and thus add little in the way of explanation, or even hinder it.

You will probably obtain more helpful answers if you can say what is your topic, what X and Y are, how many and what sorts of other predictors you are using, and whether you are analyzing for prediction or for explanation.


Working on very small groups can cause various technical issues. In that case you might want to take a look at penalised regression techniques (such as Firth's penalisation of the logistic regression) to deal with issue of data separation and finite-sample estimation bias.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.