# What is the risk of not pooling data

Neglecting sample size, is there something that I can miss if I choose to model separately each of the levels of my categorical variable?

To be more specific, I want to predict a binary outcome $$Y$$ with two predictors $$X$$ (continuous) and $$G$$ (categorical, $$n$$ levels). I could build one logistic regression model $$Y \sim X + G$$ or build $$n$$ regressions $$Y \sim X$$, one for each level of $$G$$.

What is the risk of not pooling the data in this latter scenario?

EDIT: I forgot to mention that I have good reasons to think that pooled data is not homogeneous and it's much more homogeneous inside each group.

• You will have smaller samples, hence fewer degrees of freedom in each model, leading to poorer estimates with greater standard errors. You will also have to perform $n\times$ the usual diagnostics. – Frans Rodenburg Apr 29 '19 at 14:46
• Thanks @FransRodenburg. However, I said "neglecting sample size" because I want to know if there is something else than poorer estimates and $n$X analyses that I could miss doing $n$ regressions – Patrick Apr 29 '19 at 14:50
• Another thing that comes to mind is that you will not be able to directly compare the effect sizes of each category, since they were fitted on different data. You are also more prone to overfit, unless you center each subset and estimate each model without an intercept. Why is sample size not compelling enough, though? Let's say you want to assess the predictive performance of your model. You will now need to split not just your original data, but each of the $n$ subsets into smaller subsets for validation. – Frans Rodenburg Apr 29 '19 at 15:20
• Sorry @FransRodenburg, I don't understand your overfit point. Why am I more prone to it unless I center each subset? – Patrick Apr 29 '19 at 15:28
• Have you not considered multi-level (hierarchical, mixed-effect) models? They provide a principled middle ground between pooled and independent models. – Wayne Apr 29 '19 at 22:13

The general principles behind preferring one overall model are explained quite nicely here. It's not just a matter of smaller $$N$$ with the separate analyses giving larger X-coefficient standard errors. You also typically will get variability in X-coefficient standard errors among the separate analyses, while the combined analysis takes information from all samples to get more reliable pooled error estimates. Also, if you are interested in whether the relationship of X to outcome differs among the levels of G (as you probably should be), then the pooled model would give a direct test of that possibility with an X:G interaction term in the pooled model. Attempts to pool results of separate analyses to examine interaction would be much more cumbersome and less reliable than a simple direct interaction test based on all the data.
• I realize that by adding an interaction term between $X$ and $G$, the model can have this kind of expressivity. – Patrick Apr 29 '19 at 20:17
• @Patrick yes, the interaction term allows for different slopes. And the combined analysis will typically provide a more reliable pooled error estimate for determining whether the slopes differ among levels of $G$, which is included in testing for significance of the interaction. – EdM Apr 29 '19 at 20:28