# Grouping predictor factor levels based on response variable [duplicate]

I've read that it's bad to do this, but am looking for details as to why.

Suppose we're trying to fit the linear model $Y_i = \beta_0 + \beta_1 X_{1i} + \beta_2 X_{2i} + \epsilon_i$ where $Y$ is continuous and $X$s are categorical with many levels. We don't have any opinion on how the levels should be grouped. We then calculate the average response for each level of the predictors, then collapse levels with similar averages.

How badly does this screw up parameter estimates and inferences on them, and why?

## marked as duplicate by kjetil b halvorsen, Peter Flom - Reinstate Monica♦ regression StackExchange.ready(function() { if (StackExchange.options.isMobile) return; $('.dupe-hammer-message-hover:not(.hover-bound)').each(function() { var$hover = $(this).addClass('hover-bound'),$msg = $hover.siblings('.dupe-hammer-message');$hover.hover( function() { $hover.showInfoMessage('', { messageElement:$msg.clone().show(), transient: false, position: { my: 'bottom left', at: 'top center', offsetTop: -7 }, dismissable: false, relativeToBody: true }); }, function() { StackExchange.helpers.removeMessages(); } ); }); }); May 18 '17 at 11:36

Inferences will probably be a little more tricky. Standard theory is not valid any more, since you are transforming your data after estimation. So don't look for $t$ tables to read off $p$ values. However, you are not filtering on low $p$ values, nor looking for "optimal cutpoints", but explicitly pooling levels with similar parameter estimates - so this could actually not be too far away from the standard tables.
• sorry I'm a bit slow but regarding "Standard theory is not valid any more, since you are transforming your data after estimation." -- what assumptions in particular fail to hold? is it because we have some sort of mixture distributions for the $t$ statistics due to collapsing or some multiplicity issue? – kevinykuo Apr 3 '14 at 5:11
• Look at the derivation of why $t$ statistics of regression coefficient estimates are $t$ distributed. This is entirely driven by data and observations separately. Pooling amounts to looking at the data and the observations, transforming the data (pooling factor levels) and re-estimating. The final design matrix is not independent of the data any more. – S. Kolassa - Reinstate Monica Apr 3 '14 at 7:23