Getting rid of a huge categorical factor in multiple regression I have a large regression problem with a lot of cases, but relatively few independent variables. One of them is a categorical factor with thousands of levels. Robust regression runs forever. In some cases the large number of dummy variables becomes too sparse to calculate with even "normal" lm. 
What would usually make sense is to somehow calculate the average for each level of the factor, then adjust the dependent variable accordingly, and do the regression without the big factor. A colleague of mine could remember there is a two-letter R function that does that automatically, but he cannot remember the two letter combination.
Any help would be greatly appreciated.
 A: I would think lme4 would be highly appropriate for this.  Treat your huge categorical factor as a practical random effect.  I won't go into the theoretical definitions.  Alternatively, use sparse.model.matrix() from Matrix to build the design frame and then pass that into glmnet() from glmnet package.  (lme4 naturally builds the sparse design matrix so you don't need to use the sparse.model.matrix() before you go into it.)
If you really want to do the 'average for each level' trick, then be sure to calculate each observation's average excluding itself and include a few extra observations with each factor level at the population mean.  Then use this derived variable as a feature in your models instead of the categorical variable.  If the factor was the only feature, then this result would be identical to lme4 or glmnet (assuming you solved for how many average observations to add).
There are a few blog entries out there that call the 'average for each level' trick impact coding.  Also from my experience, if there is a strong dense feature, you might want to fit a simple model on that feature and impact code the residuals by level of the huge categorical factor instead of the pure response.
As mentioned above, this is more practical advice.  Other people will probably come along with some stronger theoretical advice.
