3
$\begingroup$

I have an OLS with continuous outcome and a mix of continuous/categorical predictors. For categorical predictor X, which has 5 categories, one particular category X_5 dominates in terms of volumes --- it is 70% of the data. The other 4 categories combined are only 30%. When I create dummies for these categories X1,...,X5 and use them in the regression, does it make a difference if I decide to use X_5 as the reference category?

$\endgroup$
4
$\begingroup$

It wont make a difference in terms of model predictions or in terms of measures such as $R^2$, AIC, etc. You can think of the coefficients for the category indicators as offsets to the the intercept, and whatever your base case is there will be a set of values for the intercept and the offsets that give the same set of predictions. Parameter estimation tries to find a set of coefficients that optimize the predictions on the data (usually maximizing likelihood). Since we can get the same intercept+offset values regardless of the base case, the models will be equivalent.

Where it is going to make a difference is in evaluating the coefficients for individual predictors. The base case gets absorbed into the intercept and doesn't have a separate coefficient estimate. If homoscedasticity holds, then the sampling error will be highest for the least frequent categories. So if you keep them as separate terms you can more easily evaluate whether or not they are all statistically distinct or if you might want to merge some of the smaller categories together.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.