# How can factor-levels be automatically chosen in R to maximize the number of positive coefficients in a regression model?

I am performing simple (first-order terms) linear regression on data having several categorical variables (i.e. factors), and it is often desired that for each factor, one of the levels should add nothing to the regressand, and the other levels should add positive values to the regressand. When I perform a regression analyses, however, I often get a lot of negative coefficients.

Is there a non-manual way of choosing which levels of a factor should be used as regressor variables in order to maximize the number of positive coefficients in the equation? In other words, how can I get R to do this (somewhat tedious) task for me?

-
Find the factor with the smallest coefficient, make it the new base level, and re-run the regression (or just modify all related coefficients if all you want is their estimates). – whuber Jan 17 '12 at 23:08
i think this rather is a programming related question. whuber did give a noteworthy attempt. – Seb Jan 18 '12 at 6:39
@Seb (About the flag you raised) I believe this question should stay here because there is some statistical background. – chl Jan 18 '12 at 11:20

## 2 Answers

Here is an attampt of doing what you wanted.

 # Setting up some sample data
require(dummies)
df <- data.frame(categorial=rep(c(1,2,3), each=20), x=rnorm(60))
flevels <- dummy(df$categorial) df$categorial <- factor(df$categorial) df$y=20 + df$x*3 + flevels%*%c(3,1,2) + rnorm(60)*2  I use a regression in order to obtain the minimum factor level and then reorder:  # Now we start with trying to find the minimum cateogry. However, note that this does not work in every context! summary(helpreg <- lm(y~x+factor(categorial) - 1, data=df)) Coefficients: Estimate Std. Error t value Pr(>|t|) x 2.9944 0.2334 12.83 <2e-16 *** factor(categorial)1 22.9640 0.4472 51.35 <2e-16 *** factor(categorial)2 21.0720 0.4390 48.00 <2e-16 *** factor(categorial)3 22.1300 0.4364 50.71 <2e-16 ***  Then I start to sort out the minimum:  factors <- grep('categorial', names(coef(helpreg))) # --- replace categorial with your variable name minimumf <- which(coef(helpreg)[factors]==min(coef(helpreg)[factors]))  This is then releveled  df$categorial <- relevel(df\$categorial, ref=minimumf)


And in my case it works - probably it works for you as well....

 summary(lm(y~x+factor(categorial), data=df))

Estimate Std. Error t value Pr(>|t|)
(Intercept)          21.0720     0.4390  48.003  < 2e-16 ***
x                     2.9944     0.2334  12.828  < 2e-16 ***
factor(categorial)1   1.8920     0.6341   2.984  0.00421 **
factor(categorial)3   1.0580     0.6193   1.708  0.09310 .


Comments of course highly appreciated!

-
 This seems to work well for data having a single categorical variable (except in the case of identical minimum coefficients) -- will this extend to cases where there are more than one categorical variable, by just releveling one variable at a time? – Mark Jan 18 '12 at 15:07 I think it should work - at least if they are orthogonal onto each other... the problem is that my approach with the auxiliary regression is not probable anymore due to singularities. therefore you need to run 2 regressions in advance. seperating factors will not work if the two categorial variables are correlated. i will think about an approach where both categorials are included. – Seb Jan 18 '12 at 15:15 Another issue I noticed with your example is that it only considers the non-base levels, since it derives the list of levels from names(coef(helpreg)), and helpreg doesn't include the base levels. – Mark Jan 18 '12 at 17:43 @Mark, I'm sorry but I don't quite understand your issue. I explicitly excluded the intercept (-1) therefore all levels should be included. the first regression output above features all 3 variables (but no intercept). or didn't I get your point right? – Seb Jan 18 '12 at 18:12 see my answer below, and how I use the all_levels vector. – Mark Jan 18 '12 at 18:54
show 4 more comments

Thanks to whuber's comment, and Seb's answer, I have put together the following function which I believe does what I want. Hopefully it will be useful to someone. Comments are welcome.

# take a dataframe, and re-level it such that the levels of the factors are
# assigned positive coefficients by lm()
# NOTE: this currently only works for model-forms that don't include
#       interaction terms.
auto_relevel <- function(df, model_form)
{
# get list of categorical variables in df
catvar_indices <- get_catvar_indices(df)

# loop over categorical variables
df_colnames <- attr(df, 'names')
model_form_zeroicept <- paste(model_form, "- 1")
for (i in catvar_indices) {
catvar_name = df_colnames[i]
all_levels = attr(df[[i]], "levels")
temp_model <- lm(model_form_zeroicept, data=df)

# If at least one of the levels' coefficients is less than zero, then
# choose the one w/min coeff to be the new base-level

# put a space after catvar_name so that it doesn't match longer level-names
catvar_name <- paste(catvar_name, " ", sep="")
factors <- grep(catvar_name, names(coef(temp_model)))
coeffs <- coef(temp_model)[factors]
# remove NA's from coeffs
coeffs2 <- coeffs[! is.na(coeffs)]

if (any(coeffs2 < 0)) {
# find out where this factor is in *all_levels*
chosen_level_name <- names(coeffs2)[which(coeffs2==min(coeffs2))]
stripped_level_name <- unlist(strsplit(chosen_level_name," "))[2] # strip factor name
# add an initial space (to match all_levels)
stripped_level_name <- paste(" ", stripped_level_name, sep="")
min_level_index <- which(all_levels == stripped_level_name)
df[[i]] <- relevel(df[[i]], ref=min_level_index)
}
}

return(df)
}

-