5
$\begingroup$

I am working on a problem where I need to do regression and I have a mix of about 40 numerical variables and 40 non-numerical (aka categorical/factors) variables. Are there any established algorithms for such problems? I don't think converting the non-numerical variables into numerical variables is a good idea in this problem as they are just "labels". They don't measure anything.

$\endgroup$
  • $\begingroup$ do you really mean "dependent" in the title, or do you possibly mean "independent"? (The ease of confusing these is one reason I prefer the terminology "predictor" (= "independent") and "response" (="dependent"). If you have multiple dependent variables, then you're looking at a multivariate response, which is a much more complicated situation than having many predictor variables ... $\endgroup$ – Ben Bolker Oct 17 '16 at 1:34
  • 1
    $\begingroup$ What is the type of your dependent(outcome) variable? If your dependent(outcome) variable is categorical, you can use logistic regression. If you have continuous outcome variable, you can use, ANCOVA(Analysis of Co-variance). ANCOVA combines is the mixture of ANOVA and Regression. $\endgroup$ – Rajan Kharel Oct 17 '16 at 2:01
  • $\begingroup$ Thanks to everyone for helping me out. I used randomForest package in R and it seemed to handle the mix of numerical and factor variables better than I had thought. $\endgroup$ – morpheus Oct 17 '16 at 5:45
  • 1
    $\begingroup$ this has already been answered, check: stats.stackexchange.com/questions/267121/… $\endgroup$ – user3280076 Nov 13 '17 at 14:11
3
$\begingroup$

First off, it depends what your dependent variable (Y) is. If it is numerical then most multiple regression models would be sufficient. If it (Y) is categorical then you need a logistic regression or a similar categorical regression model.

As for how to handle independent variables, the numerical ones will fit neatly into almost any regression model. The categorical ones will need to be "factored". I use R. In R, you specify categorical variable "k" in a data frame by running

Data.Object[DollarSign]k <- factor(Data.Object[DollarSign]k)

In other languages/softwares you do it differently. But definitely make sure the categorical data is being treated as such regardless of which software you use. No, there is no magic algorithm or software that lets you wave a magic wand and factors all these for you. This is a common problem, and if you think 40 is bad, consider the problem with 100.

As for the "best" regression to run when thrown a data set like yours?...well, it depends on what you/your bosses are looking for.

The tricky part for you is interpreting the labels into something meaningful. Say you have a variable "What is your political party?" If 1 is "republican", 2 is "democrat" and 3 is "independent" then you won't have a variable for each. You'll have a variable for "democrat" and "independent" and both values equaling zero would mean "not democrat and not independent". In this case, the regression coefficient of "democrat" will show the change in Y if the individual is a democrat. The coefficient for "independent" would show the change in Y if the person is independent.

Most importantly, the base case, for all other coefficients, is a republican. So any interpretations you make for other variables should adjust for that base case. There ARE algorithms to make that adjustment easier, but I don't know any off the top of my head.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

Almost all the regression algorithms take care of both numerical and categorical variables. For categorical variables, there are different "coding" can be used.

The Simple example is binary coding. For example, for gender, you can use $0$ to represent male and $1$ to represent female. If the variables has more then $2$ values, one hot coding can be used.

Details can be found

http://www.ats.ucla.edu/stat/r/library/contrast_coding.htm

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.