13k views

### Principled way of collapsing categorical variables with many levels?

What techniques are available for collapsing (or pooling) many categories to a few, for the purpose of using them as an input (predictor) in a statistical model? Consider a variable like college ...
5k views

### Linear regression including categorical variables with hundreds of levels [duplicate]

I am trying to teach myself data science by solving some of the problems available on the internet. Currently I am trying to predict a fraud event with the aid of 4 categorical variables. Each of the ...
847 views

### Grouping predictor factor levels based on response variable [duplicate]

I've read that it's bad to do this, but am looking for details as to why. Suppose we're trying to fit the linear model $Y_i = \beta_0 + \beta_1 X_{1i} + \beta_2 X_{2i} + \epsilon_i$ where $Y$ is ...
661 views

### categorical data Analysis with too many levels [duplicate]

I am running multinomial logistic regression Model for rating Hotels. I have a variable called CHAIN ID which is a bunch of numbers but there are too many such IDs,...
147 views

### Reduce feature levels [duplicate]

I would like to know if someone knows of a way to group the number of levels of a feature that has 100's (even 1000's) of levels to a smaller number of levels - also, what number levels it should ...
76 views

### Clustering, reducing number of levels of categorical variable [duplicate]

I'm dealing with this big dataset which has: 1 categorical variable with 90 levels that represent some sort of "geographical area" 3 continuous variables What I'm trying to do is to "aggregate" the ...
56 views

### How to deal with categorical features with thousands of levels? [duplicate]

I have a query that I perform, I have a dataset that has several categorical features with thousands of levels. Applying get_dummies would generate a dataset that I could not work with. I would be ...
11k views

### R package for combining factor levels for datamining?

Wondering if anyone has run across a package/function in R that will combine levels of a factor whose proportion of all the levels in a factor is less than some threshold? Specifically, one of the ...
4k views

### Methods for merging / reducing categories in ordinal or nominal data?

I'm struggling to find a method for reducing the number of categories in nominal or ordinal data. For example, let's say that I want to build a regression model on a dataset that has a number of ...
13k views

### How to handle categorical predictors with too many levels? [duplicate]

I think it may be a problem if we directly use dummy variable for a categorical predictor having hundreds of levels. I have found one solution from the book 'Element of Statistical Learning'. The ...
5k views

### How to deal with categorical features in machine learning models?

For instance I have a set of categories from a variable called 'category'. From python, here are my categories and their corresponding counts. ...
3k views

### RandomForest factor with too many levels

I have a model with about 200,000 training observations, where I am regressing, with 4 factors and 2 continuous variables. One of my features has 927 levels, which is causing the R implementation of ...
2k views

### Pooling levels of categorical variables for regression trees

I have a data set I would like to do a regression analysis for. There are many features of both categorical and continuous types. One of the categorical features has many (>75) levels so this is an ...