I am trying to teach myself data science by solving some of the problems available on the internet.
Currently I am trying to predict a fraud event with the aid of 4 categorical variables. Each of the categorical variables have 100 of levels. Some of the levels occur frequently, some none at all.
Currently I have tried the following
- Throwing the categorical variable directly into linear regression. I get memory related errors
- Dummy encoding each categorical variable. Throw all of them into a linear regression. This also fails due to memory error
- Throwing the categorical variable into a Random ForestTM. This fails as Random Forest can only handle 52 distinct levels
None of the approaches I tried above have worked. What else can I try?
Any help is appreciated.
Update #1: The categorical variables are groupings like zip code, county, etc. In addition to these the customers are grouped into different buckets based on presence or absence of certain factors.
Update #2: I was able to use memory.limit() and increase the maximum memory size available to R. This has solved my memory issues.
Update #3: The current approach I was to use linear regression with regularization to find average fraud rate per factor level (for example average fraud rate per zip code). Now the categorical variable with hundreds of factors has been reduced to a numeric variable. I follow the same approach to convert all the 4 categorical variables into numeric. I throw these 4 numeric variables into a random forest. My results with the approach are okay with lots of room for improvement.
Update #4: I am using the R ecosystem