# How to Deal with Large Number of Dummy Variables in Machine Learning? [duplicate]

I have a cross-sectional real estate dataset with information on roughly 100000 properties, including rental price, square meter size, number of bedrooms etc. In addition, the dataset contains information about the region for each property. The total number of regions in the dataset is 400.

I would like to play around with different machine learning methods in the caret package to predict rental prices using the relevant features in the dataset. I obviously need to incorporate regional information to capture region-specific price effects.

Update: To be more formal, I have the following two model specifications in mind:

$y_i=\beta_0+\beta_1x_{i,1}+\beta_2x_{i,2}+\beta_3D_{i,1}+\beta_4D_{i,2} + \epsilon_i$

where $y_i$ refers to the rental price of property $i$, $x_{i,1}$ denotes square meter size, $x_{i,2}$ denotes the number of rooms, $D_1$ is a matrix of dummy variables capturing different characteristics like balcony, kitchen, cellar and so on, and $D_2$ is a matrix containing the 399 regional dummies.

Alternatively, the model could be specified as follows:

$y_i=\beta_0+\beta_1x_{i,1}+\beta_2x_{i,2}+\beta_3D_{i,1}+\gamma d_{j} + \epsilon_i$

where $d$ is a vector containing the median or average square meter price in each region $j$. I guess inclusion of interaction effects could also make sense (see comment below by seanv507), but I am not sure about them right now.

The obvious advantage of the last equation is that I only need one variable in my model to proxy for a regional effect. However, I also do not see too much of a problem for the first equation given the relatively large number of observations in the dataset.

I have two questions:

1. Which of the two specifications would be preferable or are there better alternative ways to deal with this issue?
2. Which machine learning methods could be promising for this type of model?
• There are many questions on this site about similar problems, such as: stats.stackexchange.com/questions/118255/… stats.stackexchange.com/questions/201287/… stats.stackexchange.com/questions/50636/… stats.stackexchange.com/questions/167697/… continued ... Aug 19, 2016 at 14:42
• Aug 19, 2016 at 14:54
• Which dataset is it? It's not the Ames housing dataset. Be careful with your model's twin assumptions and lack of two-level and three-level interactions, it could be that the living-room area matters disproportionately, or number of ground-floor rooms, or area of the master bedroom, or average area of the largest two bedrooms, etc. You'll need an interaction matrix between the $D_{i,1}$ to capture those...
– smci
Mar 2, 2020 at 9:16
• ...and if there are regional preferences for those features (e.g. Arizonians like swimming pools or solar more than northerners, who (say) value basements), you'll also need an interaction matrix between the $D_{i,1}$ and $D_{j,2}$. Btw, how large is $D_{1}$?
– smci
Mar 2, 2020 at 9:17