I am working on a Random Forest regression model to predict housing prices. I have about 500k rows of data with the following information:

1.House area in square meters.

2.Number of rooms.




6.The transaction date.

7.Type of house (single house, apartment building etc.)

8.The amount paid for the house.

I am planning on making a different model for each city, but I'm having trouble in representing the street name. I was thinking on using One Hot Encoder to represent the street name but some cities have over 1000 streets and that would give me over 1000 variables with moslty zero values.

I have read about sparse representation but I don't know how to use it in practice.

Let's say I already have a sparse representation of my data, how do I feed it to the Random Forest? Does the Random Forest Regressor from the sklearn library in Python support sparse data? If not, then is there another way to go about using Random Forest with sparse data in Python?

  • $\begingroup$ If you have a categorical variable with over 1000 levels you are in trouble. Most models would die here. Either aggregate the variable into a few sensible clusters or drop it. $\endgroup$ – user2974951 Feb 4 at 8:50
  • $\begingroup$ @user2974951 I disagree that "most models would die here", but 1,000 levels is too many to use as-is in a random forest. $\endgroup$ – shadowtalker May 14 at 0:57

There are 2 ways to approach this problem:

  1. Convert each categorical features to several binary indicators, a process known as "one-hot encoding"
  2. Apply a transformation known variously as "target encoding" or "impact coding" that replaces the categorical feature with a numerical one.

You should be able to use any of those terms to get you started in your search. The target encoding method is likely to be the most useful here; look up the library "category_encoders" for a Python implementation, and "vtreat" for R.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.