how to represent geography or zip code in machine learning model or recommender system?

Question

I am building a model and I think that geographic location is likely to be very good at predicting my target variable. I have the zip code of each of my users. I am not entirely sure about the best way to include zip code as a predictor feature in my model though. Although zip code is a number, it doesn't mean anything if the number goes up or down. I could binarize all 30,000 zip codes and then include them as features or new columns (e.g., {user_1: {61822: 1, 62118: 0, 62444: 0, etc.}}. However, this seems like it would add a ton of features to my model.

Any thoughts on the best way to handle this situation?

Answer 1

One of my favorite uses of zip code data is to look up demographic variables based on zipcode that may not be available at the individual level otherwise...

For instance, with http://www.city-data.com/ you can look up income distribution, age ranges, etc., which might tell you something about your data. These continuous variables are often far more useful than just going based on binarized zip codes, at least for relatively finite amounts of data.

Also, zip codes are hierarchical... if you take the first two or three digits, and binarize based on those, you have some amount of regional information, which gets you more data than individual zips.

As Zach said, used latitude and longitude can also be useful, especially in a tree based model. For a regularized linear model, you can use quadtrees, splitting up the United States into four geographic groups, binarized those, then each of those areas into four groups, and including those as additional binary variables... so for n total leaf regions you end up with [(4n - 1)/3 - 1] total variables (n for the smallest regions, n/4 for the next level up, etc). Of course this is multicollinear, which is why regularization is needed to do this.

Answer 2

There's 2 good options that I've seen:

1. Convert each zipcode to a dummy variable. If you have a lot of data, this can be a quick and easy solution, but you won't be able to make predictions for new zip codes. If you're worried about the number of features, you can add some regularization to your model to drop some of the zipcodes out of the model.
2. Use the latitude and longitude of the center point of the zipcode as variables. This works really well in tree-based models, as they can cut up the latitude/longitude grid into regions that are relevant to your target variable. This will also allow you to make predictions for new zipcodes, and doesn't require as much data to get right. However, this won't work well for linear models.

Personally, I really like tree-based models (such as random forest or GBMs), so I almost always choose option 2. If you want to get really fancy, you can use the lat/lon of the center of population for the zipcode, rather than the zipcode centroid. But that can be hard to get ahold of.

Answer 3

If you are calculating distance between records, as in clustering or K-NN, distances between zipcodes in their raw form might be informative. 02138 is much closer to 02139, geographically, than it is to 45809.

Answer 4

I dealt with something similar when training a classifier that used native language as a feature (how do you measure similarity between English and Spanish?) There are lots of methods out there for determining similarity among non-categorical data.

It depends on your data, but if you find that geographic distance from a zip code is not as important as whether a given input contains particular zip codes, then non-categorical methods might help.

Answer 5

I would make a choropleth map of your model's residuals at the zip code level.

The result is called a spatial residual map and it may help you choose a new explanatory variable to include in your model. This approach is called exploratory spatial data analysis (ESDA) .

One potential workflow:

1. for each zip code get the average residual
2. make a choropleth map to see the geographic distribution of the residuals
3. look for patterns that might be explained by a new explanatory variable. For example, if you see all suburban or southern or beach zipcodes with high residuals then you can add a regional dummy variable defined by the relevant zipcode grouping, or if you see high residuals for high income zipcodes then you can add an income variable.

Answer 6

You could transform your zip code into a nominal variable (string/factor). However, as far as I remember, zip code might contain other information like county, region, etc. What I would do is to understand how zip code encodes information and decode that into multiple features.

Anyway letting zip code as a numeric variable is not a good idea since some models might consider the numeric ordering or distances as something to learn.

Answer 7

Any demographic data contain lots of categories, and there are different methods to convert categorical data into numeric. However, normal encoding method like one hot encoding might cause high cardinality issue or might get some issue related to memory.

In my opinion the best way to handle zip codes is to use "HashEncoder" or "TargetEncoder" .

Both these encoders are already available in category_encoders package in python.