Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am building a model and I think that geographic location is likely to be very good at predicting my target variable. I have the zip code of each of my users. I am not entirely sure about the best way to include zip code as a predictor feature in my model though. Although zip code is a number, it doesn't mean anything if the number goes up or down. I could binarize all 30,000 zip codes and then include them as features or new columns (e.g., {user_1: {61822: 1, 62118: 0, 62444: 0, etc.}}. However, this seems like it would add a ton of features to my model.

Any thoughts on the best way to handle this situation?

Thanks for any help.

share|improve this question
    
Just a thought.. but, if zipcodes are distributed geographically then you could geographically represent zipcodes in a map and represent them with their location. With that you could also see which zipcodes are closeser.. –  Manuel Apr 23 at 18:40

5 Answers 5

up vote 7 down vote accepted

One of my favorite uses of zip code data is to look up demographic variables based on zipcode that may not be available at the individual level otherwise...

For instance, with http://www.city-data.com/ you can look up income distribution, age ranges, etc., which might tell you something about your data. These continuous variables are often far more useful than just going based on binarized zip codes, at least for relatively finite amounts of data.

Also, zip codes are hierarchical... if you take the first two or three digits, and binarize based on those, you have some amount of regional information, which gets you more data than individual zips.

As Zach said, used latitude and longitude can also be useful, especially in a tree based model. For a regularized linear model, you can use quadtrees, splitting up the United States into four geographic groups, binarized those, then each of those areas into four groups, and including those as additional binary variables... so for n total leaf regions you end up with [(4n - 1)/3 - 1] total variables (n for the smallest regions, n/4 for the next level up, etc). Of course this is multicollinear, which is why regularization is needed to do this.

share|improve this answer
1  
You can get a nice visual of the hierarchy at Ben Fry's zipdecode. –  Dimitriy V. Masterov Apr 25 at 17:54
    
Joe are you able to freely get zip level data (income etc) from this site? I was not able to see how to do this. –  B_Miner Jul 9 at 12:03
    
URLs like this: city-data.com/zips/02108.html It's not well formatted like a CSV, so you have to use regexes/scraping etc. The US census has some nicely fomatted data census.gov/epcd/www/zipstats.html and factfinder2.census.gov/faces/nav/jsf/pages/index.xhtml but they don't have the breadth. –  Joe Jul 9 at 20:39

There's 2 good options that I've seen:

  1. Convert each zipcode to a dummy variable. If you have a lot of data, this can be a quick and easy solution, but you won't be able to make predictions for new zip codes. If you're worried about the number of features, you can add some regularization to your model to drop some of the zipcodes out of the model.
  2. Use the latitude and longitude of the center point of the zipcode as variables. This works really well in tree-based models, as they can cut up the latitude/longitude grid into regions that are relevant to your target variable. This will also allow you to make predictions for new zipcodes, and doesn't require as much data to get right. However, this won't work well for linear models.

Personally, I really like tree-based models (such as random forest or GBMs), so I almost always choose option 2. If you want to get really fancy, you can use the lat/lon of the center of population for the zipcode, rather than the zipcode centroid. But that can be hard to get ahold of.

share|improve this answer
    
Definitely will go for the 2 proposal. –  andi Apr 24 at 8:51
    
#2 also appears to works with a GAM –  Affine Apr 25 at 17:59

I dealt with something similar when training a classifier that used native language as a feature (how do you measure similarity between English and Spanish?) There are lots of methods out there for determining similarity among non-categorical data.

It depends on your data, but if you find that geographic distance from a zip code is not as important as whether a given input contains particular zip codes, then non-categorical methods might help.

share|improve this answer

You could transform your zip code into a nominal variable (string/factor). However, as far as I remember, zip code might contain other information like county, region, etc. What I would do is to understand how zip code encodes information and decode that into multiple features.

Anyway letting zip code as a numeric variable is not a good idea since some models might consider the numeric ordering or distances as something to learn.

share|improve this answer
    
Thanks for the answer! However, even if zip code is a string or factor, aren't I essentially just dummy coding zip code (i.e., creating 30,000 binarized features)? I know R does this under the hood but it has to be explicitly done in scikit learn. –  ansonw Apr 23 at 18:39

I would make a choropleth map of your model's residuals at the zip code level.

The result is called a spatial residual map and it may help you choose a new explanatory variable to include in your model. This approach is called exploratory spatial data analysis (ESDA) .

One potential workflow:

  1. for each zip code get the average residual
  2. make a choropleth map to see the geographic distribution of the residuals
  3. look for patterns that might be explained by a new explanatory variable. For example, if you see all suburban or southern or beach zipcodes with high residuals then you can add a regional dummy variable defined by the relevant zipcode grouping, or if you see high residuals for high income zipcodes then you can add an income variable.
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.