I'm working with a dataset containing crimes data from Chicago. There's a lot of geographical data, and I'm looking for advice on pre-processing.

We have qualitative variables represented by integers, the blocks for example : 2 blocks with a close number aren't necessarily close to each other.

  • Will the algorithms learn based on an order that doesn't mean anything ? If so, which machine learning algorithms can/can't I use ?
  • Is it better to insert dummy variables (one column per value) ? But what if there is a lot of values (more that 30 000 here) ?
  • $\begingroup$ Treating the blocks as dummy variables completely loses all information about their geographical proximity, which is believed by most researchers to be a key factor in understanding crime. There are many ways to capture that spatial information: please refer to the literature on spatial crime analysis for some of the methods. $\endgroup$ – whuber Mar 13 '17 at 14:30
  • $\begingroup$ I don't see how the address numbers contain any more information than simply the geographic coordinates. (They could in some circumstances, e.g. sharing a large apartment complex, but that is not contained directly in the address number.) $\endgroup$ – Andy W Mar 13 '17 at 14:31
  • $\begingroup$ @andy-w So according to you, I could keep only coordinates ? I see in my dataset that some areas have a 'strange' shape and correspond to a zone covered by the same police team for example, so it obviously affects the way the crime is treated (whether an arrest is made for example). How will this information appear in my coordinates ? Am I not losing something here ? $\endgroup$ – MeanStreet Mar 14 '17 at 17:54
  • $\begingroup$ Spatial information is associated with points, linear features, or regions. One can attach that information to any given point by finding the features it lies in or next to and using the data in those features (possibly weighted by distance). Thus, having an accurate coordinate for a location gives you access to a vast amount of data curated by government agencies, scientific institutions, and more. $\endgroup$ – whuber Mar 14 '17 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.