4
$\begingroup$

I'm new to machine learning, and I'm trying to get a sense of how you optimize data for a model. I'm following this official Kaggle tutorial, which teaches the basics of machine learning through house price prediction. They use a decision tree, but I found it odd which features they feed into the model to predict the price of a house:

house_price_features = ['Rooms', 'Bathroom', 'Landsize', 'Latitude', 'Longitude']

Rooms, bathrooms, and landsize all make sense to me - but latitude and longitude? Obviously there is a correlation between location and price, but it's not going to follow a nice curve. Sometimes, going a block up will increase house prices twofold; sometimes, it'll have no effect at all. Intuitively, I feel like all a model can do with those features in predicting price is overfit. So, my question is twofold:

  1. Were they right in giving this model latitude and longitude to predict price, or is this extraneous information that can only hurt the model? Why?
  2. If the answer to the above is "no", is there any transformation of the latitude and longitude data (i.e. into neighborhood IDs) that would make the data more helpful?
$\endgroup$
3
  • $\begingroup$ I am guessing that a 5 bedroom house on 2000 square feet of land in Manhattan may be more expensive than the same house in rural South Dakota. The issue with "not going to follow a nice curve" is part of the machine learning challenge $\endgroup$
    – Henry
    Commented Aug 16, 2020 at 3:17
  • $\begingroup$ Related: stats.stackexchange.com/questions/35510/… $\endgroup$
    – Adrian
    Commented Aug 17, 2020 at 20:05
  • $\begingroup$ Should the title say "Providing" rather than "Proving"? Or "Including ... in a ..."? $\endgroup$
    – Adrian
    Commented Aug 21, 2020 at 0:14

1 Answer 1

10
$\begingroup$

The answer is yes because location usually is the main driver of house prices per square feet. Dropping it would deteriorate the model performance probably in a dramatic way.

Based on lat/lon, tree-based methods divide the map in rectangular pieces. The stronger the effect and the more data in a certain area, the smaller the pieces. In less dense regions, the pieces would not be too small.

You would not add them as linear effects in a linear regression. There, you would need to consider different approaches. A simple would be to represent lat/lon each by a cubic spline and add interaction terms between them.

$\endgroup$
2
  • $\begingroup$ When you put it that way, it makes a lot of sense! Thank you, I'll accept your answer as soon as I'm able. $\endgroup$
    – Nico A
    Commented Aug 15, 2020 at 17:03
  • $\begingroup$ "tree-based methods divide the map in rectangular pieces" - so it could be OKish but not perfect ... there are also oblique decision trees but even those would have to approximate potentially complex polygons ... $\endgroup$
    – cs0815
    Commented Jun 20, 2022 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.