5
$\begingroup$

I have a set of houses with their features (location, size, number of rooms, etc … and the y is the price). In the future I will have a new house without the price. My goal is to find the 20 closest houses from this one.

I am working on a k-NN algorithm and I don’t take into account the prices of the other houses since when I’ll use the algorithm to retrieve the 20 most similar houses for a given house, I won’t have the price for this house. But each feature haven’t the same impact so I want to set the feature weights.

I was thinking about using a linear regression to determine the weight of each feature. The features will be x and the price y. And then I’ll keep the coefficient of the regression (the parameters) as the weight for my k-NN algorithm.

I haven’t find this kind of method to determine the weight of a feature. Is there a reason for that ? Or could it be a good approximation ? Do you recommend any other method to determine weights for a k-NN algorithm like this ?

Any input will be much appreciated !

Improve this question
$\endgroup$
1
  • $\begingroup$ I came across your question, which is exactly what I'm trying to do. I have features like bed, bath, pool, garage, lot size, square foot and price. And I have neighbors for each of the houses within 1 mile distance. I need to assign weights to the features. Please what approach did you end up using and how did you go about it? Thanks! $\endgroup$
    – dsmlainlp
    Commented Jun 30, 2021 at 5:05

1 Answer 1

Reset to default
1
$\begingroup$

I think you are on the right track (feature weights make sense).

I am assuming that the goal here is to find the 20 neighbors (based on the features) with the closest price.

One thing to point out right away is that you can test the model with cross validation: take a hold out test sample and perform the nearest neighbor search over the remaining training sample. That way you can empirically determine whether a given set of feature weights leads to finding good neighbors. This should be your approach to determining the best weights.

You can then try your idea of using linear regression and see how it performs vs not setting any feature weights. Assuming the relationship is truly linear, I think this should perform well (you should standardize the features first though). I would also try just using simple correlations of the features with the DV.

A more sophisticated approach is to use something like boosted trees or random forest instead (these are popular for determining feature importance).

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1 Random Forest gives feature weights and is usually pretty accurate and easy to tune; I've used it to find important features before. I don't know whether the result should be considered ordinal or ratio though, which seems important for @mitsi's case... $\endgroup$
    – Mark
    Commented Oct 28, 2016 at 14:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.