1
$\begingroup$

I am trying to train a regression model for dataset with 500k observations and 3 features. The features are categorical and have 50, 50 and 100 levels.

Is (generally) kNN appropriate for this kind of task?

I am using R. I tried to turn my categorical variables into dummy variables but I end up with very large and sparse data set. I am using data.matrix for conversion and it sets the matrix to double by default.

Is there a way to set it to logical instead?

$\endgroup$
1
$\begingroup$

I expect you are talking about nominal categorical variables there? Ordinal variables with 100 levels are very strange. I have never seen a likert scale with 100 nuances or anything else that would warrant a 100 level ordinal variable. If you have ordinal variables with so many levels, investigate if you can reasonably transform them into interval variables. That can be done when it is reasonable to assume the distances between any two adjacent levels are the same across the scale.

If I had only nominal categorical data, I would first look at tree based models, that's where they naturally shine. With so many options within so few categorical variables, I would expect random forests to do better than single pruned trees. You can test both though.

$\endgroup$
0
$\begingroup$

If your variables are ordinal, then you can leave them by using labelencoding. However, since KNN works with distance metric, either you need to change your distance metric accordingly or use one hot encoding as you are using but as you said, it will create a huge sparse matrix.

I will suggest using a tree based algo such as random forest that need not requires one hot encoding

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.