0
$\begingroup$

I am trying to perform a grid search on the classic iris dataset. I want to find the optimal k value using grid search. I used the train() from caret library and just invoke the method. But I get a simple error and trying to figure out what is incorrect.

here is my code chunk:

library(class)
library(caret)

spec = c(train = .5, test = .25, validate = .25)

byparts = sample(cut(
  seq(nrow(iris)), 
  nrow(iris)*cumsum(c(0,spec)),
  labels = names(spec)
))

res = split(iris, byparts)

addmargins(prop.table(table(byparts)))
train_feat <- res$train[,1:4]

classifier = train(form = Species ~ ., data = train_feat, method = 'knn')

Also, I want to select this k value among 1-3-5.

Error:

Error in eval(predvars, data, env) : object 'Species' not found
9.
eval(predvars, data, env)
8.
eval(predvars, data, env)
7.
model.frame.default(form = Species ~ ., data = train_feat, na.action = na.fail)
6.
stats::model.frame(form = Species ~ ., data = train_feat, na.action = na.fail)
5.
eval(expr, p)
4.
eval(expr, p)
3.
eval.parent(m)
2.
train.formula(form = Species ~ ., data = train_feat, method = "knn", tuneGrid = expand.grid(k = c(1, 3, 5)))
1.
train(form = Species ~ ., data = train_feat, method = "knn", tuneGrid = expand.grid(k = c(1, 3, 5)))
$\endgroup$
1
$\begingroup$

Without, showing us the error, how we are supposed to help. Besides, you are not doing cv within the pipeline. Also why are you using k = 1 ? If you have an outlier as a neighbour which is from a different group that would distort your search. It is not the best rule of thumb to use uneven k's, but is a good head start: See also here: stats.stackexchange.com/questions/517054/what-happen-if-knn-has-k-1-and-there-are-2-nearest-classes-with-the-same-distanc/517057#517057

Use the kknn package of R (weighted knn) and use an internal cv and make use of tuning like this, replace my target variable and df with yours respectivley.

library(kknn)
library(caret)


set.seed(42)

levels(df_class$success) <- make.names(levels(df_class$success))


train.index <- createDataPartition(df_class$success, p = 0.75, list = FALSE)
train <- df_class[train.index, ]
test <- df_class[-train.index, ]

#your grid without using 1
params <- expand.grid(kmax=seq(3, 5, 7), 
                      kernel=c("gaussian", "rank", "optimal", "biweight"),
                      distance =c(1, 2, 3))

model_knn <- train(success ~ ., 
                   method="kknn",
                   data = train, 
                   tuneGrid = params,
                   preProcess=c("scale", "center"),
                   trControl = trainControl(classProbs = TRUE, 
                                            method="repeatedcv", 
                                            repeats = 10)
                   )
          
print(model_knn) 

Keep in mind that caret will always fix one hyper parameter in its output. Im not completely sure why, but it doesn't matter s long as the pipe is stable, you can try out as much as you like.

If a package is missing do not hesitate to ask.

$\endgroup$
6
  • $\begingroup$ My bad, that I didn't post the error earlier. Now I did. I have just one question regarding your answer, is this the nested cross validation strategy that you are talking about? $\endgroup$
    – Ranji Raj
    Apr 21 at 11:57
  • $\begingroup$ if you mean that im creating a validation set that is only used for hyperparameter tuning and will then be finally applied on the remaining 25 % of test data then yes. This is how it works with caret- The pipeline makes all subsamples in the valid. even so that hyperparam. testing will be guaranteed to work. Nothing you have to take care anymore. Glad I could help! $\endgroup$ Apr 21 at 12:01
  • $\begingroup$ The only thing that you probably wanna switch may be classProbs if you dont want predicted probas but thats it. $\endgroup$ Apr 21 at 12:02
  • $\begingroup$ yes, it pretty much helps. I am glad too there are a couple of experienced guys out there who shows up to help, Cheers! :) $\endgroup$
    – Ranji Raj
    Apr 21 at 12:03
  • $\begingroup$ Sure thing, but by the look of your error you may also consider your column namings? As Species were not found $\endgroup$ Apr 21 at 12:24

Not the answer you're looking for? Browse other questions tagged or ask your own question.