1
$\begingroup$

I am using caret glmnet to do a stock prediction project. Since I need to calculate RMSE, I use getTrainPerf(mod)$TrainRMSE to get the RMSE . However, when I use predict to get the prediction value, and deduce them by the actual value to get the RMSE, the value is different.

The time series setting is like below.

initialWindow = 80     
myTimeControl <- trainControl(method = "timeslice",
initialWindow = initialWindow,horizon = 1,fixedWindow = TRUE)

The data looks likeenter image description here The model

mod <- train(
      target ~ .,
      data = data,
      method = "glmnet",
      tuneLength=5,           
      trControl = myTimeControl
      )

The way I get RMSE from caret getTrainPerf(mod)$TrainRMSE 0.7208732

The way I calculate sqrt(sum((predict(mod,data,s =mod$bestTune$lambda )-data[,50])^2)/nrow(data)) 0.914474

So, somehow the result is bigger than caret one.The reason I have to calculate it by hand is because I have to get individual SSE in each observation. I have searched some articles and add the bestTune value, but it seems that it did not change the prediction.

My questions are 1. Are the prediction value stored in train that we can directly access? 2. If not, how can we calculate the correct SSE for each observation?

$\endgroup$

1 Answer 1

1
$\begingroup$

1st question: on caret you can access the predicted values upon training using:

mod$finalModel$predicted
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.