data = 1800 observations (rows) x 5 variables (columns)
I am using
library(caret) and training regression models using
nnet() and would like to know whether my interpretation and choice of model is valid.
My response variable is in hours (therefore continuous and non-negative)
In this example I am training three models, the only difference between them is the number of max iterations (
maxit) (to try and 'smooth' the values at the high-end of the range).
I have also split my data into a training
df_train and a test
df_test dataset using
# use 10-fold cross-validation cvCtrl <- trainControl(method="repeatedcv", repeats=3) modFit_1 <- train(v_response ~., method="nnet", trControl=cvCtrl, data=df_train, trace=TRUE, maxit=1000, linout = 1) modFit_2 <- train(v_response ~., method="nnet", trControl=cvCtrl, data=df_train, trace=TRUE, maxit=2000, linout = 1) modFit_3 <- train(v_response ~., method="nnet", trControl=cvCtrl, data=df_train, trace=TRUE, maxit=4000, linout = 1)
The plots of predicted vs actual values are for each model, and their
RMSE values are:
RMSE = 17.31634
RMSE = 16.73134
RMSE = 9.526294
As my response variable has to be non-negative would I immediately assume
modFit_2to be the 'best' model out of the three (even though
modFit_3has a lower RMSE)?
Is there a way to ensure the predicted values are non-negative (and is my
modFit_2non-negative just by chance)?
Residual vs fitted plot
df_diag <- data.frame(residuals = modFit_3$finalModel$residuals, fitted = modFit_3$finalModel$fitted.values) ggplot(data=df_diag, aes(x=fitted, y=residuals)) + geom_point()