I have a small data set of 18 persons. I have an outcome variable Y, and 200 predictors. These predictors were chosen based on biology and prior data.
I used the caret R package and split the data set in two, a training set of 10 persons and a test set of the remaining 8 persons.
I then trained a RIDGE model using glmnet on the 10 persons with 4-fold cross validation. I then predicted the outcome on the remaining 8 persons using the trained model.
The correlation between actual Y and predicted Y was r=0.933, which is amazingly good. Too good to be true?
Heres my codes:
set.seed(7)
a <- createDataPartition(dfpred$Y, list=FALSE)
training <- dfpred[a,]
test <- dfpred[-a,]
library(mlbench)
library(caret)
control <- trainControl(method = "cv", number = 4)
set.seed(7)
modelRIDGE <- train(Y~., data=training, method="glmnet", trControl=control, tuneGrid = expand.grid(alpha = 0, lambda = 0))
MEA: 0.9280647
RMSE: 1.071609
R squared: 0.9748333
hm=predict(object=modelRIDGE,newdata=test[,-1])
cor.test(hm,test$Y) # r=0.933 p<0.001
This looks very very good. So good that I worry I have overlooked something? Any input would help, thank you!
A note on the data: Its clinical relevant basic research on difficult/rare human species. The data are really unique and tries på predict body glucose uptake based on serum protein concentrations. The methods are really robust and accurate. The sample size is small, but there are no larger data sets of this type available anywhere.
I also tested some other models:
