I am dealing with fairly highly dimensional data (spectroscopic data), which I preprocess with PCA, then apply the chosen learning algorithm. When I use linear regression, using the Caret package, I get an $R^2=0.83$ on my test set of 300 PCA components however, when I use random forests on the same set:
modelFit <- train(train$age~.,data=trainPC[,1:300],method = "rf")
I get worse performance, with $R^2$ much lower on the test set. RMSE is also not as good for random forests vs linear regression.
For clarity - I am using Caret's in-built cross validation, and am evaluating the performance on an independent test set.
I get similar results when I use xgboost, neural networks etc. I know that different machine learning algorithms can perform better in different circumstances, but I am nonetheless surprised that linear regression appears to be performing best, and makes me suspect that I am approaching things incorrectly, or making a basic error with using the Caret package.
I have also tried using PLS (the main feature selection method used for my particular example) and get an $R^2=0.83$. The literature suggests that neural nets/non-linear-methods should outperform the linear methods I am using, but at the moment, I cannot reproduce this.
Does anyone have any suggests/tips on this? I also attach the data set I am using (I am trying to predict age) - data link
I also attach my current R script that I am using to do the learning - any help would be much appreciated! - R script