How to build regression models and then compare their fits with data held out from the model training-testing?

I have been building a couple different regression models using the caret package in R in order to make predictions about how fluorescent certain genetic sequences will become under certain experimental conditions. I am using a few different linear model building methods for regression, and will use postResamples in order to compare the Rsquared and RMSE of one model with other models.

However, under my current set up, I have been unable to get an Rsquared value that is higher than 0.40, in either my testing set or training set of my data. Given that I must apply these models to other data sets in the future in order to make predictions about their fluorescence score, I am not sure what I can do in order to increase the Rsquared (or decrease the RMSE). I have tried researching which different resampling methods might improve the fit, but a lot of the tutorials online seems to be geared towards improving the fit of classification and not regression models.

So what can I do in terms of modifying the traincontrol object or using a different method that may improve the quality of my fit?

I should also mention, each of the predictors is either an integer or a float, and some have ranges that may not be normally distributed.

Here is the code I thus far been using. The resampling method I have been using is assigned to the CvCtrl train control object:

library(caret)

sorted_Data<- data[order(data$fluorescence, decreasing= TRUE),] splitprob <- 0.8 traintestindex <- createDataPartition(sorted_Data$fluorescence, p=splitprob, list=F)
holdoutset <- sorted_Data[-traintestindex,]
trainingset <- sorted_Data[traintestindex,]

traindata<- trainingset[c('x1', 'x2', 'x3', 'x4', 'x5', 'fluorescence')]

cvCtrl <- trainControl(method = "repeatedcv", number= 20, repeats = 20, verboseIter = FALSE)

modelglmStepAIC <- train(fluorescence~., traindata, method = "glmStepAIC", preProc = c("center","scale"), trControl = cvCtrl)

model_rlm <- train(fluorescence~., traindata, method = "rlm", preProc = c("center","scale"), trControl = cvCtrl)

pred_glmStepAIC<- predict.lm(modelglmStepAIC$finalModel, holdoutset) pred_rlm<- predict.lm(model_rlm$finalModel, holdoutset)

glmStepAIC_summary<- postResample(pred_glmStepAIC, holdoutset$logfl) rlm_summary<- postResample(pred_rlm, holdoutset$logfl)

• I suppose this topic can be labeled off-topic, but nonetheless, you want to go here: topepo.github.io/caret/train-models-by-tag.html and browse available models by keywords such as 'regression', 'tree', 'neural'. The rest depends on your experiment design and given task. – Alexey Burnakov Oct 10 '17 at 8:15
• @AlexeyBurnakov I have browsed this page before, and I did not find it to be useful. It lists the available methods by their modeling category, and also mentions their pertinent tuning parameters, but does not detail which methods should be used for which case, or how the tuning parameters function. I'm really just looking for a resource that can help me determine what methods or resampling types I should implement in order to see if I can achieve a better fit – Bob McBobson Oct 10 '17 at 9:40
• Then I suggest using search over CrossValidated supplying available trainControl methods. Different CV types may result in different model quality. Did you just try several other options in trainControl? Did you try different vold number? I strongly believe that a proper train model choice maybe equally or more important then validation strategy. – Alexey Burnakov Oct 10 '17 at 9:47
• @AlexeyBurnakov In regards to your last statement, do you think it would be worth it for me to look into train model choices that are not based on linear regression? At this point, because I need to be able to implement said model on new data sets in order to predict their outcome, I am trying to stick to linear regression (since its coefficients are easy to parse and implement within Python). But at the same time I am aware that it is also important to have a model that fits well, so I am willing to try out other forms of modeling. – Bob McBobson Oct 10 '17 at 10:33
• It becomes a more complicated task to export a model to Python, so you limit yourself with respect to model choice. Apart from that, the top quality regression models include gradient boosted decision trees and/or linear models (XGBoost, GBM, CatBoost, LightGBM). You can also try neural network (and you can actually export weight matrix into Python). But before you go, did you try nonlinear transform of your covariates: squared, cubic, etc.? Including a nonlinearity in input terms may improve the result even if you use GLM. – Alexey Burnakov Oct 10 '17 at 10:43