I'm currently playing with caret R package, with one merged microarray affymetrix dataset(paired tissue samples), in order to build and test various classifiers, mostly based on trees-such as random forests and boosted trees to predict a binary outcome-categorical, that is disease status (normal and cancer status). In detail, my main "introductory" questions are the following:
1) Because the sample size of my dataset is somehow described as "medium"-60 samples(maybe even small), should i split my dataset like 60% training and 40% test set ? Or is more appropriate especially for random forests, because of the out of bag error-OBB( because is like an independent "testing set") to use all my dataset, and test independently in other dataset(s) with similar design/status ?
2) Moreover, except my genes-predictors, i also have 8 continuous PET variables, which also describe the same patients from my specific dataset. Thus, my main interest, is if the addition of these PET variables improve the outcome/prediction. An illustrative example of my PET variables is as follows (samples in the rows, PET variables in the columns):
Thus, after I construct a total data.frame of both genes and these PET variables in the columns, should i also transform-"preprocess" my total variables, like centering or scaling ? Or for the mentioned classification algorithms it does not matter, because RF can handle the "different nature" of the variables ? and the same applies with gbm method ? And overall is it correct to use different variables with RFs ?
3) Moreover, in case I want to build a ROC curve, to see the performance of the classifier based on the training set, how could I use similarly the pROC package from the below caret tutorial ?
In detail, from the above tutorial, the object gbmtune is the output of the train function. How could I use it similarly with the roc function, in order to create the ROC curve?
4) Finally (one more general question not explicitly related to my specific dataset), because in one other dataset I have a great number of predictors, is it appropriate to use first a correlation filter to remove redundant features, and then continue with the RFE methodology to "identify" the "best performers?
Please excuse me for any naive questions, but there are fundamental in order to have an appropriate approach on this task!