I have a data with 10 variables (continuous with log transformed values) that I am using to accurately predict in a 3 class classification.
I used RF model to select those 10 variables by first dividing the data in to 60 % (train)-40 %(validate) proportion. Now I want to plot ROC curve to further validate that my 10 variables are good (sort of significance from AUC curves).
I see many papers using multi-variable ROC curve for classification but I am unable to find an exact answer, and I am using R to run the code as I don't have SAS.
One way I found is to make a generalized linear model (glm) using those 10 variables using 'binomial' regression. I converted my 3 class classification into three different 2 class classifications (A vs ALL, B vs ALL, C vs ALL) based on posts I read on this forum.
Then I used the fitted values from glm as 'prediction', and the class as 'labels' and plotted the ROC curve using basic prediction and performance commands.
I want to know the following:
1) Is there any other method to plot ROC curve with multiple variable or using glm fitted values is correct?
2) Shall I also run train and test for glm+ROC curve on 60-40% data? Or running on combined data is ok. Can I get some sort of help with the algorithm how to approach with that?
3) I see some methods where ROC curve finds a cut off value for combined variable values. I didn't understand that part and how to obtain that?
4) Finally, I used the same data on which I performed the RF training and test. Is it wrong to do it?
I have little to no background in advanced statistics hence I would appreciate any help.