0
$\begingroup$

I have a data with 10 variables (continuous with log transformed values) that I am using to accurately predict in a 3 class classification.

I used RF model to select those 10 variables by first dividing the data in to 60 % (train)-40 %(validate) proportion. Now I want to plot ROC curve to further validate that my 10 variables are good (sort of significance from AUC curves).

I see many papers using multi-variable ROC curve for classification but I am unable to find an exact answer, and I am using R to run the code as I don't have SAS.

One way I found is to make a generalized linear model (glm) using those 10 variables using 'binomial' regression. I converted my 3 class classification into three different 2 class classifications (A vs ALL, B vs ALL, C vs ALL) based on posts I read on this forum.

Then I used the fitted values from glm as 'prediction', and the class as 'labels' and plotted the ROC curve using basic prediction and performance commands.

I want to know the following:

1) Is there any other method to plot ROC curve with multiple variable or using glm fitted values is correct?

2) Shall I also run train and test for glm+ROC curve on 60-40% data? Or running on combined data is ok. Can I get some sort of help with the algorithm how to approach with that?

3) I see some methods where ROC curve finds a cut off value for combined variable values. I didn't understand that part and how to obtain that?

4) Finally, I used the same data on which I performed the RF training and test. Is it wrong to do it?

I have little to no background in advanced statistics hence I would appreciate any help.

Thanks!

Improve this question
$\endgroup$
7
  • $\begingroup$ If you generalize your problem removing your domain specific information, you're more likely to get solution. There are far more stats experts here on this platform than genetics experts... And try to be very specific in what exactly you want. $\endgroup$
    – tired and bored dev
    Commented Feb 16, 2019 at 1:13
  • $\begingroup$ @tiredandboreddev thanks for your suggestion. I hope the post is more general now. $\endgroup$
    – Piyush Joshi
    Commented Feb 16, 2019 at 17:32
  • $\begingroup$ Is there a natural ordering of your 3 classes, such as normal/carcinoma-in-situ/invasive in cancer? Or are they simply 3 separate classes? $\endgroup$
    – EdM
    Commented Feb 16, 2019 at 17:39
  • $\begingroup$ Three molecular group of cancer. So I am looking for least number of molecular markers for with accurate diagnosis. $\endgroup$
    – Piyush Joshi
    Commented Feb 16, 2019 at 18:20
  • $\begingroup$ These links will help you: stats.stackexchange.com/questions/2151/… scikit-learn.org/stable/auto_examples/model_selection/… stats.stackexchange.com/questions/21551/… You should first study about basic ideas of machine learning. These forums can't substitute a good book/tutorial. It can only fill the gaps in knowledge. $\endgroup$
    – tired and bored dev
    Commented Feb 17, 2019 at 18:40

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.