I'm a beginner machine learning user, doing my first predictive model using random forest.
I have some questions regarding the way to measure how good a model is (Gini area from roc curve, and KS), and some about random forest algorithm. Thanks a lot for your help.
1) Which metric is better to compare models? Gini or KS? For example, I have two versions of my model, one in which gini is greater than the other but lower KS.
2) Once I run my model, I can see how significant are each variable I created using VarImpPlot in R. All of them "explain" something, but I struggle to know "when my model is optimal". For example, some times I remove one of the variables and it improves, or sometimes I remove not one but a combination of two, and it improves... but some times it gets worse. Is there any way to know when I reached an "optimal combination" given my set of variables?
Thanks a lot