I have used multivariate linear regression for one of my projects, and used r-square and p vals to evaluate the model. I couldn't find what such metric we would use for decision trees and random forests. Do we have any standard metrics for decision trees, equivalent to goodness of fit and statistical significance (like p vals for regression)?
$\begingroup$
$\endgroup$
asked Jul 16, 2015 at 16:36
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The most convenient goodness-of-fit for random forest is out-of-bag cross-validation, it can provide a R² value and e.g. std.dev of prediction. It is important to use cross-validation, as the direct goodness-of-fit is completely misleading for any non-linear machine learning model, as they can fit mostly anything also noise. Here's a link on the interpretation of R² for the randomForest package: Link!
p-values are not much used in random forest context, as the hypothesis space is so huge.
If you want to identify related variables,use variable importance. If you want to make a fair comparison on prediction performance between MLR an RF you need to design a cross-validation and embed both models. 10-fold and 10 times repeated would normally be regarded as solid. From such a cross validation a R² and std.error could be extracted.
answered Jul 17, 2015 at 10:14