2
$\begingroup$

Why do H2O Random Forest and GBM implementation use MSE (mean squared error) for node splitting metric?

As I understand from the H2O docs, no matter the response column is factor or numeric, the tree building algorithm in H2O treats it like "regression tree", it tries to predict the probability of each class. In this case, for multi-classification problem, the implementation will train a tree for each class (like one-vs-all approach) to estimate the probability for that class. Finally, the probability prediction for each class will be averaged through all the trees and then scaled to have sum of 1.

However, for classification problems, if we use Gini index or cross entropy, we only need to train one tree for all the classes, and we can still use the proportions of class predictions as our estimation of probability.

The only advantage that I can think of is that it can offer more fine-grained probability prediction and possibly higher accuracy?

Are there any other reasons that H2O only use MSE for node splitting?

$\endgroup$
1
$\begingroup$

I think there is a confusion between classification (i.e: finding the probability of being in a certain class) and regression (i.e: finding a continuous value as a prediction).

For classification, one should use Gini index or cross-entropy. It builds a split by measuring the impurity in the created leaves.

For regression, one could use mean squared error. This cost function builds a split by taking the average value of the response in the leaves and comparing it to the individuals that fall into each leaf.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.