In a CART model, why is the average of the leaf proportions equal to the total proportion only when the classes are unweighted? Suppose I want to do binary classification (the two classes are 0 and 1) and I choose to work with a CART model. I first fit this model on a training set. (Note that I am using Python, and specifically the sklearn DecisionTreeClassifier.)
Then, for a sample X, predict_proba returns [p0(X), p1(X)] where p0(X) represents the probability that X belongs to the 0-class and p1(X) represents the probability that X belongs to the 1-class.
In the sklearn documentation, it is said that p0(X) is equal to: (number of samples of the training set which belong to the leaf of X and to the 0-class) / (number of samples of the training set which belong to the leaf of X), that is to say the ratio of the 0-class in the leaf of X. And the same way for p1(X).
With this definition, I would think that the average of p0 on the training set itself, is going to be equal to: (number of samples of the training set which belong to the 0-class)/(number of samples of the training set), that is to say the ratio of the 0-class in the training set.
If I put class_weight=None, then the previous statement is true. But with any other class_weight, it is not anymore and I do not understand why.
 A: You can see why this works out in one case, but not the other, by following your nose through the arithmetic.
Say you have a tree $T$, and the terminal nodes are $T_1, T_2, \cdots, T_k$.  Let $N_1, N_2, \cdots, N_k$ denote the number of training data points in each of these terminal nodes.  Then we can write the prediction in terminal node $T_j$ as:
$$ p_j = \frac{\sum_{i \in T_j} y_i}{N_j} $$
So your training average of the predictions is
$$ \frac{1}{N} \sum_j N_j \frac{\sum_{i \in T_j} y_i}{N_j}$$
Since this looks a little gnarly, a word of explanation.  Each observation in a terminal node receives the same prediction as all the others in the same node, so to aggregate, we can sum over the terminal node predictions, and repeat each summand $N_j$ times.
Now you can see why your equality holds
$$ \frac{1}{N} \sum_j N_j \frac{\sum_{i \in T_j} y_i}{N_j} = \frac{1}{N} \sum_j \sum_{i \in T_j} y_i = \frac{1}{N}\sum_i y_i $$
When there are sample weights, the convenient cancellation does not occur, as the prediction in the terminal node is then
$$ p_j = \frac{\sum_{i \in T_j} w_i y_i}{\sum_{i \in T_j} w_i} $$
And there is no reason for $\sum_{i \in T_j} w_i = N_j$ to hold.
A similar thing happens with class weights.
