In Recursive Partitioning for Heterogeneous Causal Effects by Susan Athey, Guido W. Imbens, under section 2.5 Honest Splitting, two different datasets (called tr and est) are used for (a) creating the tree's split structure and (b) estimating leaf means. The splitting and cross-validation criteria is -EMSE(Π) which consists of two expectations (Π is essentially a tree, referred to as a partition in the paper and EMSE is a modification of estimated mean squared error). One of the expectations of -EMSE(Π) involves the following term:
$$V(\hat{\mu}^2(X_i;S^{est},\Pi))$$
The paper states:
"We wish to estimate −EMSE(Π) on the basis of the training sample S_tr and knowledge of the sample size of the estimation sample N_est"
Specifically the following is used:
$$V(\hat{\mu}^2(X_i;S^{est},\Pi))=\frac{S^2_{S^{tr}}(l(x;\Pi))}{N^{est}(l(x;\Pi))}$$
where $$S^2_{S^{tr}}$$ "is the within-leaf variance".
My questions are:
- To estimate the variance why is S^2 (in the numerator) computed from the tr set whereas N (in the denominator) is from the est set? In other words why not have both S^2 and N from the tr set once the est set is used to compute estimated leaf means?
- Normally in statistics I see that the denominator when computing an unbiased sample variance is n-1 and not n (e.g. see here). In this case why is N used in the denominator instead of N-1.
- In "We then weight this by the leaf shares pℓ to estimate the expected variance", what is the definition of leaf shares? Does "leaf shares are approximately the same" mean that the percentage of samples in each leaf is more or less the same for both the tr and est set?
