I am novice in an attempt to understand things better.
As I imagine, when one uses stratified k-fold cross validation to decide the size of a decision tree, he:
1) randomly divides data into k equally sized and proportionally distributed data sets
2) aggregates all sets leaving one set out, thus making a train and test data sets
3) using train data set builds a sequence of trees of different sizes
4) tests their results with test data set
5) records misclassification rates of all tree sizes
6) repeats 2-5 steps creating in total k combinations of train and test data sets.
7) sums up misclassification rates for each size of a tree
8) plots Xval error rate as a function of the size of a tree
9) selects the size according to 1-SE rule
If I am correct, this method is useful when data's distribution is highly uneven, since random sampling without equal distribution would result in a case where one class would be dominant or, in extreme scenario, the only one. Therefore, the results of such an effort would be crippled.
However, I can't find a function in R that uses this method to identify the best size of a tree.
So, I attempted to do stratified 10-fold cross validation manually with the aid of the caret and rpart packages, but stumbled upon a problem. I tried making trees of desired size by changing cp control parameter. However, this way each train data set lets create differently sized trees. Therefore I can't accumulate Xval error rates for desired tree sizes.
What I am doing wrong? Maybe there is a simple function to solve my problem?