Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm fitting regression and classification trees. I thought that the algorithm to fit the tree led to the same result each time. However, when I run the line below
tree1 <- rpart(log_bid_price ~ ., minbucket = 5, cp = 0.001, data = HC, method = "anova")
I get differing plots for the cross-validation parameter. I generate the plot using
plotcp(tree1)
What changes between the two plots?
You are right, but I suspect that what you see plotted there is the cross-validation error for different complexity parameters. Cross-validation leaves out some data points when fitting a model to then predict the outcome of interest for the left-out observations using the model fitted not using these data points. Thereby, it approximates something we are often interested in, namely out-of-sample performance. Now, as the data points that are left out may differ across different runs, the results need not be the same across different runs. You could help us by letting know how you generated the plot.
I believe @Christoph is correct. Yes, the algorithm is deterministic. Using printcp you can see what plotcp is plotting. The rel error is consistent between runs. This is a metric for error computed from the training data.
rpart also does cross validation to compute a more reliable error metric, and this is why you see the change between runs. So the model is the same, but the CV error metric is being approximated differently each time.
