I'm trying to train a decision tree and use cross validation doing so. (this post is quite helpful already understanding CV.)
First I did 10-fold cross validation on my data, including all attributes, with certain parameters. Then, let's say I was happy with the results of the CV, I used the whole data to train a decision tree. Now for example only 4 out of 20 attributes show up in the final decision tree since those 4 attributes already do a good classification job. Should I perform another cross validation with those 4 attributes only? I ask because during each cross validation step random (stratified) samples are taken and everytime another tree might be built. So I might have 10 different trees and average their error rate etc. But isn't the result much more reliable when doing cross validation with those 4 attributes only that eventually show up in the final decision tree instead of using 20 attributes of which some are not relevant for the final tree?
Help is much appreciated!