Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
By building the complete tree and pruning it afterward we are adopting a strategy of postpruning (or backward pruning) rather than prepruning (or forward pruning).
Prepruning would involve trying to decide during the tree-building process when to stop developing subtrees. It's attractive because allows to avoid all the work of developing subtrees only to throw them away afterward.
What are the benefits of post-prunning then?
Oh well, gotta answer my own question. Quotting "Data Mining: Practical Machine Learning Tools and Techniques",
..postpruning does seem to offer some advantages. For example, situations occur in which two attributes individually seem to have nothing to contribute but are powerful predictors when combined—a sort of combination-lock effect in which the correct combination of the two attribute values is very informative whereas the attributes taken individually are not. Most decision tree builders postprune; it is an open question whether prepruning strategies can be developed that perform as well.
So basically while building a whole decision tree (rather that a subset of it as in pre-prunning) we may often come up with powerful "combined predictors" which are plausible to notice only when the whole tree (rather than its subset) is built.
Moreover, this is the recommeded approach and forward pruning is rarely used at all.
