Gradient Boosting Tree vs Random Forest Gradient tree boosting as proposed by Friedman uses decision trees as base learners. I'm wondering if we should make the base decision tree as complex as possible (fully grown) or simpler? Is there any explanation for the choice?
Random Forest is another ensemble method using decision trees as base learners. 
Based on my understanding, we generally use the almost fully grown decision trees in each iteration. Am I right?
 A: This question is addressed in this very nice post. Please take a look at it and the references therein. http://fastml.com/what-is-better-gradient-boosted-trees-or-random-forest/
Notice in the article that the speaks about calibration, and links to another (nice) blog post about it. Still, I find that the paper Obtaining Calibrated Probabilities from Boosting gives you a better understanding of what calibration in the context of boosted classifiers is, and what are standard methods to perform it.
And finally one aspect missing (a bit more theoretical). Both RF and GBM are ensemble methods, meaning you build a classifier out a big number of smaller classifiers. Now the fundamental difference lies on the method used: 


*

*RF uses decision trees, which are very prone to overfitting. In order to achieve higher accuracy, RF decides to create a large number of them based on bagging. The basic idea is to resample the data over and over and for each sample train a new classifier. Different classifiers overfit the data in a different way, and through voting those differences are averaged out.

*GBM is a boosting method, which builds on weak classifiers. The idea is to add a classifier at a time, so that the next classifier is trained to improve the already trained ensemble. Notice that for RF each iteration the classifier is trained independently from the rest.

A: Although the above answers are really great, I would like to explain the difference in a very simple language.

*

*Bagging technique that is Bootstrap Aggregation where we build
separate decision trees using bootstrapped set of samples and average
the resulting predictions. Each individual decision tree are grown
deep without any pruning and hence each of them has high variance and
low bias but averaging them reduces the overall variance. They result
in improved accuracy over prediction with a single tree.


*Bagging technique suffers from a disadvantage that of any of the predictor is very very strong than the other predictors. Each bagged tree will look similar because most of them will use that strong predictor. Hence the predictions from the bagged trees will be highly correlated. Unfortunately, averaging many highly correlated quantities does not lead to as large of a reduction in variance as averaging many uncorrelated quantities.


*Random Forest overcome this problem by forcing each split to consider
only a subset of the predictors that are random. The main difference between bagging and random forests is the choice of predictor subset size. If a random forest is built using all the predictors, then it is equal to bagging.


*Boosting works in a similar way, except that the trees are grown
sequentially: each tree is grown using information from previously grown trees. Boosting does not involve bootstrap sampling; instead each tree is fit on a modified version of the original data set.Unlike in bagging, the construction of each tree depends strongly on the trees that have already been grown. Because the growth of a particular tree takes into account the other trees that have already been grown, smaller trees are typically sufficient. These small trees are mostly Stump which have single split.
A: $\text{error = bias + variance}$


*

*Boosting is based on weak learners (high bias, low variance). In
terms of decision trees, weak learners are shallow trees, sometimes
even as small as decision stumps (trees with two leaves). Boosting
reduces error mainly by reducing bias (and also to some extent variance, 
by aggregating the output from many models).

*On the other hand, Random Forest uses as you said fully grown
decision trees (low bias, high variance). It tackles the
error reduction task in the opposite way: by reducing variance.
The trees are made uncorrelated to maximize the decrease in variance,
but the algorithm cannot reduce bias (which is slightly higher than the 
bias of an individual tree in the forest). Hence the need for large,
unpruned trees, so that the bias is initially as low as possible.


Please note that unlike Boosting (which is sequential), RF grows trees in parallel. The term iterative that you used is thus inappropriate.
