xgboost - what is the difference between the tree booster and the linear booster? I am aware of gradient boosted trees. The extreme-gradient boosting algrithm is widely applied these days. What excactly is the difference between the tree booster (gbtree) and the linear booster (gblinear)? 
What I understand is that the booster tree grows a tree where a fit (error rate for classification, sum-of-squares for regression) is refined taking into account the complexity of the model. 
What is done differently in a linear booster?
 A: I just recently started using gradient boosted trees, please correct me if I'm wrong. I found this wiki page https://en.wikipedia.org/wiki/Gradient_boosting informative. Check out the algorithm and gradient tree boosting section.
As far as I understand, gradient boosting will of course work with most learners. Gradient boosting will in iterations($m$) train one new learner $h_m$ on the ensemble residuals of of previous iteration.
The ensemble $F_m$ is updated with 
$F_m \leftarrow F_{m-1} + \gamma_m h_m$
where $F_{m-1}$ was the previous ensemble and $\gamma_m$ is a coefficient such that,
$ \gamma_m = \underset{\gamma}{\operatorname{arg\,min}} \sum_{i=1}^n L\left(y_i, F_{m-1}(x_i) + \gamma_m h_m(x_i)\right).$
Hereby is the new learner fused with old ensemble by coefficient $\gamma_m$, such that new ensemble explains the target $y$ the most accurately(defined by loss function metric $L$)
As explained on wiki page, Friedman proposed at special modification for decision trees, where each terminal node $j$*** of the new learner $h_m$, has it own separate $\gamma_{jm}$ value. This modification would not be transferrable to most other learners, such as gblinear.
*** (wiki article describes each $\gamma_{jm}$ to cover a disjoint region(R) of the feature space. I prefer to think of its as the terminal nodes, that happens to cover each a disjoint region)
Also to mention, if you pick a strictly additive linear regression as base learner, I think the model will fail fitting interactions and non-lineairties. In example below the xgboost cannot fit $y=x_1 x_2$
library(xgboost)
X = replicate(2,rnorm(5000))
y = apply(X,1,prod)
test = sample(5000,2000)
Data = cbind(X=X)
xbm = xgboost(Data[-test,],label=y[-test],params=list(booster="gblinear"),nrounds=500)
ytest = predict(xbm,Data[test,])
plot(y[test],ytest)

