Could you explain how gradient boosting algorithm works? I have read a lot about gbm in Greedy function Approximation: A Gradient boosting Machine (pdf), but I can't code the algorithm for example LS_Boost in a simple way. Can someone explain what $h(x;a)$ is, and how to deal with it? 
 A: For implementation checkout:
https://github.com/2pc/libgbdt.git
For algorithm detail there is another graphical depiction which explained better:
http://www.lifesciencessociety.org/CSB2006/toc/PDF/43.2006.pdf

and this:

which in the protein folding case, translate to this:

A: From the FAQ in the appendix of an article I wrote with Jeremy Howard, called How to explain gradient boosting:
"Instead of creating a single powerful model, boosting combines multiple simple models into a single composite model. The idea is that, as we introduce more and more simple models, the overall model becomes stronger and stronger. In boosting terminology, the simple models are called weak models or weak learners.
To improve its predictions, gradient boosting looks at the difference between its current approximation, yhat, and the known correct target vector, y, which is called the residual, y - yhat. It then trains a weak model that maps feature vector x to that residual vector. Adding a residual predicted by a weak model to an existing model's approximation nudges the model towards the correct target. Adding lots of these nudges, improves the overall models approximation."
We put in a number of interesting visualizations that I think will help; for example, here’s one of them:

For the algorithm itself, you can take a look at our discussion of the general algorithm, but that refers to notation that you might need to look backwards in that article to get. Regardless, here is our version of the algorithm that assumes regression trees rather than any other kind of weak model:

That regression tree assumption dramatically signifies the mathematics and, besides, it's what everybody uses in practice.
