I only read that it is due to numerical reasons, e.g. on http://einspline.sourceforge.net/background.shtml, but I don't really get it. Can someone please explain it more simple? Is it because they are made of linear combinations? If so, why/how is this an advantage?
Splines are a large class of methods.
The method of B-splines is a simple method for taking a single covariate and expanding it such that it spans the set of all functions that are a polynomial of degree $d$ between all the given knots and $d-1$ differentiable everywhere. They are not the only way to achieve such an expansion of a covariate, but any other expansion will span the exact same set of functions, and B-splines have some nice numerical properties, so if you want splines that fit those smoothness conditions, there's not a good reason not to use B-splines.
B-splines are particularly popular because they are very simple and can be easily plugged into any regression model without any editing of the model to create non-linear effects without any special editing of the software.
But there's plenty of other types of splines you might want to use. To name a few:
- M-splines: splines that span strictly positive functions
- I-splines: splines that span strictly monotonic functions
- Natural splines: splines whose 1st derivative is constant outside of the knots
- P-splines: splines whose derivatives are penalized to enforce smoothness (also known as smoothing splines)
Note that M+I splines are very special cases; if you want to use them, B-splines are simply not the job. Natural splines are really a subset of B-splines. P-splines have some very nice properties, but in general require special software implementations.