# Inestability of BIC when selecting nested models

Currently I am working with spline regression and a method for selecting knots adaptively. My method gives me a set of potential knots that generally has a large number of elements. Following He et al.(2001) I would like to apply backward selection in order to prune that potential-knot-vector.

Let $$m=k+n$$ where $$k$$ is the order of the splines and $$n$$ is the number of internal knots.

Remove the $$i$$-th knot from the current vector,construct the set of basis and estimate the residual sum of squares $$RSS_{-i}$$, where $$i=1,2,...,n$$.

Choose the model with the smallest $$RSS_{-i}$$ and then replace $$m$$ by $$m-1$$ Calculate $$BIC_m$$ for the model selected in the previous step.

Continue until no internal knots are left and choose the final set of knots corresponding to the model chosen with smallest $$BIC$$.

I've repeated the simulations 20 times, and the unpruned vectors where very similar between each other. The problem is that the pruned vectors vary too much, even though in each iteration the only thing changing is the noise term.

For instance I leave you two images showing you the "optimal" (the one that minimized the BIC) in two DIFFERENT repetitions.(the error term was generated using a different seed).

As you can see the approximations are very different even though there's just one knot of difference. What I don't really understand is how the second model minimizes the BIC, given that the previous model (for the same simulation, same error term, same seed) is presented in the third image.

I know that the BIC penalises the number of parameters (in this case the number of knots) but I cannot believe that in this example the increase in RSS is smaller in weight than 1 extra knot. Any idea of what's happening here?