It is well know that the "usual" Radial Basis Function can be derived from Regularization that imposes small derivates. More precisely it is well known that the following:
$$ f(x) = \sum^{N}_{n=1} w_n \phi( \| x - x_n\| ) = \sum^{N}_{n=1} w_n e^{ - \gamma \| x - \mu_k \|^2 } $$
can be derived from the variational problem:
$$ H[h] = \sum^{N}_{n=1} (y_n - h(x_n))^2 + \lambda \sum^{\infty}_{k=0} \int^{\infty}_{- \infty} \left( \frac{d^k h }{dx^k} \right)^2 = \sum^{N}_{n=1} Loss(h(x_n), y_n) + \lambda R(h) $$
Where the nasty regularizer $R(h)$ is:
$$ R(h) = \sum^{\infty}_{k=0} \int^{\infty}_{- \infty} \left( \frac{d^k h }{dx^k} \right)^2 $$
However, one can consider a similar model where instead of considering all $N$ training points, we only consider $K$ (movable) centers $\{ \mu_1, ..., \mu_k, ... \mu_K \}$ (these movable centers can be estimated using k-means, for example).
Considering this alternative model:
$$ h(x) = \sum^{K}_{k=1} w_k \phi( \| x - x_k\| ) = \sum^{K}_{k=1} w_k e^{ - \gamma \| x - \mu_k \|^2 } $$
Is there any way to derive it from the minimization (optimization) of the empirical risk with some regularizer? Or can it be derived rigorously using Tikhonov Regukarization?
Intuitively, its clear that its not surprising that we can write the original Tikhonov Regularization solutions some kind of kernel (since Gaussians are smooth and the regularizer involves an infinite set of derivatives, the solutions is not too surprising to involve a Gaussian Kernel). This is not to surprising because of the Representer Theorem. However, I just can't see how movable centers with $K < N$ can be justified from a rigorous derivation.
I guess my main question is, how does one justify rigorously to use centers that are not the data points exactly and what its relationship is to the original regularization with radial basis functions and all the data points (as indicated by the representer theorem), since that is the only solution to be justified theoretically.
