This is not a pure programming question. I am trying to understand what's going on when I try to use RBF with 5 centers. I am using R to exemplify, see below.
My data set:
x <- c(0.0, 11.0, 17.9, 49.3, 77.4) y <- c(0.2497978, 0.5090220, 0.5373010, 0.6032853, 0.5938590)
I would like to interpolate this data with a RBF network. I start from regression (i.e., number of centers < number of points), and things look fine, but when I get to interpolation, something breaks down. So, let's start with 2 centers:
library(RSNNS) set.seed(21) rbf.model <- rbf(x, y, size = 2, maxit = 1000, linOut = TRUE) fitted(rbf.model)-y # [,1] # [1,] -2.100049e-03 # [2,] 2.320548e-02 # [3,] 2.218202e-02 # [4,] -4.328762e-02 # [5,] -1.303689e-07
Not bad, as a first attempt. Let's increase the number of centers:
size = 4 seems to do the job (the error decreases, as expected):
set.seed(21) rbf.model <- rbf(x, y, size = 4, maxit = 1000, linOut = TRUE) fitted(rbf.model)-y # [,1] # [1,] 6.058715e-08 # [2,] -2.139242e-07 # [3,] -3.251499e-07 # [4,] 1.136976e-08 # [5,] -1.115963e-08
size = 5 I should have residuals of the order of machine zero, because now my RBF network should interpolate data. This is the same as for polynomials of increasing degree - as the degree
p of the polynomial is increased up to the number of data points (sample size), the MSE decreases until it reaches 0. However...
set.seed(21) rbf.model <- rbf(x, y, size = 5, maxit = 1000, linOut = TRUE) fitted(rbf.model)-y # [,1] # [1,] -0.2098933 # [2,] -0.4351147 # [3,] -0.4650128 # [4,] -0.5511093 # [5,] -0.5425800
What the deuce?! It's not just a problem with the number of iterations, because even after increasing
maxit by two orders of magnitude (!), I still don't manage to match even the accuracy of the
size = 2 case.
set.seed(21) rbf.model <- rbf(x, y, size = 5, maxit = 100000, linOut = TRUE) fitted(rbf.model)-y # [,1] # [1,] 0.006449462 # [2,] -0.038574433 # [3,] -0.065376905 # [4,] -0.099974621 # [5,] -0.099898588
Maybe the problem could be that with
size = 5, the Gram matrix of the model becomes singular. However, there has to be a way to use RBFs for interpolation, since it's a pretty common thing.
Can you help me to understand what's happening here?