6
$\begingroup$

This is not a pure programming question. I am trying to understand what's going on when I try to use RBF with 5 centers. I am using R to exemplify, see below.

My data set:

x <- c(0.0, 11.0, 17.9, 49.3, 77.4)
y <- c(0.2497978, 0.5090220, 0.5373010, 0.6032853, 0.5938590)

I would like to interpolate this data with a RBF network. I start from regression (i.e., number of centers < number of points), and things look fine, but when I get to interpolation, something breaks down. So, let's start with 2 centers:

library(RSNNS)
set.seed(21)
rbf.model <- rbf(x, y, size = 2, maxit = 1000, linOut = TRUE)
fitted(rbf.model)-y
#                [,1]
#  [1,] -2.100049e-03
#  [2,]  2.320548e-02
#  [3,]  2.218202e-02
#  [4,] -4.328762e-02
#  [5,] -1.303689e-07

Not bad, as a first attempt. Let's increase the number of centers: size = 4 seems to do the job (the error decreases, as expected):

set.seed(21) 
rbf.model <- rbf(x, y, size = 4, maxit = 1000, linOut = TRUE)
fitted(rbf.model)-y
#                [,1]
#  [1,]  6.058715e-08
#  [2,] -2.139242e-07
#  [3,] -3.251499e-07
#  [4,]  1.136976e-08
#  [5,] -1.115963e-08

With size = 5 I should have residuals of the order of machine zero, because now my RBF network should interpolate data. This is the same as for polynomials of increasing degree - as the degree p of the polynomial is increased up to the number of data points (sample size), the MSE decreases until it reaches 0. However...

set.seed(21) 
rbf.model <- rbf(x, y, size = 5, maxit = 1000, linOut = TRUE)
fitted(rbf.model)-y
#             [,1]
#  [1,] -0.2098933
#  [2,] -0.4351147
#  [3,] -0.4650128
#  [4,] -0.5511093
#  [5,] -0.5425800

What the deuce?! It's not just a problem with the number of iterations, because even after increasing maxit by two orders of magnitude (!), I still don't manage to match even the accuracy of the size = 2 case.

set.seed(21) 
rbf.model <- rbf(x, y, size = 5, maxit = 100000, linOut = TRUE)
fitted(rbf.model)-y
#               [,1]
#  [1,]  0.006449462
#  [2,] -0.038574433
#  [3,] -0.065376905
#  [4,] -0.099974621
#  [5,] -0.099898588

Maybe the problem could be that with size = 5, the Gram matrix of the model becomes singular. However, there has to be a way to use RBFs for interpolation, since it's a pretty common thing.

Can you help me to understand what's happening here?

$\endgroup$
  • 8
    $\begingroup$ Asking for an explanation of what's going on when you try to use RBF with 5 centres is on-topic IMO, though answers needn't focus on R. $\endgroup$ – Scortchi Sep 9 '16 at 14:38
  • $\begingroup$ Error should be reduced in the 5th iteration, but it never goes to zero. $\endgroup$ – akansha Jun 26 '17 at 11:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.