I'm trying to understand Radial Basis Function Network. I have (don' know how to write proper formatter mathematical functions here..):

$x = [ -1.0000, -0.5000, 0,0.5000,1.0000]$

$y_i = f(x_i)$

$f(x) = \ \frac{1}{(1+x^2)}$

$y = [0.5000, 0.8000,1.0000, 0.8000,0.5000]$

xy = [ -1.0000    0.5000
       -0.5000    0.8000
             0    1.0000
        0.5000    0.8000
        1.0000    0.5000]

Then there is:

$Phi_j(x) = exp(-(|x - t_j|/4))$

$t_j = -1 + (j - 1)\ \times\ \frac{2}{m_1 - 1}$

$m_1 <= N$

And then G matrix is written like this:

$G = (Phi_j(x_i,t_j))$

This matrix is used to calculate weights:

$w = G^+y$

$G^+ = (G'G)^-1G'$

So it looks for me that G uses Phi function with two inputs, but Phi is written as one input function. Do I miss something here?

  • $\begingroup$ please fix your latex -- this is indecipherable. $\endgroup$ – Sycorax Nov 6 '17 at 18:30

Actually the point is that in RBF you approximate the density of probability of the function (in the case of regression) through a linear combination of kernel functions of the form,

$$f(x) = \sum_{n}w_{n}G(\lVert x-x_{n} \rVert)$$

where $x_{n}$ are the training data points. The two parameters you are referring to are the centers ($t_{n}$ in your notation) and the point you want to evaluate the function at, $x$. There are actually many possible elections for the kernel function, $G$. Each choice corresponds to a different type of regularization. One you make a choice, solving the problem reduces to a linear algebraic equation.

If you are interested on the mathematical basis of RBF, you may take a look at this paper.

| cite | improve this answer | |

You can view RBF as linear regression with a non linear transformation of the inputs. ( just as you could use x^2, x^3 etc). The G+ Matrix is just standard linear regression formula. So first step, fix the t's - this could be either by taking random x (x[r_1], x[r_2],...)samples from your inputs, or again by performing unsupervised learning on your inputs to find "prototypes". These ts are now fixed, and you calculate G by applying phi to all x[i's] and t[j]s. Finally you perform linear regression, namely the G(+)

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.