1
$\begingroup$

I was reading the following paper on hyper & radial basis function (HBFs & RBFs) networks and also this one that kind of summarizes the first one and was trying to understand how to extend their results to vector values outputs.

In the paper they mainly discuss all the result for real value outputs but I can't seem to find where they discuss the results for multi-valued outputs. On page 29 they seem to imply they did talk about this but its unclear. In particular they say:

we have so far considered GRBFs as a technique to solve the problem of approximating real values functions $f:R^n \rightarrow R^m$, ...

however, as far as I can tell they only explain and give the equations for the case $f:R^n \rightarrow R$. I was wondering if people understood how to extend HBFs networks to multiple outputs.

In particular this is more or less what I conjecture the optimization problem is and its solution (but wasn't sure if I could derive it).

So we would consider the variational optimization problem (similar to the one on the paper):

$$ H[f] = \sum^N_{i=1} \frac{1}{2} \| y_i - f(x_i) \|^2 + \lambda \| Pf \|^2$$

where $f(x_i), y_i \in R^m$.

And I would hope the solution would look something like this:

$$ y_i = [..., y_{i,j} ,...] = [..., f(x_i)_j , ...] = [..., \sum^K_{k=1} c_{k,j} G(\| x_i - t_k \|^2),...]$$

where $K$ is the number of adjustable centers (and hence number of hidden units), $c_{a,j}$ are the weights from hidden unit a to output unit $j$. In fact I hope this can be expressed in the following network:

enter image description here

Furthermore, on top of that, I think the gradient descent equations have to be re-derived.

$\endgroup$
  • $\begingroup$ Why would you do it for all coordinates simultaneously? For $f:R^n \to R^m$, take $f = (f_1,f_2,\dots,f_m)$ and optimize for each $f_j$ separately. $\endgroup$ – sandris Jul 26 '15 at 10:45
  • $\begingroup$ @sandris What do you mean by "Why would you do it for all coordinates simultaneously?"? Do you mean why I would minimize the error of the vectors? Or why would I choose operator P that correlates the coordinates? $\endgroup$ – Charlie Parker Jul 26 '15 at 14:55
  • $\begingroup$ Ah, okay, so P correlates the coordinates, makes sense then. $\endgroup$ – sandris Jul 26 '15 at 19:13
  • $\begingroup$ @sandris I think it might dependent on the specific differential operator P that is chosen. However, since it has an adjoint, it should be linear I believe.. $\endgroup$ – Charlie Parker Jul 26 '15 at 21:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.