I would like to solve the following task as an exercise: Given is the data in the image below. Each output $Y_j$ of the network is defined as:

$(*)$  $Y_j = \sum_{i}^N w_{ij}* exp(-\frac{||x-\mu_i||^2}{2\sigma_i^2})$

for the -ith neuron. **The task** is to draw an RBF network that *perfectly* classifies the data, with suitable means, covariances and weights. In a second step a point has to be taken and classified in the worked-out model.

**My ideas** We have two nodes in the input layer, one for each dimension. The hidden-layer has 4 nodes, as there are four clusters. Each of these calculates the activation given by the exponential above. The output layer has three output nodes as there are three classes. I would determine $\Sigma$ and $\mu$ as follows: For each cluster and it's datapoints $x_i$, $\mu_j$ is just the centroid of the training samples, $\sigma = \frac{1}{m}\sum_i ||x_i-\mu||$, and $\Sigma = \sigma*I_d$. I would set all weights to the correct class to 1, and everything else to zero for perfect classification.

**Questions:** 

- If we look at the architecture of the RBF neural network, what are the weights going into the hidden layers? Usually, we sum over the inputs and weights to hidden layers $\sum x_i w_i$. However it seems that for the RBF-activation function, no such summation is needed
- Where are the means of the clusters represented in the RBF architecture? Are the means actually represented in the weights to the hidden layer, and we do not sum up input and these weights but instead subtract them, i.e calculate for each dimension of the input its distance to the respective dimension of the mean?

[![enter image description here][1]][1]


  [1]: https://i.sstatic.net/I90YM.jpg