I have started to look at Radial-Basis-Function Neural Networks (RBF-NN), and would like to solve the following task as an exercise: Given the data in the image below. Each output $Y_j$ of the network is given by:

$(*)$ $Y_j = \sum_{i}^N w_{ij}* exp(-\frac{||x-\mu_i||^2}{2\sigma_i^2})$

for the -ith neuron. The task is to draw an RBF network that perfectly classifies the data, with suitable means, covariances and weights. In a second step a point has to be taken and classified accordingly.

My ideas: Here we have two nodes in the input layer - one for each dimension. The hidden-layer has as many neurons as there are training samples. Each of these calculates the activation as given by the exponential above. The output layer has three output nodes. I would determine $\Sigma$ and $\mu$ as follows: For each class and trainingsample $x_i$ of this class, $\mu_j$ is just the centroid of the training samples, $\sigma = \frac{1}{m}\sum_i ||x_i-\mu||$, and $\Sigma = \sigma*I_d$.

Questions: That does not seem to be correct however - if I have the same $\Sigma$ and $\mu$ for all the hidden nodes of the same class, each new test-input would result in the same activation for all these nodes..so what exactly are the particular $u_i$ and $\sigma_i$? Also, for perfect classification, I would set all the weights which belong to the correct class to 1, and all the others to zero - would that make sense?