Return to Question

I would like to solve the following task as an exercise: Given is the data in the image below. Each output $Y_j$ of the network is defined as:

$(*)$ $Y_j = \sum_{i}^N w_{ij}* exp(-\frac{||x-\mu_i||^2}{2\sigma_i^2})$

for the -ith neuron. The task is to draw an RBF network that perfectly classifies the data, with suitable means, covariances and weights. In a second step a point has to be taken and classified in the worked-out model.

My ideas We have two nodes in the input layer, one for each dimension. The hidden-layer has as many neurons as there are training samples. Each of these calculates the activation given by the exponential above. The output layer has three output nodes as there are three classes. I would determine $\Sigma$ and $\mu$ as follows: For each class and training-sample $x_i$ of this class, $\mu_j$ is just the centroid of the training samples, $\sigma = \frac{1}{m}\sum_i ||x_i-\mu||$, and $\Sigma = \sigma*I_d$.

Questions: That does not seem to be correct however - if I have the same $\Sigma$ and $\mu$ for all the hidden nodes of the same class, each new test-input would result in the same activation for all these nodes..so what exactly are the particular $u_i$ and $\sigma_i$? Also, for perfect classification, I would set all the weights which belong to the correct class to 1, and all the others to zero - would that make sense?

I would like to solve the following task as an exercise: Given is the data in the image below. Each output $Y_j$ of the network is defined as:

$(*)$ $Y_j = \sum_{i}^N w_{ij}* exp(-\frac{||x-\mu_i||^2}{2\sigma_i^2})$

for the -ith neuron. The task is to draw an RBF network that perfectly classifies the data, with suitable means, covariances and weights. In a second step a point has to be taken and classified in the worked-out model.

My ideas We have two nodes in the input layer, one for each dimension. The hidden-layer has 4 nodes, as there are four clusters. Each of these calculates the activation given by the exponential above. The output layer has three output nodes as there are three classes. I would determine $\Sigma$ and $\mu$ as follows: For each cluster and it's datapoints $x_i$, $\mu_j$ is just the centroid of the training samples, $\sigma = \frac{1}{m}\sum_i ||x_i-\mu||$, and $\Sigma = \sigma*I_d$. I would set all weights to the correct class to 1, and everything else to zero for perfect classification.

Questions:

• If we look at the architecture of the RBF neural network, what are the weights going into the hidden layers? Usually, we sum over the inputs and weights to hidden layers $\sum x_i w_i$. However it seems that for the RBF-activation function, no such summation is needed
• Where are the means of the clusters represented in the RBF architecture? Are the means actually represented in the weights to the hidden layer, and we do not sum up input and these weights but instead subtract them, i.e calculate for each dimension of the input its distance to the respective dimension of the mean?

I have started to look at Radial-Basis-Function Neural Networks (RBF-NN), and would like to solve the following task as an exercise: Given is the data in the image below. Each output $Y_j$ of the network is defined as:

$(*)$ $Y_j = \sum_{i}^N w_{ij}* exp(-\frac{||x-\mu_i||^2}{2\sigma_i^2})$

for the -ith neuron. The task is to draw an RBF network that perfectly classifies the data, with suitable means, covariances and weights. In a second step a point has to be taken and classified in the worked-out model.

My ideas We have two nodes in the input layer, one for each dimension. The hidden-layer has as many neurons as there are training samples. Each of these calculates the activation given by the exponential above. The output layer has three output nodes as there are three classes. I would determine $\Sigma$ and $\mu$ as follows: For each class and training-sample $x_i$ of this class, $\mu_j$ is just the centroid of the training samples, $\sigma = \frac{1}{m}\sum_i ||x_i-\mu||$, and $\Sigma = \sigma*I_d$.

Questions: That does not seem to be correct however - if I have the same $\Sigma$ and $\mu$ for all the hidden nodes of the same class, each new test-input would result in the same activation for all these nodes..so what exactly are the particular $u_i$ and $\sigma_i$? Also, for perfect classification, I would set all the weights which belong to the correct class to 1, and all the others to zero - would that make sense?

I would like to solve the following task as an exercise: Given is the data in the image below. Each output $Y_j$ of the network is defined as:

$(*)$ $Y_j = \sum_{i}^N w_{ij}* exp(-\frac{||x-\mu_i||^2}{2\sigma_i^2})$

for the -ith neuron. The task is to draw an RBF network that perfectly classifies the data, with suitable means, covariances and weights. In a second step a point has to be taken and classified in the worked-out model.

My ideas We have two nodes in the input layer, one for each dimension. The hidden-layer has as many neurons as there are training samples. Each of these calculates the activation given by the exponential above. The output layer has three output nodes as there are three classes. I would determine $\Sigma$ and $\mu$ as follows: For each class and training-sample $x_i$ of this class, $\mu_j$ is just the centroid of the training samples, $\sigma = \frac{1}{m}\sum_i ||x_i-\mu||$, and $\Sigma = \sigma*I_d$.

Questions: That does not seem to be correct however - if I have the same $\Sigma$ and $\mu$ for all the hidden nodes of the same class, each new test-input would result in the same activation for all these nodes..so what exactly are the particular $u_i$ and $\sigma_i$? Also, for perfect classification, I would set all the weights which belong to the correct class to 1, and all the others to zero - would that make sense?

I have started to look at Radial-Basis-Function Neural Networks (RBF-NN), and would like to solve the following task as an exercise: Given is the data in the image below. Each output $Y_j$ of the network is defined as:

$(*)$ $Y_j = \sum_{i}^N w_{ij}* exp(-\frac{||x-\mu_i||^2}{2\sigma_i^2})$

for the -ith neuron. The task is to draw an RBF network that perfectly classifies the data, with suitable means, covariances and weights. In a second step a point has to be taken and classified accordingly.

My ideas We have two nodes in the input layer, one for each dimension. The hidden-layer has as many neurons as there are training samples. Each of these calculates the activation given by the exponential above. The output layer has three output nodes. I would determine $\Sigma$ and $\mu$ as follows: For each class and training-sample $x_i$ of this class, $\mu_j$ is just the centroid of the training samples, $\sigma = \frac{1}{m}\sum_i ||x_i-\mu||$, and $\Sigma = \sigma*I_d$.

Questions: That does not seem to be correct however - if I have the same $\Sigma$ and $\mu$ for all the hidden nodes of the same class, each new test-input would result in the same activation for all these nodes..so what exactly are the particular $u_i$ and $\sigma_i$? Also, for perfect classification, I would set all the weights which belong to the correct class to 1, and all the others to zero - would that make sense?

I have started to look at Radial-Basis-Function Neural Networks (RBF-NN), and would like to solve the following task as an exercise: Given is the data in the image below. Each output $Y_j$ of the network is defined as:

$(*)$ $Y_j = \sum_{i}^N w_{ij}* exp(-\frac{||x-\mu_i||^2}{2\sigma_i^2})$

for the -ith neuron. The task is to draw an RBF network that perfectly classifies the data, with suitable means, covariances and weights. In a second step a point has to be taken and classified in the worked-out model.

My ideas We have two nodes in the input layer, one for each dimension. The hidden-layer has as many neurons as there are training samples. Each of these calculates the activation given by the exponential above. The output layer has three output nodes as there are three classes. I would determine $\Sigma$ and $\mu$ as follows: For each class and training-sample $x_i$ of this class, $\mu_j$ is just the centroid of the training samples, $\sigma = \frac{1}{m}\sum_i ||x_i-\mu||$, and $\Sigma = \sigma*I_d$.

Questions: That does not seem to be correct however - if I have the same $\Sigma$ and $\mu$ for all the hidden nodes of the same class, each new test-input would result in the same activation for all these nodes..so what exactly are the particular $u_i$ and $\sigma_i$? Also, for perfect classification, I would set all the weights which belong to the correct class to 1, and all the others to zero - would that make sense?