What are the main differences between two types of feedforward networks such as multilayer perceptrons (MLP) and radial basis function (RBF)?
What are the fundamental differences between these two types?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
- MLP: uses dot products (between inputs and weights) and sigmoidal activation functions (or other monotonic functions such as ReLU) and training is usually done through backpropagation for all layers (which can be as many as you want). This type of neural network is used in deep learning with the help of many techniques (such as dropout or batch normalization);
- RBF: uses Euclidean distances (between inputs and weights, which can be viewed as centers) and (usually) Gaussian activation functions (which could be multivariate), which makes neurons more locally sensitive. Thus, RBF neurons have maximum activation when the center/weights are equal to the inputs (look at the image below). Due to this property, RBF neural networks are good for novelty detection (if each neuron is centered on a training example, inputs far away from all neurons constitute novel patterns) but not so good at extrapolation. Also, RBFs may use backpropagation for learning, or hybrid approaches with unsupervised learning in the hidden layer (they usually have just 1 hidden layer). Finally, RBFs make it easier to grow new neurons during training.
$\begingroup$
$\endgroup$
RBF and MLP belong to a class of neural networks called feed-forward networks.
Hidden layer of RBF is different from MLP. It performs some computations. Each hidden unit act as a point in input space and activation/output for any instance depends on the distance between that point(Hidden Unit) and instance(Also a point in space).
Thus, RBF learns two kinds of parameters:
1) Centers and width of RBFs
2) Weights to from linear combination of outputs obtained from hidden layer
The first set of parameters can be learned independently from the second set of parameters. i.e. by performing K-means clustering, you can come with centers and width of RBFs.
In nutshell, RBFs use the distance between RBFs centers and instances as the similarity measure.
One disadvantage is RBFs give the same weight to every attribute as they are considered equally in the distance computation.
