In some literature the convolution layers of convolutional neural networks have shared weights (e.g. see "shared weights" at deeplearning.net tutorial) but I already read some papers where the weights were non-shared and rather calculated like in "normal MLPs".
So, what is the advantage of shared weights for convolutional neural networks? Are there any pros and cons? Or is it just an architecture-based decision, like "we noticed, our network works better with shared weights"?
The main advantage of shared weights, is that you can substantially lower the degrees of freedom of your problem. Take the simplest case, think of a tied autoencoder, where the input weights are $W_{x} \in \mathbb{R}^d$ and the output weights are $W_{x}^T$. You have lowered the parameters of your model by half from $2d \rightarrow d$. You can see some visualizations here: link. Similar results would be obtained in a Conv Net.
This way you can get the following results:
Therefore, it is a nice tweak to experiment with and I would suggest you to try both. I've seen cases where sharing information (sharing weights), has paved the way to better performance, and others, that made my model become significantly less flexible.
@iassael emphasized more on regularization effect as a result of reduced parameters, but I think better performance of weight sharing method is more about finding local features instead of global one. This reduces exponential possibilities to a linear scale or at least to a scale that can be more easily managed. Here is a simple example. Let's say we have an input with only 4 pixels and each pixel has only binary values '0' or '1'. there are 2^4 = 16 possible configuration for a global feature to learn. However, if a local feature, let's say it has only 1 pixel receptive field, is used, it is enough to learn 2 simple feature '0' and '1'. As receptive field size increases, number of feature needed to learnt also increases. As a result, local features reduces number of features that are needed to learnt. By convolving this local features on all input space it can be found where this features are present exaclty.
Let's apply the same analogy to an object detection test. If fully connected first layer attempts to extract all possible configurations of edges at given images, it needs to learn many combination of different edges which is plenty. However, if it tries to learn local features, number of possible edges greatly reduced to different edge orientations. Then via convolution it can reveals which location mostly activates a specific edge orientation.
A typical weight sharing technique found in CNN treats the input as a hierarchy of local regions. It imposes a general assumption (prior knowledge) that the input going to be processed by the network can be decomposed into a set of local regions with the same nature and thus each of them can be processed with the same set of transformations. With the prior assumption, we could reduce the amount of parameters in the network (as compared with a fully-connected network) and increase the network’s generalisation ability, given that this prior assumption is a correct one for the problem.
