I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this mathematically?
More specifically, I want to know what kind of decision boundary does a convolution gives to us? Like A perceptron gives us a linear boundary, is there some way we can visualize convolution as giving a decision boundary?
You may be interested in this question for the representation of a convolutional layer as a fully-connected one.
Since a convolutional layer is still a linear function in the inputs, then a convolutional layer followed by a average pool / softmax layer would still be linear.
For more complicated CNNs with commonly used relu activations and max-pooling layers, note that every component is still piece-wise linear. So the decision boundary is also piece-wise linear.
The main difficulty with visualizing this boundary is that images are very high dimensional, (almost 200K dimensions for a small 256x256 RGB image), so it's not really that feasible. You could use any number of dimensionality reducing / latent space embedding techniques to place each image in 2D-euclidean space, but this transformation is often non-linear (and thus your boundary would no longer be).
