I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this mathematically?

More specifically, I want to know what kind of decision boundary does a convolution gives to us? Like A perceptron gives us a linear boundary, is there some way we can visualize convolution as giving a decision boundary?

  • $\begingroup$ good question, i think i had the same question for a while. the answer should be Yes, but how to make it nicely in math may be a problem. $\endgroup$
    – Haitao Du
    Commented May 27, 2020 at 5:48
  • $\begingroup$ stats.stackexchange.com/a/409172/247274 $\endgroup$
    – Dave
    Commented May 1 at 10:25

1 Answer 1


You may be interested in this question for the representation of a convolutional layer as a fully-connected one.

Since a convolutional layer is still a linear function in the inputs, then a convolutional layer followed by a average pool / softmax layer would still be linear.

For more complicated CNNs with commonly used relu activations and max-pooling layers, note that every component is still piece-wise linear. So the decision boundary is also piece-wise linear.

The main difficulty with visualizing this boundary is that images are very high dimensional, (almost 200K dimensions for a small 256x256 RGB image), so it's not really that feasible. You could use any number of dimensionality reducing / latent space embedding techniques to place each image in 2D-euclidean space, but this transformation is often non-linear (and thus your boundary would no longer be).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.