According to the universal Approximation theorem, we can approximate any given function with two-layer neural networks with a sufficient number of nodes. Then Why do CNNs work well with natural data such as speech, images, and text?

  • 2
    $\begingroup$ CNNs do regularization and/or feature extraction in a clever way, fighting overfitting issues that diminish MLP performance. Have you ever drawn a CNN as a network instead of as rectangles on top of other rectangles? I found that helpful to hammer into my brain how a CNN works. $\endgroup$ – Dave Oct 17 at 17:43
  • $\begingroup$ @Dave I have tried to interpret CNN using skipped connections and parameter sharing. I think that's what you are referring to. But I can't understand how is CNN doing regularization? $\endgroup$ – Aravind P Oct 17 at 19:11
  • $\begingroup$ It biases an MLP by forcing certain parameters to be zero and certain others to be equal. $\endgroup$ – Dave Oct 17 at 19:35
  • 2
    $\begingroup$ (1) the statement of the UAT is not precise. (2) I don’t understand how your question follows from your claim about the UAT. Can you elaborate on where you’re having trouble? $\endgroup$ – Sycorax Oct 18 at 16:22