Skip to main content
4 of 7
added 35 characters in body
Aksakal
  • 62.3k
  • 6
  • 106
  • 206

I'll give you a very loose analogy (emphasis is important here) that may help you understand the intuition. There's this technical drawing tool, called a French curve, here's an example:

enter image description here

We were trained to use it in high school in a technical drawing class. These days, the same class is taught with CAD software, so you may not have encountered it. See, how to use them in this video. Compare this to a simple ruler:

enter image description here

It's more efiicient to use a French curve to draw curved lines that doing with a simple ruler. You'd have to make a lot of small lines to draft any smooth curve with the latter.

In machine learning you'd need a lot more neurons and layers with a linear activation compared to ReLU. The reasoning is similar to French curves: your solutions are nonlinear.

Aksakal
  • 62.3k
  • 6
  • 106
  • 206