Skip to main content
7 of 7
broken image fixed (click 'Inline' or 'Side-by-side' to see the difference; image retrieved via Wayback Machine); for more info, see https://git.io/JqCQN

I'll give you a very loose analogy (emphasis is important here) that may help you understand the intuition. There's this technical drawing tool, called a French curve, here's an example:

enter image description here

We were trained to use it in high school in a technical drawing class. These days, the same class is taught with CAD software, so you may not have encountered it. See, how to use them in this video.

Here's a straight ruler:

enter image description here
(source: officeworks.com.au)

Can you draw a curved line with a straight ruler? Of course, you can! However, it's more work. Take a look at this video to appreciate the difference.

It's more efficient to use a French curve to draw curved lines than with a straight ruler. You'd have to make a lot of small lines to draft any smooth curve with the latter. enter image description here

enter image description here

It's not exactly the same with machine learning, but this analogy provides you with an intuition why nonlinear activation may work better in many cases: your problems are nonlinear, and having nonlinear pieces can be more efficient when combining them into a solution to nonlinear problems.

Aksakal
  • 62.3k
  • 6
  • 106
  • 206