Return to Answer

I'll give you a very loose analogy (emphasis is important here) that may help you understand the intuition. There's this technical drawing tool, called a French curve, here's an example:

We were trained to use it in high school in a technical drawing class. These days, the same class is taught with CAD software, so you may not have encountered it. See, how to use them in this video.

Here's a straight ruler:

Can you draw a curved line with a straight ruler? Of course, you can! However, it's more work. Take a look at this video to appreciate the difference.

It's more efficient to use a French curve to draw curved lines than with a straight ruler. You'd have to make a lot of small lines to draft any smooth curve with the latter.

It's not exactly the same with machine learning, but this analogy provides you with an intuition why nonlinear activation may work better in many cases: your problems are nonlinear, and having nonlinear pieces can be more efficient when combining them into a solution to nonlinear problems.

I'll give you a very loose analogy (emphasis is important here) that may help you understand the intuition. There's this technical drawing tool, called a French curve, here's an example:

We were trained to use it in high school in a technical drawing class. These days, the same class is taught with CAD software, so you may not have encountered it. See, how to use them in this video.

Here's a straight ruler:

_{(source: officeworks.com.au)}

Can you draw a curved line with a straight ruler? Of course, you can! However, it's more work. Take a look at this video to appreciate the difference.

It's more efficient to use a French curve to draw curved lines than with a straight ruler. You'd have to make a lot of small lines to draft any smooth curve with the latter.

It's not exactly the same with machine learning, but this analogy provides you with an intuition why nonlinear activation may work better in many cases: your problems are nonlinear, and having nonlinear pieces can be more efficient when combining them into a solution to nonlinear problems.

I'll give you a very loose analogy (emphasis is important here) that may help you understand the intuition. There's this technical drawing tool, called a French curve, here's an example:

We were trained to use it in high school in a technical drawing class. These days, the same class is taught with CAD software, so you may not have encountered it. See, how to use them in this video.

Here's a straight ruler:

Can you draw a curved line with a straight ruler? Of course, you can! However, it's more work. Take a look at this video to appreciate the difference.

It's more efficient to use a French curve to draw curved lines than with a straight ruler. You'd have to make a lot of small lines to draft any smooth curve with the latter.

It's not exactly the same with machine learning, but this analogy provides you with an intuition why nonlinear activation may work better in many cases: your problems are nonlinear, and having nonlinear pieces can be more efficient when combining them into a solution to nonlinear problems.

I'll give you a very loose analogy (emphasis is important here) that may help you understand the intuition. There's this technical drawing tool, called a French curve, here's an example:

We were trained to use it in high school in a technical drawing class. These days, the same class is taught with CAD software, so you may not have encountered it. See, how to use them in this video.

Here's a straight ruler:

Can you draw a curved line with a straight ruler? Of course, you can! However, it's more work. Take a look at this video to appreciate the difference.

It's more efficient to use a French curve to draw curved lines than with a straight ruler. You'd have to make a lot of small lines to draft any smooth curve with the latter.

It's not exactly the same with machine learning, but this analogy provides you with an intuition why nonlinear activation may work better in many cases: your problems are nonlinear, and having nonlinear pieces can be more efficient when combining them into a solution to nonlinear problems.

I'll give you a very loose analogy (emphasis is important here) that may help you understand the intuition. There's this technical drawing tool, called a French curve, here's an example:

We were trained to use it in high school in a technical drawing class. These days, the same class is taught with CAD software, so you may not have encountered it. See, how to use them in this video. Compare this to a simple ruler:

It's more efiicient to use a French curve to draw curved lines that doing with a simple ruler. You'd have to make a lot of small lines to draft any smooth curve with the latter.

In machine learning you'd need a lot more neurons and layers with a linear activation compared to ReLU. The reasoning is similar to French curves: your solutions are nonlinear.

I'll give you a very loose analogy (emphasis is important here) that may help you understand the intuition. There's this technical drawing tool, called a French curve, here's an example:

We were trained to use it in high school in a technical drawing class. These days, the same class is taught with CAD software, so you may not have encountered it. See, how to use them in this video.

Here's a straight ruler:

Can you draw a curved line with a straight ruler? Of course, you can! However, it's more work. Take a look at this video to appreciate the difference.

It's more efficient to use a French curve to draw curved lines than with a straight ruler. You'd have to make a lot of small lines to draft any smooth curve with the latter.

It's not exactly the same with machine learning, but this analogy provides you with an intuition why nonlinear activation may work better in many cases: your problems are nonlinear, and having nonlinear pieces can be more efficient when combining them into a solution to nonlinear problems.