# Are all neural network activation functions differentiable?

When we design a neural network, we use gradient descent to learn the parameters. Does this require the activation function to be differentiable?

No! For example, ReLU, which is a widely used activation function, is not differentiable in $z=0$. But they are usually non-differentiable at only a small number of points and they have right derivative and left derivatives at these points. We usually use one of the one-side derivatives. This is rational since digital computers are subject to numerical errors ($z=0$ has been probably some small value rounded to zero). Read chapter 6 of the following book for more details on activation functions:
If you're going to use gradient descent to learn parameters, you need not only the activation functions to be differential almost everywhere, but ideally the gradient should be non-zero for large parts of the domain. It is not a strict requirement that the gradient be non-0 almost everywhere. For example ReLU has gradient zero for $x \le 0$, and it works pretty well. But whilst the input is in an area of zero gradient, no learning will take place. This manifests in practice in a few ways:
("almost everywhere" means with the exception of a finite, or countably infinite, set of points. eg, as Hossein points out, ReLU is not differentiable at $x = 0$)