I read all the answers and still feel that I need to write a new one.
Let us consider a linear activation function g(z)=z, which is different from Relu(z) only in the region z<0.
If all activation functions used in a network is g(z), then the network is equivalent to a simple single layer linear network,
which we know is not useful in learning complicate patterns.
We need to introduce nonlinearity into the network. So the interesting part of Relu(z) is actually its combination of this linear part with another linear part that has a different slope (specifically, zero in Relu). This introduces a nonlinearity we need, which seems to be the most simple nonlinearity that one can think of.
However simplicity itself does not imply superiorness over complexity in terms of its practical use.
Then why is this simple nonlinearity more powerful than the sigmoid function?
Both relu and sigmoid have regions of zero derivative. Other anwsers have claimed that relu has a reduced chance of encountering the vanishing gradient problem based on the facts that (1) its zero derivative region is narrower than sigmoid and (2) relu's derivative for z>0 is equal to one, which is not damped or enhanced when multiplied.
Other possible reasons for the advantage of relu over sigmoid may be that (1) Relu has larger possible range than that of the sigmoid function for z>0. (2) The exact zero values of relu for z<0 introduce sparsity effect in the network, which forces the network to learn more robust features. If this is true, something like leaky Relu, which is claimed as an improvement over relu, may be actually damaging the efficacy of Relu.
Some people consider relu very strange at first glance. It turns out that the adoption of relu is a natrual choice if we consider that (1) sigmoid is a modified version of the step function (g=0 for z<0, and g=1 for z>0) to make it continous near zero; (2) another imaginable modified version of the step function would be replacing g=1 in z>0 by g=z, which is relu.