I get that for z > 0 , the gradient of the ReLU function is 1 , hence the gradient descent algorithm can proceed faster. But what happens if z < 0? How would the gradient descent algorithm proceed then?
Both back and forward propagation are faster to evaluate for the same reason: relu doesn't need any calculation. The gradient is either 0 on the negative value or 1 for the positive.
I wouldn't say that the gradient descent moves faster due to gradient being 1 on the positive values. It's only the calculations are faster.
The gradient is zero, which means nothing gets backpropagated through them; the precise value fed to this neuron doesn't matter when $z<0$. If a RELU unit is always in the left part of the graph, you get the dying/dead RELU problem; this is one of the reasons that people have looked for alternatives to RELUs.
Sigmoid activations get a similar problem at both ends; even though they're not exactly flat like a RELU, they're close enough once you get to +/- 5 or so.